Machine Learning Implementations

This project is a collection of machine learning models that I developed during my time as a student in CS 549 Machine Learning at San Diego State University. It includes implementations of various machine learning techniques, ranging from foundational models like linear and logistic regression to advanced architectures like transformers and convolutional neural networks. Each model was implemented with a focus on understanding the underlying principles, and many were implemented from first principles using Python and NumPy. Additionally, I optimized these implementations to run on GPU-enabled systems, significantly improving computational efficiency. The code for this project can be found at the GitHub link below.

My Role

As a student, I not only met the expectations of the course by implementing these models, but I exceeded them by:

• Extending implementations to leverage GPU acceleration.
• Exploring optimization techniques for improving model training.
• Developing reusable code to streamline experimentation with various architectures.

Technical Details

• Programming Languages and Tools: Python, Jupyter Notebook, NumPy, PyTorch, Scikit Learn, Pandas, Matplotlib.
• Models Implemented:
  • Linear Regression: Implemented from first principles using NumPy for matrix operations.
  • Logistic Regression: Developed from first principles with support for gradient descent optimization.
  • Principal Component Analysis (PCA): Used for dimensionality reduction and implemented from first principles.
  • K-Means Clustering: Designed from first principles for unsupervised learning tasks.
  • Neural Networks: Built both from first principles and the PyTorch library.
  • Recurrent Neural Networks (RNNs): Implemented sequence models using PyTorch.
  • Convolutional Neural Networks (CNNs): Developed image recognition models using PyTorch that were trained on a GPU.
  • Word Embeddings: Created CBOW and GloVe embeddings used for natural language processing.
  • Transformers: Implemented a multi-head attention mechanism for transformer encoders from first principles and explored pretrained models like BERT.
• GPU Acceleration: Leveraged PyTorch and Nvidia CUDA for GPU training, improving performance on large datasets and deep architectures.
• Scikit Learn: Demonstrated how foundational models like regression and clustering can be implemented with minimal code and effort.

Challenges and Solutions

• Efficiency and Scalability:
  • Challenge: Running computationally expensive models on large datasets.
  • Solution: Enabled GPU acceleration using PyTorch to speed up training.
• Optimization:
  • Challenge: Understanding and implementing optimization techniques for neural networks.
  • Solution: Explored advanced optimization strategies, including learning rate scheduling and momentum-based gradient updates.

Outcome and Impact

• Gained a deep understanding of machine learning principles by implementing models from first principles.
• Developed technical proficiency in Python, NumPy, PyTorch, and Scikit Learn.
• Improved computational efficiency by incorporating GPU acceleration, demonstrating practical problem solving skills.

Why Include This Project?

This project demonstrates my hands-on experience with machine learning, my ability to work across diverse types of models, and my commitment to mastering both foundational and advanced techniques. It also reflects my initiative and love of optimization through GPU acceleration, highlighting my technical adaptability and problem-solving skills.