It's useful to read papers and blogs about machine learning, but I find that there's no subsitute for hands-on implementation to really understand a given topic in-depth, with all its nuances. I use Jupyter Notebooks to work through various ML and "AI engineering" topics, and I've included a subset of them here.
The provided Jupyter Notebooks cover a broad spectrum of machine learning topics, focusing on various aspects of data processing, fine-tuning, embeddings, vision and multimodal learning, as well as inference and evaluation considerations.
Copyright Note: In some notebooks, I have borrowed techniques or code snippets from other online sources, such as blogs or tutorials. I've tried to provide attribution in such cases. If you own the copyright of any content provided here, please email me at grayson@uffizzi.com, and I'll be happy to take it down.
License Note: These notebooks are included here for educational purposes only (both my own and the reader's). Commercial use of any code is prohibited without prior written consent.
Fine-tuning techniques are extensively explored, including methods such as quantized LoRA (QLoRA), which facilitates quick adaptation of models to specific tasks. Additionally, the notebooks delve into function calling, chat fine-tuning, and employing techniques like RAG fine-tuning and Direct Preference Optimization (DPO) to enhance model performance. Long context processing is also addressed, crucial for tasks requiring extensive context comprehension.
In this section, a comparison of embeddings models and other variables affecting retrieval-augmented generation (RAG) performace are evaluated. Additionally, I expore advanced RAG techniques, such as query pipelines, routing, and tool use, using LlamaIndex as an orchestrator.
In this section, I explore domains outside of NLP, focusing on vision, video, 3D models, and code. I also explore mixture of experts (MoE) models and models that support multimodality.
Lastly, considerations for inference are outlined, encompassing aspects of MLOps, like model serving and the deployment of LLMs, plus metrics, benchmarks, and scripts for evaluating inference performance. These practices are essential for deploying machine learning models into production environments efficiently, ensuring scalability, reliability, and performance optimization throughout the model lifecycle.