What is supervised fine-tuning?
Supervised fine-tuning is the customization and enhancement of pre-trained large language models for specific tasks or domains. By leveraging a proprietary knowledge base, supervised fine-tuning allows LLMs to excel in specialized applications. Unlike traditional machine learning approaches that require extensive manual feature engineering, supervised fine-tuning capitalizes on the vast knowledge and capabilities of pre-trained LLMs. Within supervised fine-tuning are specific strategies, including:
- Domain Expertise: Fine-tuning for a specific domain, such as medical applications or engineering. This could also incorporate optimizing Retrieval-Augmented Generation (RAG) embeddings.
- Task Optimization: fine-tuning for specific tasks such as summarization or sentiment analysis. For example, in sentiment analysis, fine-tuning helps the LLM better discern the emotional tone of a given text.
- Writing Styles: fine-tuning to different writing styles such as creative, technical, formal, persuasive and more. For example, fine-tuning with informative prompts that focus on conveying factual information will result in a more objective and neutral style while fine-tuning with prompts that involve storytelling or imaginative elements will likely result in a more creative style.
How does supervised fine-tuning work?
Supervised fine-tuning works by providing the LLM with a set of training data to help it learn and adjust its internal parameters to minimize the difference between its predictions and the desired outputs. It involves modifying the weights of the pre-trained LLM based on the error between the model's predictions and the labeled data. The model is trained until it reaches a point where it can accurately perform the desired task.
The amount of fine-tuning required depends on the complexity of the task and the size of the dataset. For simpler tasks, a small amount of fine-tuning may be sufficient, while more complex tasks may require more extensive fine-tuning.
How to choose the right LLM for supervised fine-tuning
Choosing the right large language model (LLM) for supervised fine-tuning is crucial to the success of your project. There's no one-size-fits-all solution, especially considering the variety of offerings within each model family. Depending on your data type, desired outcomes, and budget, one model might suit your needs better than another.
To choose the right model to get started with follow this checklist:
- What modality or modalities will your model need?
- How big is your input and output data?
- How complex are the tasks you are trying to perform?
- How important is performance versus budget?
- How critical is AI assistant safety to your use case?
- Does your company have an existing arrangement with Azure or GCP?
For instance, if you're dealing with extremely long videos or texts (hours long or hundreds of thousands of words), Gemini 1.5 Pro might be the optimal choice, providing a context window of up to 1 million tokens. Although Anthropic's Claude has been tested with windows exceeding 1 million tokens, its production limit remains at 200K tokens.
The use cases for fine-tuning and LLM are as varied as the companies developing them. Here are some of the most common ones, paired with the LLM recommended for solving the problem presented:
| Business Use Case | Problem Type | Good Model Choice |
|---|
| An assistant to ask questions about potential fantasy football games | Extremely long videos or texts | Gemini 1.5 Pro |
| A dating assistant to help with initial conversations | Cost effective | Claude 3 Haiku |
| Creating a downstream AI assistant with highly specific domain knowledge that isn’t generally available such as an Agricultural assistant for farmers in Kenya with knowledge of local plants and pests | Fine-tuning for downstream application | GPT4 |
| Creating an AI to read doctors notes to look for inconsistencies against recommended protocols | Moderate length text for complex tasks, where performance is more important than budget | Claude 3 |
| A writer's assistant to help with story writing | Moderate length text for complex tasks where budget is more important than performance | GPT4 |
| A personal assistant catering to minorities such as the physically disabled | AI assistant safety is critical | Claude |
| Early stage development or internal proof of concept development | A deep understanding of what you are doing | Llama 2 |
The benefits of supervised fine-tuning
Supervised fine-tuning offers several key advantages that make it an attractive approach for adapting large language models to specific tasks or domains.
- Improved performance on specific tasks: One of the primary benefits of supervised fine-tuning is its ability to significantly enhance the performance of a large language model on a specific task. By providing the LLM with labeled data tailored to the target task, supervised fine-tuning allows the model to learn the specific patterns and relationships required for successful task completion. This targeted training enables the model to make more accurate predictions and generate more relevant outputs, resulting in improved overall performance on the specific task.
- Reduced training time: Supervised fine-tuning can also lead to reduced training time compared to training a large language model from scratch. Since the LLM has already been pre-trained on a vast corpus of general text data, supervised fine-tuning only requires a relatively small amount of labeled data to adapt the model to the specific task. This reduced data requirement translates into shorter training times, allowing for faster model development and deployment.
- Leveraging pre-trained knowledge: Supervised fine-tuning capitalizes on the extensive pre-trained knowledge of the underlying large language model. The LLM has already acquired a vast understanding of language patterns and general world knowledge during its pre-training phase. By leveraging this pre-trained knowledge, supervised fine-tuning enables the model to transfer its existing knowledge to the specific task at hand. This transfer learning process allows the model to learn more efficiently and effectively, leading to improved performance on the target task.
- Increased accuracy and precision: Supervised fine-tuning enhances the accuracy and precision of a large language model's predictions. By exposing the LLM to labeled data, the model learns to make more accurate predictions by aligning its outputs with the desired labels. This iterative learning process helps the model refine its predictions and minimize errors, resulting in increased accuracy and precision on the specific task.
The drawbacks of supervised fine-tuning
While supervised fine-tuning can significantly improve the performance of an LLM on a specific task, it’s important to note that fine-tuning can also lead to overfitting—models that are too closely tailored to the specific training data, making them less effective in handling variations or unseen data. This can result in reduced generalization performance and make the model less adaptable to new situations.
Additionally, supervised fine-tuning can introduce bias into the model. If a training data contains biases, such as gender or racial biases, the fine-tuned model can perpetuate or even amplify these biases, leading to unfair or inaccurate predictions. Mitigating these biases requires careful data curation and analysis.
