4. Fine-Tuning with Unsloth Studio So far, we used
CS
Published · 26 slides · 0 views
1 / 1
Description
4. Fine-Tuning with Unsloth Studio So far, we used models someone else built FT is our first technique that changes the model Question: When is it worth changing the model rather than changing the prompt? The cost of entry has collapsed
Related Topics
Share
Embed code
Download this presentation From Below
"4. Fine-Tuning with Unsloth Studio So far, we used" is the property of its rightful owner. Permission is granted to download and print the materials on this website for personal, non-commercial use only, and to display it on your personal computer provided you do not modify the materials and that you retain all copyright notices contained in the materials. By downloading content from our website, you accept the terms of this agreement.
Presentation Transcript
01
4. Fine-Tuning with Unsloth Studio<br>
02
So far, we used models someone else built
FT is our first technique that changes the model
Question: When is it worth changing the model rather than changing the prompt?
The cost of entry has collapsed
Lecture: Strategy, mechanics, and judgment
Labs: Data prep, training run, and evaluation and export Fine-Tuning in a Finance Course<br>
FT is our first technique that changes the model
Question: When is it worth changing the model rather than changing the prompt?
The cost of entry has collapsed
Lecture: Strategy, mechanics, and judgment
Labs: Data prep, training run, and evaluation and export Fine-Tuning in a Finance Course<br>
03
Training of pre-trained model on a smaller, curated set of your examples
Not: Training from scratch, database, a memory, or upload
Base model already knows the language
Fine-tuning teaches style, format, or task
Change is written into the weights
New behavior persists What Is Fine-Tuning<br>
Not: Training from scratch, database, a memory, or upload
Base model already knows the language
Fine-tuning teaches style, format, or task
Change is written into the weights
New behavior persists What Is Fine-Tuning<br>
04
Pre-training
Trillions of tokens, months of compute, budgets in the millions. Produces language ability and general world knowledge.
Post-training by the lab
Instruction following, conversational behavior, safety alignment.
Your fine-tune
Hundreds to thousands of examples, minutes to hours, free Colab tier
Fine-tune is a thin third layer Fine-Tuning in Model Lifecycle<br>
Trillions of tokens, months of compute, budgets in the millions. Produces language ability and general world knowledge.
Post-training by the lab
Instruction following, conversational behavior, safety alignment.
Your fine-tune
Hundreds to thousands of examples, minutes to hours, free Colab tier
Fine-tune is a thin third layer Fine-Tuning in Model Lifecycle<br>
05
Rungs, in order:
Zero-shot prompt
Few-shot prompt
Retrieval augmented generation (RAG)
Fine-tuning
Training from scratch
↑ More control, money, setup; less flexibility
Climb only as high as the problem requires
Most deployments in finance stop at RAG plus prompts Customization Ladder<br>
Zero-shot prompt
Few-shot prompt
Retrieval augmented generation (RAG)
Fine-tuning
Training from scratch
↑ More control, money, setup; less flexibility
Climb only as high as the problem requires
Most deployments in finance stop at RAG plus prompts Customization Ladder<br>
06
Format and structure
Reliably valid JSON, a fixed memo template, consistent section ordering without repeated instruction.
Tone and voice
House style, compliance-approved phrasing, the register a specific desk uses.
Task specialization
Classifying filings or headlines into a firm's own internal taxonomy.
Implicit judgment
Learning from thousands of labeled examples what your analysts actually mean by bullish, when no written rule exists.
Efficiency
Well-tuned small model can outperform a much larger general model on a narrow task, at a fraction of the inference cost.
Every item on this list is behavior, not fact. What Fine-Tuning Does Well<br>
Reliably valid JSON, a fixed memo template, consistent section ordering without repeated instruction.
Tone and voice
House style, compliance-approved phrasing, the register a specific desk uses.
Task specialization
Classifying filings or headlines into a firm's own internal taxonomy.
Implicit judgment
Learning from thousands of labeled examples what your analysts actually mean by bullish, when no written rule exists.
Efficiency
Well-tuned small model can outperform a much larger general model on a narrow task, at a fraction of the inference cost.
Every item on this list is behavior, not fact. What Fine-Tuning Does Well<br>
07
Knowledge injection
Teaching new facts through fine-tuning is possible but inefficient, unreliable, and hard to update.
Currency
The model is frozen at training.
Attribution
Fine-tuned model cannot tell you which document an answer came from, because there is no document.
Auditability/Explainability/Interpretability
Behavior lives in weights, not in inspectable text. That is a serious problem in a regulated environment.
Classic error: Fine-tuning when retrieval needed What Fine-Tuning Does Poorly<br>
Teaching new facts through fine-tuning is possible but inefficient, unreliable, and hard to update.
Currency
The model is frozen at training.
Attribution
Fine-tuned model cannot tell you which document an answer came from, because there is no document.
Auditability/Explainability/Interpretability
Behavior lives in weights, not in inspectable text. That is a serious problem in a regulated environment.
Classic error: Fine-tuning when retrieval needed What Fine-Tuning Does Poorly<br>
08
Ask the questions in order:
Can a better prompt fix it?
Can a handful of examples fix it?
Is the gap missing information, or is it persistent behavior?
Missing information → RAG.
Persistent behavior → fine-tuning
Classify these:
Summarizing 10-K filings → prompt
Sorting SEC filings into internal categories → fine-tune
Answering questions from wealth client's account → RAG
Fine-tuning and RAG are complements: Tune behavior, retrieve facts. Decision Framework<br>
Can a better prompt fix it?
Can a handful of examples fix it?
Is the gap missing information, or is it persistent behavior?
Missing information → RAG.
Persistent behavior → fine-tuning
Classify these:
Summarizing 10-K filings → prompt
Sorting SEC filings into internal categories → fine-tune
Answering questions from wealth client's account → RAG
Fine-tuning and RAG are complements: Tune behavior, retrieve facts. Decision Framework<br>
09
Full fine-tuning updates every weight
Highest quality; highest cost
Requirement for 8B LLM: data-center hardware
Risk of catastrophic forgetting: model better at your task; worse at everything else
Parameter-efficient fine-tuning (PEFT)
Freezes the base weights; trains small add-on
LoRA dominant PEFT
Every run in our labs is PEFT Full versus Parameter-Efficient Fine-Tuning (PERT)<br>
Highest quality; highest cost
Requirement for 8B LLM: data-center hardware
Risk of catastrophic forgetting: model better at your task; worse at everything else
Parameter-efficient fine-tuning (PEFT)
Freezes the base weights; trains small add-on
LoRA dominant PEFT
Every run in our labs is PEFT Full versus Parameter-Efficient Fine-Tuning (PERT)<br>
10
Small trainable filters alongside the frozen weights of base mode
1-5% of parameters are trained; original weights frozen
Inference output: frozen base + adjustment
Adapters are small, portable, swappable
Base model can host many task-specific adapters/filters Low-Rank Adaptation (LoRA)<br>
1-5% of parameters are trained; original weights frozen
Inference output: frozen base + adjustment
Adapters are small, portable, swappable
Base model can host many task-specific adapters/filters Low-Rank Adaptation (LoRA)<br>
11
Quantization weights at lower precision
4-bit quantized base model with LoRA
7B model needing 28 GB VRAM in 6 GB
Modest quality loss vs. large gain in accessibility
Why free Colab T4 is training machine not toy QLoRA and Quantization<br>
4-bit quantized base model with LoRA
7B model needing 28 GB VRAM in 6 GB
Modest quality loss vs. large gain in accessibility
Why free Colab T4 is training machine not toy QLoRA and Quantization<br>
12
Unsloth open-source library, LoRA and 4-bit quant
Twice training speed; 70% less VRAM
500+ open-weight models
Llama, Qwen, Gemma, plus vision, audio, and embedding
Moves fine-tuning from lab budget to browser tab Why Unsloth Studio<br>
Twice training speed; 70% less VRAM
500+ open-weight models
Llama, Qwen, Gemma, plus vision, audio, and embedding
Moves fine-tuning from lab budget to browser tab Why Unsloth Studio<br>
13
Unsloth Studio Training Panel<br>
14
User interface on top of the Unsloth library
Identical engine
Point and click interface for:
Model loading, dataset formatting, hyperparameter selection, training loop, live monitoring, testing, and export
Accepts CSV, JSON, PDF, and YAML as training data
Built-in comparison chat
Runs as a Colab notebook Unsloth Studio: No-Code Layer<br>
Identical engine
Point and click interface for:
Model loading, dataset formatting, hyperparameter selection, training loop, live monitoring, testing, and export
Accepts CSV, JSON, PDF, and YAML as training data
Built-in comparison chat
Runs as a Colab notebook Unsloth Studio: No-Code Layer<br>
15
Free tier
Single T4 GPU; 15 GB of VRAM, session timeouts, ephemeral disk
Sign in with personal Gmail account
Small models in 1B-8B range, small datasets, short runs
Run of 1,000-10,000 examples on a 7B model completes in roughly 15-60 min Colab<br>
Single T4 GPU; 15 GB of VRAM, session timeouts, ephemeral disk
Sign in with personal Gmail account
Small models in 1B-8B range, small datasets, short runs
Run of 1,000-10,000 examples on a 7B model completes in roughly 15-60 min Colab<br>
16
Main determinant of FT quality: dataset, not hyperparameters
Rule of thumb: Few hundred excellent examples beat tens of thousands of mediocre ones
Every example functions as an instruction
Model learns pattern including inconsistencies and errors
Consistency matters more than volume
Inconsistent labeling reliably teaches inconsistent behavior
Dataset quality is key Data Is the Product<br>
Rule of thumb: Few hundred excellent examples beat tens of thousands of mediocre ones
Every example functions as an instruction
Model learns pattern including inconsistencies and errors
Consistency matters more than volume
Inconsistent labeling reliably teaches inconsistent behavior
Dataset quality is key Data Is the Product<br>
17
Dataset Format
Standard shape is instruction, optional input, and response
Multi-turn conversational formats follow the same logic
Chat template
Forms text into the roles and special tokens a model expects
Studio applies chat template automatically
Critical rule: Inference template = training template
Most common failure:
Test-time prompts not like the training examples Dataset Format & Chat Template<br>
Standard shape is instruction, optional input, and response
Multi-turn conversational formats follow the same logic
Chat template
Forms text into the roles and special tokens a model expects
Studio applies chat template automatically
Critical rule: Inference template = training template
Most common failure:
Test-time prompts not like the training examples Dataset Format & Chat Template<br>
18
Standard shape
Multi-turn conversation Examples: Dataset Format<br>
Multi-turn conversation Examples: Dataset Format<br>
19
Standard shape Examples: Chat Template<br>
20
Six dataset families for fine-tuning:
General instruction-following
Multi-turn chat
Preference data for alignment
Reasoning and maths
Code
Domain and multimodal sets Hugging Face Datasets<br>
General instruction-following
Multi-turn chat
Preference data for alignment
Reasoning and maths
Code
Domain and multimodal sets Hugging Face Datasets<br>
21
Epochs: Complete passes
Learning rate: Adjustment speed
LoRA rank (r): Size of filter
Alpha: Training intensity
Batch size/gradient accumulation: batch size
Max steps: How long
Data split: Training, validation, and test size
Studio has these in collapsible panels with defaults
Start from the defaults and change one thing at a time. Hyperparameters Actually Matter<br>
Learning rate: Adjustment speed
LoRA rank (r): Size of filter
Alpha: Training intensity
Batch size/gradient accumulation: batch size
Max steps: How long
Data split: Training, validation, and test size
Studio has these in collapsible panels with defaults
Start from the defaults and change one thing at a time. Hyperparameters Actually Matter<br>
22
Training/validation loss: prediction error
Loss training curve
Falling → learning
Flat → nothing happening
Loss approaching zero usually → memorization
Underfitting and overfitting are visible in curve
Catastrophic forgetting
Overtraining narrows the model, degrades general quality
Hold out data before training: Training, validation, test sets
Training loss diagnostic, not sign of success. Reading the Training Run<br>
Loss training curve
Falling → learning
Flat → nothing happening
Loss approaching zero usually → memorization
Underfitting and overfitting are visible in curve
Catastrophic forgetting
Overtraining narrows the model, degrades general quality
Hold out data before training: Training, validation, test sets
Training loss diagnostic, not sign of success. Reading the Training Run<br>
23
Eyeball test, test set performance, defined task metric
Eyeball test: Studio's comparison chat
Design evaluation before training run
Ask two questions:
Did it get better at the target task, and
Did it get worse at anything else?
Model you can’t evaluate is model you can’t deploy
Like Backtesting Evaluation, or How You Know It Worked<br>
Eyeball test: Studio's comparison chat
Design evaluation before training run
Ask two questions:
Did it get better at the target task, and
Did it get worse at anything else?
Model you can’t evaluate is model you can’t deploy
Like Backtesting Evaluation, or How You Know It Worked<br>
24
Three export choices:
LoRA adapter alone (megabytes),
Merged full model (gigabytes), or
GGUF for local inference (Ollama or LM Studio)
GGUF at Q4_K_M quantization is usual
Who runs model, where, on what hardware
In Colab, download file or push it to Hugging Face
Closes the loop: Fine-tuned model now local model you can run. Export and Deployment<br>
LoRA adapter alone (megabytes),
Merged full model (gigabytes), or
GGUF for local inference (Ollama or LM Studio)
GGUF at Q4_K_M quantization is usual
Who runs model, where, on what hardware
In Colab, download file or push it to Hugging Face
Closes the loop: Fine-tuned model now local model you can run. Export and Deployment<br>
25
Transaction categorization at scale
Classifying millions of card and ACH transactions into merchant categories, expense codes, or cashflow buckets.
Credit memo and underwriting narrative drafting
Turning structured loan application data into the prose narrative your credit committee expects.
Structured extraction from filings and contracts
Pulling risk factors from 10-Ks, covenant terms from credit agreements, or key economics from schedules and term sheets.
Financial statement variance commentary
Generating the narrative that accompanies month-end close
Client-tailored research and portfolio commentary
Producing quarterly letters or market commentary in a specific analyst's or firm's voice, at scale across many client segments.
Reconciliation break explanation
Classifying and explaining why a trade, cash, or intercompany balance failed to reconcile. Fine-Tuning in Finance.<br>
Classifying millions of card and ACH transactions into merchant categories, expense codes, or cashflow buckets.
Credit memo and underwriting narrative drafting
Turning structured loan application data into the prose narrative your credit committee expects.
Structured extraction from filings and contracts
Pulling risk factors from 10-Ks, covenant terms from credit agreements, or key economics from schedules and term sheets.
Financial statement variance commentary
Generating the narrative that accompanies month-end close
Client-tailored research and portfolio commentary
Producing quarterly letters or market commentary in a specific analyst's or firm's voice, at scale across many client segments.
Reconciliation break explanation
Classifying and explaining why a trade, cash, or intercompany balance failed to reconcile. Fine-Tuning in Finance.<br>
26
Data risk:
Fine-tuning on confidential/client data may reproduce it in outputs
Licensing:
License for base model carries through.
Model risk management:
Documentation, versioning, validation, and a named owner.
Reproducibility:
Record version, base model, hyperparameters, etc.
Bottom line:
Fine-tuning is cheap, but expensive if deployed carelessly. Risk, Governance, Takeaways<br>
Fine-tuning on confidential/client data may reproduce it in outputs
Licensing:
License for base model carries through.
Model risk management:
Documentation, versioning, validation, and a named owner.
Reproducibility:
Record version, base model, hyperparameters, etc.
Bottom line:
Fine-tuning is cheap, but expensive if deployed carelessly. Risk, Governance, Takeaways<br>