➡ Get Lifetime Access to the ADVANCEDfinetuning Repo (incl. Memorization Scripts): https://trelis.com/advancedfinetuni...
➡ Trelis Runpod Affiliate Link (supports the channel): https://runpod.io?ref=jmfkcdio
➡ Oneclick Finetuning & Inference Templates: https://github.com/TrelisResearch/one...
➡ Trelis Newsletter: https://Trelis.Substack.com
➡ Trelis Resources and Support: https://Trelis.com/About

VIDEO RESOURCES:
Slides: https://docs.google.com/presentation/...
Dataset: https://huggingface.co/datasets/Treli...

TIMESTAMPS:
0:00 Finetuning on a custom dataset
0:18 Video Overview
1:28 GPTs as statistical models
2:07 What is the reversal curse?
4:08 Synthetic dataset generation
8:28 Choosing the best batch size
14:17 What learning rate to use for finetuning?
14:50 How many epochs to train for?
16:04 Choosing the right base model
17:12 Step by step dataset generation
28:20 Finetuning script, stepbystep
40:47 Performance Ablation: Hyperparameters
42:56 Performance Ablation: Base Models
46:00 Final Recommendations for Finetuning for Memorization