This video demonstrates an innovative workflow that combines Meta's openweight Llama 3 8B model with efficient finetuning techniques (LoRA and PEFT) to deploy highly capable AI on resourceconstrained devices.

We start by using a 4bit quantized version of the Llama 3 8B model and finetune it on a custom dataset. The finetuned model is then exported in the GGUF format, optimized for efficient deployment and inference on edge devices using the GGML library.

Impressively, the finetuned Llama 3 8B model accurately recalls and generates responses based on our custom dataset when run locally on a MacBook. This demo highlights the effectiveness of combining quantization, efficient finetuning, and optimized inference formats to deploy advanced language AI on everyday devices.

Join us as we explore the potential of finetuning and efficiently deploying the Llama 3 8B model on edge devices, making AI more accessible and opening up new possibilities for natural language processing applications.

Be sure to subscribe to stay uptodate on the latest advances in AI.

My Links
Subscribe:    / @scott_ingram  
X.com:   / scott4ai  
GitHub: https://github.com/scott4ai
Hugging Face: https://huggingface.co/scott4ai

Links:
Colab Demo: https://colab.research.google.com/dri...
Dataset: https://github.com/scott4ai/llama38b...
Unsloth Colab: https://colab.research.google.com/dri...
Unsloth Wiki: https://github.com/unslothai/unsloth/...
Unsloth Web: https://unsloth.ai/