M3 Max is a Machine Learning BEAST. So I took it for a spin with some LLM's running locally.
I also show how to gguf quantizations with llama.cpp
Temperature/fan on your Mac: https://www.tunabellysoftware.com/tgp... (affiliate link)
Run Windows on a Mac: https://prf.hn/click/camref:1100libNI (affiliate)
Use COUPON: ZISKIND10
Gear Links
* New MacBook Air M1 Deal: https://amzn.to/3S59ID8
* Renewed MacBook Air M1 Deal: https://amzn.to/45K1Gmk
* ⚡ Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW
* My nvme ssd: https://amzn.to/3YLEySo
* My gear: https://www.amazon.com/shop/alexziskind
Related Videos
* RAM torture test on Mac • TRUTH about RAM vs SSD UPGRADES for M...
* Set up Conda on Mac • python environment setup on Apple Sil...
* 15" MacBook Air | developer's dream • 15" MacBook Air | developer's dream
* INSANE Machine Learning on Neural Engine • INSANE Machine Learning on Neural Eng...
* M2 MacBook Air and temps • Why SILVER is FASTER
* This is what spending more on a MacBook Pro gets you • Spend MORE on a MacBook gets you THIS
* Developer productivity Playlist • Developer Productivity
AI for Coding Playlist: • AI
Timestamps
00:00 Intro
00:40 Build from scratch manual
09:44 Bonus script automated
11:21 LM Studio one handed
Repo
https://github.com/ggerganov/llama.cpp/'>https://github.com/ggerganov/llama.cpp/
Commands
//assuming you already have a conda environment set up, and dev tools installed (see videos above for instructions)
Part 1 manual
brew install gitlfs
git lfs install
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
pip install r requirements.txt
make
git clone https://huggingface.co/teknium/OpenHe... openhermes7bv2.5
mv openhermes7bv2.5 models/
python3 convert.py ./models/openhermes7bv2.5 outfile ./models/openhermes7bv2.5/ggmlmodelf16.gguf outtype f16
./quantize ./models/openhermes7bv2.5/ggmlmodelf16.gguf ./models/openhermes7bv2.5/ggmlmodelq8_0.gguf q8_0
./quantize ./models/openhermes7bv2.5/ggmlmodelf16.gguf ./models/openhermes7bv2.5/ggmlmodelq4_k.gguf q4_k
./batchedbench ./models/openhermes7bv2.5/ggmlmodelf16.gguf 4096 0 99 0 2048 128,512 1,2,3,4
./server m models/openhermes7bv2.5/ggmlmodelq4_k.gguf port 8888 host 0.0.0.0 ctxsize 10240 parallel 4 ngl 99 n 512
Part 2 auto
bash c "$(curl s https://ggml.ai/serverllm.sh)"
MacBooks in this video
M2 Max 16" MacBook Pro 64GB/2TB
— — — — — — — — —
❤ SUBSCRIBE TO MY YOUTUBE CHANNEL
Click here to subscribe: / @azisk
— — — — — — — — —
Join this channel to get access to perks:
/ @azisk
#m3max #macbook #macbookpro
— — — — — — — — —
ALEX ON X: / digitalix