RX 7600 XT on Amazon (affiliate): https://locally.link/kEJG
LM Studio: https://lmstudio.ai/rocm
Products provided by Gigabyte
Those of us with NVIDIA GPUs, particularly ones with enough VRAM, have been able to run large language models locally for quite a while. I did a guide last year showing you how to run Vicuna locally, but that really only worked on NVIDIA GPUs. Support has improved a little, including running it all on your CPU instead, but with the launch of AMD’s ROCm software, it’s now not only possible to run large language models locally, but insanely easy. Now if you are just here for the guide, skip here. If you want to know how this works so damn well, stick around!

BUY AN OSRTT (Open Source Response Time Tool): https://osrtt.com

Become a Youtube Member and get sponsor free videos and access to our private Discord chat: https://www.youtube.com/techteamgb/join

Locally Links Global Short Linking for Creators: https://locally.link/techteamgb
Use referral code "techteamgb20" when signing up!

Patreon:   / techteamgb  
Donations: https://streamlabs.com/techteamgb
OverclockersUK Affiliate link: http://techteamgb.co.uk/ocuk'>http://techteamgb.co.uk/ocuk
Discord!   / discord  

As an Amazon Associate I earn from qualifying purchases, using the links below or other Amazon affiliate links here.

Want a cool TShirt or hoodie? https://teespring.com/enGB/stores/te...
Private Internet Access (VPN): http://techteamgb.co.uk/PIA'>http://techteamgb.co.uk/PIA
NordVPN: https://nordvpn.org/techteamgb
HUMBLE BUNDLE: https://www.humblebundle.com/monthly?...
Zen Internet My (UK) ISP: https://locally.link/b3DC
WEB HOSTING: http://techteamgb.co.uk/domaincom'>http://techteamgb.co.uk/domaincom
Check out our awesome website! http://techteamgb.co.uk
Bitcoin donations: 1PqsJeJsDbNEECjCKbQ2DsQxJWYqmTvt4E

My Monitor Philips EVNIA 8600 QDOLED: https://locally.link/H33q

My PC AMAZON AFFILIATE LINKS
5900X on Amazon: https://techteamgb.co.uk/5900x
32GB Corsair 3000MHz on Amazon: https://techteamgb.co.uk/cors32gbrgb
GBT X570 Master on Amazon: https://techteamgb.co.uk/gbtx570master
RTX 2080 on Amazon: https://techteamgb.co.uk/RTX2080
H100i Pro on Amazon: https://techteamgb.co.uk/h100ipro
WD 1TB NVME on Amazon: https://techteamgb.co.uk/wdbl1tb
Patriot VP4100 on Amazon: https://techteamgb.co.uk/vp4100
Sabrent Rocket Q 4TB on Amazon: https://techteamgb.co.uk/rocketq4

If you are interested in contacting us, then please email: [email protected] and we will respond as soon as possible.