Your local LLM is 10x slower than it should be

Here’s the one change that took mine from ~120 tok/s to 1,200+ without a new GPU.
TryHackMe just launched Cyber Security 101 (SEC1) — and for a limited time you can get 40% off with my link: https://tryhackme.com/alexsec1
Use code ALEXSEC1 for 40% off the exam fee + 3 months of Premium access.

🛒 Gear Links 🛒

🪛🪛Highly rated precision driver kit: https://amzn.to/4fkMVfg
💻☕ Favorite 15" display with magnet: https://amzn.to/3zD1DhQ
🎧⚡ Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW
🛠️🚀 My nvme ssd: https://amzn.to/3YLEySo
📦🎮 My gear: https://www.amazon.com/shop/alexziskind

🎥 Related Videos 🎥

🧳🧰 Mini PC portable setup - https://youtu.be/4RYmsrarOSw
🍎💻 Dev setup on Mac - https://youtu.be/KiKUN4i1SeU
💸🧠 Cheap mini runs a 70B LLM 🤯 - https://youtu.be/xyKEQjUzfAk
🧪🔥 RAM torture test on Mac - https://youtu.be/l3zIwPgan7M
🍏⚡ FREE Local LLMs on Apple Silicon | FAST! - https://youtu.be/bp2eev21Qfo
🧠📉 REALITY vs Apple’s Memory Claims | vs RTX4090m - https://youtu.be/fdvzQAWXU7A
🧬🐍 Set up Conda - https://youtu.be/2Acht_5_HTo
⚡💥 Thunderbolt 5 BREAKS Apple’s Upcharge - https://youtu.be/nHqrvxcRc7o
🧠🚀 INSANE Machine Learning on Neural Engine - https://youtu.be/Y2FOUg_jo7k
🧱🖥️ Mac Mini Cluster - https://youtu.be/GBR6pHZ68Ho

* 🛠️ Developer productivity Playlist - https://www.youtube.com/playli....st?list=PLPwbI_iIX3a
🔗 AI for Coding Playlist: 📚 - https://www.youtube.com/playli....st?list=PLPwbI_iIX3a


GitHub repo for this code: https://github.com/alexziskind....1/llama-throughput-l

— — — — — — — — —

❤️ SUBSCRIBE TO MY YOUTUBE CHANNEL 📺
Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1

— — — — — — — — —

Join this channel to get access to perks:
https://www.youtube.com/channe....l/UCajiMK_CY9icRhLep

— — — — — — — — —
📱 ALEX on X: https://x.com/digitalix

#macstudio #gdxspark #llamacpp