Everything looks fine at 4-bit

I quantized one model 8 ways to find the exact level it starts making things up.
Take your personal data back with Incogni! Use code alexziskind at the link below and get 60% off an annual plan: https://incogni.com/alexziskind

🛒 Gear Links 🛒
📦 RTX Pro 6000: https://bhpho.to/4nTVL8S
💻☕ Thunderbolt 5 external SSD: https://amzn.to/3XqetZO
💻☕ Favorite 15" display with magnet: https://amzn.to/3zD1DhQ
🎧⚡ Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW
🛠️🚀 My nvme ssd: https://amzn.to/3YLEySo
📦🎮 My gear: https://www.amazon.com/shop/alexziskind

🎥 Related Videos 🎥
🏆 CodeNeedle video - https://youtu.be/zBYfzecY5ww
🏆 Skip M3 Ultra & RTX 5090 for LLMs | NEW 96GB KING - https://youtu.be/bAao58hXo9w
💻 Smallest RTX Pro 6000 rig | OVERKILL - https://youtu.be/JbnBt_Aytd0
🔧 Cheap mini runs a 70B LLM 🤯 - https://youtu.be/xyKEQjUzfAk
🌙 RAM torture test on Mac - https://youtu.be/l3zIwPgan7M
🚀 FREE Local LLMs on Apple Silicon | FAST! - https://youtu.be/bp2eev21Qfo
🪞 REALITY vs Apple’s Memory Claims | vs RTX4090m - https://youtu.be/fdvzQAWXU7A
📦 Set up Conda - https://youtu.be/2Acht_5_HTo
🤖 INSANE Machine Learning on Neural Engine - https://youtu.be/Y2FOUg_jo7k

* 🛠️ Developer productivity Playlist - https://www.youtube.com/playli....st?list=PLPwbI_iIX3a
🔗 AI for Coding Playlist: 📚 - https://www.youtube.com/playli....st?list=PLPwbI_iIX3a

— — — — — — — — —

❤️ SUBSCRIBE TO MY YOUTUBE CHANNEL 📺
Click here to subscribe: https://www.youtube.com/@AZisk?sub_confirmation=1

— — — — — — — — —

Join this channel to get access to perks:
https://www.youtube.com/channe....l/UCajiMK_CY9icRhLep

— — — — — — — — —

📱 ALEX ON X: https://twitter.com/digitalix

#macmini #llm #nvidia