Better than Jev - because you can build with it

Resources: https://thenextnewthing.ai/my-resources
Presented by Zapier: https://zapier.com/
Andrew Warner and Henry Ndubuaku show how Needle brings small, private AI models directly into software and devices.

Andrew Warner talks with Henry Ndubuaku about Needle, Cactus Compute’s tiny on-device LLM built for reliable automation without requiring an internet connection or external API calls. Henry demonstrates how Needle turns natural-language requests into structured tool calls, powers software features and hardware like robot vacuums, and uses tightly scoped actions and confidence scores to reduce mistakes. They also compare Needle with Jev and explain why running an 8–29 MB model locally can matter for privacy, cost, and bringing AI into smaller devices.

Links featured:

Needle: https://github.com/cactus-compute/needle

00:00 - Needle: Run a small, free LLM locally for private and reliable automation.
00:18 - Smart Home Demo: Turn lights on and off by translating natural-language requests into structured tool calls.
01:39 - Search & Matching: Use Needle inside software for reminders, search, extraction, and other defined actions.
03:18 - Local App Integration: Add Needle to software without relying on Anthropic or another external LLM API.
04:12 - LLM.txt: Give Claude the context it needs to incorporate Needle directly into an app.
04:48 - On-Device AI: Put Needle into hardware instead of limiting it to desktop software.
05:06 - Robot Vacuum: Control a vacuum with natural-language commands while keeping the model on the device.
06:09 - Zapier SDK: Add integrations with thousands of apps alongside Needle-powered local automation.
07:12 - Reliability: Restrict Needle to carefully defined tools and actions to reduce unexpected behavior.
08:51 - Confidence Scores: Use Needle’s confidence output to decide whether an automated action should execute.
10:12 - Phones & Laptops: Organize photos, start timers, copy screen content, and automate device actions locally.
11:42 - Needle vs Jev: Compare their shared focus on constrained outputs and safer automation.
13:21 - 8–29 MB Models: Run Needle on small devices like smartwatches, remotes, AirPods, and other privacy-sensitive hardware.
14:15 - GitHub Origins: Andrew explains how Needle first caught his attention after trending among the week’s top GitHub repos.

👉 Media/Sponsorship Inquiries: https://thenextnewthing.ai/l/sponsor

👉 Join us: https://thenextnewthing.ai/