I’m getting more and more tempted to start a startup to sell a small box for a purely local LLM for HomeKit and google assistant control (and home assistant of course). I wonder if the market is strong enough for this. Sure, it won’t be GPT 4 level but it’ll definitely be better than Siri. Maybe YC 2026.
> I wonder if the market is strong enough for this
It is not.
You could not pay the average person to run a local LLM instead of just relying on Siri/Alexa/Google Assistant/whatever is built into the apps and hardware they use every day.
You could sell these to HN commenters who will play with it for a week and then put it in the cupboard to gather dust next to their Raspberry Pis.
i have an rpi zero w (i think) that came as part of a voice kit - a cardboard box, an rgb led arcade button, a speaker, and a really COOL mems multi-microphone array pi zero hat. Out of the box it had raspbian or whatever on it, and a config file you edited while following instructions that came in the kit - you have to set up a firebase account, authorize the "app" on the pi, and so on. It basically gave you access to the google ecosystem in a tiny cardboard box, like a cube a baseball might fit in.
It didn't have a wakeword, you'd hit the arcade button (which had a dim color when inactive), it'd pulse white or blue, and then the assistant voice would ask whatever google assistants ask, and the light would change red, and you could ask your question, the light would pulse, and a second or two later, the assistant would talk, replying to your request or whtaever.
now, i am working from memory, so it may have been amazon services, but i am unsure if they allow third party access, and it may have immediately waited for your question when you pushed the button. I don't remember.
If i can find one of my boxes, in a closet gathering dust next to the rest of my rpi, i'll at least try to get the model/name of the microphone array hat, because i think that's the part that will make a DIY voice assistant work rather than be a curiosity.
mine stopped working after a few weeks, and i couldn't ever figure out why. I think maybe google or whatever wanted money for API usage, or they changed the rules of how firebase worked... but the hardware still works, it just doesn't "wake up" when you push the button. the logs show the button push, etc.
anyhow, i got the last two at target like 5(?) years ago, for $10 each. i think it was called "AIY Voice Kit"
You're right. A "batteries included" device can't be sold for the actual R&D and engineering costs of the device, so you're either going to lose money on each sale, or not sell many at all; and either way that leads to anti-consumer behaviors.
Ramble follows, feel free to ignore
the AIY kits were "on clearance" when i got them, and i haven't seen similar since. But the issue isn't that people don't want to "own their data" and "DIY" - but think of all of the things you need to know how to do, to set up that device i spoke of. Linux shell. Wifi / networking. Electronics.
Figuring out the moving target of google infra services. Let's pause here; if this part is removed, we get a bunch more - running things in docker, or compiling from source, or python venvs. You have to have enough compute just laying around to do the processing of voice and tts, you still have to "hook up" all of the piping to something that can actually "do work" based on "plain english commands"
how many people on HN could set up a voice assistant without any assistance? With the archlinux wiki and stack and copilot that number might grow a magnitude or two. I do mean the full stack, but COTS hardware.
it's marginally easier these days than it was when i bought that AIY Voice Kit - transcription and TTS are downright magical, and built upon at least a decade and a half of prior art; I don't think tortoise-tts has much in common with the TI-49/A or Apple powerPC era text to speech. I don't think whisper has much in common with Dragon Naturally Speaking, or the powerPC era "short commands" that you could use with applescript.
If some startup wants to try and light investor money on fire a little slower,
it should be possible to design and build a "home assistant" device that's like a 100 TOPS tegra or functionally equivalent running linux as a "base station" and remote or satellite transceivers that have the wakewords and whatnot on them. Think like a cordless phone. Obviously to build a moat we'd use some arbitrary wireless protocol, if not proprietary. nah, it should be wifi, maybe even as part of a "home mesh wifi" system or something?
you're still gunna lose your ass trying to make something people want at a decent price.
Tech is evolving too quickly; every year the hardware will be much more powerful at the same price (as LLM optimizations reach hardware), so you’d end up replacing the device frequently.
GPUs and NPUs are gaining optimizations for the transformer architecture. It’s not “GPU is 3x faster this year”, it’s “GPU has gates specifically designed to accelerate your LLM workload”
See for instance [0], which is just starting to appear in commercial parts.
This is continuing; pretty much every low cost SoC maker is racing to build and extend ML optimizations.
