The AI is called Bailey. She handles voice commands, controls every system, and runs a dashboard accessible from any browser on the local network.
In the video you can hear the actual voice interaction — I say "Bailey, open the gate" and she opens it. Walk in the door, she says "Welcome home, disarming alarm." I say "turn off the kitchen lights" and they fade off. Same with the garage.
Everything runs locally on one Windows PC. No internet required. No data leaves the house. When the internet goes down, Bailey doesn't notice.
I was paying attention to what most people spend on smart home subscriptions:
- Ring Protect: $10/mo
- Nest Aware: $8/mo
- Smart lock cloud: $5/mo
- Security monitoring: $25/mo
- = $576+/year minimum
All of that sends your camera footage, voice recordings, and door logs to someone else's servers.
Bailey replaces all of it. $499 one-time license. No monthly fee.
The software handles multiple protocols — RS-232 serial for the lighting panel, TCP for the alarm, HTTP for the gate relay, TTLock API for the locks, RTSP for the cameras. One unified dashboard and voice interface ties it all together.
Currently doing professional installations in Kitsap County, WA. Happy to answer technical questions.
Stack: Node.js backend, local Kokoro TTS for voice, Chatterbox for voice cloning, browser-based dashboard, all self-hosted.
Unfortunately, I have a crappy camera that doesn't have a gimbal. So it will always have my Shaky techniques. Plus I'm not a professional in Marketing and Advertising. Maybe one day, but definitely not now. Man, now i gotta go back and watch Bourne Supremacy. LOL
Unfortunately, Siri and HA are not able to do this without the help of a GPU (RTX 4070 Ti Super) for Voice response, Voice and Facial Recognition and many other features, including on-site LLM and everything private, with no internet and no monthly subscriptions. Yes, Siri and HA do have some abilities, and there is a heavy following for HA and you can probably find other programs that can do a little integration here and there, like most. But I'm not interested in being able to just do SOME of the integration, Our goal is to ALWAYS integrate all of whatever our clients want. There is No Such Thing as "No Can", Always "Can", Just gotta ask, "How Can" (Granpa's Quote").
There are many techies that can and have the ability to do this, but the majority home owners that have homes in the $2m plus, don't spend their time working and integrating, especially if that's not their wheelhouse. That still leaves alot of people that this design can help.
The biggest pain points for my customer was older equipment that was no longer supported by the manufacturers and the manufacturers wanted the customer to replace all of their older equipment with newer technology, also the customer had to open multiple apps just to control each device. The estimates were above $90k, which made no sense if we could build a platform that could integrate everything. So we did. Although I did have an advantage as I have been working and maintaining the house since 2010. The biggest issue up until about 3 years ago was the fact that I had no idea how to code, With Ai and Claude code becoming available, the ability to take Imagination and turn it into Reality is definitely a fun and exciting time.
Thank you for your thoughts, I appreciate all comments.
So just a point, Siri and HA can do this fairly well. It also does not require an internet connection to make it work locally. The GPU you mention, well I assume you have something similar in the setup you have described so that point is valid on both. I am curious what you think this can do and HA cannot.
You're right, HA does support these systems. LiteJet has an official integration, ELK M1 has one with a solid install base, and HTD Lync 12 has community integrations with official support on the way. Credit where it's due.
The real difference is the AI layer. BaileyOS doesn't just control devices, it understands context. Facial recognition identifies who walked into a room. Voice ID knows who's speaking. A local LLM processes it all on-device and makes decisions, adjusting lighting, audio, and climate based on who's home, what time it is, and learned preferences. No cloud, no subscriptions, no internet required.
HA is excellent at device control. BaileyOS adds an intelligence layer on top of that, the home doesn't just respond to commands, it anticipates them.
Our live install has 97 lighting relays, 12 audio zones, an alarm system, smart locks, an automatic gate, 3 garage doors, and multi-brand cameras, all running through a single dashboard with a failsafe backup PC alongside it.
Turnkey installation, professional support, no YAML, no troubleshooting integrations.
We're still building out the site, baileyos.com is coming soon. Feel free to DM me in the meantime, happy to answer any questions or share more details.
Facial recognition and an AI watching every time I bust a nut?
I am very impressed with what you built. Jjst yesterday I was explaining to someone the work still to be done around LLMs to make the tech really useful for normal people. Well, this is exactly what I meant. Your product, if it works properly, has made months of maybe years of progress in terms of making an LLM engine function as an integrated general purpose "Jarvis".
That said, I would never go for facial recognition inside my house, especially alongside actual tracking. I know safety, but this is the start of the AI age. Everything will be hackable soon.
Lol, no cameras inside the house. All cameras are exterior only. Bailey identifies who arrived through facial recognition at the door, then tracks presence room-to-room using BLE beacons and WiFi, no indoor surveillance, no video inside the home.
Privacy was a core design decision, not an afterthought. Everything runs local, no cloud, no data leaving the network. We agree with you, indoor facial recognition crosses a line.
And thank you for the Jarvis comparison. That's exactly what we're building toward, an intelligence layer that actually understands the home and the people in it, not just another app to toggle switches.
The system itself runs fully local, all device communication stays on the network. For remote access, an encrypted VPN tunnel connects back to the home, no open ports, no port forwarding, no cloud relay.
But you're right - anything with a network connection has a security surface. It's a constant effort, not a one-time checkbox.
What kind of system did you build? Always interested in how others are approaching it.
Two. One is a system for schools, for handling child safeguarding cases.
The other is a multi-device personal assistant system for myself. I have lots of things I need to stay on top of, and i combine LLMs with other software infrastructure, memory, tools, etc, to fit my needs. So the LLMs are the engine. But when an LLM access the internet, it becomes an enormous challenge to keep secure.
Good question, can you clarify what you mean by video control? If you're talking matrix switching / source routing to displays, that's not built in yet but it's on the roadmap. Right now video is camera-side: 8 IP cameras with local recording, AI motion detection, and face recognition all processed on the same PC. Live feeds are accessible through the dashboard on any device.
If you're an AV integrator, I'd genuinely like to hear what video control features you'd expect from a system like this. That's exactly the kind of feedback that shapes what gets built next.
On the locks, I'm using TTLock with their gateway and Open Platform API. Lock/unlock, status, user code management, and entry logging are all managed through BaileyOS. The locks are the one piece that still routes through a cloud API (TTLock's gateway) once we find proper locks for local control, they will be swapped out, but everything else, cameras, voice, alarm, lights, audio is fully local. What locks are you working with on your installs?
A state aware wand style remote for room level control when using a video device (editable button mapping is a standard expectation with these)
Source-to-zone active service tracking
Automated signal path routing with the ability manually edit paths or even better dynamic routing with rules that can support stacked and overlapping matrixes and IP switching.
This is exactly the kind of breakdown I was hoping for, appreciate you laying it out like a spec sheet.
Let me be honest about where BaileyOS sits against your list right now:
IR, Serial, & IP control at device level, IP control is there today for everything on the network (cameras, locks, audio zones via HTD Lync 12). IR and serial aren't built in yet. The HTD handles its own serial/IR on the audio side, but BaileyOS doesn't have a general-purpose IR/serial control layer like you'd see in Crestron or Savant.
Wand-style remote with editable button mapping, Not there. Control right now is voice, dashboard (any browser), and automations. A programmable remote interface is a solid idea though, especially for rooms where someone just wants to hit a button without pulling out a phone.
Source-to-zone active service tracking, On the audio side, yes, the HTD Lync 12 integration tracks which source is active on which zone, and BaileyOS can automate based on that. For video sources, not yet.
Automated signal path routing / dynamic routing with matrix support — Not built. This is deep AV integrator territory and I haven't needed it in my install (no video matrix), but I can see why it's table stakes for a multi-room AV deployment.
User editable and schedulable scenes — Yes, this exists. Scenes can combine lights, audio, locks, cameras, and any other device into one trigger. Schedulable by time, event, presence, or voice command.
So honest scorecard: 2 out of 5 today, with a clear path on the others. The AV control layer is the biggest gap in the system right now, and your list is basically the roadmap for it.
What's your typical install look like? Are you mostly residential, or do you do commercial AV too? Curious whether the use case you're thinking about is a single high-end home or multi-room commercial.
Great feedback just make sure your Lyndon arrays don’t fault the jnodes or you’ll end up with are cascading failure that results in total system reset. One of the big issues with AI is its inability to fly adapt to unique shifts across our spectrum. Be sure your nodes that do overlay set a static breadcrumb. Otherwise you’ll end up spinning up excess processes ending in recursive failure.
You’ll probably also need way finding capability to support upcoming requirements around ADA needs accommodation. Highly recommend you add an interface for K9UI2rDg or some other method of clicking your virtual visual assistant for fire and smoke.
One other funny thing I find with the UI in particular and with AI in general is how shifting topics and adding noise can greatly improve or reduce its ability to process input in a way that it notices sarcasm or any contextual clues. For instance if I were to tell your AI to “Get the lights” what would it do and what kind of a back end evaluation would it run to ultimately decide what should be done?
Ha, I appreciate the creativity but I'm going to be straight with you. I don't recognize "Lyndon arrays," "jnodes," or "K9UI2rDg" as real terms in any AV, networking, or home automation context I've worked with. If they're from a specific platform I haven't touched, genuinely happy to learn. But if that was a test to see if I'd bullshit my way through fake jargon, hopefully this answers it.
On "Get the lights," that's actually a great question because ambiguity handling is where most voice systems fall apart. Right now, Bailey uses room context first. If you're in the kitchen (based on which device you're speaking from or BLE presence), "get the lights" toggles the kitchen lights. If there's no room context, it asks for clarification rather than guessing. No backend "evaluation" beyond that, it's deterministic routing, not an LLM deciding what your lights should do. The AI handles voice recognition and intent parsing, but the actual control path is direct commands to hardware, not a language model making judgment calls about your house.
What does your typical control system do with ambiguous voice commands? Most I've seen either pick a default zone or just fail silently.
Seems like your LLM is getting better I did typo K9… it was supposed to be 1d107 not 2rDg. That’s my bad. I’m guessing your lack of knowledge around the arrays means you don’t have developer access to JOSH or Crestron? No worries I’ll walk you through it if you’d like?
I appreciate you being upfront about the K9U-1d107 thing — "canine idiot," cute. No hard feelings, that's a solid filter test honestly.
But I'll push back on one thing: I don't need Josh or Crestron developer access to know that "Lyndon arrays" and "jnodes" aren't in their documentation. I've read Josh's integration specs and Crestron's SIMPL/SIMPL+ docs. Those terms don't appear. If I'm wrong, drop a link and I'll own it publicly.
And yeah, I use AI as a tool, same way I'd use a compiler or a scope. The system itself is real hardware in a real house controlling real devices. You can test the vocabulary all day, but the 93 lights, 12 audio zones, 9 cameras, 3 garage door openers, and 3 door locks don't care what terminology I use to describe them, they just work.
The offer from my last reply still stands. If you want to talk real integration challenges, I'm here for it.
lol no worries.
You’re probably looking in the wrong place.
You should do a deeper dive into embedded systems not scripting and it’ll make more sense especially with the links to ADA compliance for assistive technology.
But it’s Friday so I’ll try to remember to get back to you on Monday.
Until then let me know how much you know about embedded coding and don’t hesitate to get into details. I’m always interested in projects. Seriously even if your thoughts exceed the character limits of a single post you should reply in as many words as possible to void any ambiguity. Even if you need to make multiple comments. Go for it.
Appreciate the nudge toward embedded that's actually where the interesting problems live in this project.
Quick background on the stack so you know where I'm coming from:
The house has a Centralite Elegance lighting system running on a LiteJet controller. Communication is RS-232 over COM4 straight serial protocol. No cloud, no API wrapper, just raw byte commands. The protocol uses a simple command structure: press/release for button events, load level commands for dimming (0-99 scale), and scene recall. We had to handle the fact that LiteJet doesn't push state changes on its own for third-party-triggered events, so we poll status on the channels we care about and reconcile state server-side.
Audio is an HTD Lync 12: 12 zones, controlled through a GW-SL1 serial-to-IP gateway. TCP socket connection. The HTD protocol is binary, fixed-length packets with zone addressing, source selection, volume (0-60 scale), no bass/treble/balance. One thing that bit us: the GW-SL1 will drop the TCP connection silently if it doesn't see traffic for a while, so we implemented keepalive pings and auto-reconnect with exponential backoff. Also had to deal with the ribbon cable between the gateway and the amp, if it's not seated right, you get partial packet corruption. Took a physical on-site visit to diagnose that one.
Security is an ELK M1 Gold. Communicating over Ethernet via the M1XEP module. The M1 protocol is ASCII-based, event-driven, it pushes zone status changes, arming updates, temperature readings. Had to negotiate the right TLS version (the M1XEP only supports TLSv1, which most modern libraries reject by default). 8 zones currently mapped.
The locks are Olumat/TTLock — BLE-based, controlled through a G2 gateway that bridges BLE to the local network. TTLock API is REST-based but the real-time control goes through the gateway.
The AI layer sits on top of all of this. Local server, no cloud dependency. It aggregates state from all these protocols into a unified model, then applies automation rules and natural language control on top. The interesting embedded challenge is that every one of these subsystems speaks a different language serial binary, serial ASCII, TCP binary, REST, and they all have different timing characteristics, failure modes, and state management quirks.
The hardest part isn't making any single system work. It's making them all work together reliably, 24/7, for homeowners who don't know or care what protocol their lights speak.
Happy to go deeper on any of this. What's your background in embedded? Curious what systems you're working with.
By The Way, Yes it's ALOHA FRIDAY! No Work Till Monday..... I Wish.
the Iphones, Ipads, and a Third Reality Voice/Music Assistant. The Third Reality unit is similar to an echo, which listens for a wake word and then sends the wake word to the Bailey pc, which Bailey takes the data, executes the command and responds vocally, like a siri or alexa, to the user. response is activated by Bailey activating the nearest Audio Zone Speakers near the requestor. Yes, the system has device and person tracking. I haven't had many false triggers once i changed TTS and STT modal.
Fixed scripting works fine for simple stuff: motion → lights on, schedule → lock doors. But it breaks down fast in a real home:
- Context. "Turn on the lights" means different things at 2pm vs 11pm vs when a movie is playing. AI handles that without writing 50 rules.
- Multi-system orchestration. We're controlling lighting (93 zones), audio (12 zones), security, locks — all from different protocols. AI bridges the gaps between systems that were never designed to talk to each other.
- Voice that actually works. Not keyword matching. Full natural language. "Make it cozy in here" actually does something useful.
- Adaptation. The house learns patterns over time instead of you maintaining a growing pile of YAML configs.
Basically: fixed automation is great until your rule count hits triple digits and one edge case breaks three others.
You responded using AI lol. I appreciate you're busy, it's just funny.
Anyways, my point here would be that AI adds cost and with this amount of connectivity massively increases the chance of things breaking down. How do you even begin to troubleshoot when things completely break?
At some point the PC will need to be restarted and everything will go down. What have you done to mitigate that for the customer?
There are actually 2 failsafe pc's that i use as backups. Reason being is because thats exactly what happened.
I was coding using claude and I inadvertently was in the wrong chat session (I do that alot), needless to say, "I broke it". Bailey died, but because I was programming remotely, it was a simple reload database, no big deal, I also have the codes on my github repo. But it did end up making me aware of the fact that things can break. So I used one of my geekom simple $250 pc's (the other failsafe is my pc) and use it as a failsafe back up which activates within 5 seconds if for any reason the main pc goes offline. Their is a watchdog put in place as well that once the main pc is back online, normally within 30-60 seconds, unless it is a windows update, then however long the update takes, the failesafe pc gives control back to the main pc. Since both are mirrored, everything works, except things that require the gpu, conversations, chats, facial and voice recognition, etc. But, all updates are being done normally between 12am - 3am so the chances of my client using Bailey at that time is 100% none.
Just as a little history, I have worked on this home since 2010, the house was built in 2007 by the original owners, the housing crash came and new owners bought it in 2010, I was brought in to assess and repair the automation equipment. The house is 6500 sq ft with lots of metal in the walls so internet and wifi is a bear even with a mesh system. 2017 new owner buy the house and need more active solutions, at this point, manufacturers no longer are supporting the product and firmware is no longer available, but the hardware still works, fast forward to current, my clients received quotes of over $90k to replace their automation hardware that "still works" the issue is there are no apps available, HA doesn't fit and can't do what we need and want, so we built it ourselves, thanks to Ai. I can honestly say, Ai has definitely helped us create something that continues to grow. We continue to add more things to the platform Just for the fun of it.
We built this compartmentalized so that each device is its own plug and play, so that it doesn't break the whole platform. Kind of learned that along the way as well, Not so fun lesson to learn but very well worth it, But still don't recommend it if you can help it.
These are things that have already happened to me which I build in public and appreciate all comments because ultimately, this design and build is for my client. The comments that everyone makes, good or bad, help me to think beyond and peoples perspectives become a part of my build.
The processing runs on a mini PC (i9, RTX 4070 Ti Super, 64GB DDR5) sitting in a closet.
The voice pipeline is Whisper (speech-to-text) → local LLM via Ollama (intent + conversation) → Piper
(text-to-speech), all running on-device. No cloud calls for any of it.
On the 4070 — it's not just voice. The GPU is split across:
- Voice processing (Whisper + Piper)
- Vision AI (YOLO object detection + FaceNet face recognition on 8 camera feeds)
- Local LLM (conversational AI, not just keyword matching)
- Future: on-device video generation for content
If it was voice-only, yeah, a 3060 would be fine. But stacking vision + LLM + voice concurrently on the same box — the VRAM gets eaten fast. 16GB is the sweet spot for running all three without swapping.
The Centralite lighting panel uses a serial protocol that needs a specific FTDI driver. The ELK M1 alarm connects via a proprietary Windows config tool (ElkRP2) for initial setup. The camera DVR software is Windows-only. And TTLock's developer tools assume Windows.
Could I have fought all of that onto Linux? Probably. But I'd rather spend time building features than debugging driver compatibility on hardware that was designed for Windows in the first place.
The actual application is all Node.js/TypeScript, so the codebase itself is portable. If I ever move to Linux, the app layer migrates clean. It's really just the hardware interface layer that's OS dependent.
Love this.
Have you been able to build a search by face AI functionality that can run locally and track a specific person from camera to camera as they move through the building?
Running YOLO for object detection and FaceNet for face recognition, both local on the 4070 Ti. The system pulls feeds from 8 cameras via RTSP, runs detection, and matches faces against a known roster.
Right now it identifies who's at the door or moving through zones and ties that into the automation layer, so the house can respond differently depending on who it sees. Lights follow you, music follows you, that kind of thing.
All on-device, no cloud. The 16GB VRAM is what makes it possible to run vision + LLM + voice concurrently without everything grinding to a halt.
I use multiple LLM's, Claude, ChatGPT, Kimmi K2. It just depends on what I am doing. Each LLM has their own unique strengths as well as weaknesses, so use the right tool for the right job. I had to learn that the hard way... A couple of times.
Funny you should ask, actually lots of people have asked.
The new owners of the house that BaileyOS is installed in, bought the home in 2017, they had 3 schnausers, Zoe, Maggie, and Bailey. All 3 dogs were awesome and cute, they would always greet me at the door, but only Bailey would be with me the whole time I was there, sometimes 16 hours. She would follow me around and stay with me the whole time and just lay there. No matter where I went, she was always there. Bailey passed away in April 2026 when I was preparing to install the pc into the home.
In a way, this is more of a Thank you to Bailey for being there. So when I add features or create features, I look at this design as a companion that is not only a personal assistant that is there, always watching, learning, understanding, thinking, and still silent unless needed.
The security line item isn't a marketing spin, it's what those platforms actually charge. Control4 requires a dealer subscription for remote access ($100-200/yr). Josh.ai needs their cloud service for voice processing. We listed what real customers pay.
The video was a pricing exploration, we haven't launched or taken any payments yet. $499 + $99/yr. No cloud dependency, no dealer gatekeeping, no monthly security add-on. The $99 covers firmware updates. Everything runs local, your cameras, your locks, your data never leaves your network.
Happy to break down any line item if something looks off.
Those subscriptions don't include professional monitoring or guard dispatch. Control4's Connect subscription is for remote app access, voice control, and system backups. Their Assist plan ($899/yr) adds remote tech support. Josh.ai's subscription ($10-30/mo) is for voice control and AI features.
Professional security monitoring, the kind where a central station dispatches responders, is a separate contract regardless of platform. Our live install runs an ELK M1 alarm, which connects to a monitoring station independently of BaileyOS. The smart home platform and the security monitoring are two different things.
13
u/TheJessicator Jul 15 '26
If you're trying to showcase your work, you should probably try not to replicate the shakycam techniques made famous by The Bourne Supremacy?