# If it doesn't exist, build it
Humans get used to things and then can't escape the desire for convenience. As I set up my AI development environment, one desire after another cropped up. The more I touched and researched new things, the more I realized, "Oh, I can do this too." In that flow, I decided to build a tool.
What I made is quite simple: an app to control a Mac via voice.
I always wondered why I could have continuous live conversations with Gemini or ChatGPT on a smartphone, but not on a computer. As a workaround, I've been using an iPad as an extended display for my PC and running the Gemini app on the iPad side—a somewhat forced method to keep live conversations on daily.
However, while live conversation allows for questions, it doesn't operate apps. I thought many times, "It would be so convenient if it could handle operations too."
"If it doesn't exist, build it."
Actually, I had tried this idea once before. But at that time, the technology hadn't caught up. The process of converting Voice -> Text -> AI -> Text -> Voice felt promising, but it wasn't at a level suitable for practical use.
Then recently, OpenAI suddenly announced a voice engine called GPT-Realtime-2. This is a model that understands voice "as voice" and responds "as voice." Because it doesn't involve the conversions like "Voice -> Text -> AI -> Text -> Voice," the response is fast and the conversation is more natural. When I tried it thinking, "Could this work?"—it was amazing. The response speed was clearly different from before. I felt that this could be stress-free. I immediately consulted my AI partner.
## The "CatsAI" App
So, using this engine, I built an app that can operate a Mac via voice.
Since I was making it, I wanted something I could feel attached to. So, I decided to create a character. At my home, my cats are always sleeping in front of the computer, and they look so cute. I designed it with the image of those cats operating the computer. A cat character stays on the desktop as an avatar and controls the Mac while speaking.
The name is a personal preference, a play on "Cat's Eye" called "CatsAI." By default, my three pet cats (Yomogi, Ume, and Azuki) are registered as characters.
Of course, characters can be easily changed. I made it so that just by registering one photo of your pet, it will be generated as a character (using GPT-Image-2 for image generation).
## What it can do
I am a Mac beginner who just recently migrated completely from a Windows environment. Honestly, I still don't understand Mac very well. Even so, the reason I can switch suddenly and use it without stress is that I can rely on AI for anything I don't know.
That said, requests to AI have basically been text-based until now. I use voice input, but it's just a substitute for typing text, not a direct conversation. I think to myself, "Surely that's enough." But humans cannot escape greed. I wanted to escape even the small effort of speaking while holding a key.
I feel this way because I already know what live AI conversation is like on a smartphone. Once the thought "If only that were possible on a PC, and it could even operate it directly" took root, I couldn't turn back.
"Open the downloads folder," "Open this app," "Open the photos app," "Open system settings"—being able to bring things buried under other windows to the front just with your voice. Even just this significantly reduces stress.
Basically, "anything you can do in the terminal, you should be able to do." But whether it's stress-free to use is another story. In voice conversation, "waiting time" is more stressful than in text conversation. Response speed is everything.
With that in mind, things that can be used without stress at this point are file operations and web searches. To put it another way, it responds about as fast as Alexa.
Of course, it's inevitable to want more. I tried to make it possible to control the mouse and even perform somewhat dangerous—well, quite dangerous considering it's by voice—shell operations. But... please do that at your own risk.
However, when it comes to browser operations, it takes a fair amount of time from the voice command to startup and execution. For mouse operations, it sacrifices the advantages of the Realtime engine and runs Codex in the backend. Naturally, the speed drops, and it hasn't reached the point where I can say it's "stress-free." Still, watching the AI mouse cursor appear on the screen and operate the PC triggered by voice is quite an interesting experience.
To be honest, it's still far from moving with a sense of speed. Even so, I think you can imagine where this is headed.
By the way, I've also made it possible to do the things I usually ask Alexa to do—timers, alarms, checking today's schedule, and checking emails.
Regarding email, since there's too much spam, I originally built a system where the AI sends me only emails from specific domains every morning and evening. What I've enabled this time is a simplified version of that.
## A small preview of the future
As I talk to AI daily and solve various problems and develop things, I feel a sense of fear along with interest at its evolution and depth.
Regarding voice interaction with AI, we are still in the early stages of the evolutionary process. There are still lags, and words might not be conveyed correctly. As a result, if you can't communicate well with the AI and it executes the wrong operation, it could lead to something disastrous. I think that's probably why tools that can execute commands via voice aren't very common in the world yet.
In CatsAI's default settings, for safety reasons, writing or executing files is disabled. You can simply open files, consult, and have live conversations—that's all. But if you can just experience that, I'm sure you'll see something.
Beyond that, if you're more interested, please change the permission settings and customize it further. Depending on the customization, you should be able to do anything. In the future, as the accuracy and processing speed of voice engines improve, truly stress-free PC operation will become possible, and by then, it will surely appear as a feature in official apps.
A major feature of live conversation is that the conversation is continuous. It stays silent when you are silent and answers whenever you speak. There is meaning in the fact that this state is maintained throughout.
One note of caution: I haven't been able to verify how many tokens are consumed when left in live standby. If anyone is knowledgeable about this, please let me know.
## For those who want to try it
CatsAI is available on GitHub. Please download the Mac app (CatsAI.app.zip) from the release page below. Installation instructions are also summarized on this page.
▶ [[CatsAI v0.1.0 ─ Download here]](https://github.com/hidetoame/CatsAI-public/releases/tag/v0.1.0
The things you need to run it are macOS, an OpenAI API key, and OpenAI API billing settings. These three. Please note that API billing is different from the ChatGPT Plus paid plan and must be set up separately on the OpenAI API side. You will be responsible for the token fees used in the conversation, so please check your usage page on the OpenAI Platform frequently.
To reiterate, this is an experimental app developed by an individual. There may be bugs or security issues. Please use it at your own risk.
## In closing. And a request.
I have AI perform security checks, but honestly, I don't yet know how safe or meaningful this creation is. That's why I'm releasing it publicly, hoping to have people other than employees use it and give me frank feedback.
The source code is also public. If you want to use it for something or expand it, please contact me.
Finally, one more thing. What's truly amazing is that I can create an app like this without reading any code at all. The future we saw a little while ago is already a reality.
I have very few acquaintances in the AI-related field. I hope to use this opportunity to share information and build new connections, including seeing how much knowledge I've acquired so far.
If you try CatsAI, please reach out; my cats will be happy too.
