Written by Sarah Chieng (@milksandmatcha) and @0xSero
A personal assistant should save you time and effort. Over the past few weeks, we’ve been obsessively testing AI personal assistants on everyday tasks, from booking flights and managing calendars to buying groceries and planning meals.
We still marvel every time an assistant gets something done on its own. But watching it spend over 7 minutes on a dinner reservation we could manually make in 37 seconds definitely thins the excitement.
So we started digging into where that time goes. For this experiment, we put together a test assistant using Qwen 3.8 27B running on Cerebras, with Pi as the agent harness. With faster inference on Cerebras and harness engineering, our own assistant completed the task in 22 seconds, 19x faster than existing assistants. Here’s what we changed.
The new consumer AI assistants
OpenClaw was a glimpse into the future, and the new wave of consumer AI assistants handles more of the setup for you, so you can start handing off tasks without configuring the software underneath.
Milestones from our research notes. Instinct’s date marks observed public posts, rather than a confirmed launch date.
Where the time goes
As an experiment, we put every major assistant to the test and benchmarked them on the same dinner reservation request:
The same request sent each assistant through a different sequence of searches, checks, and browser actions. Meta Muse’s run took 4 minutes 36 seconds and included nine direct OpenTable API calls. Claude Cowork’s took 6 minutes 25 seconds and 57 tool calls. Grok Bot’s lasted 7 minutes 40 seconds. Each assistant correctly made a reservation at either A Mano, II Borgo, Doppio Zero.
Recorded runs, not a general ranking. The 22-second result is the median of two successful attempts.
With any agentic task, there’s a lot going on under the hood. The harness manages the conversation, executes tool calls, handles errors, and feeds results back to the model. To understand where the time went, let's look at how Grok Bot spent its first 2 minutes. Grok Bot started by loading skills, retrieving memories, searching, checking restaurant pages, and reporting back. One browser run alone spent 2 minutes 18 seconds checking a Mano, Il Borgo, and Doppio Zero sequentially.
For example, we can see here a closer breakdown of the Grok Bot run.
The models matter, too. Recent advances in planning, tool use, and long-horizon reasoning have made these tasks more practical. The Opus 4.5 generation paired stronger agent capabilities with context compaction: carrying a condensed account of the work forward when the context fills. Research such as CompactionRL goes further by training agents to work across those compaction steps.
And under it all, you still need a fast model to keep track of the goal, recover from mistakes, and routinely verify that the task is on track.
How we did it
We used Pi as the agent harness. Its small system prompt and four built-in tools gave us a simple starting point, with room to add our own tools for the booking task.
- Check independent options in parallel
Checking restaurant availability should be done in parallel. The booking step does depend on what those checks return. Our split: run the independent checks together, then use the results to decide how to continue.
The skill directed it to check independent options in parallel, so it could gather results from several restaurants without waiting for each check to finish before starting the next.
The browser/API portion of our optimized run was 6.8 seconds, compared with 4 minutes 31 seconds in the earlier Grok trace: roughly a 40× difference in those recorded categories. That comparison bundles changes to the model, harness, and execution path; it doesn’t isolate the effect of the skill alone.
- Speed up the remaining model calls
Every time the assistant returns to the model, it waits for another response. Faster inference on Cerebras shortens those pauses. Our optimized run used Qwen 3.8 27B; saving the website procedure also reduced how much the model had to work out during the run.
Some steps still have to happen in order. The agent needs a page result before it can decide what to do with it. Faster inference won’t make a slow website load instantly or remove a phone-verification step, but it can cut the repeated pauses between seeing, deciding, and acting.
- Save what the agent learns
Much of the wasted effort comes from discovering how the website works: call a tool, observe the page, predict the next move, call another tool. The agent pokes around, learns what happens, and then continues with its task. On the next visit, it may do that discovery work all over again.
Before running the assistant, we gave the agent more context and turned what we learned into a skill. That gave the model instructions for navigating the booking process without having to discover each step for itself. In our test, this reduced tool calls by more than 80%.
The important distinction is what gets reused. We can save the procedure for navigating the site and checking a reservation. Availability, prices, and whether a card is required still need to be checked live. The work of figuring out the route happens before the timed run; it doesn’t disappear.
A faster personal assistant
There’s more work to do before it handles every errand that well and we have a real-life Jarvis, but just imagine when these agents that already feel magical get 10x faster.
Anyways, gotta go! We have a dinner reservation to get to.
Designs by Halley Chang. A special thanks to Shintaro Matsui, Joyce Er, and Alycia Cary for their feedback.
