The Token Economy
When we talk about AI today, tokens are king. Specifically, inference tokens have emerged as the primary proxy for tracking the growth of the AI ecosystem. Public companies report monthly tokens processed to showcase their AI growth, analysts compare models’ success based on their relative token volumes, and management teams measure their commitment to & investment in AI by looking at their token usage over time.
This broad popularity makes sense; tokens are a fundamental unit of AI intelligence and computation, and growth in tokens is a good representation of overall growth of AI in the world. Tokens also abstract the complexities of inference into a single unit of measure, making it both simple to understand (it only takes 2 minutes!) and easy to track consistently over time. As a kind of lingua franca, tokens let a broad audience grasp AI's rapid, complex progress regardless of technical fluency.
More people are using AI? Tokens go up. We moved from non-reasoning models to reasoning models? Tokens go up. We moved from queries to agents? Tokens go up. Agents can now work in the background or work on long-horizon tasks? Tokens go up!
Overall absolute tokens processed increase both as a function of increased/adoption of AI, but also due to infrastructure evolutions that make models & AI form factors more “token hungry”, e.g. an agent working for an hour vs. 1 minute.
This simplicity also creates a strong growth investment thesis for venture-growth investors. All of these changes — both in adoption and token intensity of models — stack on each other to create explosive, exponential growth in overall token volumes. It’s easy to map out, and easy to believe it continues this direction with long-horizon agents and background agents on-the-come. It’s no wonder why inference has become a red hot investment category and many companies are looking to get into the inference business.
The downside of an AI poster child as legible and popular as inference is that it can crowd the field of view, so that similar emerging mega-trends trends go relatively unnoticed because they’re harder for a broad audience to see & understand.
One trend in particular is similar to inference in many ways and is posed to become a much larger part of the AI discussion as it becomes more ubiquitous and widely understood. This is the market to improve model capabilities via data, which we call the Task Economy.
The Task Economy
Over the last three years, LLMs went from answering basic queries, to reasoning through complex problems, to becoming agents that can complete real-world work over longer and longer time horizons. Early on in this journey, model improvements were made through training models on the internet’s available data with increasing amounts of compute. As we’ve 1) ran out of further data on the internet to train on and 2) saturated more and more simple/general capabilities, a clear bottleneck to further model improvements has emerged: incremental high quality data. This data will be generated and served by the Task Economy.
Tasks are the "unit of practice" in reinforcement learning: a model is given an initial state and an environment to act in, and its behavior is scored by a reward signal/verifier. Across many tasks, those scores are aggregated into a training signal that shifts the model's behavior toward what scored well. Strictly speaking, "task" refers to this RL post-training substrate. But I'll use it more loosely to stand for the unit of data-driven improvement generally, since the industry is rapidly inventing new forms that data takes in the service of making models better, and candidly because Task Economy has such a nice ring to it. I also want to distinguish this category from the dated moniker of “data labeling”, which brings to mind bounding boxes and thumbs up/down for LLM responses — the market has evolved well beyond these primitives over the last couple years into much more complex & high value tasks.
Let’s take the legal industry as quick real example. AI models trained on the open internet can garner a high-level understanding of law, know publicly available case law precedents, etc. But producing the real world work of a talented lawyer requires data not available on the internet. In order for a model to replicate high quality legal workflows, we must give the model prompts (review a contract, draft an argument), place the model in relevant environments (a legal data room), and then grade/verify the quality of the work (via a rubric, an example of which you can see here). These tasks teach the model not only what to do, but how to do it. And the more high quality tasks you expose the model to, the better the model gets.
In this way, what tokens are to inference/model usage, tasks are to model improvement efforts. Tokens are a fundamental unit of AI intelligence & compute; we should think of tasks as a fundamental unit of AI improvement. And just like tokens, tasks grow both as a function of AI adoption, and as developing frontier intelligence becomes more and more “task hungry”.
This is not precise nor comprehensive, but gives some examples of each step function increase in model intelligence requiring far more, higher complexity tasks
We’ve moved from basic preference labels to skilled experts using rubrics? Tasks go up. We’ve introduced vertical agents that replicate expert-level domain work? Tasks go up. Agents need to work across longer horizons? Tasks go up. Enterprises are adopting evals en masse? Tasks go up!
Like the inference market, these stacking growth inputs have produced similarly unprecedented growth for the Task Economy:
- OpenAI and Anthropic are scaling their data spend by 10x year over year, spending billions of dollars mobilizing experts across every domain to create data & train agents.
- Leading AI application companies & enterprises in our network are scaling their individual task-related spend to $100m+ in the near term as they recognize that data is their moat, and that Applied AI with a differentiated data strategy can beat off-the-shelf models.
- Benchmark portfolio company Mercor, the leading platform for the Task Economy, hit $1b in ARR this Feburary and then hit $2b in ARR just 4 months later.
The amount of raw tasks, the length & complexity of those tasks, and the cost per hour of the experts completing tasks is all growing, stacking to create exponential overall task spend growth
And as impressive as recent growth signals in this market have been, we are pretty clearly only in the first inning of this market's overall growth and impact. We are just barely starting to see agents that can replicate high quality work in any advanced domain, and enterprises are only beginning to scale spend this year as they wake up to the importance of data as a differentiator vis à vis the labs. Compare that against the backdrop that 99% of human knowledge relevant to the future capabilities we want AI to cover is in people’s heads. If we believe that applied AI companies of all kinds (labs, AI app companies, enterprises) are going to want to transmit that tacit knowledge into models & agents (an, we are in for many more years of rapid growth of the Task Economy across a much wider set of buyers / participants than we've had in the past.
Making the Task Mega-Trend more Legible
Tokens & tasks are important barometers of AI’s progress & evolution, and both are rapidly accelerating. Yet despite this similarly explosive growth, there are far fewer conversations about tasks than tokens online today. I think this is mainly because:
1) historically spend in this market has been concentrated in the frontier labs, who are highly secretive about their model improvement strategies which includes their spend on data/tasks. This is changing rapidly beginning this year as AI app companies and enterprises embrace the Task Economy to build differentiation vs. off-the-shelf models. These companies are more likely to market their efforts in this domain, and push the category into the regular AI conversation.
and
2) the market hasn’t had as clean of a unit of value abstraction as inference does with tokens. Part of the purpose of this piece is to change this and rally the conversation around tasks as a unit of value that we can standardize on. Tokens are a lingua franca that enable a broad audience grasp AI's progress regardless of technical fluency; tasks should act in the same way to enable a broad audience to grasp the industry's investment in advancing AI capabilities.
Given these bottlenecks, the industry doesn't have an "OpenRouter for task volume" or anything similar today that can give us a live proxy view into the scale and growth of the Task Economy over time. While it would be tremendously valuable for a company to publish something like that in the future, for now the team at Mercor was kind enough to provide a chart of historical expert hours worked per quarter on their platform as a window into the exponential growth of the market. As you can see, the real data matches the magnitude/velocity of growth we discussed in the last section:
Source: Mercor
In many ways the Task Economy is the defining market for the future of AI — the barrier to automating every task we can do on our laptops with agents is covering the full distribution of all of the apps, all of the environments, and all of the tasks that correspond to everything in the economy. This will require a massive data buildout across every professional domain, academic discipline, and consumer use case. Legal, medicine, finance, software, science, and beyond will each require their own expert-generated datasets, evaluations, and RL environments. Labs, AI app companies, and enterprises will all fight to rapidly scale this data infrastructure across the full surface area of economically useful work and the ones who succeed will continue to improve frontier capabilities and gain market share.
We will begin to track these efforts much more closely as a community as the Task Economy becomes more visible and ubiquitous in the coming years. And sometime soon, when we talk about AI, tasks will be king.
Footnote: An obvious other place where we will see improvements in genreal AI capability is in algorithmic improvements to models. I have excluded those to keep the focus of this piece on data, but it is a focus/stylistic choice rather that not thinking that we will get algorithmic improvements in the future as well.
