Token Costs Fell from $10 to $2.50 per Million — Should I Stop Caring About Cost?

Over the past few years, the AI landscape has witnessed a dramatic and welcome shift: the cost per million tokens processed by large language models (LLMs) has plummeted. Where once $10 per million tokens was standard, today costs can dip as low as $2.50 per million tokens or less, especially from newer players like Anthropic’s Claude Opus 4.7. This drop begs a critical question for AI product teams and managers: “Should I stop caring about cost?”

Having shipped multiple AI-driven features and internal agents across support, risk, and developer tools, and run countless evals and feature flag rollouts, I’m convinced that the answer—surprisingly—is a nuanced “no.” Cost remains an essential lever, but it is no longer the only or even the primary constraint as LLMs become commoditized and ubiquitous.

What Does the User Do Today? The North Star for AI Product Design

Before diving into tokens and math, it’s crucial to zoom out and start with the user. What does your user do today? How do they work? What pain points or mundane tasks do they grapple with daily? Many product teams get excited by model capabilities or cost savings and forget this fundamental step.

For example, PM Toolkit, a company focused on empowering product managers, built tools that address specific workflows—especially around feature flagging and experiment management. Their success didn’t come from just slashing token cost but from deeply understanding PMs’ work habits and designing for those workflows end-to-end.

Lowering token cost should enable you to focus less on “can I afford to do this?” and more on “how do I deliver value reliably and seamlessly in the user’s workflow?”

From $10 to $2.50 Tokens: Where Cost Fits in Today’s AI Product Landscape

The fall from $10 to $2.50 per million tokens (or even lower) primarily reflects improvements in model training efficiencies, economies of scale, and fierce competition amongst providers like Anthropic and OpenAI. Anthropic’s Claude Opus 4.7 exemplifies this trend—a reasoning model designed to reduce hallucination risk while offering competitive pricing.

Model Cost per Million Tokens Hallucination Risk Recommended Use Anthropic Claude Opus 4.7 $2.50 Low Reasoning + grounded Q&A with retrieval Legacy Model X $10.00 Moderate General purpose

But token cost alone misses the broader story. As token pricing trends downward, other challenges take center stage:

Hallucination risks: Lower-cost or larger reasoning models can still hallucinate confidently if not combined with retrieval-based grounding. Model regressions & rollout safety: New models and prompts require robust feature flagging and kill switches to avoid negative user impact. Product differentiation: When basic language generation becomes commoditized, workflow integration and user trust are the real moats.

AI Product Patterns That Survive Commoditized Models

The most impactful AI products do not just ship a “wrapper around a model.” They how to reduce hallucinations embed the model as a thoughtful part of a broader workflow that drives real outcomes for users with low friction. Here are key patterns I’ve seen work repeatedly:

Workflow-first thinking: Understand the user’s existing steps and embed AI to accelerate or automate tasks without disruption. Trust as a moat: Build transparency, correctness signals, and easy-to-use kill switches to let users regain control when AI goes awry. Eval-driven product specification: Write evaluation cases like detailed bug reports with expected outputs, not vague accuracy claims. Design for graceful regression: Use feature flags and kill switches extensively to deploy new models safely without surprising users.

Feature flags and kill switches are invaluable here. For instance, if Anthropic’s Claude Opus 4.7 introduces a subtle regression on a critical support workflow, a kill switch enables automation vs augmentation ai instant rollback while your team investigates. Gone are the days of “ship and hope”—feature flags allow iterative, data-driven deployment.

Reasoning Model Tradeoffs and Hallucination Risk

Reasoning models like Claude Opus 4.7 promise lower hallucination rates and stronger chain-of-thought capabilities. But these gains come with tradeoffs:

Cost-efficiency vs. certainty: Reasoning models can be more computationally expensive per token even if base token costs fall. Latency: Complex reasoning chains increase response time, which may hurt UX in some workflows. Hallucination vs. grounding: Without proper retrieval augmentation or anchoring data, reasoning models can hallucinate plausible but false information.

Careful eval design is your best defense. For example, embed checks in your eval sheets like:

Test Case Input Expected Output Notes Support Ticket Summarization Ticket with critical issue description Correct and concise summary matching key points Check for hallucination or missed info Risk Alert Reasoning Complex user behavior logs Flag high-risk actions with explanations Verify logical chain and absent false positives

You might notice a “retry rate” sticky note on my monitor—a reminder to watch how often prompts and model outputs require retries or manual corrections. This metric beats vague “accuracy improved” claims because it directly reflects user impact.

Design for Workflow: The Real Cost Question

When cost per million tokens drops from $10 to $2.50, many teams celebrate—but the real question shifts from “Can I afford it?” to “How do I design for the workflow such that the user trust and product value keep pace?”

Some considerations to keep top of mind:

Optimize token usage thoughtfully: Even $2.50 per million tokens can add up if your product encourages wasteful generation or unnecessary retries. Measure user impact first: Sometimes, higher token cost is justified if output quality and trust reduce manual effort downstream substantially. Invest in tooling: Feature flags and kill switches allow you to continuously test new models or prompt variants without risking user trust. Integrate retrieval or grounding: Avoid the trap of solely reasoning-based outputs without retrievable evidence to control hallucination and boost reliability.

Conclusion: Cost Per Million Tokens Is Just One Part of the Puzzle

Dropping token costs from $10 to $2.50 per million is an important enabler that expands what AI products can do and who can access them. However, the story doesn’t end there for product managers and AI teams.

A successful AI product strategy focuses on:

Starting from user workflows and designing AI augmentation to be seamless and trustworthy. Using evals as living product specifications to continuously measure and iterate on quality. Leveraging feature flags and kill switches to safely ship new models and prompt updates. Carefully balancing reasoning power and hallucination risk with augmented retrieval.

Brands like PM Toolkit succeed not by chasing the cheapest token but by owning the workflows PMs trust every day. Providers like Anthropic are pushing the boundaries on cost-performance but remind us that product design remains the essential moat in a commoditized model market.

So, should you stop caring about token cost now that we’re at $2.50 per million? No—not entirely. But you should care more about designing for workflow and trust, which ultimately unlocks the full potential of these more affordable models.