Why must companies balance AI ROI with employee experimentation?

Companies must balance AI return on investment (ROI) with employee experimentation to prevent missing innovative applications and foster the discovery of novel use cases. Restricting access or penalizing trials due to cost concerns can deter employees from finding impactful AI solutions, according to Boris Cherny, creator of Claude Code. The strategy is to manage expenses on the backend once successful applications are identified.

How is the measurement of AI's return on investment evolving?

Measuring AI ROI is evolving beyond simple metrics like the percentage of code written by AI, as models increasingly handle 100% of code. New benchmarks now include evaluating an engineer's acceleration, identifying bottlenecks in the development pipeline, and assessing the speed of idea generation. This broader perspective ensures AI contributes to overall strategic goals and enhances human capabilities.

What challenges do proprietary AI firms face from open-source alternatives?

Proprietary AI firms, including Anthropic and OpenAI, face increasing pressure from cheaper, open-source alternatives as businesses prioritize efficiency over high "tokenmaxxing" costs. This trend, exemplified by companies shifting to more cost-effective models like DeepSeek, forces leading AI providers to demonstrate clearer ROI and offer flexible cost controls. Analysts predict a period of "rationalizing of spend" in the AI market.

Boris Cherny: AI ROI, Experimentation & Innovation Balance

Boris Cherny, creator of Claude Code, states that companies must focus on AI's return on investment (ROI) while still enabling employee experimentation. He emphasized this during a Scale AI fireside chat on June 23, 2026. This approach prevents missing innovative AI applications.

The discussion highlights growing concerns among businesses about the rising costs of AI tokens, the units used to measure large language model usage. As AI models become more integrated into workflows, balancing expenditure with potential gains is a primary challenge.

This focus aims to ensure AI adoption delivers tangible business value for organizations.

Why Must Companies Balance Cost and Experimentation?

Companies must balance controlling AI token costs with fostering experimentation to unlock true innovation. Restricting access or penalizing trials for cost reasons deters employees from discovering novel, impactful AI applications, according to Boris Cherny. The goal is to manage expenses on the backend after successful use cases are identified.

Cherny stressed that preventing experimentation could lead companies to miss out on valuable ideas. These often originate from unexpected departments, like accounting or marketing. He suggested giving employees tokens and a safe environment to try out AI tools without fear of penalty.

ROI is absolutely the right framing because you don't want to just think about cost because you kind of spend something on it and you get something back.
— Boris Cherny, Creator of Claude Code at Anthropic

Once an internal use case proves effective, companies can then implement cost controls. This strategy ensures resources are directed towards proven value rather than speculative, unproven initiatives. Anthropic offers features like per-seat cost controls to help enterprises manage their budgets effectively.

How Is AI ROI Measurement Evolving?

Measuring AI ROI is evolving past simple metrics like the percentage of code written by AI, reflecting advancements in model capabilities. Boris Cherny notes that as AI increasingly writes 100% of code, new benchmarks are needed. These include evaluating an engineer's acceleration and identifying other bottlenecks in the development pipeline.

Measurement Metric	Traditional Focus	Evolving Focus
Code Generation	Percentage of code written by AI	Engineer acceleration; 100% code completion
Innovation	Specific task automation	Idea generation rate; Un-hobbling bottlenecks
Cost Control	Initial token spend	Backend cost management post-experimentation

The shift in measurement considers overall team productivity and the speed of idea generation within a company. If AI automates much of the coding, the next bottleneck becomes the quality and volume of new ideas.

This broader perspective ensures that AI contributes to overarching strategic goals, not just isolated task completion. This comprehensive approach helps organizations understand the full impact of their AI investments.

It moves beyond raw output to assess how AI enhances human capabilities and accelerates business processes. For instance, Anthropic's Claude helps modernize code, showing how these tools integrate into existing systems.

What Challenges Do AI Firms Face from Open-Source?

AI firms like Anthropic and OpenAI face increasing pressure from cheaper, open-source alternatives as companies prioritize efficiency over "tokenmaxxing." Enterprises are actively seeking more cost-effective solutions to manage their rising AI expenditures. This trend could slow growth for leading AI providers.

For example, one AI startup, Lindy, recently shifted 100% of its traffic from Anthropic's Claude models to DeepSeek, a Chinese company offering cheaper alternatives. The 34-year-old CEO highlighted the tangible cost benefits of this transition. This move dramatically reduced their costs.

This environment forces proprietary AI providers to demonstrate clearer ROI and offer flexible cost controls. Analysts like Gil Luria suggest a "rationalizing of spend" period is ahead for major AI players. This implies a more competitive market where efficiency and measurable value become paramount for adoption.