AI coding assistants have moved past autocomplete. They now read across whole repositories, generate features, write tests, and run multi-step tasks with limited supervision. Adoption is close to universal: Sonar’s 2026 survey of more than 1,100 professional developers found AI accounts for 42% of committed code, a share developers expect to reach 65% by 2027.
What has not followed is a clean productivity win. The best available evidence says AI shifts where the work happens rather than removing it — less time typing code, more time verifying it. The organizations getting real returns are the ones that rebuilt their review and quality processes to match. The ones that simply bought licenses are largely paying for faster generation of code nobody has time to check.
How It Actually Works Under the Hood
These assistants are built on large language models using the Transformer architecture, whose self-attention mechanism weighs the relative importance of tokens in a sequence, letting the model track context across both code and natural language. Given a prompt, the model predicts the most probable continuation.
Raw next-token prediction is not enough for real engineering work, which is why retrieval-augmented generation (RAG) became standard. Before generating, the tool searches the actual codebase for relevant files, functions and patterns, then feeds that context into the prompt. This grounding reduces hallucinated APIs and keeps output consistent with a project’s conventions. Tools such as Cursor surface which files were referenced, so the developer can audit the context the model was working from.
The current frontier is agentic execution. Rather than suggesting a block of code, an agent handed a refactoring task will analyze the existing module, plan an approach, write the new implementation, generate tests, run them, iterate on failures, and open a pull request. Orchestration frameworks such as LangChain and Microsoft’s Semantic Kernel let developers chain models and tools into these longer workflows. GitHub Copilot’s cloud agent and Google’s Gemini Cloud Assist are the mainstream examples.
Model evaluation, meanwhile, is in open crisis. SWE-bench Verified was the industry’s reference scoreboard until OpenAI audited 138 problems its o3 model consistently failed and found that 59.4% had material flaws in test design or problem description, alongside evidence that frontier models across multiple labs had been trained on the benchmark’s solutions. Independent work found roughly a third of successful Verified patches involved solution leakage. The recommended successor did not survive contact either: OpenAI’s follow-up audit identified 200 to 249 of SWE-bench Pro’s 731 tasks as broken, and the company withdrew its endorsement.
The practical implication for buyers is blunt. Leaderboard scores are currently close to unusable as a purchasing signal, and internal evaluation on your own repositories is the only benchmark that means anything.
The Economic and Practical Reality Check
The headline productivity claims deserve skepticism. METR recruited 16 experienced developers from large open-source repositories they had maintained for years, gave them 246 real issues from their own backlogs, and randomly assigned each to be worked with or without AI tools. Developers took 19% longer with AI. They had predicted a 24% speedup beforehand and still believed they had been faster afterwards.
The study’s limits matter: a small sample, mature codebases the developers already knew intimately, and early-2025 tooling. METR’s larger follow-up — 57 developers, 143 repositories, more than 800 tasks — could not produce a reliable estimate, because developers had become unwilling to work without AI and were selective about which tasks they submitted. METR believes developers are more sped up in 2026 than its 2025 figure suggests, but says its data is only weak evidence for the size of that gain. For the ten developers who appeared in both studies, the estimated effect remained an 18% slowdown, with a confidence interval spanning 38% slower to 9% faster.
That is the honest state of the evidence: the confident 10x numbers have no rigorous support, and neither does a confident claim of no benefit at all.
Commit-level data points to a quality cost. GitClear’s analysis of 623 million code changes from 2023 to 2026 found that heavy AI users gained about 25% in velocity against their own prior output, while duplicated code blocks rose 81% and “moved” code — the signature of refactoring — fell from 21% of changed lines in 2022 to 3.8% in 2026. Cross-file function calls, a proxy for reuse, are down 35%, and maintenance of code older than a year is down 74% against 2022 levels. Error-masking patterns, meaning catch blocks and stubs that silently swallow unexpected input, rose 47%.
Security has not kept pace with fluency either. In Veracode’s 2026 testing, models that compile almost perfectly produced secure code on only 12% of log injection tasks and 15% of cross-site scripting tasks.
Costs have also changed shape. GitHub moved all Copilot plans to usage-based billing on June 1, 2026, replacing premium request counts with AI Credits metered on token consumption; Business remains $19 per user per month and Enterprise $39, each including a matching credit allowance. Individual plans run Free, Pro at $10, Pro+ at $39 and Max at $100 per month, and because Copilot Enterprise requires GitHub Enterprise Cloud at roughly $21 per user, the real Enterprise cost lands nearer $60. The structural point for finance teams: agentic usage draws down a meter, so the seat price is a floor, not a budget.
On employment, the credible figure is smaller than the scare number but real and widening. The Stanford Digital Economy Lab’s payroll-based research finds employment of 22-to-25-year-olds in AI-exposed jobs now sits 19% below where it would be had it tracked their less-exposed peers, with no comparable gap among experienced workers. Declines concentrate in occupations where AI substitutes for human tasks; where it complements workers, employment is flat or rising. The authors are explicit that they do not see widespread, economy-wide displacement.
The verification burden is now measurable. Sonar found that 96% of developers do not fully trust AI-generated code, only 48% always verify it before committing, and 38% say reviewing AI code takes more effort than reviewing a colleague’s. Teams report spending close to a quarter of the work week checking, fixing and validating AI output. DORA’s 2025 research, drawing on nearly 5,000 technology professionals, found that AI improved delivery throughput but continued to increase delivery instability — systems have not yet adapted to safely absorb AI-accelerated development.
The bottleneck has moved. Writing code was never the expensive part; being confident enough to ship it was. AI made the cheap half nearly free and left the expensive half exactly where it was.
What This Means in 2026
For developers. Fluency with these tools is now baseline, but the evidence suggests where the benefit actually lands: unfamiliar codebases, boilerplate, greenfield work and exploration, rather than delicate changes to systems you already know well. Sonar found that junior developers report the highest productivity gains while also being likelier to say reviewing AI code takes more effort. The skill worth building is judgment about when to delegate and when the review cost exceeds the writing cost. Treat generated code as a colleague’s pull request, not as output.
For engineering leaders and buyers. Measure before you believe. The METR perception gap is the single most useful finding in this literature: developers are unreliable narrators of their own productivity, so team surveys cannot substitute for delivery metrics. Track AI code share, review latency, rework and change failure rate, and expect an early dip. DORA describes a “verification tax”: the additional effort of confirming AI-generated code is reliable, secure and architecturally coherent, a cost that leaders misread as failure when they have not budgeted for it. Its central conclusion is that returns come from the organizational system — platform quality, workflow clarity, team alignment — rather than from the tools, and that without that foundation AI produces local productivity that dissipates into downstream chaos.
Two governance items deserve attention now. Sonar found that 35% of developers access AI coding tools through personal rather than work-sanctioned accounts, a live data-handling exposure in any regulated environment. And given the state of public benchmarks, model selection should rest on internal evaluation against your own repositories.
For everyone else. The downstream effect on software quality is genuinely uncertain. Faster shipping cuts both ways, and the maintainability data suggests some of today’s velocity is a loan against tomorrow’s debugging.
Where This Goes Next
Agents across the lifecycle. Multi-step execution will extend from single tasks toward whole feature cycles, with longer-horizon memory and dynamic tool use. The constraint is no longer capability so much as the human review capacity sitting downstream of it.
Verification as the growth market. If generation is cheap and trust is scarce, value shifts to automated review, static analysis, provenance tracking and policy enforcement. Expect this layer to absorb serious investment.
Deeper platform integration. Assistants that understand build pipelines, deployment configuration and cloud infrastructure, not just source files, are the direction every major provider is moving.
Legacy modernization. Better repository-level reasoning makes migration and modernization of old systems the most plausible large commercial prize, precisely because it is work organizations have long deferred.
Honest evaluation. After the collapse of two flagship benchmarks in a single year, the field needs contamination-resistant, held-out, repository-level evaluation. Continuous measurement in production will matter more than any leaderboard.
The partnership framing is right, but the division of labor is not the one the marketing describes. The engineer’s job is moving toward specification, architecture and validation — deciding what should be built, and confirming that what was built is correct. That second half is getting harder, not easier, and it is where the next decade of tooling will be won.
