AI coding assistants have moved past autocomplete. They now read across
whole repositories, generate features, write tests, and run multi-step tasks
with limited supervision. Adoption is close to universal: Sonar’s 2026 survey
of more than 1,100 professional developers found AI accounts for 42% of
committed code, a share developers expect to reach 65% by 2027.1
What has not followed is a clean productivity win. The best available
evidence says AI shifts where the work happens rather than removing it — less
time typing code, more time verifying it. The organisations getting real
returns are the ones that rebuilt their review and quality processes to match.
The ones that simply bought licences are largely paying for faster generation
of code nobody has time to check.
[H2]How It Actually Works
Under the Hood
These assistants are built on large language models using the Transformer
architecture, whose self-attention mechanism weighs the relative importance of
tokens in a sequence, letting the model track context across both code and
natural language. Given a prompt, the model predicts the most probable
continuation.
Raw next-token prediction is not enough for real engineering work, which
is why retrieval-augmented generation (RAG) became standard. Before generating,
the tool searches the actual codebase for relevant files, functions and
patterns, then feeds that context into the prompt. This grounding reduces
hallucinated APIs and keeps output consistent with a project’s conventions.
Tools such as Cursor surface which files were referenced, so the developer can
audit the context the model was working from.
The current frontier is agentic execution. Rather than suggesting a block
of code, an agent handed a refactoring task will analyse the existing module,
plan an approach, write the new implementation, generate tests, run them,
iterate on failures, and open a pull request. Orchestration frameworks such as
LangChain and Microsoft’s Semantic Kernel let developers chain models and tools
into these longer workflows. GitHub Copilot’s cloud agent and Google’s Gemini
Cloud Assist are the mainstream examples.
Model evaluation, meanwhile, is in open crisis. SWE-bench Verified was
the industry’s reference scoreboard until OpenAI audited 138 problems its o3
model consistently failed and found that 59.4% had material flaws in test
design or problem description, alongside evidence that frontier models across
multiple labs had been trained on the benchmark’s solutions.2
Independent work found roughly a third of successful Verified patches involved
solution leakage.3 The recommended successor did not survive contact
either: OpenAI’s follow-up audit identified 200 to 249 of SWE-bench Pro’s 731
tasks as broken, and the company withdrew its endorsement.4
The practical implication for buyers is blunt. Leaderboard scores are
currently close to unusable as a purchasing signal, and internal evaluation on
your own repositories is the only benchmark that means anything.
[H2]The Economic and
Practical Reality Check
The headline productivity claims deserve scepticism. METR recruited 16
experienced developers from large open-source repositories they had maintained
for years, gave them 246 real issues from their own backlogs, and randomly
assigned each to be worked with or without AI tools. Developers took 19% longer
with AI. They had predicted a 24% speedup beforehand and still believed they
had been faster afterwards.5
The study’s limits matter: a small sample, mature codebases the
developers already knew intimately, and early-2025 tooling. METR’s larger
follow-up — 57 developers, 143 repositories, more than 800 tasks — could not
produce a reliable estimate, because developers had become unwilling to work
without AI and were selective about which tasks they submitted. METR believes
developers are more sped up in 2026 than its 2025 figure suggests, but says its
data is only weak evidence for the size of that gain. For the ten developers
who appeared in both studies, the estimated effect remained an 18% slowdown,
with a confidence interval spanning 38% slower to 9% faster.6
That is the honest state of the evidence: the confident 10x numbers have
no rigorous support, and neither does a confident claim of no benefit at all.
Commit-level data points to a quality cost. GitClear’s analysis of 623
million code changes from 2023 to 2026 found that heavy AI users gained about
25% in velocity against their own prior output, while duplicated code blocks
rose 81% and “moved” code — the signature of refactoring — fell from 21% of
changed lines in 2022 to 3.8% in 2026. Cross-file function calls, a proxy for
reuse, are down 35%, and maintenance of code older than a year is down 74%
against 2022 levels. Error-masking patterns, meaning catch blocks and stubs
that silently swallow unexpected input, rose 47%.7
Security has not kept pace with fluency either. In Veracode’s 2026
testing, models that compile almost perfectly produced secure code on only 12%
of log injection tasks and 15% of cross-site scripting tasks.8
Costs have also changed shape. GitHub moved all Copilot plans to
usage-based billing on 1 June 2026, replacing premium request counts with AI
Credits metered on token consumption; Business remains $19 per user per month
and Enterprise $39, each including a matching credit allowance.9
Individual plans run Free, Pro at $10, Pro+ at $39 and Max at $100 per month,
and because Copilot Enterprise requires GitHub Enterprise Cloud at roughly $21
per user, the real Enterprise cost lands nearer $60.10 The
structural point for finance teams: agentic usage draws down a meter, so the
seat price is a floor, not a budget.
On employment, the credible figure is smaller than the scare number but
real and widening. The Stanford Digital Economy Lab’s payroll-based research
finds employment of 22-to-25-year-olds in AI-exposed jobs now sits 19% below
where it would be had it tracked their less-exposed peers, with no comparable
gap among experienced workers.11 Declines concentrate in occupations
where AI substitutes for human tasks; where it complements workers, employment
is flat or rising. The authors are explicit that they do not see widespread,
economy-wide displacement.12
The verification burden is now measurable. Sonar found that 96% of
developers do not fully trust AI-generated code, only 48% always verify it
before committing, and 38% say reviewing AI code takes more effort than
reviewing a colleague’s. Teams report spending close to a quarter of the work
week checking, fixing and validating AI output.1 DORA’s 2025
research, drawing on nearly 5,000 technology professionals, found that AI
improved delivery throughput but continued to increase delivery instability —
systems have not yet adapted to safely absorb AI-accelerated development.13
13.0pt;line-height:133%;color:#333333">The bottleneck has moved. Writing code
was never the expensive part; being confident enough to ship it was. AI made
the cheap half nearly free and left the expensive half exactly where it was.
What This Means in 2026
For developers. Fluency with these tools is now baseline, but the
evidence suggests where the benefit actually lands: unfamiliar codebases,
boilerplate, greenfield work and exploration, rather than delicate changes to
systems you already know well. Sonar found that junior developers report the
highest productivity gains while also being likelier to say reviewing AI code
takes more effort.14 The skill worth building is judgement about
when to delegate and when the review cost exceeds the writing cost. Treat
generated code as a colleague’s pull request, not as output.
For engineering leaders and buyers. Measure before you believe.
The METR perception gap is the single most useful finding in this literature:
developers are unreliable narrators of their own productivity, so team surveys
cannot substitute for delivery metrics. Track AI code share, review latency,
rework and change failure rate, and expect an early dip. DORA describes a
“verification tax”: the additional effort of confirming AI-generated code is
reliable, secure and architecturally coherent, a cost that leaders misread as
failure when they have not budgeted for it.15 Its central conclusion
is that returns come from the organisational system — platform quality,
workflow clarity, team alignment — rather than from the tools, and that without
that foundation AI produces local productivity that dissipates into downstream
chaos.16
Two governance items deserve attention now. Sonar found that 35% of
developers access AI coding tools through personal rather than work-sanctioned
accounts, a live data-handling exposure in any regulated environment.14
And given the state of public benchmarks, model selection should rest on
internal evaluation against your own repositories.
For everyone else. The downstream effect on software quality is
genuinely uncertain. Faster shipping cuts both ways, and the maintainability
data suggests some of today’s velocity is a loan against tomorrow’s debugging.
Where This Goes Next
Agents across the lifecycle. Multi-step execution will extend from
single tasks toward whole feature cycles, with longer-horizon memory and
dynamic tool use. The constraint is no longer capability so much as the human
review capacity sitting downstream of it.
Verification as the growth market. If generation is cheap and
trust is scarce, value shifts to automated review, static analysis, provenance
tracking and policy enforcement. Expect this layer to absorb serious
investment.
Deeper platform integration. Assistants that understand build
pipelines, deployment configuration and cloud infrastructure, not just source
files, are the direction every major provider is moving.
Legacy modernisation. Better repository-level reasoning makes
migration and modernisation of old systems the most plausible large commercial
prize, precisely because it is work organisations have long deferred.
Honest evaluation. After the collapse of two flagship benchmarks
in a single year, the field needs contamination-resistant, held-out,
repository-level evaluation. Continuous measurement in production will matter
more than any leaderboard.
The partnership framing is right, but the division of labour is not the
one the marketing describes. The engineer’s job is moving toward specification,
architecture and validation — deciding what should be built, and confirming
that what was built is correct. That second half is getting harder, not easier,
and it is where the next decade of tooling will be won.