Insights

Key claims, evidence, and primary sources extracted from my articles.

Every claim below was checked against the source PDF by a deterministic verifier. The method is documented in a deposited paper.DOI 10.5281/zenodo.22122106

Insights

Then the line that reframes the whole cost question: "If that sounds like a lot of surface area, it is. And it is the team's surface area, not the model provider's."

The number of validation evaluations a builder ran was essentially uncorrelated with final accuracy (Pearson r=0.17), while the builder's own reasoning effort improved scaffold quality monotonically.

Concentrating evaluation on tasks where candidate harnesses actually disagree matched full-set search while cutting evaluations by 80%, and at a 20% budget it beat full-set search outright on text classification.

Instructions and rule files. Tools and the prose that tells the model when to call them. Sandboxes and execution environments. Orchestration logic for sub-agent spawning and model routing. Deterministic hooks that fire before a tool call or after a file edit.

Their equation is blunt: agent equals model plus harness.

The tasks every candidate already passes, and the ones every candidate still fails, were consuming most of the budget and producing almost no signal.

The techniques that separated strong scaffolds from weak ones were the ones requiring real task analysis: polarity and negation logic (+0.09), structured state extraction (+0.06), hybrid fallback (+0.04).

Against the weak GPT-5.4-mini target, no builder regressed below baseline on any benchmark, 0 of 20 matched cases.

The share of rollout groups reaching genuinely different evidence fell from about 86% to below 10% by training step 30.

Harness-G's own scale split reproduces that pattern in a different architecture, raising average F1 by 10.74 points at 1.5B against 3.98 points at 3B.

Harness-G shows it in the reward signal.

Each builder model saw only a 195-item validation sample, 5% of the data, and never the 3,900-item hidden test set.

Against the already-strong Gemini-3.5-flash, every builder regressed somewhere, 9 of 20 cases, with Hi-ToM down 0.04 and near-saturated MuMA-ToM down 0.02 on average.

BigToM was almost fully offloadable at a mean determinism fraction around 0.94, Hi-ToM sat near 0.51, MMToM-QA near 0.44, and MuMA-ToM, built on free-form dialogue reasoning, near 0.36.

The prevalence table in the Salesforce study puts format enforcement in 100% of scaffolds, temperature control in 98% and benchmark routing in 95%.

The best scaffolded run reached 0.912.

A paired McNemar test over the 3,900 items shows the shape of that gain: the scaffold fixed 1,717 baseline errors while breaking 105 previously correct ones.

A raw model is not an agent at all; it becomes one once something gives it state, tool execution, feedback loops and enforceable constraints.

And scaffold code size related only weakly to accuracy (r≈0.22), while the determinism fraction, the share of items answered entirely by code or rules, correlated at r=0.72.

The authors' own recipe is to build two or three scaffolds and ship the best validation performer, which is cheap insurance for a team already running an agentic SDLC.

Holding the graph, the reward, the training budget and the GRPO configuration fixed, that interface change alone improves F1 by more than 17 points, and by more than 35 points on MuSiQue under outcome-only training.

Miyai, Aizawa and Yamasaki put a ceiling on the effect, citing evidence that changing the harness around a fixed model yields up to a 6× performance difference on the same benchmark.

Realized uplift tracked the target's available headroom at r=0.75, what the authors call a headroom law.

The unscaffolded GPT-5.4, the bigger model a team in that meeting would have bought, scores 0.619.

Across 57 scaffolded runs the mean macro-average was 0.763, an uplift of +0.275, and every single run beat the baseline.

Build variance is real: the mean standard deviation across repeats was 0.036 against a +0.275 mean uplift, but the widest setting spanned 0.201, with the widest spreads coming from deterministic-solver strategies where one logic error shifts accuracy by tens of points.

The honest limit on all of this is that scaffolding is a competence-recovery mechanism, not a capability generator.

They took GPT-5.4-mini, which scores a macro-average accuracy of 0.488 across four Theory-of-Mind benchmarks, and left its weights alone.

A skill is a portable, natural-language artifact that packages procedures, domain heuristics, tool policies, output constraints, and failure modes.

If skills are portable, they can be transferred across model scales, execution harnesses, and nearby domains without further optimization.

In their work on SkillOpt, they introduced a text-space optimizer for agent skills that applies deep-learning principles—like learning rates, validation gates, and momentum—to the text documents that guide agents.

Instead of a developer manually tweaking a prompt when an edge case fails, a separate optimizer model can analyze execution trajectories, propose bounded add/delete/replace edits to the skill document, and accept those edits only if they improve performance on a held-out validation set.

It acts as the external state for a frozen agent, allowing the agent to adapt and improve without requiring weight updates.

As the McKinsey authors note, the symbiotic enterprise requires redesigning roles and redeploying talent.

A 2026 report by QuantumBlack, McKinsey's AI arm, outlines the rise of the "symbiotic enterprise," suggesting that nearly 60 percent of work hours are theoretically automatable when cognitive and physical AI combine.

The United Nations Foundation's 2026 analysis warns that AI diffusion alone can widen inequality if transition management isn't built in from the start.

They opened it on about a third of the days they were on the platform.

The platform produced faster math gains than the comparison group by year two, and the researchers say that benefit did not seem to come from the AI.

Under the floor, homework scores were very high and exam scores extremely low.

An Alan Turing Institute survey of 780 UK children aged 8 to 12 found that 52% of those at private institutions had used generative AI, against 18% at state-funded ones.

It links surveys to national tax and test records for 4,497 users in their final primary year, in a system where adaptive AI tools are standard.

Family socio-economic status kept a strong direct link to performance (B = 0.23) that ran through neither.

Family background barely predicted digital literacy at all; the coefficient was slightly negative.

When they did, many sent off-topic messages or tried to get it to hand over the answer.

The authors classed anything under 50 minutes as outsourcing: 58% of AI users overall, and 81% after more than five months of use.

The paper reports adopters and non-adopters as well balanced on demographics and prior exam scores.

In the study, non-AI users generally needed at least 50 minutes per assignment.

Scores on monthly closed-book exams fell by 20% within six months.

The authors add a caveat: the data dates from spring 2023, before generative AI was widely integrated, and it captures institution-sanctioned tools only.

Among AI users with above-median homework scores, higher homework scores went with lower exam scores.

On 25 August 2026, Chalkbeat reported on a two-year experiment across 18 sites in one Tennessee district, where low-performing users worked in Khan Academy with the Khanmigo AI assistant one click away.

Among staff aware of AI use in assigned work, 47% at private institutions reported users submitting AI-generated work as their own, against 60% at state-funded ones.

AI usage intensity had no significant relationship with test performance (B = 0.03).

The Chinese authors reach the same place from the demand side: AI tools built to coach already exist there at low or zero cost, and most users pick general-purpose tools that give quick answers.

Losses were larger for high achievers: 24% for the top third by prior score, against 16% for the bottom third.

Digital literacy did (B = 1.78).

"Access was nearly universal but engagement was thin," the researchers wrote.

AI skill has not become a marker of family income, at least in the one study that measured it directly.

AI users who kept spending as much time on homework as non-users reached similar exam scores, and they were not stronger performers to begin with.

They suggest monitoring inputs such as homework time over outputs such as homework scores.

There, 53% of 15-year-olds use an AI chatbot to help them learn at least once a week, against an OECD average of 46%.

In a Chinese panel of 26,811 users, homework scores rose 18% and closed-book exam scores fell 20% within six months of adopting generative AI.

More than half of AI users finished in 20 to 50 minutes, faster than even the fastest unaided users.

The losses concentrated in the roughly 80% of AI users whose behaviour was consistent with outsourcing.

Openness, extraversion and perseverance each predicted digital literacy (B = 0.19, 0.17 and 0.05), and digital literacy carried part of their link to performance.

In their June 2026 working paper, adoption raised homework scores by 18% and cut completion time from 64 to 45 minutes.

Since the study period, Khan Academy has redesigned its interface to integrate the assistant, and Sal Khan wrote that the team "had to make productive struggle harder to sidestep."

A Dutch study of 4,497 users found no significant link between AI usage and test results, a clear link for digital literacy, and a family-background advantage that persisted regardless of AI.

Only 7% say they never use one for assigned work, half the OECD figure of 14%.

Strömberg, Lei and Wu tracked 26,811 secondary-level users in one Chinese county for 30 months, as reported AI use went from nearly zero to around 80%.

The same OECD country note, released with the PISA 2025 results on 8 September 2026, reports that 37% of Indonesian 15-year-olds reach Level 2 or higher in science.

High-stakes entrance exam scores fell by 18% and 24%, and that loss took about two years to show in full.

Between 50 and 65 minutes, AI and non-AI users had similar exam scores.

For the most profitable banks in the top quantiles, the effect flipped entirely.

Fintech investments actually increased non-performing loans for the healthiest, lowest-risk institutions.

They found that a 1% increase in fintech development boosts macroeconomic productivity by 0.044%.

The research shows that fintech promotes macroeconomic productivity primarily through the rationalization of industrial structures.

For banks in the lower profit quantiles—the regional institutions struggling with legacy debt and bloated operations—fintech investment yielded the strongest positive profitability effects.

Meanwhile, the investments significantly reduced risk for the most unstable banks, who used the tools to distribute their loan portfolio risk and automate credit scoring.