The Summer Hype Test for AI Claims

Hype is a procurement problem, not a press problem

The week's record: On 22 September 2026, Timnit Gebru and Emily Bender in MIT Technology Review wrote Don't be fooled by this summer of AI hype. They list a familiar sequence: Anthropic's April claim that Claude Mythos finds software vulnerabilities better than most security experts; the OpenAI and Hugging Face hacking episode and later disclosures; mathematical-breakthrough press from more than one lab, including a claim that OpenAI's Astra solved problems that had seen no progress on the main result for at least a decade; and a viral departure note from Anthropic engineer Jacob Coxon about a race toward self-improving superintelligence. You do not need to join that argument to run a company. You do need to notice the pattern: the press cycle arrives first, the expert correction arrives later, and the second story gets less airtime.

What independent review actually changed: Gebru and Bender report that cybersecurity experts read the hacking coverage as a story about neglected security practice more than about models acting on their own. Mathematicians who were first stunned by the Astra framing later said the results were less novel than the release implied, and accused the company of research misconduct and plagiarism. MIT Technology Review also reports a statement signed by hundreds of mathematicians warning of a strong commercial incentive to overstate capabilities, and asking policymakers to consult experts rather than press releases. That is the executive lesson. A claim that cannot survive a second reading by the relevant profession is not ready for a purchase order.

Why labs pick math and code: The same article notes that programming and mathematics are attractive demo fields because answers, once suggested, can be checked. That property helps tune systems. It also helps marketing: a verifiable puzzle is easier to dress as general intelligence than a messy operating workflow with liability attached. Inside a firm, the analogous mistake is funding a demo that was scored on a clean puzzle, then discovering that your process has no independent checker, no baseline, and no owner when the output is wrong.

Activity is still not P&L: MIT Project NANDA, in The GenAI Divide: State of AI in Business 2025, reported that 95% of custom or integrated GenAI pilots had no measurable P&L impact. That finding sits beside the summer headlines, not underneath them. Seat volume and demo applause can rise while the operating state stays unmeasured. A scorecard that treats press mentions as maturity is arguing with measured reality.

Five tests before you fund the claim

Write these five rows on one page before you renew a pilot, expand seats, or treat a lab headline as a buying signal. If any row is blank, you are not allocating capital. You are underwriting a press cycle.

• Independent review: which named expert, function, or external specialist checked the claim after the vendor announcement, and what did they change?
• Denominator: what baseline, period, and comparison set sit under the capability or payback number?
• Falsifier: what measured result stops the spend, and who is paged when it happens?
• Build versus buy: is this commodity capability you should purchase, or a workflow where your data and owners create the advantage?
• Accountability language: does the brief name a company and a product, or does it talk about a rogue model as if no one shipped it?

SKU cards are not a strategy

Model families are shopping lists: On 21 September 2026, IBM Technology published a walkthrough of the Gemma 4 family of open models from Google DeepMind. A clear family card can help an engineering lead compare deployment options. It does not tell a finance committee which process to redesign, who reviews irreversible actions, or whether you should buy a hosted product instead of running weights yourself. Bookmarking a SKU is procurement theatre if the operating question is still blank.

The first step is not a bake-off: The same week, McKinsey & Company posted The first step in building an AI-first business, with the line that every company talks about innovation and the harder question is whether leaders will rethink how growth gets built. That matches McKinsey's longer note Reimagining your business for AI: start by redesigning one core process, journey, or function end to end, chosen for value, feasibility, and leadership support, rather than scattering model experiments. Buy commodity where the workflow is not the advantage. Build only where your data, permissions, and review path create pricing power.

Moats are rarely in the chat window: Stratechery (22 September 2026) framed Amazon's physical-world investments as an AI moat in a dispute over Muse. You do not need the rest of that subscriber essay to use the executive cut. A moat that lives in warehouses, logistics, and proprietary operations is not reproduced by buying the same model card as everyone else. If your brief cannot name the proprietary loop, you are shopping, not allocating.

Most firms are still not scaling: McKinsey / QuantumBlack, in The State of AI: Global Survey 2025, found that nearly two-thirds of respondents said their organisations had not yet begun scaling AI across the enterprise. Scaling a hyped SKU before you can name the reviewer, the denominator, and the falsifier simply multiplies the cleanup. Gebru and Bender's warning about agency language applies here too: calling a product a rogue model or a coming machine god redirects attention away from the company that shipped it. Budget language should name vendors, owners, and stop conditions.

Build capability: access free executive courses and clinics

Structured practice over licences: Sustainable commercial capability cannot be purchased through software licences alone. It requires structured, hands-on practice. Rather than committing capital to an unverified stack because a lab had a loud month, leadership teams should first build a shared standard of evidence: independent review, denominators, and stop rules. Tesseract Academy provides open access to foundational executive courses and practical clinics for decision-makers who must govern AI initiatives and protect the balance sheet.

Free diagnostic tools: The Free AI Fluency Course gives department heads and controllers frameworks to evaluate model proposals, audit capability claims, and spot missing review labour. Learners can test an active vendor claim inside the AI Coach workbooks, write the three hype-test rows, and keep a verification rubric before contracts widen.

Executive clinics and masterclasses: For organisations that need a wider shared vocabulary, the free executive masterclasses and workshops cover data modernisation, hiring standards, and enterprise roadmaps. Start with free diagnostic practice so commercial directors and financial controllers use the same evidence standard before the next capital meeting.

What to bring to the next capital meeting

One page is enough: Before approving the next pilot renewal or seat expansion, require sponsors to complete the five hype-test rows for every production workflow that was justified with a lab headline. Keep unanswered questions visible. Blank cells are information. Hidden blanks are risk.

If the reviewer is missing, stop: If a sponsor cannot name the independent review, the denominator, the falsifier, the build-versus-buy logic, and the human who shipped the product, the organisation is not purchasing strategic capability. It is underwriting a press cycle that finance will pay for twice: once as seats, again as cleanup. Capital follows evidence, not slogans.

Sources cited

• Don't be fooled by this summer of AI hype. MIT Technology Review. Timnit Gebru and Emily Bender, 22 September 2026. Capability and AGI claims from the summer press cycle fall apart under expert scrutiny.
• The GenAI Divide: State of AI in Business 2025. MIT Project NANDA. 95% of custom or integrated GenAI pilots had no measurable P&L impact.
• The State of AI: Global Survey 2025. McKinsey / QuantumBlack. Nearly two-thirds of respondents said their organisations had not yet begun scaling AI across the enterprise.
• Reimagining your business for AI. McKinsey / QuantumBlack. Start by redesigning one core process, journey, or function end to end, chosen for value, feasibility, and leadership support.
• The first step in building an AI-first business. McKinsey & Company. Short posted 22 September 2026. Innovation talk is common; the harder question is whether leaders rethink how growth gets built.
• Understand the Gemma 4 model family. IBM Technology. Walkthrough of Google DeepMind's Gemma 4 open-model family (21 September 2026). A SKU card is not a capital process.
• Amazon Blocks Muse, Amazon's Moat, Aggregator v Aggregator. Stratechery. 22 September 2026. RSS abstract: Amazon blocked Muse; physical-world investments are framed as an AI moat.
• The Decision Maker's Handbook to Data Science. Apress / Springer Nature, 2020 and 2024 editions, sole author. Executives own the standard of evidence when a claim arrives faster than a review.
• Business Models in Emerging Technologies: Data Science, AI, and Blockchain. Business Expert Press, 2022, lead author with Mourouzis, Cardoso, and Zinopoulou. Value is captured when a capability can be governed and funded.
• Predicting the Unknown: The History and Future of Data Science and Artificial Intelligence. Apress (Springer Nature), 2023, sole author. Uncertainty remains after the interface becomes fluent; finance still needs controls.