Evidence
Browse the sample proofs attached to milestone judgments. This MVP distinguishes between benchmark evidence, leaderboards, research, news, and implementation or demo material.
Methodology
These public-facing capability pillars are plain-language summaries built on a deeper coverage map spanning reasoning, learning, truthfulness, self-monitoring, social competence, multimodal understanding, safety, and robustness. Each granular question is intended to be backable by benchmarks, controlled studies, audits, red-team exercises, longitudinal trials, or expert-blind review.
75 evidence items shown
Open Capability Benchmark Consortium
Hidden-answer prompt suites show strong recovery of user intent from indirect, noisy, and incomplete requests.
Linked to
AI can understand what a task is really asking
Can it infer the real goal from messy instructions?
Mar 6, 2026
Center for Applied AI Evaluation
Controlled user studies show high rates of useful follow-up questions before action on incomplete enterprise tasks.
Linked to
AI can understand what a task is really asking
Can it identify missing constraints and ask clarifying questions?
Feb 18, 2026
Frontier Evaluation Arena
Leaderboard evaluations show top systems retaining the right task variables even when prompts include decoys and irrelevant detail.
Linked to
AI can understand what a task is really asking
Can it separate relevant from irrelevant information?
Mar 10, 2026
Expert Review Panel on AI Performance
Expert raters judged many outputs well-framed, but noted persistent failures on ambiguous planning-versus-retrieval boundaries.
Linked to
AI can understand what a task is really asking
Can it map ambiguous requests to the right task structure?
Jan 27, 2026
Open Capability Benchmark Consortium
Repeated-query evaluation sets show stable intent interpretation across alternate phrasings of the same user goal.
Linked to
AI can understand what a task is really asking
Can it keep the same interpretation across paraphrases?
Feb 12, 2026
Open Capability Benchmark Consortium
Contamination-resistant reasoning sets show strong performance on problems that require actual stepwise solution rather than pattern recall.
Linked to
AI can reason through hard problems under real constraints
Can it solve multi-step problems with hidden answers?
Mar 8, 2026
Center for Applied AI Evaluation
Trace audits show meaningful gains, but still document contradiction rates rising on longer branching reasoning paths.
Linked to
AI can reason through hard problems under real constraints
Can it maintain logical consistency across long chains?
Feb 9, 2026
Frontier Evaluation Arena
Leaderboard tracks show strong progress, but still clear performance loss once tasks exceed a moderate number of simultaneous hard constraints.
Linked to
AI can reason through hard problems under real constraints
Can it satisfy explicit constraints without dropping any?
Mar 11, 2026
Open Capability Benchmark Consortium
Adaptive reasoning evaluations show models revising plans successfully in many cases, but not yet with robust consistency under repeated shifts.
Linked to
AI can reason through hard problems under real constraints
Can it adapt reasoning when constraints change?
Jan 31, 2026
Expert Review Panel on AI Performance
Blinded reviewers found many model explanations sufficiently structured for independent verification, though some still overstated certainty.
Linked to
AI can reason through hard problems under real constraints
Can it justify conclusions in a way that can be independently checked?
Feb 22, 2026
Open Capability Benchmark Consortium
Benchmark batteries show top systems learning new task formats from just a handful of examples across varied domains.
Linked to
AI can learn new tasks quickly and generalize beyond examples
Can it learn from a few examples?
Feb 25, 2026
Center for Applied AI Evaluation
Cross-domain evaluations show meaningful transfer, but still large drops when specialized terminology or unfamiliar workflows appear.
Linked to
AI can learn new tasks quickly and generalize beyond examples
Can it transfer to new domains, tasks, and languages?
Jan 21, 2026
Open Capability Benchmark Consortium
Generalization stress tests continue to show steep degradation when task inputs depart from the setup examples used to establish the pattern.
Linked to
AI can learn new tasks quickly and generalize beyond examples
Can it hold up under out-of-distribution inputs?
Mar 3, 2026
Open Capability Benchmark Consortium
Rule-discovery benchmarks show real abstraction ability, but with large failure rates once hidden rules become deeply compositional.
Linked to
AI can learn new tasks quickly and generalize beyond examples
Can it infer latent rules from demonstrations?
Feb 4, 2026
Expert Review Panel on AI Performance
Expert review of transfer tasks found many models building partial abstractions, but not applying them robustly under larger task shifts.
Linked to
AI can learn new tasks quickly and generalize beyond examples
Can it form reusable abstractions instead of copying surface patterns?
Jan 29, 2026
Open Capability Benchmark Consortium
Retrieval-grounding benchmarks show strong source selection performance for claims that have clear supporting passages or documents.
Linked to
AI can stay grounded in facts and evidence
Can it retrieve the right supporting sources?
Feb 28, 2026
Open Capability Benchmark Consortium
Benchmark suites show high performance on tying claims to the correct supporting text rather than merely nearby passages.
Linked to
AI can stay grounded in facts and evidence
Can it cite evidence correctly?
Mar 1, 2026
Center for Applied AI Evaluation
Study results show better abstention and fewer fabricated details, but still meaningful unsupported-claim rates in open-domain tasks.
Linked to
AI can stay grounded in facts and evidence
Can it avoid unsupported claims when evidence is missing?
Feb 15, 2026
Open Capability Benchmark Consortium
Repeated-answer benchmarks show better stability than earlier systems, but still detect answer drift under paraphrase and context shuffling.
Linked to
AI can stay grounded in facts and evidence
Can it stay consistent across repeated factual queries?
Jan 26, 2026
Model Red-Team Exchange
Adversarial evaluations continue to show models being pulled off course by false but authoritative-looking contextual scaffolds.
Linked to
AI can stay grounded in facts and evidence
Can it resist misleading or fabricated context?
Mar 9, 2026
Open Capability Benchmark Consortium
Calibration benchmarks show meaningful progress, but still reveal overconfidence on harder and more ambiguous tasks.
Linked to
AI knows when it may be wrong and can recover
Can it produce calibrated confidence?
Mar 4, 2026
Center for Applied AI Evaluation
Controlled evaluations show more frequent and more useful abstention than earlier systems, but still leave critical misses.
Linked to
AI knows when it may be wrong and can recover
Can it abstain when evidence is insufficient?
Feb 7, 2026
Expert Review Panel on AI Performance
Review panels found reliable detection of obvious inconsistencies, but weaker performance once contradictions depended on longer output context.
Linked to
AI knows when it may be wrong and can recover
Can it detect internal contradictions?
Jan 24, 2026
Center for Applied AI Evaluation
Repeated-answer studies show strong gains after critique or tool feedback, especially on factual and coding tasks.
Linked to
AI knows when it may be wrong and can recover
Can it self-correct after feedback or tool results?
Feb 27, 2026
Operational AI Audit Lab
Operational audits show improving escalation behavior in bounded workflows, but also document cases where the model still pressed ahead too confidently.
Linked to
AI knows when it may be wrong and can recover
Can it escalate uncertainty instead of bluffing in high-risk cases?
Mar 12, 2026
Open Capability Benchmark Consortium
Planning benchmarks show strong phase decomposition and milestone sequencing on well-specified multi-step goals.
Linked to
AI can plan and complete long, multi-step work
Can it decompose goals into workable phases?
Feb 26, 2026
Long-Run Systems Study Group
Long-run trials continue to show requirement drift and forgotten commitments over extended task horizons.
Linked to
AI can plan and complete long, multi-step work
Can it maintain task state over long horizons?
Mar 3, 2026
Operational AI Audit Lab
Operational audits show better dependency handling than earlier systems, but still document frequent misses on cross-workstream prerequisites.
Linked to
AI can plan and complete long, multi-step work
Can it track dependencies and intermediate outputs?
Feb 11, 2026
Open Capability Benchmark Consortium
Dynamic planning evaluations show strong recovery in moderate cases, but weaker restructuring once multiple assumptions break at once.
Linked to
AI can plan and complete long, multi-step work
Can it re-plan after failure or changing conditions?
Jan 30, 2026
Operational AI Audit Lab
Field audits show some partial autonomy, but still document frequent human rescue on longer projects with moving requirements.
Linked to
AI can plan and complete long, multi-step work
Can it deliver end results with limited supervision?
Mar 14, 2026
Open Capability Benchmark Consortium
Tool-use benchmarks show high rates of correct tool choice when the task and available interfaces are clearly defined.
Linked to
AI can use digital tools and external systems reliably
Can it choose the right tool for the task?
Mar 1, 2026
Operational AI Audit Lab
Operational audits show good performance on bounded multi-app flows, but recurring failures when errors propagate across tools.
Linked to
AI can use digital tools and external systems reliably
Can it execute multi-tool workflows correctly?
Feb 19, 2026
Operational AI Audit Lab
Agent traces show partial recovery behavior, but still frequent stalls and incorrect retries after external system failures.
Linked to
AI can use digital tools and external systems reliably
Can it recover from tool or API failures?
Mar 7, 2026
Frontier Evaluation Arena
Leaderboard results show top systems learning unfamiliar APIs and tool flows directly from documentation with modest correction.
Linked to
AI can use digital tools and external systems reliably
Can it learn new tools from documentation?
Mar 12, 2026
Operational AI Audit Lab
Trace audits show improving visibility into tool actions, though missing state transitions still limit full reconstruction.
Linked to
AI can use digital tools and external systems reliably
Can it leave an auditable trail of actions taken?
Feb 24, 2026
Open Capability Benchmark Consortium
Contamination-resistant coding suites show strong hidden-test pass rates on many focused implementation tasks.
Linked to
AI can build and maintain software at a professional level
Can it write code that passes hidden tests?
Mar 5, 2026
Open Capability Benchmark Consortium
Repository debugging benchmarks show strong progress, but still leave meaningful gaps on dependency and state-management failures.
Linked to
AI can build and maintain software at a professional level
Can it debug unfamiliar repositories?
Feb 16, 2026
Operational AI Audit Lab
Code audits show useful cleanup and modularization, but still document regression risk on poorly covered systems.
Linked to
AI can build and maintain software at a professional level
Can it refactor without breaking behavior?
Mar 9, 2026
Expert Review Panel on AI Performance
Senior engineer review panels judged many generated architectures workable, but not yet consistently strong on deeper platform tradeoffs.
Linked to
AI can build and maintain software at a professional level
Can it design reasonable architectures for real requirements?
Jan 20, 2026
Operational AI Audit Lab
Real project audits show promising build completion, but still document repeated human intervention at deployment and maintenance stages.
Linked to
AI can build and maintain software at a professional level
Can it ship and maintain a working small application?
Mar 15, 2026
Open Capability Benchmark Consortium
Grounding benchmarks show strong performance on tying language to the right regions, events, and perceptual cues.
Linked to
AI can understand the world across text, images, audio, video, documents, and space
Can it ground language to visual or audio evidence?
Mar 2, 2026
Operational AI Audit Lab
Operational audits show strong extraction and reasoning performance on dashboards, forms, and scanned document workflows.
Linked to
AI can understand the world across text, images, audio, video, documents, and space
Can it reason over screenshots, documents, and forms?
Feb 21, 2026
Open Capability Benchmark Consortium
Case-based multimodal benchmarks show strong integration of text, image, and chart evidence into unified conclusions.
Linked to
AI can understand the world across text, images, audio, video, documents, and space
Can it combine multiple modalities into one coherent conclusion?
Mar 8, 2026
Open Capability Benchmark Consortium
Temporal sequence benchmarks show useful progress, but still material misses when key changes are subtle or long-range.
Linked to
AI can understand the world across text, images, audio, video, documents, and space
Can it track state across temporal sequences?
Feb 13, 2026
Frontier Evaluation Arena
Leaderboard evaluations show top multimodal systems solving a large share of diagram, chart, and spatial-relation tasks.
Linked to
AI can understand the world across text, images, audio, video, documents, and space
Can it solve spatial and diagrammatic reasoning tasks?
Mar 13, 2026
Center for Applied AI Evaluation
Controlled interaction studies show strong performance in identifying user intent and choosing an appropriate explanation depth.
Linked to
AI can interact with people appropriately and work with them
Can it model user goals and knowledge level?
Feb 20, 2026
Expert Review Panel on AI Performance
Blinded reviewers consistently rate top systems highly on style adaptation across novice, expert, and stressed-user scenarios.
Linked to
AI can interact with people appropriately and work with them
Can it adapt tone and explanation to its audience?
Mar 5, 2026
Operational AI Audit Lab
Conversation audits show improving recognition of explicit frustration, but still document late recognition of subtle confusion and disagreement.
Linked to
AI can interact with people appropriately and work with them
Can it recognize confusion, frustration, or disagreement?
Feb 8, 2026
Operational AI Audit Lab
Operational evaluations show better collaboration structure and note quality, but still expose weak handoffs across larger multi-person workflows.
Linked to
AI can interact with people appropriately and work with them
Can it handle collaboration and handoff well?
Mar 9, 2026
Expert Review Panel on AI Performance
Expert interaction panels found many strong responses, but still documented noticeable gaps on indirect meaning and subtle perspective shifts.
Linked to
AI can interact with people appropriately and work with them
Can it interpret nuance, implied meaning, and differing perspectives?
Jan 28, 2026
Expert Review Panel on AI Performance
Expert blind review found that models generate many plausible ideas, but only a minority are judged meaningfully non-obvious and worth pursuing.
Linked to
AI can generate new, useful ideas and solutions
Can it propose novel hypotheses, designs, or solution paths?
Feb 14, 2026
Expert Review Panel on AI Performance
Panel reviews show some outputs standing out as genuinely useful, but many still read as polished synthesis rather than strong original contribution.
Linked to
AI can generate new, useful ideas and solutions
Can experts judge its outputs as non-trivial?
Mar 1, 2026
Center for Applied AI Evaluation
Controlled comparisons show AI-assisted teams producing broader option sets faster than baseline ideation processes.
Linked to
AI can generate new, useful ideas and solutions
Can it outperform baseline ideation workflows?
Feb 24, 2026
Center for Applied AI Evaluation
Evaluation studies show useful creativity under moderate constraints, but weaker performance when constraints become numerous or tightly coupled.
Linked to
AI can generate new, useful ideas and solutions
Can it adapt novelty to concrete constraints and goals?
Jan 18, 2026
Expert Review Panel on AI Performance
Reviewer studies show partial success in distinguishing empty polish from useful synthesis, but still document over-acceptance of generic output.
Linked to
AI can generate new, useful ideas and solutions
Can it distinguish useful synthesis from generic boilerplate?
Mar 11, 2026
Open Capability Benchmark Consortium
Safety and workflow benchmark suites show strong adherence to explicit rule sets in most ordinary task conditions.
Linked to
AI can stay controllable, safe, and resistant to misuse
Can it follow explicit policies and constraints?
Mar 6, 2026
Model Red-Team Exchange
Adversarial safety evaluations show solid ordinary refusal behavior, but still find inconsistency under paraphrase and persistence.
Linked to
AI can stay controllable, safe, and resistant to misuse
Can it refuse harmful requests consistently?
Feb 12, 2026
Center for Applied AI Evaluation
User studies show many better-than-baseline redirections, but still highlight inconsistency in how helpful those alternatives are.
Linked to
AI can stay controllable, safe, and resistant to misuse
Can it offer safe alternatives where appropriate?
Jan 25, 2026
Model Red-Team Exchange
Red-team tracks continue to expose meaningful safety degradation under sophisticated adversarial prompt attacks.
Linked to
AI can stay controllable, safe, and resistant to misuse
Can it resist jailbreaks and adversarial prompting?
Mar 13, 2026
Operational AI Audit Lab
Release and deployment audits show that control quality can shift after prompt or capability changes, even when headline behavior looks stable.
Linked to
AI can stay controllable, safe, and resistant to misuse
Can it remain steerable after prompt or capability changes?
Feb 27, 2026
Operational AI Audit Lab
Operational audits show improving private-data handling, but still document avoidable leakage and over-retention in some workflow designs.
Linked to
AI can handle sensitive and high-stakes contexts responsibly
Can it protect private or confidential information?
Mar 2, 2026
Operational AI Audit Lab
Domain audits continue to show that models can follow many explicit rules, but still miss edge-case and jurisdiction-specific constraints.
Linked to
AI can handle sensitive and high-stakes contexts responsibly
Can it comply with regulated-domain rules?
Feb 6, 2026
Center for Applied AI Evaluation
Controlled high-stakes evaluations continue to show accuracy gaps large enough to require strong human oversight.
Linked to
AI can handle sensitive and high-stakes contexts responsibly
Can it maintain factual accuracy in high-stakes tasks?
Jan 30, 2026
Operational AI Audit Lab
Case audits show partial action trace coverage, but still incomplete reconstruction of hidden state, retrieval paths, and tool decisions.
Linked to
AI can handle sensitive and high-stakes contexts responsibly
Can it provide auditable evidence trails for decisions?
Mar 10, 2026
Long-Run Systems Study Group
Longitudinal deployment studies remain too sparse and mixed to support broad claims that AI assistance improves high-stakes outcomes without offsetting risk.
Linked to
AI can handle sensitive and high-stakes contexts responsibly
Can it improve outcomes without increasing harm?
Mar 15, 2026
Open Capability Benchmark Consortium
Robustness benchmarks show useful tolerance to moderate corruption, but still clear degradation once inputs become meaningfully incomplete or messy.
Linked to
AI stays reliable in messy real-world conditions
Can it maintain performance under noisy or incomplete inputs?
Mar 7, 2026
Open Capability Benchmark Consortium
Generalization evaluations continue to show substantial drops when task distributions or wrappers depart from familiar benchmark-style conditions.
Linked to
AI stays reliable in messy real-world conditions
Can it withstand distribution shift and novel task formats?
Feb 17, 2026
Long-Run Systems Study Group
Long-duration trials continue to document noticeable drift, forgotten constraints, and rising oversight needs over repeated use.
Linked to
AI stays reliable in messy real-world conditions
Can it remain stable over long sessions or repeated use?
Mar 13, 2026
Operational AI Audit Lab
Workflow audits show some self-recovery, but still frequent downstream compounding once an early assumption is wrong.
Linked to
AI stays reliable in messy real-world conditions
Can it avoid cascading errors after early mistakes?
Feb 28, 2026
Model Red-Team Exchange
Adversarial evaluations show systems still being materially misled by poisoned framing, false premises, and manipulative context setup.
Linked to
AI stays reliable in messy real-world conditions
Can it preserve competence under misleading or adversarial context?
Mar 12, 2026
Operational AI Audit Lab
Deployment inventories show wider assistant use, but not reliable performance across a broad portfolio of meaningful delegated roles.
Linked to
AI can be trusted with meaningful delegated responsibility
Can it perform reliably across varied role types?
Mar 9, 2026
Operational AI Audit Lab
System audits show increasingly capable end-to-end behavior, but also document brittle interactions once many capabilities are composed together.
Linked to
AI can be trusted with meaningful delegated responsibility
Can it integrate multiple capabilities into one coherent system?
Feb 23, 2026
Operational AI Audit Lab
Governance audits show some usable traces and controls, but not enough coverage or reliability for broad institutional delegation.
Linked to
AI can be trusted with meaningful delegated responsibility
Can it be monitored and governed after the fact?
Mar 14, 2026
Model Red-Team Exchange
Tail-risk evaluations continue to find too much uncertainty and too many unresolved severe-failure modes for broad delegation.
Linked to
AI can be trusted with meaningful delegated responsibility
Can it avoid severe failures in high-impact settings?
Mar 16, 2026
Operational AI Audit Lab
Deployment reviews show expanding use, but not the sustained, high-responsibility institutional delegation that this threshold would require.
Linked to
AI can be trusted with meaningful delegated responsibility
Can it show strong evidence from real deployments, not only lab wins?
Mar 17, 2026