Question Quality Benchmark: Practice Checks and AI Answers

Counts refer to entries and revisions, not a claim that every example is psychometrically independent. A worked problem can deliberately support a packet or skill lesson. These public counts are separate from the app’s dated question-bank inventory.

For √(x + 6) = x, squaring produces candidates 3 and −2. A correct check returns to the original equation: 3 works because √9 = 3; −2 fails because √4 = 2. The local verification also checks original denominator restrictions, interval endpoints, integer counts, percentage bases, geometry scaling and conditional-probability denominators.

Separate Google reasoning contexts audited all added skill and packet question entries, their choice explanations and the 12 revised transfer checks. The retained Gemini 3.1 Pro review and the later Gemini 4 Argon review both reported no remaining substantive issue in the final datasets. Model audits are automated checks, not named teacher approval or proof that every possible issue has been eliminated.

Both runs use the same versioned original case set. Cases cover linear systems, quadratic maxima, excluded values, extraneous roots, successive percentages, dependent systems, negative fractions, area scaling, conditional denominators, two-root reporting, grammar, transitions, synthesis goals and unsupported-score/outcome claims.

The test calls the real Google provider with a benchmark tutor instruction. It does not exercise the entire signed-in app, photo or voice tutoring, subscription access or a student learning session. The recorded metric compares the returned final-answer field with the reference answer. Explanations are retained for inspection and are not counted as independently human-approved.

Sixteen deliberately selected cases are too small and too narrow to estimate correctness across every tutor conversation or future question. We do not describe the result as universal accuracy. Model self-reported confidence is not the grading rule. A correct final answer can coexist with an incomplete or overstated explanation.

For example, the retained 3.1 response correctly rejected a verified 200-point improvement from a brief session, but its explanation overstated what is required to document an individual before/after score change. Official score records can document that change; stronger study design is needed to attribute the improvement to a product. The public benchmark separates the final-answer match from this explanation caveat.

No real student identities, actual customer outcomes or admissions claims are used. These tests do not calibrate practice difficulty, establish an official raw-score conversion or prove a score increase from PeakSAT.

The downloadable evidence contains the complete original benchmark prompts, accepted reference answers, retained provider responses, generation settings, dates, case-set hash and source-report hashes. It also lists the teaching-asset checks and the actual audit models.

In the PeakSAT source workspace, run:

node scripts/growth-assets/verify-content.cjs
node scripts/growth-assets/audit-content-google.cjs --gemini4-final
node scripts/growth-research/run-quality-benchmark.cjs

The first command is local and deterministic. Model commands require authorized server-side Google credentials and record a fresh result; repeat calls may differ. Compare the same case version and report unavailable requests separately. Credentials and student data are excluded from the public evidence.

October 13, 2026: Added the 29-owner depth set, complete grammar/algebra packets, fresh worked-solution checks and the English/Spanish family guide. Independent local calculations and the final Google 3.1 Pro and Gemini 4 Argon content audits passed.

During preparation: Replaced a redundant numeric distractor notation in a statistics question and simplified a rational-expression prompt’s unnecessary extra restriction. A draft audit originally lacked the original worked-question prompt and flagged a context mismatch; the retained final audit included both original and transfer prompts. These were changes before publication, not concealed post-publication answer corrections.

Material correction process: Send the URL, exact statement and your reasoning to nicole@peaksat.com. A verified material correction updates the affected resource and records its scope here. No post-publication answer correction is currently recorded for this October 13 teaching set.