Research Deep-Dive · XiaoHu Explains

Dartmouth Tested AI Grading on Real Coursework: Students Called It Rigid, But the Ones Who Used It Scored Higher

151 students tested it: short-answer questions moved the needle far more than multiple-choice, while almost nobody touched the AI Q&A sidebar
Quick Take
  • Dartmouth College piloted an AI learning platform called Phosphor with 151 students in a statistics course — a fully optional, ungraded textbook replacement where Claude Sonnet 4.6 handled grading on short-answer questions.
  • 90.2% of students used the platform at least once, more than 6x the course's self-reported textbook reading compliance rate (10 to 15%).
  • Full engagement (completing all 24 lessons plus passing all 3 cross-lesson module reviews) versus zero engagement produced a final-exam gap of 0.71 standard deviations after controlling for prior performance, or 1.30 standard deviations without controls.
  • Due to student feedback, the exam format for the three modules changed twice, accidentally creating a natural experiment: only the module requiring students to write their own short answers retained a "more practice, higher scores" relationship — in the pure multiple-choice module, that relationship nearly vanished.
  • The platform's built-in AI Q&A chat sidebar was queried just 72 times all semester, with only 14 students using it more than once.
1 Dartmouth's Textbook Experiment

They Put AI Grading Inside the Textbook — and 90% of Students Bought In Voluntarily

A statistics teaching team at Dartmouth College recently piloted an AI learning platform called Phosphor with 151 students, embedding short-answer and multiple-choice quizzes directly into the reading flow, with Claude Sonnet 4.6 grading short answers against rubrics the instructors defined.

Under a fully optional, ungraded setup, 90.2% of students used the platform at least once; students who fully engaged scored up to 1.3 standard deviations higher on the final exam than those with zero engagement.
◆
Why it matters: that 90.2% voluntary usage rate is more than 6x this course's self-reported textbook reading compliance rate (10 to 15%). It's evidence that AI grading can genuinely embed itself into daily teaching, get used by students, and correlate with final grades.

This platform is aimed at a well-established contradiction. On one side, college students basically don't read their textbooks: reading compliance has been sliding since the 1980s, students actively resist reading assignments, and objectively measured compliance runs far below self-reported numbers. In this course (MATH 010, Introductory Statistics), students self-reported about 15% reading compliance, the instructor estimated only 10%, and individual student comments ranged from "literally nobody reads this" to "wait, this needs to be recorded too?"

On the other side, letting students use AI unsupervised can actually hurt learning. Bastani et al.'s randomized controlled trial with nearly a thousand students found that giving unrestricted GPT-4 access actually dropped scores by 17% once the tool was taken away — students had turned it into a crutch rather than a learning tool; only the version with pedagogical guardrails avoided that side effect. Meanwhile, the share of students using AI for coursework keeps climbing: a 2026 Higher Education Policy Institute (HEPI) survey found 94% of college students had used generative AI on graded assignments, up from just 53% two years earlier.

−17%
Score drop after tool removal in an ~1,000-student RCT using unrestricted GPT-4
94%
Share of college students who used AI on graded assignments in 2026 (vs. 53% two years prior)
10–15%
Self-reported textbook reading compliance rate in this course

What Phosphor (formerly named Spongium) is trying to do is turn "active answering" into a structural step baked into the reading experience: read the text, then immediately take an AI-graded mini-quiz. Its design premise is that AI is most useful when embedded directly into the content delivery system itself — echoing the well-documented "doer effect" from education research, which we'll get to below.

Used the Phosphor platform
(optional, ungraded)
90.2%
Traditional textbook reading compliance
(student/instructor self-report)
10–15%
2 How the Platform Works

Finish the Reading, Take the Quiz, Get Graded by AI on the Spot

Phosphor is a web app whose course content is organized into "lessons," each paired with a question bank of 15 to 20 items. After finishing a lesson, students take a quiz on the same page, and it's graded instantly by a mix of automated scoring and AI.

Read lesson Short-answer quiz Module review Final exam
1
Read a lesson — the sidebar shows the full course outline and progress on each lesson. The course content was written by the team itself, based on open educational resources.
2
Randomly draw 4 questions from that lesson's question bank for the quiz. Multiple-choice (MCQ) is graded automatically by the system; short-answer (CRQ) is handed to Claude Sonnet 4.6, which grades against the rubric and reference answer the instructor set for each question, and explains why an answer was right or wrong. Scoring above 75% counts as a "pass." 40% of the question bank is short-answer, 60% multiple-choice. There's no reading gate — students can read and quiz in any order, and retakes are unlimited.
3
Once enough lessons accumulate, a cross-lesson module review triggers: 10 questions covering the whole module across multiple lessons, requiring 90% to pass, also with unlimited retakes. Reviews default to multiple-choice only; students can opt into an "all question types" mode that adds short-answer items back in.
4
The course ends with a paper-and-pen final exam, held under strict in-person proctoring. The full semester lines up with two midterms and one cumulative final, which this study uses to measure outcomes.