What is it about?
AI coding tools can now write, run and fix whole programs, and many students use them for coursework. Standard autograders only check whether the final code passes its tests, which says little about how the student worked with the AI. We propose a framework that scores a recorded coding session on five skills: problem solving, planning, prompting, checking the AI's work, and directing an AI agent. Each score links to the moment in the session replay that supports it, so instructors and students can see the evidence.
Featured Image
Photo by Igor Omilaev on Unsplash
Why is it important?
When an AI writes the code, a passing program no longer shows that the student understood or directed the work. Instructors need ways to assess the skills that matter when AI is allowed. The framework works at three levels of AI access: none, hints only, or a full AI agent. It runs in an assessment platform we built, and it is early work: we describe a planned study comparing its scores with those of human graders.
Perspectives
This paper started with a question I could not answer. In 2024, while I trained for the ICPC regional contest, models began solving the same problems in minutes, and I no longer knew what my practice was for. Courses face the same question when they grade. Test suites check only the final code, so a student who ships code they do not understand can score the same as one who planned the work, prompted with care, and checked every change. Our framework scores the session itself, with each score linked to the moment in the replay that earned it. Next, we plan a study that compares its scores with those of human graders.
Eli Young
AlgoArena
Read the Original
This page is a summary of: A Competency Framework for Assessing AI-Assisted Programming Work, November 2026, ACM (Association for Computing Machinery),
DOI: 10.1145/3795860.3846260.
You can read the full text:
Resources
Contributors
The following have contributed to this page
