Ihza MahendraResume
Back to Evaluation & calibration
Evaluation & calibration

Automatic grading system that can't be gamed by blind resubmission

A submission system that reads whatever a person turns in (a document, image, code file, or link), grades it against a rubric, and catches people who resubmit the same broken work hoping it slips through the second time.

What it is

A real-time submission system that runs when a person turns in a piece of work for review. It extracts the actual content out of whatever they submitted (a PDF, an image, a code file, or a URL), grades that content against a rubric, and returns a structured verdict: what was good, what wasn't, and what to improve to go beyond the bar. If a person resubmits a failed attempt without actually fixing the underlying problem, the system catches that and refuses to flip the result to a pass.

What it's for

Submission is the moment of truth: someone hands over their work and the system has to say yes or no. If the grading is wrong, the person loses trust in it. If the grading can be gamed, the person learns the wrong lesson (resubmit blindly until it happens to pass). If grading is slow, the person disengages while waiting. So this has to be fast, has to be accurate, and has to refuse to reward a resubmission that didn't actually fix anything.

How it was built

On each submission, an initial check (is this even on-topic) and the full grading run at the same time to start shaping the verdict the moment the file lands, then stream back so it doesn't feel like a long wait. Extraction handles whatever gets submitted: PDFs and images go through a multimodal reading step, code files through structured parsing, links through a fetch-and-read pipeline. Grading runs through a chain of models, with backups if the primary is unavailable, so one provider going down doesn't stall every submission. The result is a structured verdict: a pass/fail status, what was good, what needs fixing with its exact location in the file, and how to exceed the bar. Underneath, every submitted file gets a fingerprint, and on any resubmission the new fingerprint is compared against the last one and the new set of issues is compared against the previous unresolved ones. If the file is essentially unchanged and the same issues remain, the system blocks the pass so a person can't win by simply resubmitting the same thing.

My role

Sole author of the real-time evaluator, the extraction pipeline, the rubric grading, the anti-gaming guard, and the structured feedback shape.

Built with
PythonFastAPIWebSocketsClaudeGeminiOpenAI fallbackSHA-256 fingerprintingFuzzy matchingMultimodal extraction

Want the full technical depth, the tradeoffs, what broke, what I'd do differently? Ask the agent about this project.