Methods are blinded. Judge only A relative to B; do not infer identities from appearance.
Video A
Video B
A-relative-to-B scale: −2 = B much better · −1 = B slightly better · 0 = tie/indistinguishable · +1 = A slightly better · +2 = A much better.
Critical failures in A
Critical failures in B
Short evidence
Protocol and rubric
Enter a unique rater ID before scoring.
Watch both complete videos at normal speed. Revisit ambiguous motion at 0.5×.
Score all six dimensions independently. A visual improvement must not compensate for a physical failure in another field.
Mark explicit critical failures and write concise frame/event evidence.
Do not discuss assignments or judgments until all reviewers export their files.
Completion requires all six ratings and the “watched fully at 1×” confirmation for every pair. Progress is stored locally in this browser. Send the exported JSONL—not the progress backup—to the study coordinator.