Comparative Study of Subjective Video Quality Assessment Test Methods in Crowdsourcing for Varied Use Cases

Babak Naderi, Ross Cutler · 2025

Read Comparative Study of Subjective Video Quality Assessment Test Methods in Crowdsourcing for Varied Use Cases by Babak Naderi, Ross Cutler (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-20118.txt.

Description

Babak Naderi and Ross Cutler wrote this 2025 arXiv paper on video quality testing. Companies often ask crowds of online viewers to rate video quality. Raters can use a plain rating scale, a scale with a hidden clean reference, or a comparison scale. The paper tests which of these three methods works best in practice.

The team ran six studies that follow the P.910 standard for such tests. They used fifteen talking-head clips with flaws like blur, compression, scaling, and freezing. The plain and hidden-reference methods agreed closely at the level of whole conditions. The comparison method was more sensitive and caught gains beyond the reference. The hidden-reference method was about twice as fast and cheaper, but it squeezed the rating scale. The authors end with practical advice on picking a method for each use case.

Catalog record

CREATORBabak Naderi, Ross Cutler
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE15 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-20118

Browse the full library · About the Tower of Babel