time, I began a multi-part post to discuss techniques used in measuring
video quality. I
introduced the concept of a Mean Opinion Score (MOS), which has been
used for years as a measure of voice quality, and is now being used in a
similar way to quantify the quality of a video stream.
I'll continue this discussion by talking about two alternative methods
for measuring video quality - "full-reference" and "reference-free"
My hope is that this information may help you if you find
yourself evaluating different video quality measurement solutions from
If you imagine your user
experience as you watch a stream of IP video arriving at your laptop or
on your mobile device, your mind probably pictures something that is not
perfect - the video probably pauses to buffer sometimes, the image
itself might sometimes freeze or appear "blocky" or pixelated. This is often the
nature of today's video as it streams through bandwidth-constrained
this environment, vendors are developing automated systems to analyze
the video and provide some measurement of the video quality. But they are choosing
to do it in different ways.
Some vendors are developing
solutions using what they call a "full reference" measurement model. This means that they
have a "reference" copy of the video - a "golden" copy, if you will -
that existed before the video got compressed, sent through the network,
and decompressed. These
vendors choose to measure the video quality by comparing the output
from the network against the "reference" copy... in this way, the system
can identify the areas of the image that have changed.
approach delivers excellent results, but is computationally expensive. Sometimes, each pixel
of each image must be compared to get a true result, and these
calculations can often not be completed in real time. And not all applications allow
for a reference copy - user-generated content that is streamed from one
iPhone to another does not allow for a comparison against a reference.
most common application for full-reference video quality measurement is
in pre-service network testing. This is an extremely useful tool for verifying that video
processing equipment can do its job, or that a network has been
engineered to support the transport of high-quality video content. These types of systems
are generally deployed in lab environments, and can be used to measure
the results from different network and equipment configurations.
alternative approach is called "reference-free" measurement. In this case, there is
no "golden" copy of the content - instead, the video is analyzed
dynamically for characteristics like blurriness or blockiness. In fact, in some
reference-free systems, the vendor actually measures network
characteristics and then predicts the effects that these network
conditions will cause on the video stream.
solutions can typically complete their analysis in real-time with
acceptable computational requirements. The drawback is that their
analysis is not as precise as that provided in a full-reference model. For example, the
original video may not have been very good - the reference free method
will (accurately) claim that the video coming from the network has poor
quality (and so the network might be suspect). The full-reference
implementation will tell you that the video coming out of the network is
the same as what went in, and so the network and equipment are working
methods are typically deployed today for in-service video quality
measurement systems that require real-time results.
two methods have adopted different philosophies, and each is
appropriate for different target applications. Full-reference systems make
great sense in pre-service lab testing, where precise quality
measurement is important and real-time results are not necessary. A reference-free
measurement system is a better choice for in-service monitoring, since
it can perform its job in real-time and give good information about the
video quality. The
key is to identify the needs of your business, and choose the model
that best fits your requirements.