my last post, I discussed the importance of quality in the world of
mobile video; I claimed that consumers who watch video on their cell
phones (or other mobile device) would not accept a low-quality
have to admit that I expected some disagreement with my position -
after all, wireless voice consumers have shown a willingness to put up
with all kinds of nonsense (incoherent speech quality, missing
syllables, dropped calls), all for the sake of the convenience that
mobile telephony provides. So why would consumers demand higher quality when watching video? I
guess I must have made a convincing argument, because the feedback has
been pretty sparse; if you would like to disagree with my position (and
state the reasons for the disagreement) I would welcome your comments here.
But moving on...
I want to begin a multi-part discussion about the ways that service
providers (and especially wireless service providers) can measure the
quality of the experience that they provide to their consumers of video
content. I'll start with a definition of one of the most commonly-used techniques for comparing media quality.
For years, there has been a standard measurement technique used in the field of multimedia - the "Mean Opinion Score" or MOS. (Note that sometimes people refer to a "MOS Score" but that is redundant.) The MOS is a numerical value between 1 - 5, with 5 representing the best quality. Traditionally, the MOS was calculated in a very manual, time-consuming way: Groups of people were recruited and invited to listen to a set of different recordings (typically audio). After each audio clip, each person gave a numeric score to the quality of that clip. The average of the scores across the group became the MOS for that piece of content. Typically a score around 4.1 was considered very good, while a score of 2.7 was pretty lousy.
that created codecs (software to COmpress and DECompress audio or video
content so it could move efficiently through a network) then used these
scores in marketing their products. Any
vendor that created a "new and improved" codec would generate mean
opinion scores that showed that their product was superior to the
previous generation and would trumpet those results in advertising.
One important thing to keep in mind with these numeric scores: MOS is, by definition, a relative measurement. It is properly used as a comparative measure of quality. So, it is correct to say, "In this test, my product has a MOS of 4.17 while the competition has a MOS of 4.08." However, it is technically incorrect to say, "My new product has a MOS of 4.1." This
is because the group of people generating the scores may be "easy
graders" or "tough graders" so it is important to use the relative
measurement to counter this bias.
These days, while the use of MOS in advertising continues, the method of generating the scores has changed. Companies have created algorithms that can estimate a MOS based on the characteristics of the media stream. So
instead of recruiting people to sit in a room and listen to (or watch)
clips, companies can use measurement tools to generate mean opinion
scores automatically. The results are typically very reliable, and correlate well with the results that would have been generated by humans.
this environment, then, the use of automatically-generated mean opinion
scores as a measure of video quality is becoming more common. Several vendors are offering products that can estimate the MOS based on network characteristics. In
the next installment of this series, I'll discuss some of the types of
impairments that may affect a video MOS, and some of the conditions (in
the network or elsewhere) that may cause these impairments. Stay tuned!