Standardization Trends in Video/Multimedia Quality Assessment

Arthur Webster, Philip Corriveau, Vittorio Baroncini, Margaret Pinson

Abstract

This tutorial will explore the recent history of Video/Multimedia Quality Assessment Standardization and describe some of the current work in the field. The talk will focus on longstanding as well as newer organizations including the Video Quality Experts Group (VQEG), the International Telecommunications Union Groups (ITU-R WP6C, ITU-T SGs 9 and 12), the Motion Picture Experts Group (MPEG), and 3D@Home. An introduction to technical aspects of the field of video quality assessment will be given including both subjective and objective methodologies. Current trends in the progression of the state of the art of quality assessment will be detailed by participants active in the above named groups. We will touch upon the the relative merits of competitive versus collaborative approaches to technical standardization efforts.

Speakers

Arthur Webster

Webster

Mr. Webster is the Chairman of the International Telecommunications Union, Telecommunications Standardization Sector's (ITU-T) Study Group (SG) 9. ITU-T SG9 is responsible for international standardization activities related to "Television and sound transmission and integrated broadband cable networks." Mr. Webster serves in this position for the current study period (2009-2012). He has for many years participated in many technical standards groups--most of which are devoted to the standardization of video and multimedia quality assessment methods. Mr. Webster began participating regularly in ITU-T in 1994 and was a Rapporteur (in SG9 or SG12) from 1995 to 2009. Mr. Webster is also a Co-chair of the Video Quality Experts Group (VQEG), a position he as held since its founding in 1997.

Mr. Webster contributes technically to video and multimedia quality assessment as well as to the development and standardization of priority, reliability, and security aspects of emergency telecommunications services over IP networks. He is co-author of several approved and draft ITU-T Recommendations and Technical Reports in the areas of priority telecommunications, quality assessment, and security. Mr. Webster has participated in ATIS PRQC, (Performance, Reliability, and Security Committee), and its predecessor committees (T1Q1, T1A1) since 1990. He served as Chair of the ATIS PRQC Security Task Force for several years. He periodically participates in other Standards groups such as the SCTE, IETF, IEEE, ITU-T Study Groups 12 and 16, ITU-R Study Group 6, TIA, and ATIS IIF. Mr. Webster has chaired sessions and given presentations at several ITU and ETSI workshops.

Arthur holds a BA in English and a MS in Electrical Engineering. His past work included signal and image processing in radar, gyroscope testing, laser speckle imaging, IP networking, and emergency communications, but the majority of his career involves video and multimedia quality assessment and standardization. Since 1990 he has worked for the U.S. Department of Commerce, National Telecommunications and Information Administration (NTIA), Institute for Telecommunication Sciences (ITS) at their Boulder Laboratories. NTIA/ITS is a part of the United States Department of Commerce. From 2005 to 2008 he was the Standards Development Team Leader for the NTIA/ITS.P Division. He manages 2 technical projects at NTIA/ITS, both of which have strong standards outputs. Mr. Webster is a member of IEEE and ACM and holds two U.S. patents for innovations in objective assessment of video quality.

Philip Corriveau

Corriveau

Philip Corriveau is a Principal Engineer in the technology arm of the Interaction & Experience Research group in Intel Labs. Philip received his Bachelors of Science Honors at Carleton University, Ottawa Canada in 1990. He immediately started his career at the Canadian Government Communications Research Center performing end-user subjective testing in support of the ATSC HD standard for North America. In January 2009 he was awarded a National Academy of Television Arts & Science, Technology & Engineering Emmy® Award for User Experience Research for the Standardization of the ATSC Digital System.

Philip moved to Intel in 2001 to seed a research capability called the Media and Acoustics Perception Laboratory designed to address fundamental perceptual aspects of platform and product design. He now manages a team of human factors engineers in the Experience Metrics & Quality Evaluation group conducting user experience research across Intel technologies, platforms and product lines.

Philip is currently the Chair of Steering Team 5 for 3D@Home addressing Human Factors issues surrounding the development of 3D technologies for end-users. He founded and still participates in the Video Quality Experts Group, aimed at developing, testing and recommending for standardization objective video quality metrics.

Vittorio Baroncini

Baroncini

Education baccalaureate in Physics, in digital electronics. Work experience began work in 1976 at the ITT R&D Laboratory of Pomezia (Italy), as HW designer. And in 1984, began to work at the "Instituto Superiore delle Poste e Telecomunicazioni", the technical branch of the Italian Government's Telecommunication Ministry. Worked in the area of national regulatory activity, dealing with line transmission system on digital networks (PDH). In 1986, began work at the Fondazione Ugo Bordoni (FUB), and moved to the TV group of FUB, with responsibility for the design of HW. At the same time, began participating in international standards work, by contributing to the works of the CCIR (now ITU-R) Committee. Began activity in the International Standards Organization (ISO) in 1992, participating in the works of the SC29-WG11 (MPEG) group dealing with the MPEG-2 verification tests. In 1995, he took responsibility of FUB involvement in the EC projects "QUOVADIS", and "MOSQUITO", both of them related to the automatic control of QoS in TV digital telecommunication networks. In these projects he contributed with a new algorithm for a "reduced reference" QoS metric to be applied on digital TV network capable of a return channel on the "control network". Co-founder (1997) of the Video Quality Expert Group (VQEG), a joint group of T and R Sector experts of ITU, aiming at the promotion of new metrics for the objective video quality assessment, he lead the group to the first world wide international standard of objective quality metric for digital TV (ITU-R Recommendation BT- 1683). Currenlty co-chair of the HDTV and of the Multimedia II project and member of the ILG (Independent Laboratory Group). From 2000 in the steering board of two prominent Working Parties in the ITU-R: chair of WP6Q (Performance assessment and quality control) and vice-chair of TG6/9 (Digital Cinema). On 2008 he became Vice-Chair of ITU-R WP6C and Chair of the MPEG Test Group. On 2010 he managed the Subjective Quality Evaluation test issued by ISO and ITU-T in the joint effort for the definition of the new HVC standard in video coding; following that successful experience he was assigned the position of test coordinator of the subjective test for the 3DV Call for Proposal. He is the technical responsible for video quality assessment and of the HDTV, 3DTV and D-Cinema projects in FUB.

Author of many papers to conferences and journals he is also co-author of two books on MPEG.

Margaret H. Pinson

Pinson

Margaret H. Pinson (margaret@its.bldrdoc.gov) has over twenty years of experience in video quality research. She received her B.S and M.S. degrees in computer science from the University of Colorado at Boulder, USA, in 1988 and 1990, respectively.

Since 1988, she has been working as a researcher at the Institute for Telecommunication Sciences (ITS) in Boulder, Colorado. Margaret H. Pinson joined the Video Quality Program in 1989. She is project leader of the Video Quality Research Program. Her goal is to develop automated metrics for assessing the performance of video systems and actively transfer this technology to end users, standards bodies, and U.S. industry. She is a Co-Chair of the Independent Lab Group (ILG) in the Video Quality Experts Group (VQEG) and an Associate Rapporteur of Question 12 in ITU-T Study Group 9. She has served as technical editor for VQEG Final Reports and ITU-T Recommendations. She administers the Consumer Digital Video Library.

Patrick Le Callet

Le Callet

Patrick LE CALLET received M.Sc. degree PhD degree in image processing from Ecole polytechnique de l'université de Nantes. He was also student at the Ecole Normale Superieure de Cachan where he get the "Aggrégation" (credentialing exam) in electronics of the French National Education. He has working as an Assistant professor from 1997 to 1999 and as a full time lecturer from 1999 to 2003 at the department of Electrical engineering of Technical Institute of University of Nantes (IUT). Since 2003 is teaching at Ecole polytechnique de l'université de Nantes (Engineer School) in the Electrical Engineering and the Computer Science department where is now Full Professor. Since 2006, he is the head of the Image and VideoCommunication lab at CNRS IRCCyN, an academic research group of more than 35 researchers. He is mostly engaged in research dealing with the application of human vision modeling in image and video processing. His current centers of interest are 3D image and video quality assessment, watermarking techniques and visual attention modeling and applications. He is co-author of more than 190 publications and communications and co-inventor of 13 international patents on these topics. He has coordinated and is currently managing for IRCCyN several National or European collaborative research programs representing grants of more than 2,5 million euros. He is serving in VQEG (Video Quality Expert Group) where is co-chairing the "HDR Group" and "3DTV" activities. He is currently serving as associate editor for IEEE transactions on Circuit System and Video Technology, SPIE Journal of Electronic Imaging and SPRINGER EURASIP Journal on Image and Video Processing.