MPEG1 - City University of Hong Kong City Moving Picture Expert Group Established in 1988 by the Joint ISO/IEC Technical Committee on IT Mission To

Info iconThis preview shows pages 1–6. Sign up to view the full content.

View Full Document Right Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon

Info iconThis preview has intentionally blurred sections. Sign up to view the full version.

View Full DocumentRight Arrow Icon
This is the end of the preview. Sign up to access the rest of the document.

Unformatted text preview: City University of Hong Kong City Moving Picture Expert Group - Established in 1988 by the Joint ISO/IEC Technical Committee on IT. Mission - To develop standards for coded representation of motion pictures and audio at a bit rate of up to 1.5Mb/s. MPEG-1 was issued in 1992. MPEG-2 (1994) - higher quality (not lower than NTSC and PAL) with bit rates between 2-10Mb/s. Applications - Digital CATV and Terrestrial digital broadcasting distribution, Video recording and retrieval. City University of Hong Kong City Lossy compression Trade off image quality with bit rate according to objective or subjective criteria Video sequences usually contains large statistical redundancies in both temporal and spatial directions Intraframe coding Interframe coding Subsampling of Chrominance - Human eye is more sensitive to luminance than chrominance City University of Hong Kong City Encoding of a single picture Similar to JPEG Discrete Cosine Transform- Converts spatial to frequency domain Quantization of spectral coefficients DPCM to encode DC terms Zigzag scan to group zeros into long sequences, followed by run-length coding Lossless, Variable Length Coding to encode AC coefficients City University of Hong Kong City Remove temporal redundancies between frames Use extensively in MPEG-1 and MPEG-2 Based on estimation of motion between video frames Use of motion vectors to describe displacement of pixels from one frame to the next Spatial correlation between motion vectors are high One motion vector can represent the motion of a block of pixels. City University of Hong Kong City For each image block in the current frame, Find its nearest counterpart in the previous frame. Record the displacement vector Figure 1 Previous frame Current frame City University of Hong Kong City Search Window Previous Block Location Current Block Location mv Frame N-1 Frame N Figure 2 Only the prediction error (residual) images are transmitted Good prediction reduces information content in residual images Partition the previous and the current images into nonoverlapping square blocks of size NxN Previous frame Current frame Partition the previous and the current images into nonoverlapping square blocks of size NxN Previous frame Current frame e.g., N=8 Partition the previous and the current images into nonoverlapping square blocks of size NxN Previous frame Represent each block with a 2D matrix: f(x,y) for previous frame g(x,y) for current frame Current frame Partition the previous and the current images into non-overlapping square blocks of size NxN Previous frame Represent each block with a 2D matrix: f(x,y) for previous frame g(x,y) for current frame Current frame Difference between any two blocks is given by 1 D( f , g ) = 2 N Row = 0 Col = 0 [ f ( x, y ) − g ( x, y ) ] 2 ∑∑ N N Partition the previous and the current images into non-overlapping square blocks of size NxN Previous frame Represent each block with a 2D matrix: f(x,y) for previous frame g(x,y) for current frame Current frame The lower the difference, the more similar is the pair of blocks 1 D( f , g ) = 2 N Row = 0 Col = 0 [ f ( x, y ) − g ( x, y ) ] 2 ∑∑ N N A motion vector is computed for ‘EVERY’ blocks in the current frame. HOW? Previous frame Current frame A motion vector is computed for ‘EVERY’ blocks in the current frame. HOW? Previous frame Current frame Each block in the current frame matched against all the blocks in the previous frame, the closest one is taken to be its counterpart. MV=(-2,-3) x y Previous frame Current frame Each block in the current frame matched against all the blocks in the previous frame, the closest one is taken to be its counterpart. MV=(-1,-3) x y Previous frame Current frame Each block in the current frame matched against all the blocks in the previous frame, the closest one is taken to be its counterpart. MV=(-1,-2) x y Previous frame Current frame Each block in the current frame matched against all the blocks in the previous frame, the closest one is taken to be its counterpart. Previous frame Current frame The method is slow, especially if the image resolution and N are large. A motion vector is computed for ‘EVERY’ blocks in the near neighborhood of the current frame. Previous frame Current frame For example, only the blocks that are adjacent to the current one is tested. The method is faster but the search area is restricted A motion vector is computed for ‘EVERY’ blocks in the near neighborhood of the current frame. Search Window Previous frame Current frame For example, only the blocks that are adjacent to the current one is tested. The method is faster but the search area is restricted A motion vector is computed for ‘EVERY’ blocks in the near neighborhood of the current frame. Search Window Previous frame Current frame Assumption: changes between frames are small and are restricted within the search window. However, the search time is still long Given a block in the current frame, search the best match in the previous frame along the vertical direction Search Window Previous frame Current frame Given a block in the current frame, search the best match in the previous frame along the vertical direction Search Window Previous frame Current frame Best Match Search the best match in the previous frame along the horizontal direction Search Window Previous frame Current frame Solution Search the best match in the previous frame along the horizontal direction Search Window Previous frame Current frame Solution Non-optimal solution with the assumption of smooth intensity distribution City University of Hong Kong 3 3 3 3 3 Applications - multimedia and video transmission Based on JPEG and H.261 standards Flexible picture size and frame rate specified by users Video source - Non-interlaced video signals. Minimum requirements on decoders – resolution of 720X576 – 30 frames/s – 1.86Mb/s City University of Hong Kong Layer structure in MPEG bitstream 3 3 3 3 3 3 Sequence Group Of Pictures (GOP) Picture Slice Macroblock Block City University of Hong Kong Video Sequence Group Of Pictures A Picture A Slice Macroblock Block City University of Hong Kong 3 3 3 3 3 3 Partitioning of images into Macroblocks (MB) Intraframe coding on one out of every K images Motion estimation on MBs Generate (K-1) predicted frames Encode residual error images Conditional Replenishment of Macroblocks City University of Hong Kong 3 3 3 An image is partitioned into Macroblocks of size 16X16 1 MB = 4 luminance (Y) and 2 chrominance blocks (U,V) The sampling ratio between Y, U and V is 4:1:1 I P P P Y1 Y2 Y3 Y4 U V 1 2 3 4 Y:U:V = 4:1:1 Figure 3 Chrominance format: 4:4:4, 4:2:2, 4:1:1, 4:2:0 4:4:4 Assuming 8bits for Y, U and V components 4*8 (Y) + 4*8 (U) + 4*8 (V) = 96 bits Bits per pixel = 96/4 = 24 bpp Chrominance format: 4:4:4, 4:2:2, 4:1:1, 4:2:0 4:2:2 Assuming 8bits for Y, U and V components 4*8 (Y) + 2*8 (U) + 2*8 (V) = 64 bits Bits per pixel = 64/4 = 16 bpp Chrominance format: 4:4:4, 4:2:2, 4:1:1, 4:2:0 4:1:1 Assuming 8bits for Y, U and V components 4*8 (Y) + 1*8 (U) + 1*8 (V) = 48 bits Bits per pixel = 48/4 = 12 bpp Chrominance format: 4:4:4, 4:2:2, 4:1:1, 4:2:0 4:2:0 Assuming 8bits for Y, U and V components 4*8 (Y) + 1*8 (U) + 1*8 (V) = 48 bits Bits per pixel = 48/4 = 12 bpp City University of Hong Kong 3 3 3 3 DCT Weighted (I-frame)/Uniform (P-frame) Quantization DPCM on DC terms Zigzag scan + runlength + VLC Macroblock Block to be encoded sz: Step Size DCT Q AC ZigZag Scanning DC DPCM JPEG encoded DC 8 8 8 8 sz Figure 4 Runlength Encoding VLC JPEG encoded AC 3 3 3 3 Previous I or P frame is stored in both encoder and decoder Motion Compensation is performed on a macroblock basis One motion vector (mv) is generated for each macroblock The mvs are coded and transmitted to the receiver Motion prediction error of pixels in each macroblock is calculated 3 Error blocks (size 8X8) are encoded in the same manner as those in the I-Picture 3 A video buffer plus step size adjustment maintain a constant target bit-rate 3 e(n) x(n) + Coder • Current signal x(n) is predicted from previous sample x(n-1). Predicted value is x (n) ec(n) Decoder p epq(n) xp(n) + Predictor + Predicted error e(n)=x(n)-x (n) is • p xr(n) compressed (encode) and transmitted City University of Hong Kong City Block to be encoded 8 8 + DCT CONTROL sz Q RLC VLC VB ENCODED RESIDUAL ERROR Q-1 DCT-1 + MC + FRAME STORE + MOTION VECTOR Figure 5 City University of Hong Kong City 3 3 3 3 3 I-Pictures are encoded independently I-Pictures can therefore be used as access point for random access, fast-forward (FF) or fast-reverse (FR) P-Pictures cannot be decoded alone, hence cannot be used as an access point B-Pictures are constructed with the nearest I or P Pictures Backward prediction requires the presence of the start and end frames, both can be used as access points I B B P B B P Figure 6a Order of Coding 1 3 4 2 6 7 5 I Pictures only I and P Pictures I, P and B Pictures Compression Low Medium High Random Access Highest Low Medium Coding Delay Low Medium High I I I I I I I I P P I P P I Figure 6b I P B I P B I I Pictures only I and P Pictures I, P and B Pictures Compression Low Medium High Random Access Highest Low Medium Coding Delay Low Medium High City University of Hong Kong City 3 3 3 3 3 Only macroblocks that have been changed in the decoder are updated Three types of MB are classified in MPEG standard Skipped MB - Zero motion vector, the MB is neither encoded nor transmitted Inter MB - Motion Prediction is valid, the MB type and address, motion vector and the coded DCT coefficients are transmitted Intra MB - Encoded DCT coefficients of the MB are transmitted. No Motion Compensation is used Q C No Q Non I No C MC Q I MB C No MC No C No Q No Q Q Pred-mcq Pred-mc Pred-m Intra-q Intra-d Pred-cq Pred-c Skipped Q C No Q Non I No C MC Q I MB C No MC No C No Q No Q Q Pred-mcq Pred-mc Pred-m Intra-q Intra-d Pred-cq Pred-c Skipped Non-zero motion vector, Error coded with defined quantization Q C No Q Non I No C MC Q I MB C No MC No C No Q No Q Q Pred-mcq Pred-mc Pred-m Intra-q Intra-d Pred-cq Pred-c Skipped Non-zero motion vector, Error coded with default quantization Q C No Q Non I No C MC Q I MB C No MC No C No Q No Q Q Pred-mcq Pred-mc Pred-m Intra-q Intra-d Pred-cq Pred-c Skipped Non-zero motion vector, Error not coded Q C No Q Non I No C MC Q I MB C No MC No C No Q No Q Q Pred-mcq Pred-mc Pred-m Intra-q Intra-d Pred-cq Pred-c Skipped Macroblock intracoded with defined quantization Q C No Q Non I No C MC Q I MB C No MC No C No Q No Q Q Pred-mcq Pred-mc Pred-m Intra-q Intra-d Pred-cq Pred-c Skipped Macroblock intracoded with default quantization Q C No Q Non I No C MC Q I MB C No MC No C No Q No Q Q Pred-mcq Pred-mc Pred-m Intra-q Intra-d Pred-cq Pred-c Skipped MV = 0 (not predicted) Error coded with defined quantization Q C No Q Non I No C MC Q I MB C No MC No C No Q No Q Q Pred-mcq Pred-mc Pred-m Intra-q Intra-d Pred-cq Pred-c Skipped MV = 0 (not predicted) Error coded with default quantization Q C No Q Non I No C MC Q I MB C No MC No C No Q No Q Q Pred-mcq Pred-mc Pred-m Intra-q Intra-d Pred-cq Pred-c Skipped Macroblock copied from predictor picture Q C No Q Non I No C MB Q I No Q Pred-f/b/i cq Pred-f/b/i c Skipped Pred-f/b/i Intra-q Intra-d Reverse process of the encoder Figure 7 ENCODED DCT DATA VB RLD VLD Q-1 Decoded Block MC+FRAME STORE MOTION VECTOR DCT-1 + + 8 8 Note: MC for P-Frames only City University of Hong Kong City 3 3 3 3 3 3 A superset of MPEG-1 and backward compatible to the latter Support interlaced video signals Scalable video-coding property, can be decoded by receivers with different capabilities Permits partial implementation defined by Profiles and Levels A Profile defines a new set of algorithms added as a superset to the algorithms in the profile that follows A Level specifies the range of parameters supported by the implementation City University of Hong Kong LEVEL High High 1440 Main Low Pel/Line 1920 1440 720 352 Pels/Frame Frame Rate (f/s) Bit-rate (Mb/s) 1152 60 80 1152 60 60 576 30 15 288 30 4 PROFILE MAIN Figure 8 Non-scalable Coding Interlaced Video B-Picture 4:2:0 YUV format MAIN + 2 layers SNR Scalable Coding 2 layers Spatial Scalable Coding 4:0:0 YUV format 3 layers SNR and Spatial Scalable Coding 4:2:2 YUV format B-Picture Prediction SNR SCALABLE SPATIAL SCALABLE SNR SCALABLE + HIGH SPATIAL SCALABLE + SIMPLE MAIN - City University of Hong Kong City 3 Main Profile: MPEG-2 Non-scalable Coding Mode 3 A straightforward extension of MPEG-1 to accomodate interlaced video signals 3 Field/Frame Macroblocks 3 Two types of prediction Á Frame Prediction: Prediction based on one or more previously decoded frames Á Field Prediction : Prediction of individual field based on one or more previously decoded field City University of Hong Kong City o e o e o e o e A stationary scene image The four sub-blocks of a Frame Macroblock Figure 9a City University of Hong Kong City o e o e o e o e A moving scene image The four sub-blocks of a Frame Macroblock Figure 9b City University of Hong Kong City 3 The object shape is changed with motion because of the interlacing mechanism 3 Same object may appear different in successive frames because of the above reason - prediction is not accurate 3 Simple image patterns may become complicated 3 More AC coefficients are required to describe each component in the frame Macroblock City University of Hong Kong City o e o e o e o e A moving scene image The four sub-blocks of a Frame Macroblock Figure 9c City University of Hong Kong City • Compute Field-based Variance • Compute Frame-based Variance • If Field-based Variance < Frame-based Variance MB coded with Field-based DCT City University of Hong Kong City var1=0; for(m=0;m<COL;m++) for(n=0;n<ROW-2;n++) { D1=x(m,n)-x(m,n+1); D2=x(m,n+1)-x(m,n+2) var1+=(D1*D1)+(D2*D2); } n 0 1 2 3 o e o e ROW-1 City University of Hong Kong City var1=0; for(m=0;m<COL;m++) for(n=0;n<ROW-2;n++) { D1=x(m,n)-x(m,n+2); D2=x(m,n+1)-x(m,n+3) var1+=(D1*D1)+(D2*D2); } n 0 1 2 3 o e o e ROW-1 3 A top field is predicted from either previously coded top or bottom field with Motion Compensation (MC) 3 Bottom fields are predicted from previously coded top field with MC 3 Combine frame and field prediction is used in MPEG-2 City University of Hong Kong City 3 Provide interoperability between different services and systems 3 Base layer - encodes downscaled video at reduced bitstream 3 Enhance layer - encodes the difference between original signal and the upscaled base-layer video Video in ENHANCEMENT ENCODER Enhancement layer bitstream DOWNSCALING UPSCALING BASE LAYER ENCODER Basic layer bitstream Figure 10 Video in ENHANCEMENT ENCODER Enhancement layer bitstream DOWNSCALING UPSCALING BASE LAYER ENCODER Basic layer bitstream City University of Hong Kong City 3 A 2-layer DCT, VLC and MC encoder 3 Both layers encoded video signal at the same resolution 3 Base layer - DCT coefficients are coarsely quantized and is protected from transmission error 3 Enhancement layer: DCT coefficients are finely quantized and their difference with the base layer is transmitted Quantized level L x Quantizer De-quantizer x =s * L Q Step size s Quantized level L x Quantizer De-quantizer x =s * L Quantization error Q Step size s Quantized level L x Quantizer De-quantizer x =S * L Quantization error E Q Coarse step size S L Quantizer De-quantizer E =s * L E can be used to R Fine step size s Q Q Image Block + - DCT Q Q-1 VLC VB Base Layer Bit Stream + - Q VLC VB Enhancement Layer Bit Stream + + DCT-1 + MC FS + Q-1 Base Layer Bit Stream VB VB VLD + VLD Enhancement Layer Bit Stream + DCT-1 + MC FS + Image Blocks Figure 11 City University of Hong Kong 3 Temporal prediction from previous frame 3 Estimation of mv of lost MB from neighbouring Mbs 3 Add mvs to MBs of I-Frame for error concealment 3 2 Layer Coding using Data Partitioning, Spatial and Frequency scalability City University of Hong Kong Applications of MPEG Encoding Standard Cable and Interactive TV Distribution Satellite and Digital Terrestrial TV Broadcasting Remote Surveillance Video Conferencing/Telephony Standalone or Computer Based Multimedia Systems HDTV DVD VCD Digital Camera ...
View Full Document

This note was uploaded on 01/11/2011 for the course EE 4115 taught by Professor Waimingtsang during the Spring '10 term at City University of Hong Kong.

Page1 / 77

MPEG1 - City University of Hong Kong City Moving Picture Expert Group Established in 1988 by the Joint ISO/IEC Technical Committee on IT Mission To

This preview shows document pages 1 - 6. Sign up to view the full document.

View Full Document Right Arrow Icon
Ask a homework question - tutors are online