Recherche avancée

Sur d’autres sites (7108)

• Inside WebM Technology : VP8 Intra and Inter Prediction

  20 juillet 2010, par noreply@blogger.com (Lou Quillio)
  Continuing our series on WebM technology, I will discuss the use of prediction methods in the VP8 video codec, with special attention to the TM_PRED and SPLITMV modes, which are unique to VP8.

  
  

  First, some background. To encode a video frame, block-based codecs such as VP8 first divide the frame into smaller segments called macroblocks. Within each macroblock, the encoder can predict redundant motion and color information based on previously processed blocks. The redundant data can be subtracted from the block, resulting in more efficient compression.

  
  Image by Fido Factor, licensed under Creative Commons Attribution License.
  Based on a work at www.flickr.com

  
  

  A VP8 encoder uses two classes of prediction :

  • Intra prediction uses data within a single video frame
  • Inter prediction uses data from previously encoded frames

  The residual signal data is then encoded using other techniques, such as transform coding.

  
  

  VP8 Intra Prediction Modes

  VP8 intra prediction modes are used with three types of macroblocks :

  • 4x4 luma
  • 16x16 luma
  • 8x8 chroma

  Four common intra prediction modes are shared by these macroblocks :
  

  • H_PRED (horizontal prediction). Fills each column of the block with a copy of the left column, L.
  • V_PRED (vertical prediction). Fills each row of the block with a copy of the above row, A.
  • DC_PRED (DC prediction). Fills the block with a single value using the average of the pixels in the row above A and the column to the left of L.
  • TM_PRED (TrueMotion prediction). A mode that gets its name from a compression technique developed by On2 Technologies. In addition to the row A and column L, TM_PRED uses the pixel P above and to the left of the block. Horizontal differences between pixels in A (starting from P) are propagated using the pixels from L to start each row.

  For 4x4 luma blocks, there are six additional intra modes similar to V_PRED and H_PRED, but correspond to predicting pixels in different directions. These modes are outside the scope of this post, but if you want to learn more see the VP8 Bitstream Guide.

  
  

  As mentioned above, the TM_PRED mode is unique to VP8. The following figure uses an example 4x4 block of pixels to illustrate how the TM_PRED mode works :

  

  Where C, As and Ls represent reconstructed pixel values from previously coded blocks, and X₀₀ through X₃₃ represent predicted values for the current block. TM_PRED uses the following equation to calculate X_ij :

  
  

  X_ij = L_i + A_j - C (i, j=0, 1, 2, 3)

  
  

  Although the above example uses a 4x4 block, the TM_PRED mode for 8x8 and 16x16 blocks works in the same fashion.

  TM_PRED is one of the more frequently used intra prediction modes in VP8, and for common video sequences it is typically used by 20% to 45% of all blocks that are intra coded. Overall, together with other intra prediction modes, TM_PRED helps VP8 to achieve very good compression efficiency, especially for key frames, which can only use intra modes (key frames by their very nature cannot refer to previously encoded frames).

  
  

  VP8 Inter Prediction Modes

  
  

  In VP8, inter prediction modes are used only on inter frames (non-key frames). For any VP8 inter frame, there are typically three previously coded reference frames that can be used for prediction. A typical inter prediction block is constructed using a motion vector to copy a block from one of the three frames. The motion vector points to the location of a pixel block to be copied. In most video compression schemes, a good portion of the bits are spent on encoding motion vectors ; the portion can be especially large for video encoded at lower datarates.

  
  

  Like previous VPx codecs, VP8 encodes motion vectors very efficiently by reusing vectors from neighboring macroblocks (a macroblock includes one 16x16 luma block and two 8x8 chroma blocks). VP8 uses a similar strategy in the overall design of inter prediction modes. For example, the prediction modes "NEAREST" and "NEAR" make use of last and second-to-last, non-zero motion vectors from neighboring macroblocks. These inter prediction modes can be used in combination with any of the three different reference frames.

  
  

  In addition, VP8 has a very sophisticated, flexible inter prediction mode called SPLITMV. This mode was designed to enable flexible partitioning of a macroblock into sub-blocks to achieve better inter prediction. SPLITMV is very useful when objects within a macroblock have different motion characteristics. Within a macroblock coded using SPLITMV mode, each sub-block can have its own motion vector. Similar to the strategy of reusing motion vectors at the macroblock level, a sub-block can also use motion vectors from neighboring sub-blocks above or left to the current block. This strategy is very flexible and can effectively encode any shape of sub-macroblock partitioning, and does so efficiently. Here is an example of a macroblock with 16x16 luma pixels that is partitioned to 16 4x4 blocks :

  
  

  

  
  

  where New represents a 4x4 bock coded with a new motion vector, and Left and Above represent a 4x4 block coded using the motion vector from the left and above, respectively. This example effectively partitions the 16x16 macroblock into 3 different segments with 3 different motion vectors (represented below by 1, 2 and 3) :

  
  

  

  
  

  Through effective use of intra and inter prediction modes, WebM encoder implementations can achieve great compression quality on a wide range of source material. If you want to delve further into VP8 prediction modes, read the VP8 Bitstream Guide or examine the reconintra.c and rdopt.c files in the VP8 source tree.

  
  

  Yaowu Xu, Ph.D. is a codec engineer at Google.

  
  

  

• how to prevent the hls video player dont refresh when m3u8 changes

  6 juin 2024, par Leo

  here i am using ffmpeg to use camera and audio to make a hls stream on server as the stream continious old m3u8 components deletes and new ones gets added to main.m3u8. but as we insert the url of hls stream file main.m3u8. the player refreshes as soon as the file gets rewritten because of new and old ones. so

  


  i have tried to change the players like hls.js or videojs etc none were to solve this. how to solve this and make sure the stream runs smoothly.

  


  Server.js

  


  const startStreaming = (viddev,auddev) => {
    if (ffmpegProcess) {
        ffmpegProcess.kill();
    }
    const segmentDuration = 10;
    const outputFilename = './video/output.m3u8';
    const ffmpegCommand = `ffmpeg -f dshow -i video="${viddev}" -f dshow -i audio="${auddev}" -codec:v libx264 -preset ultrafast -tune zerolatency -codec:a aac -b:a 128k -hls_time ${segmentDuration} -hls_list_size 3 -hls_flags delete_segments -start_number 0 -hls_segment_type mpegts ${outputFilename}`;

    ffmpegProcess = exec(ffmpegCommand);

    ffmpegProcess.stderr.on('data', (data) => {
        console.error(`ffmpeg stderr: ${data}`);
    });

    ffmpegProcess.on('close', (code) => {
        console.log(`ffmpeg process exited with code ${code}`);
    });
};

app.use('/video', express.static(path.join(__dirname, 'video')));

// Endpoint to list audio and video devices
app.get('/devices', async (req, res) => {
    const data=await parseDevices()
    // console.log(data.cameras[0].name)
    startStreaming(data.cameras[0].name,data.microphones[0].name);
    res.json(data)
});


  


  index.html

  


  &#xA;&#xA;&#xA;    &#xA;    &#xA;    &#xA;    &#xA;    &#xA;&#xA;&#xA;&#xA;    &#xA;        <source src="https://localhost:3000/video/output.m3u8" type="application/x-mpegURL">&#xA;    &#xA;&#xA;    <code class="echappe-js">&lt;script src=&quot;https://vjs.zencdn.net/8.10.0/video.min.js&quot;&gt;&lt;/script&gt;&#xA;    &lt;script&gt;&amp;#xA;        // Initialize Video.js player&amp;#xA;        var player = videojs(&amp;#x27;video&amp;#x27;, {&amp;#xA;            autoplay: &amp;#x27;play&amp;#x27;,&amp;#xA;            liveui: true  // Enable the live UI for live streams&amp;#xA;        });&amp;#xA;&amp;#xA;        // Seek to live when the player is ready&amp;#xA;        player.ready(function() {&amp;#xA;            player.liveTracker.on(&amp;#x27;liveedgechange&amp;#x27;, function() {&amp;#xA;                player.currentTime(player.liveTracker.liveCurrentTime());&amp;#xA;            });&amp;#xA;        });&amp;#xA;&amp;#xA;        // Handle any errors encountered by Video.js&amp;#xA;        player.on(&amp;#x27;error&amp;#x27;, function() {&amp;#xA;            console.error(&amp;#x27;Video.js encountered an error:&amp;#x27;, player.error());&amp;#xA;        });&amp;#xA;    &lt;/script&gt;&#xA;&#xA;&#xA;&#xA;&#xA;

  


• ffmpeg export multichannel audio Stereo won't play [migrated]

  9 avril 2014, par speedyrazor

  I am trying to export all audio channels from a multichannel quicktime file with ffmpeg which has the following audio configuration, but am unsure if the command below is correct. All the files look and play correct in Quicktime player except the L+R_Total.wav which refuses to play in Quicktime player or Final Cut but plays fine in VLC, so i'm sure my ffmpeg command is not correct for the stereo on Track 7. The Stereo file is about 1.8GB in size. Here is my track layout in the Quicktime file :

  Track 1 - mono
  
  Track 2 - mono
  
  Track 3 - mono
  
  Track 4 - mono
  
  Track 5 - mono
  
  Track 6 - mono
  
  Track 7 - stereo

  I am using :

  /Users/me/Desktop/python/ffmpeg/ffmpeg -i /Users/me/Desktop/test.mov -acodec pcm_s24le -map 0:1 -y Left.wav -acodec pcm_s24le -map 0:2 Right.wav -acodec pcm_s24le -map 0:3 Center.wav -acodec pcm_s24le -map 0:4 LFE.wav -acodec pcm_s24le -map 0:5 Left_Surround.wav -acodec pcm_s24le -map 0:6 Right_Surround.wav -acodec pcm_s24le -map 0:7 Left+Right_Total.wav

  Here is the printout from the terminal :

      MacBook-Pro:~ me$ /Users/me/Desktop/python/ffmpeg/ffmpeg -i /Users/me/Desktop/test.mov -acodec pcm_s24le -map 0:1 -y Left.wav -acodec pcm_s24le -map 0:2 Right.wav -acodec pcm_s24le -map 0:3 Center.wav -acodec pcm_s24le -map 0:4 LFE.wav -acodec pcm_s24le -map 0:5 Left_Surround.wav -acodec pcm_s24le -map 0:6 Right_Surround.wav -acodec pcm_s24le -map 0:7 Left+Right_Total.wav
  
  ffmpeg version 2.2 Copyright (c) 2000-2014 the FFmpeg developers
  
    built on Mar 25 2014 15:00:55 with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
  
    configuration: --prefix=/Volumes/Ramdisk/sw --enable-gpl --enable-pthreads --enable-version3 --enable-libspeex --enable-libvpx --disable-decoder=libvpx --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-avfilter --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-filters --enable-libgsm --arch=x86_64 --enable-runtime-cpudetect
  
    libavutil      52. 66.100 / 52. 66.100
  
    libavcodec     55. 52.102 / 55. 52.102
  
    libavformat    55. 33.100 / 55. 33.100
  
    libavdevice    55. 10.100 / 55. 10.100
  
    libavfilter     4.  2.100 /  4.  2.100
  
    libswscale      2.  5.102 /  2.  5.102
  
    libswresample   0. 18.100 /  0. 18.100
  
    libpostproc    52.  3.100 / 52.  3.100
  
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/Users/me/Desktop/test.mov':
  
    Metadata:
  
      major_brand     : qt  
  
      minor_version   : 537199360
  
      compatible_brands: qt  
  
      creation_time   : 2014-03-06 11:25:34
  
    Duration: 00:01:03.48, start: 0.000000, bitrate: 153967 kb/s
  
      Stream #0:0(eng): Video: prores (apch / 0x68637061), yuv422p10le, 1920x1080, 144704 kb/s, SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 23976 tbn, 23976 tbc (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
        timecode        : 00:00:00:00
  
      Stream #0:1(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 1 channels (FL), s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
      Stream #0:2(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 1 channels (FR), s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
      Stream #0:3(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, mono, s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
      Stream #0:4(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 1 channels (LFE), s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
      Stream #0:5(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 1 channels (BL), s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
      Stream #0:6(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, 1 channels (BR), s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
      Stream #0:7(eng): Audio: pcm_s24le (in24 / 0x34326E69), 48000 Hz, downmix, s32, 2304 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
      Stream #0:8(eng): Data: none (tmcd / 0x64636D74) (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:26:04
  
        handler_name    : Apple Alias Data Handler
  
        timecode        : 00:00:00:00
  
  Output #0, wav, to 'Left.wav':
  
    Metadata:
  
      major_brand     : qt  
  
      minor_version   : 537199360
  
      compatible_brands: qt  
  
      ISFT            : Lavf55.33.100
  
      Stream #0:0(eng): Audio: pcm_s24le ([1][0][0][0] / 0x0001), 48000 Hz, 1 channels (FL), s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
  Output #1, wav, to 'Right.wav':
  
    Metadata:
  
      major_brand     : qt  
  
      minor_version   : 537199360
  
      compatible_brands: qt  
  
      ISFT            : Lavf55.33.100
  
      Stream #1:0(eng): Audio: pcm_s24le ([1][0][0][0] / 0x0001), 48000 Hz, 1 channels (FR), s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
  Output #2, wav, to 'Center.wav':
  
    Metadata:
  
      major_brand     : qt  
  
      minor_version   : 537199360
  
      compatible_brands: qt  
  
      ISFT            : Lavf55.33.100
  
      Stream #2:0(eng): Audio: pcm_s24le ([1][0][0][0] / 0x0001), 48000 Hz, mono, s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
  Output #3, wav, to 'LFE.wav':
  
    Metadata:
  
      major_brand     : qt  
  
      minor_version   : 537199360
  
      compatible_brands: qt  
  
      ISFT            : Lavf55.33.100
  
      Stream #3:0(eng): Audio: pcm_s24le ([1][0][0][0] / 0x0001), 48000 Hz, 1 channels (LFE), s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
  Output #4, wav, to 'Left_Surround.wav':
  
    Metadata:
  
      major_brand     : qt  
  
      minor_version   : 537199360
  
      compatible_brands: qt  
  
      ISFT            : Lavf55.33.100
  
      Stream #4:0(eng): Audio: pcm_s24le ([1][0][0][0] / 0x0001), 48000 Hz, 1 channels (BL), s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
  Output #5, wav, to 'Right_Surround.wav':
  
    Metadata:
  
      major_brand     : qt  
  
      minor_version   : 537199360
  
      compatible_brands: qt  
  
      ISFT            : Lavf55.33.100
  
      Stream #5:0(eng): Audio: pcm_s24le ([1][0][0][0] / 0x0001), 48000 Hz, 1 channels (BR), s32, 1152 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
  Output #6, wav, to 'Left+Right_Total.wav':
  
    Metadata:
  
      major_brand     : qt  
  
      minor_version   : 537199360
  
      compatible_brands: qt  
  
      ISFT            : Lavf55.33.100
  
      Stream #6:0(eng): Audio: pcm_s24le ([1][0][0][0] / 0x0001), 48000 Hz, downmix, s32, 2304 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-03-06 11:25:34
  
        handler_name    : Apple Alias Data Handler
  
  Stream mapping:
  
    Stream #0:1 -> #0:0 (pcm_s24le -> pcm_s24le)
  
    Stream #0:2 -> #1:0 (pcm_s24le -> pcm_s24le)
  
    Stream #0:3 -> #2:0 (pcm_s24le -> pcm_s24le)
  
    Stream #0:4 -> #3:0 (pcm_s24le -> pcm_s24le)
  
    Stream #0:5 -> #4:0 (pcm_s24le -> pcm_s24le)
  
    Stream #0:6 -> #5:0 (pcm_s24le -> pcm_s24le)
  
    Stream #0:7 -> #6:0 (pcm_s24le -> pcm_s24le)
  
  Press [q] to stop, [?] for help
  
  size=    8927kB time=00:01:03.48 bitrate=1152.0kbits/s    
  
  video:0kB audio:71415kB subtitle:0 data:0 global headers:0kB muxing overhead -87.499863%