Sur d’autres sites (10470)

• How to enable LHLS in FFMPEG 4.1 ?

  27 décembre 2020, par mehdi.r

  I am trying to create a low latency CMAF video stream using FFMPEG.
To do so, I would like to enable the lhls option in FFMPEG in order to have the #EXT-X-PREFETCH tag written in the HLS manifest.

  



  From the FFMPEG doc :

  



  https://www.ffmpeg.org/ffmpeg-all.html

  



  


  Enable Low-latency HLS(LHLS). Adds #EXT-X-PREFETCH tag with current >segment’s URI. Apple doesn’t have an official spec for LHLS. Meanwhile >hls.js player folks are trying to standardize a open LHLS spec. The >draft spec is available in https://github.com/video-dev/hlsjs->rfcs/blob/lhls-spec/proposals/0001-lhls.md This option will also try >to comply with the above open spec, till Apple’s spec officially >supports it. Applicable only when streaming and hls_playlist options >are enabled. This is an experimental feature.

  


  



  I am using the following command with FFMPEG 4.1 :

  



  ffmpeg -re -i ~/Documents/videos/BigBuckBunny.mp4 \
    -map 0 -map 0 -map 0 -c:a aac -c:v libx264 -tune zerolatency \
    -b:v:0 2000k -s:v:0 1280x720 -profile:v:0 high \
    -b:v:1 1500k -s:v:1 640x340  -profile:v:1 main \
    -b:v:2 500k -s:v:2 320x170  -profile:v:2 baseline \
    -bf 1 \
    -keyint_min 24 -g 24 -sc_threshold 0 -b_strategy 0 -ar:a:1 22050 -use_timeline 1 -use_template 1 \
    -window_size 5 -adaptation_sets "id=0,streams=v id=1,streams=a" \
    -hls_playlist 1 -seg_duration 1 -streaming 1  -strict experimental -lhls 1 -remove_at_exit 1 \
    -f dash manifest.mpd



  



  The kind of HLS manifest I obtained for a specific resolution :

  



  #EXTM3U
#EXT-X-VERSION:6
#EXT-X-TARGETDURATION:1
#EXT-X-MEDIA-SEQUENCE:8
#EXT-X-MAP:URI="init-stream0.mp4"
#EXTINF:0.998458,
#EXT-X-PROGRAM-DATE-TIME:2019-06-21T18:13:56.966+0900
chunk-stream0-00008.mp4
#EXTINF:0.998458,
#EXT-X-PROGRAM-DATE-TIME:2019-06-21T18:13:57.964+0900
chunk-stream0-00009.mp4
#EXTINF:0.998458,
#EXT-X-PROGRAM-DATE-TIME:2019-06-21T18:13:58.963+0900
chunk-stream0-00010.mp4
#EXTINF:0.998458,
#EXT-X-PROGRAM-DATE-TIME:2019-06-21T18:13:59.961+0900
chunk-stream0-00011.mp4
#EXTINF:1.021678,
#EXT-X-PROGRAM-DATE-TIME:2019-06-21T18:14:00.960+0900
chunk-stream0-00012.mp4
...



  



  As you can see the #EXT-X-PREFETCH tag is missing.

  



  Any help would be highly appreciated.

  



  Edit

  



  I also compiled FFmpeg from its master branch by doing the following :

  



  nasm

  



  sudo apt-get install nasm mingw-w64


  



  Codecs

  



  sudo apt-get install libx265-dev libnuma-dev libx264-dev libvpx-dev libfdk-aac-dev libmp3lame-dev libopus-dev


  



  FFmpeg

  



  mkdir lhls
cd lhls 
git init 
git clone https://github.com/FFmpeg/FFmpeg.git
cd FFmpeg 
git checkout master


  



  AOM (inside FFmpeg dir)

  



  git -C aom pull 2> /dev/null || git clone --depth 1 https://aomedia.googlesource.com/aom && \
mkdir -p aom_build && \
cd aom_build && \
PATH="$HOME/bin:$PATH" cmake -G "Unix Makefiles" -DCMAKE_INSTALL_PREFIX="$HOME/ffmpeg_build" -DENABLE_SHARED=off -DENABLE_NASM=on ../aom && \
PATH="$HOME/bin:$PATH" make && \
make install
cd..


  



  Compiling

  



  PATH="$HOME/bin:$PATH" PKG_CONFIG_PATH="$HOME/ffmpeg_build/lib/pkgconfig" ./configure \
  --prefix="$HOME/ffmpeg_build" \
  --pkg-config-flags="--static" \
  --extra-cflags="-I$HOME/ffmpeg_build/include" \
  --extra-ldflags="-L$HOME/ffmpeg_build/lib" \
  --extra-libs="-lpthread -lm" \
  --bindir="$HOME/bin" \
  --enable-gpl \
  --enable-libaom \
  --enable-libass \
  --enable-libfdk-aac \
  --enable-libfreetype \
  --enable-libmp3lame \
  --enable-libopus \
  --enable-libvorbis \
  --enable-libvpx \
  --enable-libx264 \
  --enable-libx265 \
  --enable-nonfree && \
PATH="$HOME/bin:$PATH" make 


  



  Unfortunately, the #EXT-X-PREFETCH is still missing in the HLS Manifest.

  



  I also tried nightly builds from https://ffmpeg.zeranoe.com/builds/ , same result.

  



  Any help would be highly appreciated.

  



  EDIT 2 :resolved

  



  Thanks to @aergistal and @Gyan , the #EXT-X-PREFETCH tag is now present in my HLS manifest.

  



  Here the FFMPEG command I am using :

  



  ./ffmpeg -re -i ~/videos/BigBuckBunny.mp4 -loglevel debug \
  -map 0 -map 0 -map 0 -c:a aac -c:v libx264 -tune zerolatency \
  -b:v:0 2000k -s:v:0 1280x720 -profile:v:0 high -b:v:1 1500k -s:v:1 640x340  -profile:v:1 main -b:v:2 500k -s:v:2 320x170  -profile:v:2 baseline -bf 1 \
 -keyint_min 24 -g 24 -sc_threshold 0 -b_strategy 0 -ar:a:1 22050 -use_timeline 1 -use_template 1 -window_size 5 \
 -adaptation_sets "id=0,streams=v id=1,streams=a" -hls_playlist 1 -seg_duration 3 -streaming 1 \
 -strict experimental -lhls 1 -remove_at_exit 0 -master_m3u8_publish_rate 3 \
 -f dash -method PUT -http_persistent 1  https://example.com/manifest.mpd


  



  Apparently the mime types are not passed to the server & FFmpeg seems to ignore the -headers option.

  


• Performance issue in streaming desktop out of a Raspberry with FFMPEG

  30 avril 2021, par skynet

  I'm quite a newbie in FFMPEG and I apologize in advance for any inaccuracy I may write.

  


  My goal is to stream over UDP a Full HD (1920x1080) desktop connected to a Raspberry PI 4 - 4 GB RAM.

  


  I made many attempts, and currently this the setup with better performance I found (I used the knowledge in https://www.willusher.io/general/2020/11/15/hw-accel-encoding-rpi4).

  


  

  1. I installed the 64-bit Raspbian OS on the PI from this link : https://www.raspberrypi.org/forums/viewtopic.php?t=275370

  


  


  This because the HW H264 encoder (h264_v4l2m2m) is faster than the 32-bit encoder h264_omx

  


  

  2. I downloaded and installed FFMPEG 4.4, using this configuration
./configure —prefix="$HOME/ffmpeg_build" —pkg-config-flags="pkg-config —static" —extra-cflags="-I$HOME/ffmpeg_build/include" —extra-ldflags="-L$HOME/ffmpeg_build/lib" —extra-libs="-lpthread -lm" —bindir="$HOME/bin" —enable-gpl —enable-gnutls —disable-libaom —enable-libass —enable-libfdk-aac —enable-libfreetype —enable-libmp3lame —enable-libopus —enable-libvorbis —enable-libvpx —enable-libx264 —disable-libx265 —enable-nonfree —arch=aarch64 —disable-libxml2 —enable-libwebp—enable-libdrm

  


  


  I used FFMPEG 4.4 because of compatibility issues with h264_v4l2m2m of the shipped FFMPEG version 4.1.4

  


  


  3. I use this command

     


     ffmpeg -f x11grab -probesize 42M -s 1920x1080 -i :0.0 -c:a copy -c:v h264_v4l2m2m -num_output_buffers 32 -num_capture_buffers 16 -b:v 8M -minrate 8M -maxrate 8M -pix_fmt rgb24 -f mpegts udp ://239.255.90.60:5004 ?pkt_size=1316

     


  


  


  The -pix_fmt rgb24 flag is needed because x11grab in 64 bit OS use BGR0 chroma subsampling... which means red and blue colours are inverted !

  


  This is the output :

  


  ffmpeg version 4.4 Copyright (c) 2000-2021 the FFmpeg developers  
  built with gcc 8 (Debian 8.3.0-6)  
  configuration: --prefix=/home/pi/ffmpeg_build --pkg-config-flags='pkg-config --static' --extra-cflags=-I/home/pi/ffmpeg_build/include --extra-ldflags=-L/home/pi/ffmpeg_build/lib --extra-libs='-lpthread -lm' --enable-gpl --enable-gnutls --disable-libaom --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --disable-libx265 --enable-nonfree --arch=aarch64 --disable-libxml2 --enable-libwebp --enable-libdrm  
  libavutil      56. 70.100 / 56. 70.100  
  libavcodec     58.134.100 / 58.134.100  
  libavformat    58. 76.100 / 58. 76.100  
  libavdevice    58. 13.100 / 58. 13.100  
  libavfilter     7.110.100 /  7.110.100  
  libswscale      5.  9.100 /  5.  9.100  
  libswresample   3.  9.100 /  3.  9.100  
  libpostproc    55.  9.100 / 55.  9.100  
Input #0, x11grab, from ':0.0':  
  Duration: N/A, start: 1619778505.990981, bitrate: 1988667 kb/s  
  Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1920x1080, 1988667 kb/s, 29.97 fps, 56   tbr, 1000k tbn, 1000k tbc  
Stream mapping:  
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (h264_v4l2m2m))  
Press [q] to stop, [?] for help  
[h264_v4l2m2m @ 0x5598d14070] Using device /dev/video11  
[h264_v4l2m2m @ 0x5598d14070] driver 'bcm2835-codec' on card 'bcm2835-codec-encode' in mplane mode  
[h264_v4l2m2m @ 0x5598d14070] requesting formats: output=RGB3 capture=H264  
[h264_v4l2m2m @ 0x5598d14070] Failed to set gop size: Invalid argument  
Output #0, mpegts, to 'udp://239.255.90.60:5004?pkt_size=1316':  
  Metadata:  
    encoder         : Lavf58.76.100  
  Stream #0:0: Video: h264, rgb24(pc, progressive), 1920x1080, q=2-31, 8000 kb/s, 56 fps, 90k tbn  
    Metadata:  
      encoder         : Lavc58.134.100 h264_v4l2m2m  
[mpegts @ 0x5598d12c10] Non-monotonous DTS in output stream 0:0; previous: 0, current: 0; changing   to 1. This may result in incorrect timestamps in the output file.  
frame=  799 fps= 16 q=-0.0 size=   16180kB time=00:00:49.19 bitrate=2694.2kbits/s speed=   1x 


  


  As you can see my issue is that the fps value is about 15 when streaming an HD video with VLC in full-screen mode (the fps value depends on what is displayed on screen, which I find odd being the encoding process should be done in HW).

  


  So the question is : any hope I can get close to 25 fps, so to have a smooth display on the receiver ? Either using a better FFMPEG command or tweaking the Raspberry ?

  


  Thanks for any help !

  


• C++, FFmpeg, save AVPacket infomation into videostate structure in ffplay.c

  28 mai 2015, par Yoohoo

  I am currently working on a project that tests video streaming. In the project, video stream is encoded with H.264 before send and decoded after receive, using FFmpeg codec and functions.

  I can encode video stream by

  init_video_encode(AV_CODEC_ID_H264);

  where

  static void init_video_encode(AVCodecID codec_id){
  
  codec = avcodec_find_encoder(codec_id);
  
  if (!codec) {
  
      fprintf(stderr, "Codec not found\n");
  
      exit(1);
  
  }
  
  
  
  c = avcodec_alloc_context3(codec);
  
  if (!c) {
  
      fprintf(stderr, "Could not allocate video codec context\n");
  
      exit(1);
  
  }
  
  
  
  /* put sample parameters */
  
  c->bit_rate = 400000;
  
  /* resolution must be a multiple of two */
  
  c->width = 640;
  
  c->height = 480;
  
  /* frames per second */
  
  c->time_base= (AVRational){1,25};
  
  c->gop_size = 10; /* emit one intra frame every ten frames */
  
  c->max_b_frames=max_f;
  
  c->pix_fmt = AV_PIX_FMT_YUV420P;
  
  
  
  if(codec_id == AV_CODEC_ID_H264)
  
      av_opt_set(c->priv_data, "preset", "slow", 0);
  
  
  
  /* open it */
  
  if (avcodec_open2(c, codec, NULL) < 0) {
  
      fprintf(stderr, "Could not open codec\n");
  
      exit(1);
  
  }
  
  
  
  frame = avcodec_alloc_frame();
  
  if (!frame) {
  
      fprintf(stderr, "Could not allocate video frame\n");
  
      exit(1);
  
  }
  
  frame->format = c->pix_fmt;
  
  frame->width  = c->width;
  
  frame->height = c->height;
  
  
  
  /* the image can be allocated by any means and av_image_alloc() is
  
   * just the most convenient way if av_malloc() is to be used */
  
  ret = av_image_alloc(frame->data, frame->linesize, c->width, c->height,
  
                       c->pix_fmt, 32);
  
  
  
  /* get the delayed frames */
  
  if (ret < 0) {
  
      fprintf(stderr, "Could not allocate raw picture buffer\n");
  
      exit(1);
  
  }
  
      av_init_packet(&pkt);
  
  
  
  //}
  
  
  
      pkt.data = NULL;    // packet data will be allocated by the encoder
  
      pkt.size = 0;
  
      //cout<<"\nBefore YUV\n";
  
      if(count == 0)
  
      read_yuv420(frame->data[0]);
  
      count ++;
  
     // cout<<"\nAfter YUV\n";
  
      if(count == SUBSITY) {
  
      count = 0;
  
      }
  
  
  
      frame->pts = i++;
  
  
  
      /* encode the image */
  
      ret = avcodec_encode_video2(c, &pkt, frame, &got_output);
  
      if (ret < 0) {
  
           fprintf(stderr, "Error encoding frame\n");
  
           return -1;
  
      }
  
      //cout<<"\nRecord the Video\n";
  
      if (got_output) {
  
           //printf("Write frame %3d (size=%5d)\n", i, pkt.size);
  
           //cout<<"\nBefore Memcpy\n\n\n";
  
  
  
           memcpy(inbufout+totalSize,pkt.data,pkt.size);
  
           //cout<<"\nAfter Memcpy\n\n\n";
  
           totalSize+=pkt.size;

  The video encoder works very well, if I write the encoded packet into a .h264 file, it can be played. But at the decoder side, I cannot decode the received packet with :

      codec = avcodec_find_decoder(AV_CODEC_ID_H264);
  
      if (!codec) {
  
          fprintf(stderr, "Codec not found\n");
  
          exit(1);
  
      }
  
  
  
      c = avcodec_alloc_context3(codec);
  
      if (!c) {
  
          fprintf(stderr, "Could not allocate video codec context\n");
  
          exit(1);
  
      }
  
  
  
      if(codec->capabilities&CODEC_CAP_TRUNCATED)
  
          c->flags|= CODEC_FLAG_TRUNCATED; 
  
  
  
      /* open it */
  
      if (avcodec_open2(c, codec, NULL) < 0) {
  
          fprintf(stderr, "Could not open codec\n");
  
          exit(1);
  
      }
  
  
  
      frame = avcodec_alloc_frame();
  
      if (!frame) {
  
          fprintf(stderr, "Could not allocate video frame\n");
  
          exit(1);
  
      }
  
  len = avcodec_decode_video2(avctx, frame, &got_frame, pkt);
  
      if (len < 0) {
  
          fprintf(stderr, "Error while decoding frame %d\n", *frame_count);
  
          return len;
  
      }

  The reason of failure is lacking parser, I have tried to build a parser but failed......

  Therefore I am wondering using ffplay.c as a header file in my receiver program so that I can use it as the decoder and player.

  I have took a look at ffplay.c, it actually fetch file into a videostate structure and processing it. The fetching part is from line 3188 of ffplay.c :

  VideoState *is;
  
  
  
      is = av_mallocz(sizeof(VideoState));
  
      if (!is)
  
          return NULL;
  
      av_strlcpy(is->filename, filename, sizeof(is->filename));
  
      is->iformat = iformat;
  
      is->ytop    = 0;
  
      is->xleft   = 0;
  
  
  
      /* start video display */
  
      if (frame_queue_init(&is->pictq, &is->videoq, VIDEO_PICTURE_QUEUE_SIZE, 1) < 0)
  
          goto fail;
  
      if (frame_queue_init(&is->subpq, &is->subtitleq, SUBPICTURE_QUEUE_SIZE, 0) < 0)
  
          goto fail;
  
      if (frame_queue_init(&is->sampq, &is->audioq, SAMPLE_QUEUE_SIZE, 1) < 0)
  
          goto fail;
  
  
  
      packet_queue_init(&is->videoq);
  
      packet_queue_init(&is->audioq);
  
      packet_queue_init(&is->subtitleq);
  
  
  
      is->continue_read_thread = SDL_CreateCond();
  
  
  
      init_clock(&is->vidclk, &is->videoq.serial);
  
      init_clock(&is->audclk, &is->audioq.serial);
  
      init_clock(&is->extclk, &is->extclk.serial);
  
      is->audio_clock_serial = -1;
  
      is->av_sync_type = av_sync_type;
  
      is->read_tid     = SDL_CreateThread(read_thread, is);
  
      if (!is->read_tid) {
  
  fail:
  
          stream_close(is);
  
          return NULL;
  
      }

  Now instead of fetching file, I want to modify ffplay.c code so that let it fetch the received packet, I can save received packet to AVPacket by

  static AVPacket avpkt;
  
  avpkt.data = inbuf;

  My question is : how to put AVPacket information into videostate structure ?