Sur d’autres sites (5866)

• How to HLS-live-stream incoming batches of individual frames, "appending" to a m3u8 playlist in real time, with ffmpeg ?

  20 novembre 2024, par Rob

  My overall goal :

  



  Server-side :

  



  

  • I have batches of sequential, JPEG-encoded frames (8-16) arriving from time to time, generated at roughly 2 FPS.

  


  • I would like to host an HLS live stream, where, when a new batch of frames arrives, I encode those new frames as h264 .ts segments with ffmpeg, and have the new .ts segments automatically added to an HLS stream (e.g. .m3u8 file).

  


  



  Client/browser-side :

  



  

  • When the .m3u8 is updated, I would like the video stream being watched to simply "continue", advancing from the point where new .ts segments have been added.

  


  • I do not need the user to scrub backwards in time, the client just needs to support live observation of the stream.

  


  



  



  My current approach :

  



  Server-side :

  



  To generate the "first" few segments of the stream, I'm attempting the below (just command-line for now to get ffmpeg working right, but ultimately will be automated via a Python script) :

  



  For reference, I'm using ffmpeg version 3.4.6-0ubuntu0.18.04.1.

  



  ffmpeg -y -framerate 2 -i /frames/batch1/frame_%d.jpg \
       -c:v libx264 -crf 21 -preset veryfast -g 2 \
       -f hls -hls_time 4 -hls_list_size 4 -segment_wrap 4 -segment_list_flags +live video/stream.m3u8


  



  where the /frames/batch1/ folder contains a sequence of frames (e.g. frame_01.jpg, frame_02.jpg, etc...). This already doesn't appear to work correctly, because it keeps adding #EXT-X-ENDLIST to the end of the .m3u8 file, which as I understand is not correct for a live HLS stream - here's what that generates :

  



  #EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:4.000000,
stream0.ts
#EXTINF:4.000000,
stream1.ts
#EXTINF:2.000000,
stream2.ts
#EXT-X-ENDLIST


  



  I can't figure out how to suppress #EXT-X-ENDLIST here - this is problem #1.

  



  Then, to generate subsequent segments (e.g. when new frames become available), I'm trying this :

  



  ffmpeg -y -framerate 2 -start_number 20 -i /frames/batch2/frame_%d.jpg \
       -c:v libx264 -crf 21 -preset veryfast -g 2 \
       -f hls -hls_time 4 -hls_list_size 4 -segment_wrap 4 -segment_list_flags +live video/stream.m3u8


  



  Unfortunately, this does not work the way I want it to. It simply overwrites stream.m3u8, does and does not advance #EXT-X-MEDIA-SEQUENCE, it does not index the new .ts files correctly, and it also includes the undesirable #EXT-X-ENDLIST - this is the output of that command :

  



  #EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:4.000000,
stream0.ts
#EXTINF:4.000000,
stream1.ts
#EXTINF:3.000000,
stream2.ts
#EXT-X-ENDLIST


  



  Fundamentally, I can't figure out how to "append" to an existing .m3u8 in a way that makes sense for HLS live streaming. That's essentially problem #2.

  



  For hosting the stream, I'm using a simple Flask app - which appears to be working the way I intend - here's what I'm doing for reference :

  



  @app.route('/video/')
def stream(file_name):
    video_dir = './video'
    return send_from_directory(directory=video_dir, filename=file_name)


  



  Client-side :

  



  I'm trying HLS.js in Chrome - basically boils down to this :

  



  <video></video>&#xA;&#xA;...&#xA;&#xA;<code class="echappe-js">&lt;script src=&quot;https://cdn.jsdelivr.net/npm/hls.js@latest&quot;&gt;&lt;/script&gt;&#xA;&lt;script&gt;&amp;#xA;   var video = document.getElementById(&amp;#x27;video1&amp;#x27;);&amp;#xA;   if (Hls.isSupported()) {&amp;#xA;     var hls = new Hls();&amp;#xA;     hls.loadSource(&amp;#x27;/video/stream.m3u8&amp;#x27;);&amp;#xA;     hls.attachMedia(video);&amp;#xA;     hls.on(Hls.Events.MANIFEST_PARSED, function() {&amp;#xA;       video.play();&amp;#xA;     });&amp;#xA;   }&amp;#xA;   else if (video.canPlayType(&amp;#x27;application/vnd.apple.mpegurl&amp;#x27;)) {&amp;#xA;     video.src = &amp;#x27;/video/stream.m3u8&amp;#x27;;&amp;#xA;     video.addEventListener(&amp;#x27;loadedmetadata&amp;#x27;, function() {&amp;#xA;       video.play();&amp;#xA;     });&amp;#xA;   }&amp;#xA;&lt;/script&gt;   &#xA;

  



  I'd like to think that what I'm trying to do doesn't require a more complex approach than what I'm trying above, but since what I'm trying to far definitely isn't working, I'm starting to think I need to come at this from a different angle. Any ideas on what I'm missing ?

  



  Edit :

  



  I've also attempted the same (again in Chrome) with video.js, and am seeing similar behavior - in particular, when I manually update the backing stream.m3u8 (with no #EXT-X-ENDLIST tag), videojs never picks up the new changes to the live stream, and just buffers/hangs indefinitely.

  



  <video class="video-js vjs-default-skin" muted="muted" controls="controls">&#xA;    <source type="application/x-mpegURL" src="/video/stream.m3u8">&#xA;</source></video>&#xA;&#xA;...&#xA;&#xA;<code class="echappe-js">&lt;script&gt;&amp;#xA;    var player = videojs(&amp;#x27;video1&amp;#x27;);&amp;#xA;    player.play();&amp;#xA;&lt;/script&gt;&#xA;

  



  For example, if I start with this initial version of stream.m3u8 :

  



  #EXTM3U
#EXT-X-PLAYLIST-TYPE:EVENT
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:8
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:4.000000,
stream0.ts
#EXTINF:4.000000,
stream1.ts
#EXTINF:2.000000,
stream2.ts


  



  and then manually update it server-side to this :

  



  #EXTM3U
#EXT-X-PLAYLIST-TYPE:EVENT
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:8
#EXT-X-MEDIA-SEQUENCE:3
#EXTINF:4.000000,
stream3.ts
#EXTINF:4.000000,
stream4.ts
#EXTINF:3.000000,
stream5.ts


  



  the video.js control just buffers indefinitely after only playing the first 3 segments (stream*.ts 0-2), which isn't what I'd expect to happen (I'd expect it to continue playing stream*.ts 3-5 once stream.m3u8 is updated and video.js makes a request for the latest version of the playlist).

  


• Is there any possibility to transcode a h264 hls/m3u8 live stream into another HEVC (h265) hls/m3u8 live stream using FFmpeg in real time ?

  2 septembre 2024, par Ghenadie Pripa

  The command I used :

  


  ffmpeg -i  https://tezaurtv.md/wp-content/uploads/live/index.m3u8 -c:v libx265 -b:v 1500k -tag:v hvc1 -c:a aac -b:a 128k -f hls -hls_time 10 -hls_list_size 10 -hls_flags delete_segments+append_list+program_date_time -hls_playlist_type event -hls_segment_type mpegts -hls_segment_filename "D:/BRUT/HLS/test/stream_%d.ts" -vsync 2 -rtbufsize 500M -loglevel debug D:/BRUT/HLS/test/stream.m3u8


  


  I tried a lot of commands and parameters but nothing helps. The experiment is on local Windows. Transcode with ffmpeg and play with FFplay.

  


  The problem is the following : After I start stream.m3u8 to play after a few seconds the picture and sound stops. What is wrong in the FFmpeg command ?

  


  ffmpeg version 7.0.2-full_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.0 (Rev5, Built by MSYS2 project)


  


  ffplay version 7.0.2-full_build-www.gyan.dev Copyright (c) 2003-2024 the FFmpeg developers
  built with gcc 13.2.0 (Rev5, Built by MSYS2 project)


  


  An error from FFplay :

  


  [hevc @ 0000023d8c320940] Output frame with POC 239.
[hevc @ 0000023d8c320940] Decoded frame with POC 241.
[hevc @ 0000023d8c320e40] nal_unit_type: 35(AUD), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0000023d8c320e40] nal_unit_type: 0(TRAIL_N), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0000023d8c320e40] Output frame with POC 240.
[hevc @ 0000023d8c320e40] Decoded frame with POC 240.
[hevc @ 0000023d855aae80] nal_unit_type: 35(AUD), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0000023d855aae80] nal_unit_type: 0(TRAIL_N), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0000023d855aae80] Output frame with POC 241.
[hevc @ 0000023d855aae80] Decoded frame with POC 242.
[hevc @ 0000023d8c2cab40] nal_unit_type: 35(AUD), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0000023d8c2cab40] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0000023d8c2cab40] Output frame with POC 242.
[hevc @ 0000023d8c2cab40] Decoded frame with POC 247.
[hevc @ 0000023d8c2da9c0] nal_unit_type: 35(AUD), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0000023d8c2da9c0] nal_unit_type: 1(TRAIL_R), nuh_layer_id: 0, temporal_id: 0
[hevc @ 0000023d8c2da9c0] Output frame with POC 243.
[hevc @ 0000023d8c2da9c0] Decoded frame with POC 245.
[hls @ 0000023d85544b80] Opening 'D:\BRUT\HLS\test\stream.m3u8' for reading
[hls @ 0000023d85544b80] Skip ('#EXT-X-VERSION:3')
[hls @ 0000023d85544b80] Skip ('#EXT-X-DISCONTINUITY')
[hls @ 0000023d85544b80] Skip ('#EXT-X-PROGRAM-DATE-TIME:2024-08-30T16:29:28.175+0300')
[AVIOContext @ 0000023d8c327040] Statistics: 207 bytes read, 0 seeks
[hls @ 0000023d85544b80] Opening 'D:\BRUT\HLS\test\stream.m3u8' for reading
[hls @ 0000023d85544b80] Skip ('#EXT-X-VERSION:3')sq=    0B
[hls @ 0000023d85544b80] Skip ('#EXT-X-DISCONTINUITY')
[hls @ 0000023d85544b80] Skip ('#EXT-X-PROGRAM-DATE-TIME:2024-08-30T16:29:28.175+0300')
[AVIOContext @ 0000023d8c326980] Statistics: 207 bytes read, 0 seeks
[hls @ 0000023d85544b80] Opening 'D:\BRUT\HLS\test\stream.m3u8' for reading
[hls @ 0000023d85544b80] Skip ('#EXT-X-VERSION:3')
[hls @ 0000023d85544b80] Skip ('#EXT-X-DISCONTINUITY')
[hls @ 0000023d85544b80] Skip ('#EXT-X-PROGRAM-DATE-TIME:2024-08-30T16:29:28.175+0300')
[AVIOContext @ 0000023d8c326f00] Statistics: 207 bytes read, 0 seeks
  11.30 A-V:-18.499 fd=   3 aq=    0KB vq=    0KB sq=    0B


  


  I tried to transcode in real time hls h264 to hls 265.

  


• Why when using ffmpeg to create in real time avi video file from images the avi file is playing with purple noisy color ?

  30 juin 2015, par Brubaker Haim

  This is my Ffmpeg class i did some time ago

  using System;
  
  using System.Windows.Forms;
  
  using System.Collections.Generic;
  
  using System.Linq;
  
  using System.Text;
  
  using System.Threading.Tasks;
  
  using System.Drawing;
  
  using System.IO.Pipes;
  
  using System.Runtime.InteropServices;
  
  using System.Diagnostics;
  
  using System.IO;
  
  using DannyGeneral;
  
  
  
  namespace Manager
  
  {
  
      class Ffmpeg
  
      {
  
          NamedPipeServerStream p;
  
          String pipename = "mytestpipe";
  
          System.Diagnostics.Process process;
  
          string ffmpegFileName = "ffmpeg.exe";
  
          string workingDirectory;
  
  
  
          public Ffmpeg()
  
          {
  
              workingDirectory = Path.GetDirectoryName(Application.ExecutablePath);
  
              Logger.Write("workingDirectory: " + workingDirectory);
  
              if (!Directory.Exists(workingDirectory))
  
              {
  
                  Directory.CreateDirectory(workingDirectory);
  
              }
  
              ffmpegFileName = Path.Combine(workingDirectory, ffmpegFileName);
  
              Logger.Write("FfmpegFilename: " + ffmpegFileName);
  
          }
  
  
  
          public void Start(string pathFileName, int BitmapRate)
  
          {
  
              try
  
              {
  
  
  
                  string outPath = pathFileName;
  
                  p = new NamedPipeServerStream(pipename, PipeDirection.Out, 1, PipeTransmissionMode.Byte);
  
  
  
                  ProcessStartInfo psi = new ProcessStartInfo();
  
                  psi.WindowStyle = ProcessWindowStyle.Hidden;
  
                  psi.UseShellExecute = false;
  
                  psi.CreateNoWindow = false;
  
                  psi.FileName = ffmpegFileName;
  
                  psi.WorkingDirectory = workingDirectory;
  
                  psi.Arguments = @"-f rawvideo -pix_fmt yuv420p -video_size 1920x1080 -i \\.\pipe\mytestpipe -map 0 -c:v mpeg4 -r " + BitmapRate + " " + outPath;
  
                  process = Process.Start(psi);
  
                  process.EnableRaisingEvents = false;
  
                  psi.RedirectStandardError = true;
  
                  p.WaitForConnection();
  
              }
  
              catch (Exception err)
  
              {
  
                  Logger.Write("Exception Error: " + err.ToString());
  
              }
  
          }
  
  
  
          public void PushFrame(Bitmap bmp)
  
          {
  
              try
  
              {
  
                  int length;
  
                  // Lock the bitmap's bits.
  
                  //bmp = new Bitmap(1920, 1080);
  
                  Rectangle rect = new Rectangle(0, 0, bmp.Width, bmp.Height);
  
                  //Rectangle rect = new Rectangle(0, 0, 1280, 720);
  
                  System.Drawing.Imaging.BitmapData bmpData =
  
                      bmp.LockBits(rect, System.Drawing.Imaging.ImageLockMode.ReadOnly,
  
                      bmp.PixelFormat);
  
  
  
                  int absStride = Math.Abs(bmpData.Stride);
  
                  // Get the address of the first line.
  
                  IntPtr ptr = bmpData.Scan0;
  
  
  
                  // Declare an array to hold the bytes of the bitmap.
  
                  //length = 3 * bmp.Width * bmp.Height;
  
                  length = absStride * bmpData.Height;
  
                  byte[] rgbValues = new byte[length];
  
  
  
                  //Marshal.Copy(ptr, rgbValues, 0, length);
  
                  int j = bmp.Height - 1;
  
                  for (int i = 0; i < bmp.Height; i++)
  
                  {
  
                      IntPtr pointer = new IntPtr(bmpData.Scan0.ToInt32() + (bmpData.Stride * j));
  
                      System.Runtime.InteropServices.Marshal.Copy(pointer, rgbValues, absStride * (bmp.Height - i - 1), absStride);
  
                      j--;
  
                  }
  
                  p.Write(rgbValues, 0, length);
  
                  bmp.UnlockBits(bmpData);
  
              }
  
              catch(Exception err)
  
              {
  
                  Logger.Write("Error: " + err.ToString());
  
              }
  
  
  
          }
  
  
  
          public void Close()
  
          {
  
              p.Close();
  
          }
  
      }
  
  }

  And i’m using it in form1 in a button click event :

  private void button1_Click(object sender, EventArgs e)
  
          {
  
              timer1.Start();
  
          }

  the directroy screenshots is where i’m taking a screenshot every 100ms in the timer1 tick event :

      ScreenShot shot = new ScreenShot();
  
      public static int counter = 0;
  
      private void timer1_Tick(object sender, EventArgs e)
  
      {
  
          counter++;
  
          shot.GetScreenShot(@"e:\screenshots\", "screenshot");
  
          if (counter == 1200)
  
          {
  
              timer1.Stop();
  
          }
  
      }

  I’m calling the method PushFrame from inside the ScreenShot class where i save the screenshots.

  Ffmpeg fmpeg;

  Then :

  fmpeg = new Ffmpeg();
  
  fmpeg.Start(@"e:\screenshots\test.avi", 25);

  And :

  public Bitmap GetScreenShot(string folder, string name)
  
      {
  
          _screenShot = new Bitmap(GetScreen());
  
          System.GC.Collect();
  
          System.GC.WaitForPendingFinalizers();
  
          string ingName = folder + name + Elgato_Video_Capture.counter.ToString("D6") + ".bmp";
  
          _screenShot.Save(ingName);
  
          fmpeg.PushFrame(_screenShot);
  
          _screenShot.Dispose();
  
  
  
          return _screenShot;
  
      }

  All the images on the hard disk are fine i can edit/open them and watch them no problems.
  They are also same size.

  The result in the end is one big avi file 1.08 GB size.
  But when i play it i see many windows running inside very fast and all painted with noisy purple color.

  Here a screenshot from the video file when playing it :

  avi file playing screenshot

  I think the problem is somewhere in the Ffmpeg class where i give parameters to the ffmpeg.exe

  psi.Arguments = @"-f rawvideo -pix_fmt yuv420p -video_size 1920x1080 -i \\.\pipe\mytestpipe -map 0 -c:v mpeg4 -r " + BitmapRate + " " + outPath;

  Not sure what make this avi file to look like that.

  This is the video file the result i got : https://www.youtube.com/watch?v=fdxPus-Xv1k&feature=youtu.be