Recherche avancée

Sur d’autres sites (7153)

• Use ffmpeg to transform rtsp stream containing B-Frames and no DTS into fragmented mp4 [closed]

  13 octobre 2023, par Stuntman11

  The goal is to play a rtsp video stream in the chrome browser using Websocket and MSE. The structure of the system looks almost identical to FFmpeg live streaming for media source extensions(MSE). This setup works in most situations but the moment B-Frames come into play everything breaks. FFmpeg starts to output Non-monotonous DTS in output stream and shortly after the video played via MSE freezes for a few seconds. As a layman my understanding of the situation is that :

  


  


  1. There are decode timestamp (DTS) and presentation timestamp (PTS) and an RTSP stream does not carry DTS information but allows somehow to relay PTS information.

     


  


  2. Without B-Frames DTS=PTS but when B-Frames occur, the frames have to be decoded ahead of time for later frames to be displayed correctly, resulting in a PTS > DTS for these frames.

     


  


  3. A mp4 muxer requires monotonously increasing DTS and DTS <= PTS. As the RTSP does not have DTS ffmpeg deduces DTS from PTS by default which results in jumps when a B-Frame occurs, hinted by the output Non-monotonous DTS in output stream. FFmpeg repairs the DTS by increment from the last valid DTS.

     &#xA;

  &#xA;

  4. Now DTS is running behind video playback time the video presentation breaks down. Even the mode segments does not help here as ffmpeg rewrote DTS to make it monotonous for the mp4 container and the underlying h264 does not include any timestamps so reordering can no longer happen.

     &#xA;

  &#xA;

  &#xA;

  As I have no control over the existence of B-Frames in the RTSP stream, I must assist FFmpeg in the deduction of DTS or at least fix the decoding order. Important : The live stream can only have a latency around one second so that expensive operations like reencoding the stream without B-Frames is not a valid option ! I would also prefer to not build a specific application with ffmpeg's lib and instead use FFmpeg only via command line.

  &#xA;

  Idea 1 : Buffer around one second of the rtsp packets, reoder them by PTS and then generate DTS based on PTS. The reordering should effectivly prevent the issues of non monotonous dts increase and presentation. While I think this is the most promising solution, it does not seem to be possible with ffmpeg cli ?

  &#xA;

  Idea 2 : Offset PTS by one second and set DTS to decode as soon as possible that frames are always ready for presentation. This assumes that B-Frames can be decoded after their respective partners are already decoded. I am not sure if this is even possible because the decoder may uses B-Frames as direct dependecies for the frames afterwards ?

  &#xA;

  Idea 3 : Somehow export ISO BMFF with non monotonous DTS for each packet and let the MediaSource sort the segments by DTS. As far as I understand the segments mode reorders by PTS and not DTS, but maybe someone has more knowledge here ?

  &#xA;

  I already did a ton of research but was not able to implement the mentioned ideas with ffmpeg cli. Does someone know how to do it or has someone successfully used a similar system ? I greatly appreciate other suggestions or technical feedback to improve my understanding. Solutions using ffmpeg as a programming library are also welcome but I am more interested in cli.

  &#xA;

  Here the current ffmpeg command that works well without B-Frames :

  &#xA;

  ffmpeg -nostats -hide_banner -loglevel fatal -i  -c copy -an -f mp4 -frag_duration 50000 -movflags +frag_keyframe+empty_moov+default_base_moof pipe:1


  &#xA;

  My understanding is based on these sources :
  &#xA;How to write a video stream containing B-frame and no DTS to a MP4 container ?
  &#xA;Does RTP timestamp represents the PTS of the audio/video present in media ?
  &#xA;h264 RTP timestamp
  &#xA;https://www.ietf.org/rfc/rfc3550.txt
  &#xA;https://www.w3.org/TR/mse-byte-stream-format-isobmff/
  

  &#xA;

• How can I stream a jpeg file that is continually created to a mjpeg stream that can be read as a webcam from Raspbery Pi ? [closed]

  10 novembre 2023, par Andrei Ignat

  I am using OpenCV to process frames I catch from a MJPEG stream, aprox. 1-3fps for a burst of 30-60s and want to stream the result back to view it side by side as a webcam in the webinterface.&#xA;The Webinterface reading the stream is Mainsail and the OpenCV file is created by a Python script in Klipper.

  &#xA;

  I tried saving the frame to "frame.jpeg" and stream using FFMPEG :

  &#xA;

   ffmpeg -loop 1 -i frame.jpg -framerate 15 -c:v mjpeg -f mjpeg -listen 1 http://*:8282/cv2


  &#xA;

  But it gives me this error when the webcam client connects :

  &#xA;

  Input #0, image2, from 'frame.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 10017 kb/s
  Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 640x480 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> mjpeg (native))
Press [q] to stop, [?] for help
Output #0, mjpeg, to 'http://*:8282/cv2':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown, progressive), 640x480 [SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn
    Metadata:
      encoder         : Lavc59.37.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
av_interleaved_write_frame(): Broken pipe time=00:00:00.04 bitrate=3496.4kbits/s speed=2e+04x
Error writing trailer of http://*:8282/cv2: Broken pipe
frame=    2 fps=0.0 q=2.7 Lsize=      17kB time=00:00:00.08 bitrate=1748.2kbits/s speed=1.27x
video:43kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[http @ 0x29fd660] URL read error: End of file
Error closing file http://*:8282/cv2: End of file
Conversion failed!


  &#xA;

• When I attempt to live a video in an infinite loop using the ffmpeg command, it automatically terminates after a few hours

  13 décembre 2023, par Isteyak Ali

  When attempting to continuously loop a video using the ffmpeg command with the following syntax :

  &#xA;

  ffmpeg -re -stream_loop -1 -i abc.mp4 -c copy -f flv rtmps://abc.xyz/.....

  &#xA;

  I encounter an issue where the process automatically terminates after a few hours. The error message includes the following details :

  &#xA;

  **av_interleaved_write_frame(): Broken pipe
[flv @ 0x55d73001dd00] Failed to update header with correct duration.
[flv @ 0x55d73001dd00] Failed to update header with correct filesize.
Error writing trailer of rtmp://a.rtmp.youtube.com/live2/........................: Broken pipe
frame=1812023 fps= 30 q=-1.0 Lsize= 4902934kB time=16:47:41.10 bitrate= 664.3kbits/s speed=   1x    
video:3879605kB audio:944717kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.629512%
Conversion failed!**


  &#xA;

  It seems to be related to a "Broken pipe" error during the writing of the trailer, and issues updating the header with correct duration and filesize. The overall conversion fails with a muxing overhead of 1.629512%.=

  &#xA;

  I attempted to address the issue by incorporating ffmpeg's reconnect command options, specifically using the following command :

  &#xA;

  ffmpeg -stream_loop -1 -re -i abc.mp4 -c copy -f flv -reconnect 1 -reconnect_streamed 1 -reconnect_delay_max 2 rtmps://abc.xyz/......

  &#xA;

  However, despite including the reconnect commands such as -reconnect 1, -reconnect_streamed 1, and -reconnect_delay_max 2, the error persisted, and the issue of automatic termination after a few hours remained unresolved.

  &#xA;

  I suspect that the problem may be related to network issues. Although I have included the reconnect command to solve this, I have not yet found a successful solution. I would appreciate assistance in understanding the root cause of this problem and any suggested solutions.

  &#xA;