Recherche avancée

Sur d’autres sites (5460)

• FFmpeg - RTMP streaming from NodeJS, stream is faster than realtime

  6 avril 2023, par Weston

  My goal is to render a canvas in Node, and stream that canvas to an RTMP server (Twitch ultimately, but for now I'm testing on a local RTMP server). The standard way to stream to RTMP seems to be ffmpeg, so I'm using that, spawned as a child process from within NodeJS.

  


  I've tried a bunch of different combinations of techniques and ffmpeg params to get a consistent framerate and a stream at "realtime" speed, but can't figure it out. Here's the paths I've gone down so far

  


  Render canvas and send input in continuous interval

  


  import { createCanvas } from 'canvas';

const canvas = createCanvas(1920, 1080);
const ctx = canvas.getContext('2d');

const fps = 30;
const ffmpeg = spawn('ffmpeg', [
  '-re',
  '-framerate', String(.fps),
  '-r', String(fps),

  '-i', '-',
  
  '-vcodec', 'libx264',
  '-r', String(fps),
  '-s', '1920x1080',
  '-g:v', String(2*fps),
  '-c:a', 'aac',
  '-f', 'flv', 'rtmp://127.0.0.1/live'
]);
ffmpeg.stdout.pipe(process.stdout)
ffmpeg.stderr.pipe(process.stderr)


const send = () => {
  ctx.fillStyle = 'red'
  ctx.fillRect(0, 0, 1920, 1080);
  ctx.font = '100px Arial';
  ctx.fillStyle = 'black'
  ctx.fillText(new Date().toLocaleString(), 500, 500);
  ffmpeg.stdin.write(canvas.toBuffer())
  setImmediate(() => send())
}
send()


  


  Observations

  


  

  • Took about 35 seconds for the stream to actually start (I think because of ffmpeg needing some amount of time to analyze the input ?)

  


  • Frame rate extremely below what I set it to, and "speed" also very low, although I'm not 100% sure what this means. example log Frame=  906 fps=3.9 q=29.0 size= 311kB time=00:00:27.83 bitrate= 91.5kbits/s speed=0.119x

  


  • Stream behavior

    

    • Takes about a minute to load once opened in VLC

    


    • Timer on the stream starts about 1 minute behind real time, stays stuck on a single second for 30+ seconds, then shoots up a few seconds quickly, and gets stuck again

    


    


  


  


  I had a hunch here that at least some of the reason for the strange behavior was that rendering the canvas in the same loop that I send input to ffmpeg in was too slow to achieve 30 FPS.

  


  Render canvas in separate interval from ffmpeg input interval

  


  Only render canvas FPS-times per second

  


  Continue sending input to ffmpeg as fast as possible

  


  import { createCanvas } from 'canvas';

const canvas = createCanvas(1920, 1080);
const ctx = canvas.getContext('2d');

let buffer = canvas.toBuffer();

const fps = 30;
const ffmpeg = spawn('ffmpeg', [
  ...same as before
]);

const render = () => {
  ctx.fillStyle = 'red'
  ctx.fillRect(0, 0, 1920, 1080);
  ctx.font = '100px Arial';
  ctx.fillStyle = 'black'
  ctx.fillText(new Date().toLocaleString(), 500, 500);
  buffer = canvas.toBuffer();
  setTimeout(() => render(), 1/fps)
}
render();

const send = () => {
  ffmpeg.stdin.write(buffer)
  setImmediate(() => send())
}
send()


  


  Observations

  


  

  • ffmpeg starts streaming almost immediately

  


  • fps starts out around 16, takes a couple seconds to hit 28, and then 30 more seconds to hit 30fps. speed much closer to 1x, but not quite all the way. example log frame=15421 fps= 30 q=29.0 size=    4502kB time=00:08:31.66 bitrate=  72.1kbits/s speed=0.994x      

  


  • Stream behavior

    

    • Takes about 5 seconds to load once opened in VLC

    


    • Timer stays stuck on the same second for multiple minutes

    


    


  


  


  My hunch here for the stream being stuck on 1 timestamp is that while ffmpeg is sending frames out at 30 frames per second, I'm sending it frames much quicker than that. So in the first 1st of a second of streaming

  


  

  1. Canvas renders with timestamp T 30 times

  


  2. send runs N times where N is likely way higher than 30, sending ffmpeg N frames with the current timestamp

  


  3. ffmpeg now has N frames with timestamp T on them, but can only send them out 30 per second, so it takes more than 1 second for the timestamp on the screen to change

  


  


  Only send ffmpeg a frame every 1/FPS second

  


  Same as before, but instead of sending ffmpeg frames as quickly as possible, only send it FPS frames every second.

  


  import { createCanvas } from 'canvas';

const canvas = createCanvas(1920, 1080);
const ctx = canvas.getContext('2d');

let buffer = canvas.toBuffer();

const fps = 30;
const ffmpeg = spawn('ffmpeg', [
  ...same as before
]);

const render = () => {
  ...same as before
}
render();

const send = () => {
  ffmpeg.stdin.write(buffer)
  setTimeout(() => send(), 1/fps)
}
send()


  


  Observations

  


  

  • ffmpeg takes a few seconds to start streaming

  


  • fps starts out high, around 28, and over the next minute or so drops down to 16. Speed drops along with it. example log frame= 1329 fps= 16 q=29.0 size=     463kB time=00:00:41.93 bitrate=  90.5kbits/s speed= 0.5x

  


  • Stream behavior

    

    • Takes about 10 seconds to load once opened in VLC

    


    • Timer increases about twice as fast as expected, then gets hung on one second for a bit, and then starts increasing again at same rate

    


    


  


  


  ---

  


  I'll stop there, but tl ;dr I've tried a whole bunch of different combinations of -re, -framerate, -fps_mode, -r ffmpeg args, and some other techniques in the code like continuing to use setImmediate to send frames, but use a date comparison to actually send a frame at an FPS rate. I'm sure there's probably some fundamental video streaming knowledge I'm missing, so I'm just looking for any sort of guidance on how I can get my canvas to stream at a "realtime" speed, or whatever I could be missing here.

  


• ffmpeg : playing udp stream

  14 novembre 2013, par deimus

  I'm playing udp stream on iDevice using ffmpeg.
  It does play the video and audio successfully.

  The only issue I've got here that the following function call does take a long time

  avformat_find_stream_info

  It takes about 10 secs to complete the execution of this function.
  The media that I'm playing has following properties :

  MPEG-4 VIDEO v3 (DIV3)
  
  RESOLUTION : 640x480
  
  Frame rate : 25

  Any ideas how to workaround this delay ?

• Revision 84af0486f9 : Fix rd_pick_partition search loop for 4x4 blocks The partition search for 4x4 b

  25 juillet 2014, par Jingning Han

  Changed Paths :
   Modify /vp9/encoder/vp9_encodeframe.c

  
  
  Fix rd_pick_partition search loop for 4x4 blocks

  The partition search for 4x4 blocks takes unnecessary steps to
  reconstruct pixels and an extra partition type update. This commit
  removes such operations. No visible compression/speed difference.
  Thanks to Yue (yuec@) for finding this issue.

  Change-Id : I3f83824aa3fd3717d63be0b280fa57258939a70a