Recherche avancée

Sur d’autres sites (9099)

• FFmpeg - RTMP streaming from NodeJS, stream is faster than realtime

  6 avril 2023, par Weston

  My goal is to render a canvas in Node, and stream that canvas to an RTMP server (Twitch ultimately, but for now I'm testing on a local RTMP server). The standard way to stream to RTMP seems to be ffmpeg, so I'm using that, spawned as a child process from within NodeJS.

  


  I've tried a bunch of different combinations of techniques and ffmpeg params to get a consistent framerate and a stream at "realtime" speed, but can't figure it out. Here's the paths I've gone down so far

  


  Render canvas and send input in continuous interval

  


  import { createCanvas } from 'canvas';

const canvas = createCanvas(1920, 1080);
const ctx = canvas.getContext('2d');

const fps = 30;
const ffmpeg = spawn('ffmpeg', [
  '-re',
  '-framerate', String(.fps),
  '-r', String(fps),

  '-i', '-',
  
  '-vcodec', 'libx264',
  '-r', String(fps),
  '-s', '1920x1080',
  '-g:v', String(2*fps),
  '-c:a', 'aac',
  '-f', 'flv', 'rtmp://127.0.0.1/live'
]);
ffmpeg.stdout.pipe(process.stdout)
ffmpeg.stderr.pipe(process.stderr)


const send = () => {
  ctx.fillStyle = 'red'
  ctx.fillRect(0, 0, 1920, 1080);
  ctx.font = '100px Arial';
  ctx.fillStyle = 'black'
  ctx.fillText(new Date().toLocaleString(), 500, 500);
  ffmpeg.stdin.write(canvas.toBuffer())
  setImmediate(() => send())
}
send()


  


  Observations

  


  

  • Took about 35 seconds for the stream to actually start (I think because of ffmpeg needing some amount of time to analyze the input ?)

  


  • Frame rate extremely below what I set it to, and "speed" also very low, although I'm not 100% sure what this means. example log Frame=  906 fps=3.9 q=29.0 size= 311kB time=00:00:27.83 bitrate= 91.5kbits/s speed=0.119x

  


  • Stream behavior

    

    • Takes about a minute to load once opened in VLC

    


    • Timer on the stream starts about 1 minute behind real time, stays stuck on a single second for 30+ seconds, then shoots up a few seconds quickly, and gets stuck again

    


    


  


  


  I had a hunch here that at least some of the reason for the strange behavior was that rendering the canvas in the same loop that I send input to ffmpeg in was too slow to achieve 30 FPS.

  


  Render canvas in separate interval from ffmpeg input interval

  


  Only render canvas FPS-times per second

  


  Continue sending input to ffmpeg as fast as possible

  


  import { createCanvas } from 'canvas';

const canvas = createCanvas(1920, 1080);
const ctx = canvas.getContext('2d');

let buffer = canvas.toBuffer();

const fps = 30;
const ffmpeg = spawn('ffmpeg', [
  ...same as before
]);

const render = () => {
  ctx.fillStyle = 'red'
  ctx.fillRect(0, 0, 1920, 1080);
  ctx.font = '100px Arial';
  ctx.fillStyle = 'black'
  ctx.fillText(new Date().toLocaleString(), 500, 500);
  buffer = canvas.toBuffer();
  setTimeout(() => render(), 1/fps)
}
render();

const send = () => {
  ffmpeg.stdin.write(buffer)
  setImmediate(() => send())
}
send()


  


  Observations

  


  

  • ffmpeg starts streaming almost immediately

  


  • fps starts out around 16, takes a couple seconds to hit 28, and then 30 more seconds to hit 30fps. speed much closer to 1x, but not quite all the way. example log frame=15421 fps= 30 q=29.0 size=    4502kB time=00:08:31.66 bitrate=  72.1kbits/s speed=0.994x      

  


  • Stream behavior

    

    • Takes about 5 seconds to load once opened in VLC

    


    • Timer stays stuck on the same second for multiple minutes

    


    


  


  


  My hunch here for the stream being stuck on 1 timestamp is that while ffmpeg is sending frames out at 30 frames per second, I'm sending it frames much quicker than that. So in the first 1st of a second of streaming

  


  

  1. Canvas renders with timestamp T 30 times

  


  2. send runs N times where N is likely way higher than 30, sending ffmpeg N frames with the current timestamp

  


  3. ffmpeg now has N frames with timestamp T on them, but can only send them out 30 per second, so it takes more than 1 second for the timestamp on the screen to change

  


  


  Only send ffmpeg a frame every 1/FPS second

  


  Same as before, but instead of sending ffmpeg frames as quickly as possible, only send it FPS frames every second.

  


  import { createCanvas } from 'canvas';

const canvas = createCanvas(1920, 1080);
const ctx = canvas.getContext('2d');

let buffer = canvas.toBuffer();

const fps = 30;
const ffmpeg = spawn('ffmpeg', [
  ...same as before
]);

const render = () => {
  ...same as before
}
render();

const send = () => {
  ffmpeg.stdin.write(buffer)
  setTimeout(() => send(), 1/fps)
}
send()


  


  Observations

  


  

  • ffmpeg takes a few seconds to start streaming

  


  • fps starts out high, around 28, and over the next minute or so drops down to 16. Speed drops along with it. example log frame= 1329 fps= 16 q=29.0 size=     463kB time=00:00:41.93 bitrate=  90.5kbits/s speed= 0.5x

  


  • Stream behavior

    

    • Takes about 10 seconds to load once opened in VLC

    


    • Timer increases about twice as fast as expected, then gets hung on one second for a bit, and then starts increasing again at same rate

    


    


  


  


  ---

  


  I'll stop there, but tl ;dr I've tried a whole bunch of different combinations of -re, -framerate, -fps_mode, -r ffmpeg args, and some other techniques in the code like continuing to use setImmediate to send frames, but use a date comparison to actually send a frame at an FPS rate. I'm sure there's probably some fundamental video streaming knowledge I'm missing, so I'm just looking for any sort of guidance on how I can get my canvas to stream at a "realtime" speed, or whatever I could be missing here.

  


• How to stack several videos using ffmpeg ?

  13 juillet 2019, par John

  I have four cameras that are recording videos and wand to stack all videos into one video file. The final video must be a 2x2 grid.

  

  THE HARD PART is that sometimes cameras are not recording simultaneously.

  Here you can see one situation of how cameras are recording :

  Camera 1 timeline: |recording....................................|
  
  Camera 2 timeline: |stopped....|recording........................|
  
  Camera 3 timeline: |recording.........|stopped....|recording.....|
  
  Camera 4 timeline: |recording..................|stopped..........|

  Explanation :

  • Camera 1 is recording all time.
  • Camera 2 is starting recording later.
  • Camera 3 is recording, then stopped a period of time, then again recording.
  • Camera 4 is recording a period of time, then stopped.

  In the final video I want to synchronize each video by timestamps.

  If a camera is stopped a period of time then at its position must be a black portion.

  How can I do this ?

• ffmpeg_kit_flutter operation not permitted for audio operations

  25 août 2023, par Black Eyed Beans

  I'm trying to trim an audio file using ffmpeg_kit_flutter but I keep getting the error :

  


  audio/path/output.mp3: Operation not permitted.


  


  This is the ffmpeg command that I'm using :

  


  final cmd="-y -i \"$audioPath\" -ss $audioStartTime -to $audioEndTime -c:a libmp3lame $outPutName";


  


  And I've also tried :

  


  final cmd="-y -i \"$audioPath\" -ss $audioStartTime -to $audioEndTime -c copy $outPutName";


  


  But the error is still the same.
I'm using the ffmpeg_kit_flutter_full_gpl package.