Recherche avancée

Sur d’autres sites (8550)

• Pipe video frames from ffmpeg to canvas without loading the entire video into memory

  1er janvier 2024, par Aviato

  I am working on a project that involves frame manipulation and I decided to choose node canvas API for that. I used to work with OpenCV Python and there was a cv2.VideoCapture class that takes a video as input and prepares to read the frames of the video and we can loop through the frames one at a time without having to load all the frames at once in memory.
Now I tried a lot of ways to replicate the same using ffmpeg, i.e. trying to load frames from a video in an ordered, but "on-demand," fashion.

  


  I tried using ffmpeg as a child process to process frames and standout the frames.

  


  const spawnProcess = require('child_process').spawn,
    ffmpeg = spawnProcess('ffmpeg', [
        '-i', 'test.mp4',
        '-vcodec', 'png',
        '-f', 'rawvideo',
        '-s', '1920*1080', // size of one frame
        'pipe:1'
    ]);
ffmpeg.stdout.on('data', (data) => {
    try {
        // console.log(tf.node.decodeImage(data).shape)
        console.log(`${++i} frames read`)
        //context.drawImage(data, 0, 0, width, height)
        
        
    } catch(e) {
        console.log(e)
    } 
})


  


  The value in the console shows something around 4000 + console logs, but the video only had 150 frames, after much investigating and console logging the data, I found that was buffer data, and it's not processing it for each frame. The on-data function returns the buffer data in an unstructured way
I want to read frames from a video and process each one at a time in memory, I don't want to hold all the frames at once in memory or in the filesystem.

  


  

  


  I also want to pipe the frames in a format that could be rendered on top of a canvas using drawImage

  


• How to read frames of a video and write them on another video output using FFMPEG and nodejs

  29 décembre 2023, par Aviato

  I am working on a project where I need to process video frames one at a time in Node.js. I aim to avoid storing all frames in memory or the filesystem due to resource constraints. I plan to use the ffmpeg from child processes for video processing.
I tried reading a video file and then output frames of it in the filesystem first for testing purposes :-

  


  const ffmpegProcess = spawn('ffmpeg', [
  '-i', videoFile,
  'testfolder/%04d.png' // Output frames to stdout
]);


  


  and the above code works fine, it saves the video frames as png files in the filesystem. Now instead of saving them in the file system, I want to read the frames on at a time and use a image manipulation library and than write the final edited frames to another video as output

  


  I tried this :-

  


  const ffmpegProcess = spawn('ffmpeg', [
  '-i', videoFile,
  'pipe:1' // Output frames to stdout
]);

const ffmpegOutputProcess = spawn('ffmpeg', [
  '-i', '-',
  'outputFileName.mp4'
  ]);

ffmpegProcess.stdout.on('data', (data) => {
  // Process the frame data as needed
  console.log('Received frame data:');
  ffmpegOutputProcess.stdin.write(data)
});

ffmpegProcess.on('close', (code) => {
  if (code !== 0) {
    console.error(`ffmpeg process exited with code ${code}`);
  } else {
    console.log('ffmpeg process successfully completed');
    
  }
});

// Handle errors
ffmpegProcess.on('error', (err) => {
  console.error('Error while spawning ffmpeg:', err);
});


  


  But when I tried above code and also some other modifications in the input and output suffix in the command I got problems as below :-

  


  

  1. ffmpeg process exited with code 1

  


  2. The final output video was corrupted when trying to initializing the filters for commands :-

  


  


  
const ffmpegProcess = spawn('ffmpeg', [
 '-i', videoFile,
 '-f', 'rawvideo',
 '-pix_fmt', 'rgb24',
 'pipe:1' // Output frames to stdout
]);

const ffmpegOutputCommand = [
 '-f', 'rawvideo',
 '-pix_fmt', 'rgb24',
 '-s', '1920x1080',
 '-r', '30',
 '-i', '-',
 '-c:v', 'libx264',
 '-pix_fmt', 'yuv420p',
 outputFileName
];


  


  Thank you so much in advance :)

  


• Download youtube video as stream Readable object

  26 décembre 2023, par Abraam Emad

  in this function it download youtube video as a file out.mp4 on hard disk i need to download it as a Readable Object to upload it

  


  private async downloadVideo(videoId: string) {
// Buildin with nodejs
const cp = require('child_process');
const readline = require('readline');
// External modules
const ytdl = require('ytdl-core');
const ffmpeg = require('ffmpeg-static');
// Global constants
const ref = `https://www.youtube.com/watch?v=${videoId}`;
const tracker = {
  start: Date.now(),
  audio: { downloaded: 0, total: Infinity },
  video: { downloaded: 0, total: Infinity },
  merged: { frame: 0, speed: '0x', fps: 0 },
};

// Get audio and video streams
const audio = ytdl(ref, { quality: 'highestaudio' })
  .on('progress', (_, downloaded, total) => {
    tracker.audio = { downloaded, total };
  });
const video = ytdl(ref, { quality: 'highestvideo' })
  .on('progress', (_, downloaded, total) => {
    tracker.video = { downloaded, total };
  });

// Prepare the progress bar
let progressbarHandle = null;
const progressbarInterval = 1000;
const showProgress = () => {
  readline.cursorTo(process.stdout, 0);
  const toMB = i => (i / 1024 / 1024).toFixed(2);

  process.stdout.write(`Audio  | ${(tracker.audio.downloaded / tracker.audio.total * 100).toFixed(2)}% processed `);
  process.stdout.write(`(${toMB(tracker.audio.downloaded)}MB of ${toMB(tracker.audio.total)}MB).${' '.repeat(10)}\n`);

  process.stdout.write(`Video  | ${(tracker.video.downloaded / tracker.video.total * 100).toFixed(2)}% processed `);
  process.stdout.write(`(${toMB(tracker.video.downloaded)}MB of ${toMB(tracker.video.total)}MB).${' '.repeat(10)}\n`);

  process.stdout.write(`Merged | processing frame ${tracker.merged.frame} `);
  process.stdout.write(`(at ${tracker.merged.fps} fps => ${tracker.merged.speed}).${' '.repeat(10)}\n`);

  process.stdout.write(`running for: ${((Date.now() - tracker.start) / 1000 / 60).toFixed(2)} Minutes.`);
  readline.moveCursor(process.stdout, 0, -3);
};

// Start the ffmpeg child process
const ffmpegProcess = cp.spawn(ffmpeg, [
  // Remove ffmpeg's console spamming
  '-loglevel', '8', '-hide_banner',
  // Redirect/Enable progress messages
  '-progress', 'pipe:3',
  // Set inputs
  '-i', 'pipe:4',
  '-i', 'pipe:5',
  // Map audio & video from streams
  '-map', '0:a',
  '-map', '1:v',
  // Keep encoding
  '-c:v', 'copy',
  // Define output file
  '-f', 'mpegts', // Use MPEG-TS format for streaming
  'out.mp4'
], {
  windowsHide: true,
  stdio: [
    /* Standard: stdin, stdout, stderr */
    'inherit', 'inherit', 'inherit',
    /* Custom: pipe:3, pipe:4, pipe:5 */
    'pipe', 'pipe', 'pipe',
  ],
});
ffmpegProcess.on('close', () => {
  console.log('done');
  // Cleanup
  process.stdout.write('\n\n\n\n');
  clearInterval(progressbarHandle);
});
// Link streams
// FFmpeg creates the transformer streams and we just have to insert / read data
ffmpegProcess.stdio[3].on('data', chunk => {
  // Start the progress bar
  if (!progressbarHandle) progressbarHandle = setInterval(showProgress, progressbarInterval);
  // Parse the param=value list returned by ffmpeg
  const lines = chunk.toString().trim().split('\n');
  const args: any = {};
  for (const l of lines) {
    const [key, value] = l.split('=');
    args[key.trim()] = value.trim();
  }
  tracker.merged = args;
});
audio.pipe(ffmpegProcess.stdio[4]);
video.pipe(ffmpegProcess.stdio[5]);


  


  }`