Recherche avancée

Sur d’autres sites (4983)

• Play a video with ffmpeg and SDL2 on a Raspberry Pi 5

  18 février 2024, par aforino

  I want to create a python script that decodes a h264 1080p video and outputs it via SDL2 on a Raspberry Pi 5. The Raspberry Pi 5 is able to play a h264 1080p video without problem using VLC. Total CPU load with VLC is about 10%. However decoding with ffmpeg and outputting via SDL2 uses around 70% CPU load. Since I want to be able to switch seamlessly between two output videos I will need to decode two videos at the same time. Therefore 70% CPU load for one transcoded 1080p video is not acceptable. How can I make the code more efficient and why is VLC so much more efficient ?

  


  This is my current python script :

  


  import numpy as np
import ffmpeg  # ffmpeg-python
import sdl2.ext

in_file = ffmpeg.input('bbb1080_x264.mp4', re=None)

width = 1920
height = 1080

process1 = (
    in_file
    .output('pipe:', format='rawvideo', pix_fmt='bgra')
    .run_async(pipe_stdout=True)
)

sdl2.ext.init()
window = sdl2.ext.Window("Hello World!", size=(width, height))
window.show()
windowsurface = sdl2.SDL_GetWindowSurface(window.window)
windowArray = sdl2.ext.pixels3d(windowsurface.contents)

sdl2.ext.mouse.hide_cursor()

while True:
    in_bytes = process1.stdout.read(width * height * 4)

    if not in_bytes:
        break

    in_frame = (
        np
        .frombuffer(in_bytes, np.uint8)
        .reshape([height, width, 4])
        .transpose(1, 0, 2)
    )

    for event in sdl2.ext.get_events():
        if event.type == sdl2.SDL_QUIT:
            exit()

    windowArray[:] = in_frame
    window.refresh()

process1.wait()


  


  Also it is interesting to note that when I start VLC on a Raspberry Pi 5 this is the output on the terminal

  


  [00007fff78c1a550] avcodec decoder error: cannot start codec (h264_v4l2m2m)
Fontconfig warning: ignoring UTF-8: not a valid region tag
[00007fff68002d70] gles2 generic error: parent window not available
[00007fff68002d70] xcb generic error: window not available
[00007fff680013f0] mmal_xsplitter vout display: Try drm
[00007fff68002d70] drm_vout generic: <<< OpenDrmVout: Fmt=I420
[00007fff68002d70] drm_vout generic error: Failed to get xlease`


  


  It indicates that VLC is not using the h264_v4l2m2m hardware acceleration.

  


• How to force A/V sync using mkvmerge and external time-codes ?

  19 avril 2017, par b..

  Background

  I’m working on a project where video and audio are algorithmic interpretations of an MKV source file where I use ffmpeg -ss and -t to extract a particular region of audio and video to separate files. I use scene changes in the video in the audio process (i.e. the audio changes on video scene change), so sync is crucial.

  Audio is 48khz, using 512 sample blocks.
  Video is 23.976fps (I also tried 24).

  I store the frame onset of sceneChanges in a file in terms of cumulative blocks :

  blocksPerFrame = (48000 / 512) / 23.976
  
  sceneOnsetBlock = sceneOnsetFrame*blocksPerFrame

  I use these blocks in my audio code to treat the samples associated with each scene as a group.

  When I combine the audio and video back together (currently using ffmpeg to generate mp4(v) mp3(a) in an MKV container), the audio and video start off in sync but increasingly drifts until it ends up being 35 seconds off. The worst part is that the audio lag is nonlinear ! By non-linear, I mean that if I plot the lag against the location of that lag in time, I don’t get a line, but what you see in the image below). I can’t just shift or scale the audio to fit the video because of this nonlinearity. I cannot figure out the cause of this nonlinearly increasing audio delay ; I’ve double and triple checked my math.

  

  Since I know the exact timing of scene changes, I should be able to generate "external timecodes" (from the blocks above) for mkvmerge to perfectly sync the output !

  Subquestions :

  1. Is this the best approach (beyond trying to figure out what went wrong in the first place) ? As I’m using my video frames as a
     reference, if I use the scene changes as timecodes for the audio,
     will it force the video to match the audio or vice versa ? I’m much less concerned with the duration than the sync. The video was much more laborious to produce, so I’d rather loose some sound than some frames.

  2. I’m not clear on what numbers to use in the timecodes file.
     According to mkvmerge documentation "For video this is exactly
     one frame, for audio this is one packet of the specific audio type."
     Since I’m using MP3, what is the packet size ? Ideally, I could specify a packetsize (in the audio-encoder ?) that matches my block size (512) to keep things consistent and simple. Can I do this with ffmpeg ?

  Thank you !

• Command failed ffmpeg in Lambda function (Node js)

  14 février 2019, par Arun

  I am trying to convert a video file to an audio file inside the lambda function. But I keep getting FFmpeg command failed error. I put FFmpeg binary exe file inside the project directory. but still, I am getting the same error. I tried this ( Lambda not connecting to ffmpeg ) but my issue is not solving yet. Any helps ?? thanks,

  Here is my lambda function code

  process.env.PATH = process.env.PATH + ':/tmp/';
  
  process.env['FFMPEG_PATH'] = '/tmp/ffmpeg';
  
  const BIN_PATH = process.env['LAMBDA_TASK_ROOT'];
  
  process.env['PATH'] = process.env['PATH'] + ':' + BIN_PATH;
  
  
  
  const fs = require('fs');
  
  const AWS = require('aws-sdk');
  
  
  
  AWS.config.update({
  
      region : 'us-east-2'
  
  });
  
  const s3 = new AWS.S3({apiVersion: '2006-03-01'});
  
  
  
  
  
  exports.handler = (event, context, callback) => {
  
      require('child_process').exec(
  
          'cp /var/task/ffmpeg /tmp/.; chmod 755 /tmp/ffmpeg;',
  
          function (error, stdout, stderr) {
  
              if (error) {
  
                  console.log('Erro occured',error);
  
              } else {
  
                  var ffmpeg = require('ffmpeg');
  
                  var params = {
  
                      Bucket: "bucket_name",
  
                      Key: event.Records[0].s3.object.key
  
                  };
  
                  s3.getObject(params, function(err, data) {
  
                      if (err) {
  
                          console.log("Error", err);
  
                      }
  
                      fs.writeFile("/tmp/vid.mp4", data.Body, function (err) {
  
                          if (err) console.log(err.code, "-", err.message);
  
                          return callback(err);
  
                      }, function() {
  
                          try {
  
                              var stats = fs.statSync("/tmp/vid.mp4");
  
                              console.log("size of the file1 ", stats["size"]);
  
                              try {
  
                                  console.log("Yeah");
  
                                  var process = new ffmpeg('/tmp/vid.mp4');
  
                                  process.then(function (video) {
  
                                      // Callback mode
  
                                      console.log("video function ", video);
  
                                      video.fnExtractSoundToMP3('/tmp/video.mp3', function (error, file) {
  
                                          if (!error)
  
                                              console.log('Audio file: ' + file);
  
                                          else console.log('error video ', error);
  
                                      });
  
                                  }, function (err) {
  
                                      console.log('Error: ' + err);
  
                                  });
  
                              } catch (e) {
  
                                  console.log(e.code);
  
                                  console.log(e.msg);
  
                              }
  
                          } catch (e) {
  
                              console.log("file is not complete", e);
  
                          }
  
                      });
  
                      return callback(err);
  
                  });
  
              }
  
          }
  
      )
  
  }

  Error message

  { Error: Command failed: ffmpeg -i /tmp/vid.mp4 -vn -ar 44100 -ac 2 -ab 192 -f mp3 /tmp/video.mp3
  
  
  
      at ChildProcess.exithandler (child_process.js:275:12)
  
      at emitTwo (events.js:126:13)
  
      at ChildProcess.emit (events.js:214:7)
  
      at maybeClose (internal/child_process.js:925:16)
  
      at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
  
      killed: false,
  
          code: null,
  
      signal: 'SIGSEGV',
  
      cmd: 'ffmpeg -i /tmp/vid.mp4 -vn -ar 44100 -ac 2 -ab 192 -f mp3 /tmp/video.mp3' }