Recherche avancée

Sur d’autres sites (8964)

• Pipe video frames from ffmpeg to canvas without loading the entire video into memory

  1er janvier 2024, par Aviato

  I am working on a project that involves frame manipulation and I decided to choose node canvas API for that. I used to work with OpenCV Python and there was a cv2.VideoCapture class that takes a video as input and prepares to read the frames of the video and we can loop through the frames one at a time without having to load all the frames at once in memory.
Now I tried a lot of ways to replicate the same using ffmpeg, i.e. trying to load frames from a video in an ordered, but "on-demand," fashion.

  


  I tried using ffmpeg as a child process to process frames and standout the frames.

  


  const spawnProcess = require('child_process').spawn,
    ffmpeg = spawnProcess('ffmpeg', [
        '-i', 'test.mp4',
        '-vcodec', 'png',
        '-f', 'rawvideo',
        '-s', '1920*1080', // size of one frame
        'pipe:1'
    ]);
ffmpeg.stdout.on('data', (data) => {
    try {
        // console.log(tf.node.decodeImage(data).shape)
        console.log(`${++i} frames read`)
        //context.drawImage(data, 0, 0, width, height)
        
        
    } catch(e) {
        console.log(e)
    } 
})


  


  The value in the console shows something around 4000 + console logs, but the video only had 150 frames, after much investigating and console logging the data, I found that was buffer data, and it's not processing it for each frame. The on-data function returns the buffer data in an unstructured way
I want to read frames from a video and process each one at a time in memory, I don't want to hold all the frames at once in memory or in the filesystem.

  


  

  


  I also want to pipe the frames in a format that could be rendered on top of a canvas using drawImage

  


• tools : Don't include the direct library names when linking

  28 septembre 2023, par Martin Storsjö

  tools : Don't include the direct library names when linking
  When linking the main tools, the object files to link are set up
  
  via the variable OBJS-<name>, but for the tools, we've only
  
  used the target's list of dependencies.
  In most cases, this has been fine, but it has caused specifying
  
  the libraries to link in a duplicate fashion ; the linking command
  
  has looked like this :
      $CC -Llibavutil ... tools/tool.o libavutil/libavutil.a -lavutil
  Normally, the libraries to link are handled with "-Llibavutil -lavutil" ;
  
  when linking the main fftools, this is how they are linked.
  In the case of the binaries under the "tools" directory (within the
  
  make variable TOOLS), we've passed the full set of dependencies
  
  to the linker, via $^, which does contain the names of the
  
  dependency libraries as well.
  When libraries are built as regular static libraries, or shared
  
  unix libraries, this has all worked fine. When libraries are
  
  built as DLLs for Windows, though, the norm is not to pass the
  
  actual DLL to the linker, but an import library.
  Mingw tools generally can handle linking directly against a DLL
  
  as well, but MSVC tools don't support that, and error out with
  
  a very cryptic error message :
      libavdevice\avdevice.dll : fatal error LNK1107 : invalid or corrupt file : cannot read at 0x2D8
  By omitting these parts of the dependencies, linking of these tool
  
  executables succeed in MSVC builds with shared libraries enabled.
  Signed-off-by : Martin Storsjö <martin@martin.st>
  

  • [DH] Makefile

• avcodec/h264dec : Fix data race when updating decode_error_flags

  12 septembre 2023, par Andreas Rheinhardt

  avcodec/h264dec : Fix data race when updating decode_error_flags
  When using multi-threaded decoding, every decoding thread
  
  has its own DBP consisting of H264Pictures and each of these
  
  points to its own AVFrames. They are synced during
  
  update_thread_context via av_frame_ref() and therefore
  
  the threads actually decoding (as well as all the others)
  
  must not modify any field that is copied by av_frame_ref()
  
  after ff_thread_finish_setup().
  Yet this is exactly what happens when an error occurs
  
  during decoding and the AVFrame's decode_error_flags are updated.
  
  Given that these errors only become apparent during decoding,
  
  this can't be set before ff_thread_finish_setup() without
  
  defeating the point of frame-threading ; in practice,
  
  this meant that the decoder did not set these flags correctly
  
  in case frame-threading was in use. (This means that e.g.
  
  the ffmpeg cli tool fails to output its "corrupt decoded frame"
  
  message in a nondeterministic fashion.)
  This commit fixes this by adding a new H264Picture field
  
  that is actually propagated across threads ; the field
  
  is an AVBufferRef* whose data is an atomic_int ; it is
  
  atomic in order to allow multiple threads to update it
  
  concurrently and not to provide synchronization
  
  between the threads setting the field and the thread
  
  ultimately returning the AVFrame.
  This unfortunately has the overhead of one allocation
  
  per H264Picture (both the original one as well as
  
  creating a reference to an existing one), even in case
  
  of no errors. In order to mitigate this, an AVBufferPool
  
  has been used and only if frame-threading is actually
  
  in use. This expense will be removed as soon as
  
  a proper API for refcounted objects (not based upon
  
  AVBuffer) is in place.
  Signed-off-by : Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
  

  • [DH] libavcodec/h264_picture.c
  • [DH] libavcodec/h264_slice.c
  • [DH] libavcodec/h264dec.c
  • [DH] libavcodec/h264dec.h