Recherche avancée

Sur d’autres sites (7341)

• How to extract motion vectors from h264 without a full decode on the CPU

  25 septembre 2020, par Adrian May

  I'm trying to use my nose as a pointing device. The plan is to encode the video stream from a webcam pointed at my face as h264 or the like, get the motion vectors, cook the numbers a bit and chuck them into /dev/uinput to make the mouse pointer move about. The uinput bit was easy.

  


  This has to work with zero discernable latency. This, for instance :

  


  #!/bin/bash
[ -p pipe.mkv ] || mkfifo pipe.mkv
ffmpeg -y -rtbufsize 1M -s 640x360 -vcodec mjpeg -i /dev/video0 -c h264_nvenc pipe.mkv &
ffplay -flags2 +export_mvs -vf codecview=mv=pf+bf+bb pipe.mkv


  


  shows that the vectors are there but with a latency of several seconds which is unusable in a mouse. I know that the first ffmpeg step is working very fast by using the GPU, so either the pipe or the h264 decode in the second step is introducing the latency.

  


  I tried MV Tractus (same as mpegflow I think) in a similar pipe arrangement and it was also very slow. They do a full h264 decode on the CPU and I think that's the problem cos I can see them imposing a lot of load on one CPU. If the pipe had caused the delay by buffering badly then the CPU wouldn't have been loaded. I guess ffplay also did the decoding on the CPU and I couldn't persuade it not to, but it only wants to draw arrows which are no use to me.

  


  I think there are several approaches, and I'd like advice on which would be best, or if there's something even better I don't know about. I could :

  


  

  • Decode in hardware and get the motion vectors. So far this has failed. I tried combining ffmpeg's extract_mvs.c and hw_decode.c samples but no motion vectors turn up. vdpau is the only decoder I got working on my linux box. I have a nvidia gpu.

  


  • Do a minimal parse of the h264 to fish out the motion vectors only, ignoring all the other data. I think this would mean putting some kind of "motion only" option in libav's parser, but I'm not at all familiar with that code.

  


  • Find some other h264 parsing library that has said option and also unpacks the container.

  


  • Forget about hardware accelerated encoding and use a stripped down encoder to make only the motion vectors on either CPU or GPU. I suspect this would be slow cos I think calculating the motion vectors is the hardest part of the algorithm.

  


  


  I'm tending towards the second option but I need some help figuring out where in the libav code to do it.

  


• Video editing multi customer host in server instances

  1er août 2020, par General Omosco

  I created a multi conference live stream web app using rtc and ffmpeg C api for media stream, videos/images/texts files mixed up together in realtime sending the output to rtmp multiple destinations, but unfortunately my server could not handle more than 2 conference room smoothly.

  


  The server which couldn't handle more than two conference room running ffmpeg api

  


  It is Comfort plan in OVH
Type : vps,
Processor : 4 vCore,
Ram : 8gb

  


  Usage
CPU : 100% //playing smoothly or not.
Ram : 3%

  


  My question here is that. Is it possible to be creating self isolated instance on fly and automatically deploy the app in the instance for each customer ? And the host provider that can accept that.

  


• Using FFMPEG library with iPhone SDK for video encoding

  7 août 2020, par user203349

  I need to encode several pictures shot by the iPhone camera into a mp4 video file and I know FFMPEG can do this (the application TimeLapser and ReelMoments do it already). I plan to use this in my app iMotion (available in the appstore).

  


  I successfully install and compile the ffmpeg for the iphone SDK with this link :
http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2009-October/076618.html

  


  But now I'm stuck here in my XCode project. What should I do next to use the FFMPEG library for video encoding ? The Apple documentation about external library using is very light and I just can find any tutorial on the web which explains how to do this.