Recherche avancée

Sur d’autres sites (3054)

• libavcodec initialization to achieve real time playback with frame dropping when necessary

  20 octobre 2019, par Blake Senftner

  I have a C++ computer vision application linking with the ffmpeg libraries that provides frames from video streams to analysis routines. The idea being one can provide a moderately generic video stream identifier, and that video source will be decompressed and passed frame after frame to an analysis routine (which runs the user’s analysis functions.) The "moderately generic video identifier" covers 3 generic video stream types : paths to video files on disk, IP video streams (cameras or video streaming services), and USB webcam pins with desired format & rate.

  My current video player is generic as possible : video only, ignoring audio and other streams. It has a switch case for retrieving a stream’s frame rate based upon the stream’s source and codec, which is used to estimate the delay between decompressing frames. I’ve had many issues with trying to get reliable timestamps from the streams, so I am currently ignoring pts and dts. I know ignoring pts/dts is bad for variable frame rate streams. I plan to special case them later. The player currently checks to see if the last decompressed frame is more than 2 frames late (assuming a constant frame rate), and if so "drops the frame" - does not pass it to the user’s analysis routine.

  Essentially, the video player’s logic is determining when to skip frames (not pass them to the time consuming analysis routine) so the analysis is fed video frames in as close as possible to real time.

  I am looking for examples or discussions how one can initialize and/or maintain their AVFormatContext, AVStream, and AVCodecContext using (presumably but not limited to) AVDictionary options such that frame dropping as is necessary to maintain real time is performed at the libav libraries level, and not at my video player level. If achieving this requires separate AVDictionaies (or more) for each stream type and codec, then so be it. I am interested in understanding the pros and cons of both approachs : dropping frames at the player level or at the libav level.

  (When some analysis requires every frame, the existing player implementation with frame dropping disabled is fine. I suspect if I can get frame dropping to occur at the libav level, I’ll save the packet to frame decompression time as well, reducing the processing more than my current version.)

• Salty Game Music

  31 mai 2011, par Multimedia Mike — General

  Have you heard of Google’s Native Client (NaCl) project ? Probably not. Basically, it allows native code modules to run inside a browser (where ‘browser’ is defined pretty narrowly as ‘Google Chrome’ in this case). Programs are sandboxed so they aren’t a security menace (or so the whitepapers claim) but are allowed to access a variety of APIs including video and audio. The latter API is significant because sound tends to be forgotten in all the hullabaloo surrounding non-Flash web technologies. At any rate, enjoy NaCl while you can because I suspect it won’t be around much longer.

  After my recent work upgrading some old music synthesis programs to user more modern audio APIs, I got the idea to try porting the same code to run under NaCl in Chrome (first Nosefart, then Game Music Emu/GME). In this exercise, I met with very limited success. This blog post documents some of the pitfalls in my excursion.

  
  
  

  Infrastructure
  People who know me know that I’m rather partial — to put it gently — to straight-up C vs. C++. The NaCl SDK is heavily skewed towards C++. However, it does provide a Python tool called init_project.py which can create the skeleton of a project and can do so in C with the '-c' option :

  ./init_project.py -c -n saltynosefart
  

  This generates something that can be built using a simple ‘make’. When I added Nosefart’s C files, I learned that the project Makefile has places for project-necessary CFLAGS but does not honor them. The problem is that the generated Makefile includes a broader system Makefile that overrides the CFLAGS in the project Makefile. Going into the system Makefile and changing "CFLAGS =" -> "CFLAGS +=" solves this problem.

  Still, maybe I’m the first person to attempt building something in Native Client so I’m the first person to notice this ?

  Basic Playback
  At least the process to create an audio-enabled NaCl app is well-documented. Too bad it doesn’t seem to compile as advertised. According to my notes on the matter, I filled in PPP_InitializeModule() with the appropriate boilerplate as outlined in the docs but got a linker error concerning get_browser_interface().

  Plan B : C++
  Obviously, the straight C stuff is very much a second-class citizen in this NaCl setup. Fortunately, there is already that fully functional tone generator example program in the limited samples suite. Plan B is to copy that project and edit it until it accepts Nosefart/GME audio instead of a sine wave.

  The build system assumes all C++ files should have .cc extensions. I have to make some fixes so that it will accept .cpp files (either that, or rename all .cpp to .cc, but that’s not very clean).

  Making Noise
  You’ll be happy to know that I did successfully swap out the tone generator for either Nosefart or GME. Nosefart has a slightly fickle API that requires revving the emulator frame by frame and generating a certain number of audio samples. GME’s API is much easier to work with in this situation — just tell it how many samples it needs to generate and give it a pointer to a buffer. I played NES and SNES music play through this ad-hoc browser plugin, and I’m confident all the other supported formats would have worked if I went through the bother of converting the music data files into C headers to be included in the NaCl executable binaries (dynamically loading data via the network promised to be a far more challenging prospect reserved for phase 3 of the project).

  Portable ?
  I wouldn’t say so. I developed it on Linux and things ran fine there. I tried to run the same binaries on the Windows version of Chrome to no avail. It looks like it wasn’t even loading the .nexe files (NaCl executables).

  Thinking About The (Lack Of A) Future
  As I was working on this project, I noticed that the online NaCl documentation materialized explicit banners warning that my NaCl binaries compiled for Chrome 11 won’t work for Chrome 12 and that I need to code to the newly-released 0.3 SDK version. Not a fuzzy feeling. I also don’t feel good that I’m working from examples using bleeding edge APIs that feature deprecation as part of their naming convention, e.g., pp::deprecated::ScriptableObject().

  Ever-changing API + minimal API documentation + API that only works in one browser brand + requiring end user to explicitly enable feature = … well, that’s why I didn’t bother to release any showcase pertaining to this little experiment. Would have been neat, but I strongly suspect that this is yet another one of those APIs that Google decides to deprecate soon.

  See Also :

  • I eventually got a second wind and made this work using the C interface.

• Evolution #4300 (Nouveau) : Rendre facultatif les plugins-dist

  1er mars 2019, par Franck D

  Hello :-)
  Le commit de toutati https://zone.spip.net/trac/spip-zone/changeset/114194/spip-zone m’a fait repenser à ce vieux sujet (2009) https://blog.spip.net/SPIP-Core-tout-change-rien-ne-change.html
  Jluc avait fait un sujet à ce propos https://contrib.spip.net/plugins-dist-indispensables-ou-non
  Je me doute que cela représente un boulot de malade... mais ne faudrait’il pas dire au moins dans la F.A.Q. https://www.spip.net/fr_rubrique170.html les plugins-dist qui sont encore obligatoire ?

  J’ai fait le choix de "evolution" et non de "documentation" concernant le ticket, car sans ticket personne n’y pense :-D
  Pour spip 3.3, c’est sans doute un peu tard, mais à chaque sortie majeure, ne serait’il pas possible de rendre au moins "un" plugins-dist optionnel ?

  Pour le moment, en spip 3.3 [24240], il y a comme plugins-dist https://zone.spip.net/trac/spip-zone/browser/spip-zone/_core_/plugins
  Mais tous ne sont pas obligatoirement installer avec la dist (grenier, ?)
  Faudrait rayer les plugs qui sont en options

  Aide
  Archiviste
  Breves
  Compagnon
  Compresseur
  dev
  dist
  dump
  filtre_images
  forum
  grenier
  jquery_ui
  mediabox
  medias
  mots
  msie_compat
  organiseur
  petitions
  plan
  porte_plume
  revisions
  safehtml
  sites
  squelettes_par-rubrique
  statistique
  svp
  textwhell
  url_etendues
  vertèbres

  Franck