Recherche avancée

Sur d’autres sites (4094)

• FFmpeg and Code Coverage Tools

  21 août 2010, par Multimedia Mike — FATE Server, Python

  Code coverage tools likely occupy the same niche as profiling tools : Tools that you’re supposed to use somewhere during the software engineering process but probably never quite get around to it, usually because you’re too busy adding features or fixing bugs. But there may come a day when you wish to learn how much of your code is actually being exercised in normal production use. For example, the team charged with continuously testing the FFmpeg project, would be curious to know how much code is being exercised, especially since many of the FATE test specs explicitly claim to be "exercising XYZ subsystem".

  The primary GNU code coverage tool is called gcov and is probably already on your GNU-based development system. I set out to determine how much FFmpeg source code is exercised while running the full FATE suite. I ran into some problems when trying to use gcov on a project-wide scale. I spackled around those holes with some very ad-hoc solutions. I’m sure I was just overlooking some more obvious solutions about which you all will be happy to enlighten me.

  Results
  I’ve learned to cut to the chase earlier in blog posts (results first, methods second). With that, here are the results I produced from this experiment. This Google spreadsheet contains 3 sheets : The first contains code coverage stats for a bunch of FFmpeg C files sorted first by percent coverage (ascending), then by number of lines (descending), thus highlighting which files have the most uncovered code (ffserver.c currently tops that chart). The second sheet has files for which no stats were generated. The third sheet has "problems". These files were rejected by my ad-hoc script.

  Here’s a link to the data in CSV if you want to play with it yourself.

  Using gcov with FFmpeg
  To instrument a program for gcov analysis, compile and link the target program with the -fprofile-arcs and -ftest-coverage options. These need to be applied at both the compile and link stages, so in the case of FFmpeg, configure with :

    ./configure \
      —extra-cflags="-fprofile-arcs -ftest-coverage" \
      —extra-ldflags="-fprofile-arcs -ftest-coverage"
  

  The building process results in a bunch of .gcno files which pertain to code coverage. After running the program as normal, a bunch of .gcda files are generated. To get coverage statistics from these files, run 'gcov sourcefile.c'. This will print some basic statistics as well as generate a corresponding .gcov file with more detailed information about exactly which lines have been executed, and how many times.

  Be advised that the source file must either live in the same directory from which gcov is invoked, or else the path to the source must be given to gcov via the '-o, --object-directory' option.

  Resetting Statistics
  Statistics in the .gcda are cumulative. Should you wish to reset the statistics, doing this in the build directory should suffice :

    find . -name "*.gcda" | xargs rm -f
  

  Getting Project-Wide Data
  As mentioned, I had to get a little creative here to get a big picture of FFmpeg code coverage. After building FFmpeg with the code coverage options and running FATE,

  for file in `find . -name "*.c"` \
  do \
    echo "*****" $file \
    gcov -o `dirname $file` `basename $file` \
  done > ffmpeg-code-coverage.txt 2>&1
  

  After that, I ran the ffmpeg-code-coverage.txt file through a custom Python script to print out the 3 CSV files that I later dumped into the Google Spreadsheet.

  Further Work
  I’m sure there are better ways to do this, and I’m sure you all will let me know what they are. But I have to get the ball rolling somehow.

  There’s also TestCocoon. I’d like to try that program and see if it addresses some of gcov’s shortcomings (assuming they are indeed shortcomings rather than oversights).

  Source for script : process-gcov-slop.py

  PLAIN TEXT

  PYTHON :

  1. # !/usr/bin/python
  2.  
  3. import re
  4.  
  5. lines = open("ffmpeg-code-coverage.txt").read().splitlines()
  6. no_coverage = ""
  7. coverage = "filename, % covered, total lines\n"
  8. problems = ""
  9.  
  10. stats_exp = re.compile(’Lines executed :(\d+\.\d+)% of (\d+)’)
  11. for i in xrange(len(lines)) :
  12.   line = lines[i]
  13.   if line.startswith("***** ") :
  14.     filename = line[line.find(’./’)+2 :]
  15.     i += 1
  16.     if lines[i].find(":cannot open graph file") != -1 :
  17.       no_coverage += filename + ’\n’
  18.     else :
  19.       while lines[i].find(filename) == -1 and not lines[i].startswith("***** ") :
  20.         i += 1
  21.       try :
  22.         (percent, total_lines) = stats_exp.findall(lines[i+1])[0]
  23.         coverage += filename + ’, ’ + percent + ’, ’ + total_lines + ’\n’
  24.       except IndexError :
  25.         problems += filename + ’\n’
  26.  
  27. open("no_coverage.csv", ’w’).write(no_coverage)
  28. open("coverage.csv", ’w’).write(coverage)
  29. open("problems.csv", ’w’).write(problems)

• Linux Media Player Survey Circa 2001

  2 septembre 2010, par Multimedia Mike — General

  Here’s a document I scavenged from my archives. It was dated September 1, 2001 and I now publish it 9 years later. It serves as sort of a time capsule for the state of media player programs at the time. Looking back on this list, I can’t understand why I couldn’t find MPlayer while I was conducting this survey, especially since MPlayer is the project I eventually started to work for a few months after writing this piece.

  For a little context, I had been studying multimedia concepts and tech for a year and was itching to get my hands dirty with practical multimedia coding. But I wanted to tackle what I perceived as unsolved problems– like playback of proprietary codecs. I didn’t want to have to build a new media playback framework just to start working on my problems. So I surveyed the players available to see which ones I could plug into and use as a testbed for implementing new decoders.

  Regarding Real Player, I wrote : “We’re trying to move away from the proprietary, closed-source “solutions”. Heh. Was I really an insufferable open source idealist back in the day ?

  Anyway, here’s the text with some Where are they now ? commentary [in brackets] :

  ---

  Towards an All-Inclusive Media Playing Solution for Linux

  I don’t feel that the media playing solutions for Linux set their sights high enough, even though they do tend to be quite ambitious.

  I want to create a media player for Linux that can open a file, figure out what type of file it is (AVI, MOV, etc.), determine the compression algorithms used to encode the audio and video chunks inside (MPEG, Cinepak, Sorenson, etc.) and replay the file using the best audio, video, and CPU facilities available on the computer.

  Video and audio playback is a solved problem on Linux ; I don’t wish to solve that problem again. The problem that isn’t solved is reliance on proprietary multimedia solutions through some kind of WINE-like layer in order to decode compressed multimedia files.

  Survey of Linux solutions for decoding proprietary multimedia
  updated 2001-09-01

  AVI Player for XMMS
  This is based on Avifile. All the same advantages and limitations apply.
  [Top Google hit is a Freshmeat page that doesn’t indicate activity since 2001-2002.]

  Avifile
  This player does a great job at taking apart AVI and ASF files and then feeding the compressed chunks of multimedia data through to the binary Win32 decoders.

  The program is written in C++ and I’m not very good at interpreting that kind of code. But I’m learning all over again. Examining the object hierarchy, it appears that the designers had the foresight to include native support for decoders that are compiled into the program from source code. However, closer examination reveals that there is support for ONE source decoder and that’s the “decoder” for uncompressed data. Still, I tried to manipulate this routine to accept and decode data from other codecs but no dice. It’s really confounding. The program always crashes when I feed non-uncompressed data through the source decoder.
  [Lives at http://avifile.sourceforge.net/ ; not updated since 2006.]

  Real Player
  There’s not much to do with this since it is closed source and proprietary. Even though there is a plugin architecture, that’s not satisfactory. We’re trying to move away from the proprietary, closed-source “solutions”.
  [Still kickin’ with version 11.]

  XAnim
  This is a well-established Unix media player. To his credit, the author does as well as he can with the resources he has. In other words, he supports the non-proprietary video codecs well, and even has support for some proprietary video codecs through binary-only decoders.

  The source code is extremely difficult to work with as the author chose to use the X coding format which I’ve never seen used anywhere else except for X header files. The infrastructure for extending the program and supporting other codecs and file formats is there, I suppose, but I would have to wrap my head around the coding style. Maybe I can learn to work past that. The other thing that bothers me about this program is the decoding approach : It seems that each video decoder includes routines to decompress the multimedia data into every conceivable RGB and YUV output format. This seems backwards to me ; it seems better to have one decoder function that decodes the data into its native format it was compressed from (e.g., YV12 for MPEG data) and then pass that data to another layer of the program that’s in charge of presenting the data and possibly converting it if necessary. This layer would encompass highly-optimized software conversion routines including special CPU-specific instructions (e.g., MMX and SSE) and eliminate the need to place those routines in lots of other routines. But I’m getting ahead of myself.
  [This one was pretty much dead before I made this survey, the most recent update being in 1999. Still, we owe it much respect as the granddaddy of Unix multimedia playback programs.]

  Xine
  This seems like a promising program. It was originally designed to play MPEGs from DVDs. It can also play MPEG files on a hard drive and utilizes the Xv extensions for hardware YUV playback. It’s also supposed to play AVI files using the same technique as Avifile but I have never, ever gotten it to work. If an AVI file has both video and sound, the binary video decoder can’t decode any frames. If the AVI file has video and no sound, the program gets confused and crashes, as far as I can tell.

  Still, it’s promising, and I’ve been trying to work around these crashes. It doesn’t yet have the type of modularization I’d like to see. Right now, it tailored to suit MPEG playback and AVI playback is an afterthought. Still, it appears to have a generalized interface for dropping in new file demultiplexers.

  I tried to extend the program for supporting source decoders by rewriting w32codec.c from scratch. I’m not having a smooth time of it so far. I’m able to perform some manipulations on the output window. However, I can’t get the program to deal with an RGB image format. It has trouble allocating an RGB surface with XvShmCreateImage(). This isn’t suprising, per my limited knowledge of X which is that Xv applies to YUV images, but it could also apply to RGB images as well. Anyway, the program should be able to fall back on regular RGB pixmaps if that Xv call fails.

  Right now, this program is looking the most promising. It will take some work to extend the underlying infrastructure, but it seems doable since I know C quite well and can understand the flow of this program, as opposed to Avifile and its C++. The C code also compiles about 10 times faster.
  [My home project for many years after a brief flirtation with MPlayer. It is still alive ; its latest release was just a month ago.]

  XMovie
  This library is a Quicktime movie player. I haven’t looked at it too extensively yet, but I do remember looking at it at one point and reading the documentation that said it doesn’t support key frames. Still, I should examine it again since they released a new version recently.
  [Heroine Virtual still puts out some software but XMovie has not been updated since 2005.]

  XMPS
  This program compiles for me, but doesn’t do much else. It can play an MP3 file. I have been able to get MPEG movies to play through it, but it refuses to show the full video frame, constricting it to a small window (obviously a bug).
  [This project is hosted on SourceForge and is listed with a registration date of 2003, well after this survey was made. So the project obviously lived elsewhere in 2001. Meanwhile, it doesn’t look like any files ever made it to SF for hosting.]

  XTheater
  I can’t even get this program to compile. It’s supposed to be an MPEG player based on SMPEG. As such, it probably doesn’t hold much promise for being easily extended into a general media player.
  [Last updated in 2002.]

  GMerlin
  I can’t get this to compile yet. I have a bug report in to the dev group.
  [Updated consistently in the last 9 years. Last update was in February of this year. I can’t find any record of my bug report, though.]

• How Much H.264 In Each Encoder ?

  8 septembre 2010, par Multimedia Mike — General

  Thanks to my recent experiments with code coverage tools, I have a powerful new — admittedly somewhat specious — method of comparing programs. For example, I am certain that I have read on more than one occasion that Apple’s H.264 encoder sucks compared to x264 due, at least in part, to the Apple encoder’s alleged inability to exercise all of H.264′s features. I wonder how to test that claim ?

  Experiment
  Use code coverage tools to determine which H.264 encoder uses the most features.

  Assumptions

  • Movie trailers hosted by Apple will all be encoded with the same settings using Apple’s encoder.
  • Similarly, Yahoo’s movie trailers will be encoded with consistent settings using an unknown encoder.
  • Encoding a video using FFmpeg’s libx264-slow setting will necessarily throw a bunch of H.264′s features into the mix (I really don’t think this assumption holds much water, but I also don’t know what “standard” x264 settings are).

  Methodology

  • Grab a random Apple-hosted 1080p movie trailer and random Yahoo-hosted 1080p movie trailer from Dave’s Trailer Page.
  • Use libx264/FFmpeg with the ‘slow’ preset to encode Big Buck Bunny 1080p from raw PNG files.
  • Build FFmpeg with code coverage enabled.
  • Decode each file to raw YUV, ignore audio decoding, generate code coverage statistics using gcovr, reset stats after each run by deleting *.gcda files.

  Results

  • x264 1080p video : 9968 / 134203 lines
  • Apple 1080p trailer : 9968 / 134203 lines
  • Yahoo 1080p trailer : 9914 / 134203 lines

  I also ran this old x264-encoded file (ImperishableNightStage6Low.mp4) through the same test. It demonstrated the most code coverage with 10671 / 134203 lines.

  Conclusions
  Conclusions ? Ha ! Go ahead and jump all over this test. I’m already fairly confident that it’s impossible (or maybe just very difficult) to build a single H.264-encoded video that exercises every feature that FFmpeg’s decoder supports. For example, is it possible for a file to use both CABAC and CAVLC entropy methods ? If it’s possible, does any current encoder do that ?