Recherche avancée

Sur d’autres sites (7190)

• Announcing the first free software Blu-ray encoder

  25 avril 2010, par Dark Shikari — blu-ray, x264

  For many years it has been possible to make your own DVDs with free software tools. Over the course of the past decade, DVD creation evolved from the exclusive domain of the media publishing companies to something basically anyone could do on their home computer.

  But Blu-ray has yet to get that treatment. Despite the “format war” between Blu-ray and HD DVD ending over two years ago, free software has lagged behind. “Professional” tools for Blu-ray video encoding can cost as much as $100,000 and are often utter garbage. Here are two actual screenshots from real Blu-rays : I wish I was making this up.

  But today, things change. Today we take the first step towards a free software Blu-ray creation toolkit.

  Thanks to tireless work by Kieran Kunyha, Alex Giladi, Lamont Alston, and the Doom9 crowd, x264 can now produce Blu-ray-compliant video. Extra special thanks to The Criterion Collection for sponsoring the final compliance test to confirm x264′s Blu-ray compliance.

  With x264′s powerful compression, as demonstrated by the incredibly popular BD-Rebuilder Blu-ray backup software, it’s quite possible to author Blu-ray disks on DVD9s (dual-layer DVDs) or even DVD5s (single-layer DVDs) with a reasonable level of quality. With a free software encoder and less need for an expensive Blu-ray burner, we are one step closer to putting HD optical media creation in the hands of the everyday user.

  To celebrate this achievement, we are making available for download a demo Blu-ray encoded with x264, containing entirely free content !

  On this Blu-ray are the Open Movie Project films Big Buck Bunny and Elephant’s Dream, available under a Creative Commons license. Additionally, Microsoft has graciously provided about 6 minutes of lossless HD video and audio (from part of a documentary project) under a very liberal license. This footage rounds out the Blu-ray by adding some difficult live-action content in addition to the relatively compressible CGI footage from the Open Movie Project. Finally, we used this sound sample, available under a Creative Commons license.

  You may notice that the Blu-ray image is only just over 2GB. This is intentional ; we have encoded all the content on the disk at appropriate bitrates to be playable from an ordinary 4.7GB DVD. This should make it far easier to burn a copy of the Blu-ray, since Blu-ray burners and writable media are still relatively rare. Most Blu-ray players will treat a DVD containing Blu-ray data as a normal Blu-ray disc. A few, such as the Playstation 3, will not, but you can still play it as a data disc.

  Finally, note that (in accordance with the Blu-ray spec) the disc image file uses the UDF 2.5 filesystem, which may be incompatible with some older virtual drive and DVD burning applications. You’ll also need to play it on an actual Blu-ray player if you want to get the menus and such working correctly. If you’re looking to play it on a PC, a free trial of Arcsoft TMT is available here.

  What are you waiting for ? Grab a copy today !

  UPDATE : Here is an AVCHD-compliant version of the above, which should work better when burned on a DVD-5 instead of a BD-R. (mirror)
  

  What’s left before we have a fully free software Blu-ray creation toolkit ? Audio is already dealt with ; AC3 audio (aka Dolby Digital), the format used in DVD, is still supported by Blu-ray, and there are many free software AC3 encoders. The primary missing application is a free software Blu-ray authoring tool, to combine the video and audio streams to create a Blu-ray file structure with the menus, chapters, and so forth that we have all come to expect. But the hardest part is dealt with : we can now create compatible video and audio streams.

  In the meantime, x264 can be used to create streams to be authored using Blu-Print, Scenarist, Encore or other commercial authoring tools.

  More detailed documentation on the new Blu-ray support and how to use it can be found in the official commit message. Do keep in mind that you have to export to raw H.264 (not MKV or MP4) or else the buffering information will be slightly incorrect. Finally, also note that the encoding settings given as an example are not a good choice for general-purpose encoding : they are intentionally crippled by Blu-ray restrictions, which will significantly reduce compression for ordinary non-Blu-ray encoding.

  In addition to Blu-ray support, the aforementioned commit comes with a lot of fun extras :

  x264 now has native variable-framerate ratecontrol, which makes sure your encodes get a correct target bitrate and proper limiting of maximum bitrate even if the duration of every frame is different and the “framerate” is completely unknown. This helps a lot when encoding from variable-framerate container formats such as FLV and WMV, along with variable-framerate content such as anime.

  x264 now supports pulldown (telecine) in much the same fashion as it is handled in MPEG-2. The calling application can pass in flags representing how to display a frame, allowing easy transcoding from MPEG-2 sources with pulldown, such as broadcast television. The x264 commandline app contains some examples of these (such as the common 3:2 pulldown pattern).

  x264 now also exports HRD timing information, which is critical for compliant transport stream muxing. There is currently an active project to write a fully DVB-compatible free software TS muxer that will be able to interface with x264 for a seamless free software broadcast system. It will likely also be possible to repurpose this muxer as part of a free software Blu-ray authoring package.

  All of this is now available in the latest x264.

  http://mirror05.x264.nl/Dark/bluray%20fail.png

• Announcing the first free software Blu-ray encoder

  25 avril 2010, par Dark Shikari — blu-ray, x264

  For many years it has been possible to make your own DVDs with free software tools. Over the course of the past decade, DVD creation evolved from the exclusive domain of the media publishing companies to something basically anyone could do on their home computer.

  But Blu-ray has yet to get that treatment. Despite the “format war” between Blu-ray and HD DVD ending over two years ago, free software has lagged behind. “Professional” tools for Blu-ray video encoding can cost as much as $100,000 and are often utter garbage. Here are two actual screenshots from real Blu-rays : I wish I was making this up.

  But today, things change. Today we take the first step towards a free software Blu-ray creation toolkit.

  Thanks to tireless work by Kieran Kunyha, Alex Giladi, Lamont Alston, and the Doom9 crowd, x264 can now produce Blu-ray-compliant video. Extra special thanks to The Criterion Collection for sponsoring the final compliance test to confirm x264′s Blu-ray compliance.

  With x264′s powerful compression, as demonstrated by the incredibly popular BD-Rebuilder Blu-ray backup software, it’s quite possible to author Blu-ray disks on DVD9s (dual-layer DVDs) or even DVD5s (single-layer DVDs) with a reasonable level of quality. With a free software encoder and less need for an expensive Blu-ray burner, we are one step closer to putting HD optical media creation in the hands of the everyday user.

  To celebrate this achievement, we are making available for download a demo Blu-ray encoded with x264, containing entirely free content !

  On this Blu-ray are the Open Movie Project films Big Buck Bunny and Elephant’s Dream, available under a Creative Commons license. Additionally, Microsoft has graciously provided about 6 minutes of lossless HD video and audio (from part of a documentary project) under a very liberal license. This footage rounds out the Blu-ray by adding some difficult live-action content in addition to the relatively compressible CGI footage from the Open Movie Project. Finally, we used this sound sample, available under a Creative Commons license.

  You may notice that the Blu-ray image is only just over 2GB. This is intentional ; we have encoded all the content on the disk at appropriate bitrates to be playable from an ordinary 4.7GB DVD. This should make it far easier to burn a copy of the Blu-ray, since Blu-ray burners and writable media are still relatively rare. Most Blu-ray players will treat a DVD containing Blu-ray data as a normal Blu-ray disc. A few, such as the Playstation 3, will not, but you can still play it as a data disc.

  Finally, note that (in accordance with the Blu-ray spec) the disc image file uses the UDF 2.5 filesystem, which may be incompatible with some older virtual drive and DVD burning applications. You’ll also need to play it on an actual Blu-ray player if you want to get the menus and such working correctly. If you’re looking to play it on a PC, a free trial of Arcsoft TMT is available here.

  What are you waiting for ? Grab a copy today !

  UPDATE : Here is an AVCHD-compliant version of the above, which should work better when burned on a DVD-5 instead of a BD-R. (mirror)
  

  What’s left before we have a fully free software Blu-ray creation toolkit ? Audio is already dealt with ; AC3 audio (aka Dolby Digital), the format used in DVD, is still supported by Blu-ray, and there are many free software AC3 encoders. The primary missing application is a free software Blu-ray authoring tool, to combine the video and audio streams to create a Blu-ray file structure with the menus, chapters, and so forth that we have all come to expect. But the hardest part is dealt with : we can now create compatible video and audio streams.

  In the meantime, x264 can be used to create streams to be authored using Blu-Print, Scenarist, Encore or other commercial authoring tools.

  More detailed documentation on the new Blu-ray support and how to use it can be found in the official commit message. Do keep in mind that you have to export to raw H.264 (not MKV or MP4) or else the buffering information will be slightly incorrect. Finally, also note that the encoding settings given as an example are not a good choice for general-purpose encoding : they are intentionally crippled by Blu-ray restrictions, which will significantly reduce compression for ordinary non-Blu-ray encoding.

  In addition to Blu-ray support, the aforementioned commit comes with a lot of fun extras :

  x264 now has native variable-framerate ratecontrol, which makes sure your encodes get a correct target bitrate and proper limiting of maximum bitrate even if the duration of every frame is different and the “framerate” is completely unknown. This helps a lot when encoding from variable-framerate container formats such as FLV and WMV, along with variable-framerate content such as anime.

  x264 now supports pulldown (telecine) in much the same fashion as it is handled in MPEG-2. The calling application can pass in flags representing how to display a frame, allowing easy transcoding from MPEG-2 sources with pulldown, such as broadcast television. The x264 commandline app contains some examples of these (such as the common 3:2 pulldown pattern).

  x264 now also exports HRD timing information, which is critical for compliant transport stream muxing. There is currently an active project to write a fully DVB-compatible free software TS muxer that will be able to interface with x264 for a seamless free software broadcast system. It will likely also be possible to repurpose this muxer as part of a free software Blu-ray authoring package.

  All of this is now available in the latest x264.

  http://mirror05.x264.nl/Dark/bluray%20fail.png

• Visualizing Call Graphs Using Gephi

  1er septembre 2014, par Multimedia Mike — General

  When I was at university studying computer science, I took a basic chemistry course. During an accompanying lab, the teaching assistant chatted me up and asked about my major. He then said, “Computer science ? Well, that’s just typing stuff, right ?”

  My impulsive retort : “Sure, and chemistry is just about mixing together liquids and coming up with different colored liquids, as seen on the cover of my high school chemistry textbook, right ?”

  
  
  

  In fact, pure computer science has precious little to do with typing (as is joked in CS circles, computer science is about computers in the same way that astronomy is about telescopes). However, people who study computer science often pursue careers as programmers, or to put it in fancier professional language, software engineers.

  So, what’s a software engineer’s job ? Isn’t it just typing ? That’s where I’ve been going with this overly long setup. After thinking about it for long enough, I like to say that a software engineer’s trade is managing complexity.

  A few years ago, I discovered Gephi, an open source tool for graph and data visualization. It looked neat but I didn’t have much use for it at the time. Recently, however, I was trying to get a better handle on a large codebase. I.e., I was trying to manage the project’s complexity. And then I thought of Gephi again.

  Prior Work
  One way to get a grip on a large C codebase is to instrument it for profiling and extract details from the profiler. On Linux systems, this means compiling and linking the code using the -pg flag. After running the executable, there will be a gmon.out file which is post-processed using the gprof command.

  GNU software development tools have a reputation for being rather powerful and flexible, but also extremely raw. This first hit home when I was learning how to use the GNU tool for code coverage — gcov — and the way it outputs very raw data that you need to massage with other tools in order to get really useful intelligence.

  And so it is with gprof output. The output gives you a list of functions sorted by the amount of processing time spent in each. Then it gives you a flattened call tree. This is arranged as “during the profiled executions, function c was called by functions a and b and called functions d, e, and f ; function d was called by function c and called functions g and h”.

  How can this call tree data be represented in a more instructive manner that is easier to navigate ? My first impulse (and I don’t think I’m alone in this) is to convert the gprof call tree into a representation suitable for interpretation by Graphviz. Unfortunately, doing so tends to generate some enormous and unwieldy static images.

  Feeding gprof Data To Gephi
  I learned of Gephi a few years ago and recalled it when I developed an interest in gaining better perspective on a large base of alien C code. To understand what this codebase is doing for a particular use case, instrument it with gprof, gather execution data, and then study the code paths.

  How could I feed the gprof data into Gephi ? Gephi supports numerous graphing formats including an XML-based format named GEXF.

  Thus, the challenge becomes converting gprof output to GEXF.

  Which I did.

  Demonstration
  I have been absent from FFmpeg development for a long time, which is a pity because a lot of interesting development has occurred over the last 2-3 years after a troubling period of stagnation. I know that 2 big video codec developments have been HEVC (next in the line of MPEG codecs) and VP9 (heir to VP8’s throne). FFmpeg implements them both now.

  I decided I wanted to study the code flow of VP9. So I got the latest FFmpeg code from git and built it using the options "--extra-cflags=-pg --extra-ldflags=-pg". Annoyingly, I also needed to specify "--disable-asm" because gcc complains of some register allocation snafus when compiling inline ASM in profiling mode (and this is on x86_64). No matter ; ASM isn’t necessary for understanding overall code flow.

  After compiling, the binary ‘ffmpeg_g’ will have symbols and be instrumented for profiling. I grabbed a sample from this VP9 test vector set and went to work.

  ./ffmpeg_g -i vp90-2-00-quantizer-00.webm -f null /dev/null
  gprof ./ffmpeg_g > vp9decode.txt
  convert-gprof-to-gexf.py vp9decode.txt > /bigdisk/vp9decode.gexf
  

  Gephi loads vp9decode.gexf with no problem. Using Gephi, however, can be a bit challenging if one is not versed in any data exploration jargon. I recommend this Gephi getting starting guide in slide deck form. Here’s what the default graph looks like :

  
  
  

  Not very pretty or helpful. BTW, that beefy arrow running from mid-top to lower-right is the call from decode_coeffs_b -> iwht_iwht_4x4_add_c. There were 18774 from the former to the latter in this execution. Right now, the edge thicknesses correlate to number of calls between the nodes, which I’m not sure is the best representation.

  Following the tutorial slide deck, I at least learned how to enable the node labels (function symbols in this case) and apply a layout algorithm. The tutorial shows the force atlas layout. Here’s what the node neighborhood looks like for probing file type :

  
  
  

  Okay, so that’s not especially surprising– avprobe_input_format3 calls all of the *_probe functions in order to automatically determine input type. Let’s find that decode_coeffs_b function and see what its neighborhood looks like :

  
  
  

  That’s not very useful. Perhaps another algorithm might help. I select the Fruchterman–Reingold algorithm instead and get a slightly more coherent representation of the decoding node neighborhood :

  
  
  

  Further Work
  Obviously, I’m just getting started with this data exploration topic. One thing I would really appreciate in such a tool is the ability to interactively travel the graph since that’s what I’m really hoping to get out of this experiment– watching the code flows.

  Perhaps someone else can find better use cases for visualizing call graph data. Thus, I have published the source code for this tool at Github.