Recherche avancée

Sur d’autres sites (11442)

• Encode Frames to Video with C Library

  31 juillet 2018, par NetherGranite

  For the sake of continuity, let us assume "RGB values" are the following :

  typedef struct RGB {
  
      uint8_t r, g, b;
  
  } rgb;

  However, if you feel that a different color space is more appropriate for this question, please use that instead.

  How might I go about writing 2D arrays of RGB values to a video in C given an output format and framerate ?

  Before I continue, I should specify that I wish to be able to do this all within one program. I am trying to add functionality to an application that would allow it to compile videos frame by frame without having to leave it.

  Additionally, my needs for this functionality are extremely basic ; I simply need to be able to set individual pixels to certain colors.

  The closest I have come to a solution so far is the C library FFmpeg. Allow me to describe what I was able to learn on my own :

  After looking through its documentation, I came across the function avcodec_send_frame(avctx, frame), whose parameters are of the types AVCodexContext* and const AVFrame* respectively. If these are not the right tools for what I am trying to do, please ignore the rest of the question and instead point me towards what I should be using.

  However, I do not know which fields of avctx and frame must be set manually and which do not. The reason I assume some do not is because both are extremely large structures, but correct me if I am wrong.

  Question 1 : What values of an AVCodecContext and AVFrame must be set ? Of these, what is/are the recommended value(s) for each of them ?

  Additionally, I was only able to find instructions on how to initialize an AVFrame (using av_frame_alloc() and av_frame_get_buffer()) but not for an AVCodexConstant.

  Question 2 : Is there a proper way to initialize an AVCodexConstant ? And just in case, is the method of initializing an AVFrame described above correct ? Do any of the fields of either have a proper method of initialization ?

  Also, I was not able to find official documentation on how to take this AVCodexConstant (which I assume contains the video information) and turn it into a video. I apologize if the documentation for this is easy to find and I just missed it.

  Question 3 : How do I turn an AVCodexConstant into a file of a given format ?

  And, given my limited knowledge :

  Question 4 : Are there any other parts to this process that I am missing, and do I have any of the above parts wrong ?

  ---

  Please keep in mind that I found out about FFmpeg for the first time very recently, and as a result, I am a complete beginner to this. Additionally, my experience with C is very limited, so I would greatly appreciate it if you could note which files need to be included with #include.

  Feel free to even go as far as recommending something other than FFmpeg, just as long as it is written in C. I do not need power-user options, but I would greatly prefer flexibility in what audio and video file types the library can handle.

  ---

  Addressing Potential Duplicates

  I appologize for how long this section is ; I just want to have my bases covered. I heavily apologize, however, if this is in fact a duplicate of a question that I was just unable to find.

  • ffmpeg C API documentation/tutorial [closed] — This question was too open-ended and received answers pointing the asker towards a tutorial at dranger.com, a tutorial that confusingly muddied the waters by focusing heavily on a graphics library of choice. Please do not take this as me saying it is bad ; I am just enough of a beginner that I could not wade through it all.
  • Encoding frames to video with ffmpeg — Although this question seems to have been asking the same thing, it is geared towards Unreal Engine 4, and the asker provided sample code, making it difficult for me to understand which of parts of the accepted answer were necessary for me and which were not.
  • How to write frames to a video file ? — While this also asked the same thing, the accepted answer simply provides a command instead of an explanation of code.
  • YUV Raw frames to video stream — While the accepted answer for this question is a command, the question states that it is looking for a way to encode frames generated by C++ code. Is there some way to run commands in code that I haven’t been able to find ?
  • Converting sequenced frames to video — Not only is the asker’s code written in Python, but it also seems to use already-existing image files as frames.
  • How to write bitmaps as frames to H.264 with x264 in C\C++ ? — The accepted answer seems to describe a process that would take multiple applications, but I could be wrong as I am enough of a beginner that I am not sure exactly what it means other than Step 3.
  • How to write bitmaps as frames to Ogg Theora in C\C++ ? — Although it isn’t a problem that the question specifies the ogg format, it is a problem that the accepted answer suggests libtheora, which appears to only work with ogg files.

• Decoding VP8 On A Sega Dreamcast

  20 février 2011, par Multimedia Mike — Sega Dreamcast, VP8

  I got Google’s libvpx VP8 codec library to compile and run on the Sega Dreamcast with its Hitachi/Renesas SH-4 200 MHz CPU. So give Google/On2 their due credit for writing portable software. I’m not sure how best to illustrate this so please accept this still photo depicting my testbench Dreamcast console driving video to my monitor :

  
  
  

  Why ? Because I wanted to try my hand at porting some existing software to this console and because I tend to be most comfortable working with assorted multimedia software components. This seemed like it would be a good exercise.

  You may have observed that the video is blue. Shortest, simplest answer : Pure laziness. Short, technical answer : Path of least resistance for getting through this exercise. Longer answer follows.

  Update : I did eventually realize that the Dreamcast can work with YUV textures. Read more in my followup post.

  Process and Pitfalls
  libvpx comes with a number of little utilities including decode_to_md5.c. The first order of business was porting over enough source files to make the VP8 decoder compile along with the MD5 testbench utility.

  Again, I used the KallistiOS (KOS) console RTOS (aside : I’m still working to get modern Linux kernels compiled for the Dreamcast). I started by configuring and compiling libvpx on a regular desktop Linux system. From there, I was able to modify a number of configuration options to make the build more amenable to the embedded RTOS.
  
  I had to create a few shim header files that mapped various functions related to threading and synchronization to their KOS equivalents. For example, KOS has a threading library cleverly named kthreads which is mostly compatible with the more common pthread library functions. KOS apparently also predates stdint.h, so I had to contrive a file with those basic types.

  So I got everything compiled and then uploaded the binary along with a small VP8 IVF test vector. Imagine my surprise when an MD5 sum came out of the serial console. Further, visualize my utter speechlessness when I noticed that the MD5 sum matched what my desktop platform produced. It worked !

  Almost. When I tried to decode all frames in a test vector, the program would invariably crash. The problem was that the file that manages motion compensation (reconinter.c) needs to define MUST_BE_ALIGNED which compiles byte-wise block copy functions. This is necessary for CPUs like the SH-4 which can’t load unaligned data. Apparently, even ARM CPUs these days can handle unaligned memory accesses which is why this isn’t a configure-time option.

  Showing The Work
  I completed the first testbench application which ran the MD5 test on all 17 official IVF test vectors. The SH-4/Dreamcast version aces the whole suite.

  However, this is a video game console, so I had better be able to show the decoded video. The Dreamcast is strictly RGB— forget about displaying YUV data directly. I could take the performance hit to convert YUV -> RGB. Or, I could just display the intensity information (Y plane) rendered on a random color scale (I chose blue) on an RGB565 texture (the DC’s graphics hardware can also do paletted textures but those need to be rearranged/twiddled/swizzled).

  Results
  So, can the Dreamcast decode VP8 video in realtime ? Sure ! Well, I really need to qualify. In the test depicted in the picture, it seems to be realtime (though I wasn’t enforcing proper frame timings, just decoding and displaying as quickly as possible). Obviously, I wasn’t bothering to properly convert YUV -> RGB. Plus, that Big Buck Bunny test vector clip is only 176x144. Obviously, no audio decoding either.

  So, realtime playback, with a little fine print.

  On the plus side, it’s trivial to get the Dreamcast video hardware to upscale that little blue image to fullscreen.

  I was able to tally the total milliseconds’ worth of wall clock time required to decode the 17 VP8 test vectors. As you can probably work out from this list, when I try to play a 320x240 video, things start to break down.

  1. Processed 29 176x144 frames in 987 milliseconds.
  2. Processed 49 176x144 frames in 1809 milliseconds.
  3. Processed 49 176x144 frames in 704 milliseconds.
  4. Processed 29 176x144 frames in 255 milliseconds.
  5. Processed 49 176x144 frames in 339 milliseconds.
  6. Processed 48 175x143 frames in 2446 milliseconds.
  7. Processed 29 176x144 frames in 432 milliseconds.
  8. Processed 2 1432x888 frames in 2060 milliseconds.
  9. Processed 49 176x144 frames in 1884 milliseconds.
  10. Processed 57 320x240 frames in 5792 milliseconds.
  11. Processed 29 176x144 frames in 989 milliseconds.
  12. Processed 29 176x144 frames in 740 milliseconds.
  13. Processed 29 176x144 frames in 839 milliseconds.
  14. Processed 49 175x143 frames in 2849 milliseconds.
  15. Processed 260 320x240 frames in 29719 milliseconds.
  16. Processed 29 176x144 frames in 962 milliseconds.
  17. Processed 29 176x144 frames in 933 milliseconds.

• avcodec : add DFPWM1a codec

  8 mars 2022, par Jack Bruienne

  avcodec : add DFPWM1a codec
  From the wiki page (https://wiki.vexatos.com/dfpwm) :
  
  > DFPWM (Dynamic Filter Pulse Width Modulation) is an audio codec
  
  > created by Ben “GreaseMonkey” Russell in 2012, originally to be used
  
  > as a voice codec for asiekierka's pixmess, a C remake of 64pixels.
  
  > It is a 1-bit-per-sample codec which uses a dynamic-strength one-pole
  
  > low-pass filter as a predictor. Due to the fact that a raw DPFWM decoding
  
  > creates a high-pitched whine, it is often followed by some post-processing
  
  > filters to make the stream more listenable.
  It has recently gained popularity through the ComputerCraft mod for
  
  Minecraft, which added support for audio through this codec, as well as
  
  the Computronics expansion which preceeded the official support. These
  
  both implement the slightly adjusted 1a version of the codec, which is
  
  the version I have chosen for this patch.
  This patch adds a new codec (with encoding and decoding) for DFPWM1a.
  
  The codec sources are pretty simple : they use the reference codec with
  
  a basic wrapper to connect it to the FFmpeg AVCodec system.
  To clarify, the codec does not have a specific sample rate - it is
  
  provided by the container (or user), which is typically 48000, but has
  
  also been known to be 32768. The codec does not specify channel info
  
  either, and it's pretty much always used with one mono channel.
  
  However, since it appears that libavcodec expects both sample rate and
  
  channel count to be handled by either the codec or container, I have
  
  made the decision to allow multiple channels interleaved, which as far
  
  as I know has never been used, but it works fine here nevertheless. The
  
  accompanying raw format has a channels option to set this. (I expect
  
  most users of this will not use multiple channels, but it remains an
  
  option just in case.)
  This patch will be highly useful to ComputerCraft developers who are
  
  working with audio, as it is the standard format for audio, and there
  
  are few user-friendly encoders out there, and even fewer decoders. It
  
  will streamline the process for importing and listening to audio,
  
  replacing the need to write code or use tools that require very
  
  specific input formats.
  You may use the CraftOS-PC program (https://www.craftos-pc.cc) to test
  
  out DFPWM playback. To use it, run the program and type this command :
  
  "attach left speaker" Then run "speaker play <file.dfpwm>" for each file.
  
  The app runs in a sandbox, so files have to be transferred in first ;
  
  the easiest way to do this is to simply drag the file on the window.
  
  (Or copy files to the folder at https://www.craftos-pc.cc/docs/saves.)
  Sample DFPWM files can be generated with an online tool at
  
  https://music.madefor.cc. This is the current best way to encode DFPWM
  
  files. Simply drag an audio file onto the page, and it will encode it,
  
  giving a download link on the page.
  I've made sure to update all of the docs as per Developer§7, and I've
  
  tested it as per section 8. Test files encoded to DFPWM play correctly
  
  in ComputerCraft, and other files that work in CC are correctly decoded.
  
  I have also verified that corrupt files do not crash the decoder - this
  
  should theoretically not be an issue as the result size is constant with
  
  respect to the input size.
  Signed-off-by : Jack Bruienne <jackbruienne@gmail.com>
  

  • [DH] Changelog
  • [DH] MAINTAINERS
  • [DH] doc/general_contents.texi
  • [DH] libavcodec/Makefile
  • [DH] libavcodec/allcodecs.c
  • [DH] libavcodec/codec_desc.c
  • [DH] libavcodec/codec_id.h
  • [DH] libavcodec/dfpwmdec.c
  • [DH] libavcodec/dfpwmenc.c
  • [DH] libavcodec/utils.c
  • [DH] libavcodec/version.h