Recherche avancée

Sur d’autres sites (7575)

• Copying a FFMPEG AVFrame to a buffer for use with SDL

  13 août 2021, par ryan

  for the past couple of days I've been trying to get a decoded AVFrame* converted to an SDL_Surface. In order to do that, I need to get the pure RGBA data from the frame. Here is the code involved with copying.

  


   std::uint32_t numBytes = av_image_get_buffer_size(AV_PIX_FMT_BGR32,
                                                   codecCtx->width,
                                                   codecCtx->height,
                                                   32);
/// ...
std::vector nBuf(numBytes);
auto e = av_image_copy_to_buffer(nBuf.data(), nBuf.size(),
                                 rgbFrame->data, rgbFrame->linesize,
                                 AV_PIX_FMT_BGR32, codecCtx->width,
                                 codecCtx->height, 32);
auto surf = SDL_CreateRGBSurfaceFrom(reinterpret_cast(nBuf.data()),
                                     codecCtx->width, codecCtx->height,
                                     32, 4 * codecCtx->width, 0x000000FF, 0x0000FF00,
                                     0x00FF0000,0xFF000000);
                


  


  However, this creates a corrupted and messed up image. I've also tried using 24bit RGB as well with the same results, and I've also tried using std::copy and std::memcpy instead of using av_image_copy_to_buffer, like this :

  


  for(auto i = 0; i < codecCtx->height; i++)
    std::memcpy(nBuf.data() + i * codecCtx->width, rgbFrame->data[0] + i * rgbFrame->linesize[0], codecCtx->width * 4);


  


  with similarly corrupted results. I've also tried feeding SDL_CreateRGBSurfaceFrom rgbFrame->data[0] directly, and I got the same corrupted image.

  


  I know that rgbFrame is correct, I output it to a raw RGB format and the image was right.

  


  I need to copy this data manually since I cannot have SDL_Renderer and SDL_Textures do this for me since I am using SDL_GPU as my renderer ; the SDL_Surface is just so I can convert the image to a GPU_Image.

  


  If anyone could show me what I am doing wrong or show me an alternate way to get this as a GPU_Image that would be great !

  


• Batch mixing audio, given timestamps. Multiple offsets, only two sounds. How to do it efficiently ?

  3 août 2021, par Evil

  I have two stereo sounds, 1.wav and 2.wav, these sounds are less than 1 second long and list of timestamps (miliseconds from start of recording). Recording of pure video (recording.mp4) is several hours long and there are thousands (20 000 - 30 000) of timestamps per sounds.

  


  I want to convert list of timestamps and sounds into one recording, merging it with video. The part of merging audio with video is easy with ffmpeg, so this is not part of the question.

  


  The list of timestamps is tsv, for example :

  


  


  1201\t1.wav
  
1501\t2.wav
1603\t1.wav
  
and so on, up to 50 000

  


  


  I can convert it to anything, I am generating this file.

  


  I have seen mixing sound with padding and mixing audio to existing video, but I have to batch process a lots of samples, running sox that many times is not feasible. Mere constructing input for ffmpeg or sox is a cumbersome task.

  


  


  sox -M f2.wav f3.wav f1.wav out.wav delay 4 4 8 8 remix 1,3,5 2,4,6
  
(assuming stereo), or

  


  


  


  sox -m f1.wav "|sox f2.wav -p pad 4" "|sox f3.wav -p pad 8" out.wav

  


  


  Cool for three files. Not feasible for 50 000+. First one needs to read file multiple times (even if it is the same one) and remix channels. Second executes 50 000 sox invocations, also reading the same two files (1.wav, 2.wav) over and over.

  


  I do not use any effects on sounds. There is no explicit support in sox to take one input and play it multiple times (echo / echos destroys the material). Also creating padding or delay takes a lot of time. FFMPEG also needs long query to make it happen.

  


  Since muxing two files is easy, I have tried to record two sounds separately, but still it takes a lot of time to process.

  


  Is there simpler / faster way ?

  


  Taking advice from fdcpp, since wav is PCM coded I also consider writing C program to parse it. I will update code, when I am done.
  
This extends question : is there way to encode offsets in wav format ?

  


• Mixing audio, given timestamps. How to do it efficiently ?

  2 août 2021, par Evil

  I have two stereo sounds, 1.wav and 2.wav, these sounds are less than 1 second long and list of timestamps (miliseconds from start of recording). Recording of pure video (recording.mp4) is several hours long and there are thousands (20 000 - 30 000) of timestamps per sounds.

  


  I want to convert list of timestamps and sounds into one recording, merging it with video. The part of merging audio with video is easy with ffmpeg, so this is not part of the question.

  


  The list of timestamps is tsv, for example :

  


  


  1201\t1.wav
  
1501\t2.wav
1603\t1.wav
  
and so on, up to 50 000

  


  


  I can convert it to anything, I am generating this file.

  


  I have seen mixing sound with padding and mixing audio to existing video, but I have to batch process a lots of samples, running sox that many times is not feasible. Mere onstructing input for ffmpeg or sox is a cumbersome task.

  


  I do not use any effects on sounds. There is no explicit support in sox to take one input and play it multiple times (echo / echos destroys the material). Also creating padding or delay takes a lot of time. FFMPEG also needs long query to make it happen.

  


  Since muxing two files is easy, I have tried to record two sounds separately, but still it takes a lot of time to process.

  


  Is there simpler / faster way ?