Recherche avancée

Sur d’autres sites (8774)

• Losing audio quality when merging multiple videos with FFmpeg

  30 mai 2021, par Charly

  I am trying to merge multiple videos (a few seconds each) into one, using FFmpeg.

  


  Merging all at once

  


  I first tried merging all videos with one FFmpeg command, but it only seems to work with up to a certain amount of videos. On my Raspberry Pi 4 (2BG RAM), it worked for 16 videos and broke (ran out of memory ?) while merging 36 videos (command and ouput below)

  


  ffmpeg -i 1Rbrdn1Xp.mp4 -i G7eYqdFPb.mp4 -i -Xo-6ABm3.mp4 -i 6As9IlqbW.mp4 -i tz9tVKFA-.mp4
-i MS6DHZoJE.mp4 -i CeJYs99-G.mp4 -i Hsv5X48IC.mp4 -i Gm_DtjNU4.mp4 -i PZytbInZL.mp4
-i qsIJuvKO2.mp4 -i c5wxGk_Fx.mp4 -i j4K1qA7ih.mp4 -i AIpVWuhGV.mp4 -i yiLk0snSh.mp4
-i q1zmytaXZ.mp4 -i VRwQvLxaW.mp4 -i 0_3yLrokH.mp4 -i fW28LPQsk.mp4 -i La6WsGeE8.mp4
-i FlroQnCHq.mp4 -i c7SiL_qSU.mp4 -i aaGTUMJXD.mp4 -i hvXYBaCSM.mp4 -i CMDaFrWM4.mp4
-i _YAGDRyQN.mp4 -i RQXyshs_o.mp4 -i 96pKlIZxW.mp4 -i HVOOeCUYG.mp4 -i pJFxq7BTx.mp4
-i VxMJQqbK5.mp4 -i mkgbViPTI.mp4 -i zrjj93b5c.mp4 -i AL9oG1F36.mp4 -i IsUnC0NPb.mp4
-i Qhj2bhab9.mp4 -y -filter_complex concat=n=36:v=1:a=1 -vsync 2 kn-dQSN-v.mp4


  


  ffmpeg version 4.1.6-1~deb10u1+rpt1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Raspbian 8.3.0-6+rpi1)
(...)
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1Rbrdn1Xp.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.20.100
  Duration: 00:00:08.08, start: 0.000000, bitrate: 602 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 533 kb/s, 57.62 fps, 60 tbr, 15360 tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 22050 Hz, mono, fltp, 64 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'G7eYqdFPb.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.20.100
  Duration: 00:00:21.00, start: 0.000000, bitrate: 310 kb/s
    Stream #1:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 229 kb/s, 59.09 fps, 60 tbr, 15360 tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #1:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 22050 Hz, mono, fltp, 72 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
(...)
Stream mapping:
  Stream #0:0 (h264) -> concat:in0:v0
  Stream #0:1 (aac) -> concat:in0:a0
  Stream #1:0 (h264) -> concat:in1:v0
  Stream #1:1 (aac) -> concat:in1:a0
(...)
  concat:out:v0 -> Stream #0:0 (libx264)
  concat:out:a0 -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
[libx264 @ 0x17cb5e0] using SAR=1/1   0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A    
[libx264 @ 0x17cb5e0] using cpu capabilities: ARMv6 NEON
[libx264 @ 0x17cb5e0] profile High, level 3.2
[libx264 @ 0x17cb5e0] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
(...)
Output #0, mp4, to 'kn-dQSN-v.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.20.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 60 fps, 15360 tbn, 60 tbc (default)
    Metadata:
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 22050 Hz, mono, fltp, 69 kb/s (default)
    Metadata:
      encoder         : Lavc58.35.100 aac
frame=15952 fps= 52 q=31.0 size=   10240kB time=00:04:33.02 bitrate= 307.3kbits/s speed=0.844x
[h264 @ 0x3472d80] get_buffer() failed40kB time=00:04:33.02 bitrate= 307.3kbits/s speed=0.844x    
[h264 @ 0x3472d80] thread_get_buffer() failed
[h264 @ 0x3472d80] decode_slice_header error
[h264 @ 0x3472d80] no frame!
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x34f54d0] get_buffer() failed
[h264 @ 0x34f54d0] thread_get_buffer() failed
[h264 @ 0x34f54d0] decode_slice_header error
[h264 @ 0x34f54d0] no frame!
[h264 @ 0x3472d80] get_buffer() failed
[h264 @ 0x3472d80] thread_get_buffer() failed
(...)
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x3472d80] reference picture missing during reorder
[h264 @ 0x3472d80] Missing reference picture, default is 65660
[h264 @ 0x34c4540] mmco: unref short failure
[h264 @ 0x34f54d0] get_buffer() failed08kB time=00:04:45.37 bitrate= 316.0kbits/s speed=0.845x    
[h264 @ 0x34f54d0] thread_get_buffer() failed
[h264 @ 0x34f54d0] decode_slice_header error
[h264 @ 0x34f54d0] no frame!
[h264 @ 0x3472d80] decode_slice_header error
[h264 @ 0x3472d80] no frame!
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x34f54d0] mmco: unref short failure
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x3472d80] get_buffer() failed
[h264 @ 0x3472d80] thread_get_buffer() failed
[h264 @ 0x3472d80] decode_slice_header error
[h264 @ 0x3472d80] no frame!
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x3472d80] get_buffer() failed
[h264 @ 0x3472d80] thread_get_buffer() failed
[h264 @ 0x3472d80] decode_slice_header error
[h264 @ 0x3472d80] no frame!
[h264 @ 0x34f54d0] reference picture missing during reorder
    Last message repeated 1 times
[h264 @ 0x34f54d0] Missing reference picture, default is 65708
    Last message repeated 1 times
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x34dccb0] reference picture missing during reorder
[h264 @ 0x34dccb0] Missing reference picture, default is 65716
[h264 @ 0x3472d80] mmco: unref short failure
[h264 @ 0x34f54d0] decode_slice_header error
[h264 @ 0x34f54d0] no frame!
Error while decoding stream #15:0: Invalid data found when processing inputbits/s speed=0.845x    
[h264 @ 0x3480010] get_buffer() failed
[h264 @ 0x3480010] thread_get_buffer() failed
[h264 @ 0x3480010] decode_slice_header error
[h264 @ 0x3480010] no frame!
[h264 @ 0x34c4540] decode_slice_header error
[h264 @ 0x34c4540] no frame!
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x3480010] mmco: unref short failure
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x34c4540] get_buffer() failed
[h264 @ 0x34c4540] thread_get_buffer() failed
[h264 @ 0x34c4540] decode_slice_header error
[h264 @ 0x34c4540] no frame!
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x34c4540] get_buffer() failed
[h264 @ 0x34c4540] thread_get_buffer() failed
[h264 @ 0x34c4540] decode_slice_header error
[h264 @ 0x34c4540] no frame!
[h264 @ 0x34f54d0] decode_slice_header error
[h264 @ 0x34f54d0] no frame!
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x3480010] reference picture missing during reorder
    Last message repeated 1 times
[h264 @ 0x3480010] Missing reference picture, default is 65772
    Last message repeated 1 times
Error while decoding stream #15:0: Invalid data found when processing input
[h264 @ 0x3472d80] reference picture missing during reorder
[h264 @ 0x3472d80] Missing reference picture, default is 65780
[h264 @ 0x34c4540] mmco: unref short failure
av_interleaved_write_frame(): Cannot allocate memory
[mp4 @ 0x295c7e0] Application provided duration: 3689347710924763697 / timestamp: 6291454 is out of range for mov/mp4 format
Segmentation fault


  


  Merging videos 2 by 2

  


  I then tried gradually merging videos into one, using multiple FFmpeg commands.

  


  # Pseudocode
final = merge(clip1, clip2)
final = merge(final, clip3)
final = merge(final, clip4)
final = merge(final, clip5)
...


  


  with each command, looking like this :

  


  ffmpeg -i 1Rbrdn1Xp.mp4 -i G7eYqdFPb.mp4 -y -filter_complex concat=n=2:v=1:a=1 -vsync 2 UMVDl9jki.mp4


  


  ffmpeg -i UMVDl9jki.mp4 -i -Xo-6ABm3.mp4 -y -filter_complex concat=n=2:v=1:a=1 -vsync 2 azE1Uombp.mp4


  


  This works but there is a loss in audio quality on the first clips, and it is slowly improving over time. I am guessing this is due to the first videos being merged more times than the last ones. You can see this effect happen in this video (weird artefacts in the voice).

  


  ---

  


  So, am I doing something wrong ?
  
Is there a way of merging a lot of videos, without running out of memory or worsening the audio quality ?

  


• Subtitling Sierra RBT Files

  2 juin 2016, par Multimedia Mike — Game Hacking

  This is part 2 of the adventure started in my Subtitling Sierra VMD Files post. After I completed the VMD subtitling, The Translator discovered a wealth of animation files in a format called RBT (this apparently stands for “Robot” but I think “Ribbit” format could be more fun). What are we going to do ? We had come so far by solving the VMD subtitling problem for Phantasmagoria. It would be a shame if the effort ground to a halt due to this.

  Fortunately, the folks behind the ScummVM project already figured out enough of the format to be able to decode the RBT files in Phantasmagoria.

  In the end, I was successful in creating a completely standalone tool that can take a Robot file and a subtitle file and create a new Robot file with subtitles. The source code is here (subtitle-rbt.c). Here’s what the final result looks like :

  
  
  “What’s in the refrigerator ?” I should note at this juncture that I am not sure if this particular Robot file even has sound or dialogue since I was conducting these experiments on a computer with non-working audio.
  

  The RBT Format
  I have created a new MultimediaWiki page describing the Robot Animation format based on the ScummVM source code. I have not worked with a format quite like this before. These are paletted animations which consist of a sequence of independent frames that are designed to be overlaid on top of static background. Because of these characteristics, each frame encodes its own unique dimensions and origin coordinate within the frame. While the Phantasmagoria VMD files are usually 288×144 (which are usually double-sized for the benefit of a 640×400 Super VGA canvas), these frames are meant to be plotted on a game field that was roughly 576×288 (288×144 doublesized).
  

  For example, 2 minimalist animation frames from a desk investigation Robot file :

  
  
  100×147

  
  101×149
  

  As for compression, my first impression was that the algorithm was the same as VMD. This is wrong. It evidently uses an unmodified version of a standard algorithm called Lempel-Ziv-Stac (LZS). It shows up in several RFCs and was apparently used in MS-DOS’s transparent disk compression scheme.

  Approach
  Thankfully, many of the lessons I learned from the previous project are applicable to this project, including : subtitle library interfacing, subtitling in the paletted colorspace, and replacing encoded frames from the original file instead of trying to create a new file.

  Here is the pitch for this project :

  • Create a C program that can traverse through an input file, piece by piece, and generate an output file. The result of this should be a bitwise identical file.
  • Adapt the LZS compression decoding algorithm from ScummVM into the new tool. Make the tool dump raw Portable NetMap (PNM) files of varying dimensions and ensure that they look correct.
  • Compress using LZS.
  • Stretch the frames and draw subtitles.
  • More compression. Find the minimum window for each frame.

  Compression
  Normally, my first goal is to decompress the video and store the data in a raw form. However, this turned out to be mathematically intractable. While the format does support both compressed and uncompressed frames (even though ScummVM indicates that the uncompressed path is yet unexercised), the goal of this project requires making the frames so large that they overflow certain parameters of the file.

  A Robot file has a sequence of frames and 2 tables describing the size of each frame. One table describes the entire frame size (audio + video) while the second table describes just the video frame size. Since these tables only use 16 bits to specify a size, the maximum frame size is 65536 bytes. Leaving space for the audio portion of the frame, this only leaves a per-frame byte budget of about 63000 bytes for the video. Expanding the frame to 576×288 (165,888 pixels) would overflow this limit.

  Anyway, the upshot is that I needed to compress the data up front.

  Fortunately, the LZS compressor is pretty straightforward, at least if you have experience writing VLC-oriented codecs. While the algorithm revolves around back references, my approach was to essentially write an RLE encoder. My compressor would search for runs of data (plentiful when I started to stretch the frame for subtitling purposes). When a run length of n=3 or more of the same pixel is found, encode the pixel by itself, and then store a back reference of offset -1 and length (n-1). It took a little while to iron out a few problems, but I eventually got it to work perfectly.

  I have to say, however, that the format is a little bit weird in how it codes very large numbers. The length encoding is somewhat Golomb-like, i.e., smaller values are encoded with fewer bits. However, when it gets to large numbers, it starts encoding counts of 15 as blocks of 1111. For example, 24 is bigger than 7. Thus, emit 1111 into the bitstream and subtract 8 from 23 -> 16. Still bigger than 15, so stuff another 1111 into the bitstream and subtract 15. Now we’re at 1, so stuff 0001. So 24 is 11111111 0001. 12 bits is not too horrible. But the total number of bytes (value / 30). So a value of 300 takes around 10 bytes (80 bits) to encode.

  Palette Slices
  As in the VMD subtitling project, I took the subtitle color offered in the subtitle spec file as a suggestion and used Euclidean distance to match to the closest available color in the palette. One problem, however, is that the palette is a lot smaller in these animations. According to my notes, for the set of animations I scanned, only about 80 colors were specified, starting at palette index 55. I hypothesize that different slices of the palette are reserved for different uses. E.g., animation, background, and user interface. Thus, there is a smaller number of colors to draw upon for subtitling purposes.

  Scaling
  One bit of residual weirdness in this format is the presence of a per-frame scale factor. While most frames set this to 100 (100% scale), I have observed 70%, 80%, and 90%. ScummVM is a bit unsure about how to handle these, so I am as well. However, I eventually realized I didn’t really need to care, at least not when decoding and re-encoding the frame. Just preserve the scale factor. I intend to modify the tool further to take scale factor into account when creating the subtitle.

  The Final Resolution
  Right around the time that I was composing this post, The Translator emailed me and notified me that he had found a better way to subtitle the Robot files by modifying the scripts, rendering my entire approach moot. The result is much cleaner :

  
  
  Turns out that the engine supported subtitles all along
  

  It’s a good thing that I enjoyed the challenge or I might be annoyed at this point.

  See Also

  • Subtitling Sierra VMD Files : My effort to subtitle the main FMV files found in Sierra games.

  The post Subtitling Sierra RBT Files first appeared on Breaking Eggs And Making Omelettes.

• Subtitling Sierra RBT Files

  2 juin 2016, par Multimedia Mike — Game Hacking

  This is part 2 of the adventure started in my Subtitling Sierra VMD Files post. After I completed the VMD subtitling, The Translator discovered a wealth of animation files in a format called RBT (this apparently stands for “Robot” but I think “Ribbit” format could be more fun). What are we going to do ? We had come so far by solving the VMD subtitling problem for Phantasmagoria. It would be a shame if the effort ground to a halt due to this.

  Fortunately, the folks behind the ScummVM project already figured out enough of the format to be able to decode the RBT files in Phantasmagoria.

  In the end, I was successful in creating a completely standalone tool that can take a Robot file and a subtitle file and create a new Robot file with subtitles. The source code is here (subtitle-rbt.c). Here’s what the final result looks like :

  
  
  “What’s in the refrigerator ?” I should note at this juncture that I am not sure if this particular Robot file even has sound or dialogue since I was conducting these experiments on a computer with non-working audio.
  

  The RBT Format
  I have created a new MultimediaWiki page describing the Robot Animation format based on the ScummVM source code. I have not worked with a format quite like this before. These are paletted animations which consist of a sequence of independent frames that are designed to be overlaid on top of static background. Because of these characteristics, each frame encodes its own unique dimensions and origin coordinate within the frame. While the Phantasmagoria VMD files are usually 288×144 (which are usually double-sized for the benefit of a 640×400 Super VGA canvas), these frames are meant to be plotted on a game field that was roughly 576×288 (288×144 doublesized).
  

  For example, 2 minimalist animation frames from a desk investigation Robot file :

  
  
  100×147

  
  101×149
  

  As for compression, my first impression was that the algorithm was the same as VMD. This is wrong. It evidently uses an unmodified version of a standard algorithm called Lempel-Ziv-Stac (LZS). It shows up in several RFCs and was apparently used in MS-DOS’s transparent disk compression scheme.

  Approach
  Thankfully, many of the lessons I learned from the previous project are applicable to this project, including : subtitle library interfacing, subtitling in the paletted colorspace, and replacing encoded frames from the original file instead of trying to create a new file.

  Here is the pitch for this project :

  • Create a C program that can traverse through an input file, piece by piece, and generate an output file. The result of this should be a bitwise identical file.
  • Adapt the LZS compression decoding algorithm from ScummVM into the new tool. Make the tool dump raw Portable NetMap (PNM) files of varying dimensions and ensure that they look correct.
  • Compress using LZS.
  • Stretch the frames and draw subtitles.
  • More compression. Find the minimum window for each frame.

  Compression
  Normally, my first goal is to decompress the video and store the data in a raw form. However, this turned out to be mathematically intractable. While the format does support both compressed and uncompressed frames (even though ScummVM indicates that the uncompressed path is yet unexercised), the goal of this project requires making the frames so large that they overflow certain parameters of the file.

  A Robot file has a sequence of frames and 2 tables describing the size of each frame. One table describes the entire frame size (audio + video) while the second table describes just the video frame size. Since these tables only use 16 bits to specify a size, the maximum frame size is 65536 bytes. Leaving space for the audio portion of the frame, this only leaves a per-frame byte budget of about 63000 bytes for the video. Expanding the frame to 576×288 (165,888 pixels) would overflow this limit.

  Anyway, the upshot is that I needed to compress the data up front.

  Fortunately, the LZS compressor is pretty straightforward, at least if you have experience writing VLC-oriented codecs. While the algorithm revolves around back references, my approach was to essentially write an RLE encoder. My compressor would search for runs of data (plentiful when I started to stretch the frame for subtitling purposes). When a run length of n=3 or more of the same pixel is found, encode the pixel by itself, and then store a back reference of offset -1 and length (n-1). It took a little while to iron out a few problems, but I eventually got it to work perfectly.

  I have to say, however, that the format is a little bit weird in how it codes very large numbers. The length encoding is somewhat Golomb-like, i.e., smaller values are encoded with fewer bits. However, when it gets to large numbers, it starts encoding counts of 15 as blocks of 1111. For example, 24 is bigger than 7. Thus, emit 1111 into the bitstream and subtract 8 from 23 -> 16. Still bigger than 15, so stuff another 1111 into the bitstream and subtract 15. Now we’re at 1, so stuff 0001. So 24 is 11111111 0001. 12 bits is not too horrible. But the total number of bytes (value / 30). So a value of 300 takes around 10 bytes (80 bits) to encode.

  Palette Slices
  As in the VMD subtitling project, I took the subtitle color offered in the subtitle spec file as a suggestion and used Euclidean distance to match to the closest available color in the palette. One problem, however, is that the palette is a lot smaller in these animations. According to my notes, for the set of animations I scanned, only about 80 colors were specified, starting at palette index 55. I hypothesize that different slices of the palette are reserved for different uses. E.g., animation, background, and user interface. Thus, there is a smaller number of colors to draw upon for subtitling purposes.

  Scaling
  One bit of residual weirdness in this format is the presence of a per-frame scale factor. While most frames set this to 100 (100% scale), I have observed 70%, 80%, and 90%. ScummVM is a bit unsure about how to handle these, so I am as well. However, I eventually realized I didn’t really need to care, at least not when decoding and re-encoding the frame. Just preserve the scale factor. I intend to modify the tool further to take scale factor into account when creating the subtitle.

  The Final Resolution
  Right around the time that I was composing this post, The Translator emailed me and notified me that he had found a better way to subtitle the Robot files by modifying the scripts, rendering my entire approach moot. The result is much cleaner :

  
  
  Turns out that the engine supported subtitles all along
  

  It’s a good thing that I enjoyed the challenge or I might be annoyed at this point.

  See Also

  • Subtitling Sierra VMD Files : My effort to subtitle the main FMV files found in Sierra games.