Recherche avancée

Sur d’autres sites (6880)

• Video files recorded in Google Chrome have stuttering audio

  4 juin 2018, par maxpaj

  Background

  I’m developing a platform where users can record videos of themselves or their screen and send them as video messages to customers / clients.

  I have limited users to only using my application with Google Chrome and I’m using the MediaRecorder API to record the video data from the users screen or webcamera. The codecs that are used for recording are VP8/OPUS (WEBM container).

  I need the videos to run in as many browsers as possible, so I’m using a 3rd party service to transcode videos from whatever format I’m getting from the users to a H.265/AAC MP4 container (caniuse MPEG-4/H.264).

  Issue

  Lately I’ve seen that some videos recorded on Mac OSX machines have the video and audio out of sync or that the video and audio stutters, depending on which player I’m using. I call these video files corrupt, for lack of a better word. Playing a corrupt file in Google Chrome renders smooth playing audio. Playing the video in VLC on my Windows machine renders stuttering audio.

  When I run the corrupt video files through the transcoding service I get video files with stuttering audio, no matter which player I’m using.

  This is an unwanted result and pretty much unacceptable since the audio needs to be smooth in order for the recipient of a video to not be bothered with the quality.

  Debugging

  According to the transcoding service support, this happens because of their mechanisms that try to sync up the audio and video from the corrupt file :

  Inspecting our encoding logs, I’ve noticed the following kind of
  warnings :

  [2018-05-16 14:08:38.009] [pcm_s16le @ 0x1d608c0] pcm_encode_frame :
  filling in for 5856 missing samples (122 ms) before pts 40800 to
  correct sync ! [2018-05-16 14:08:38.009] [pcm_s16le @ 0x1d608c0]
  pcm_encode_frame : dropping 2880 samples (60 ms) at pts 43392 to help
  correct sync to -3168 samples (-66 ms) !

  The problem here comes from the way that the audio in the original
  source file is encoded.

  -

  you should ensure that the audio is not out of sync (audio timestamps
  are correct) in your source file before submitting the job

  Running a corrupt file through ffmpeg on my own machine, re-encoding with the same codecs, produces the same kind of stuttering video. The logs produce an alarming amount of errors. Here is a sample of the log output :

  [libopus @ 0000029938e24d80] Queue input is backward in timeitrate= 194.8kbits/s dup=0 drop=5 speed=0.31x
  
  [webm @ 0000029938e09b00] Non-monotonous DTS in output stream 0:1; previous: 15434, current: 15394; changing to 15434. This may result in incorrect timestamps in the output file.
  
  [webm @ 0000029938e09b00] Non-monotonous DTS in output stream 0:1; previous: 15434, current: 15414; changing to 15434. This may result in incorrect timestamps in the output file.
  
  [libopus @ 0000029938e24d80] Queue input is backward in timeitrate= 193.3kbits/s dup=0 drop=5 speed=0.309x
  
  [webm @ 0000029938e09b00] Non-monotonous DTS in output stream 0:1; previous: 15539, current: 15499; changing to 15539. This may result in incorrect timestamps in the output file.
  
  [webm @ 0000029938e09b00] Non-monotonous DTS in output stream 0:1; previous: 15539, current: 15519; changing to 15539. This may result in incorrect timestamps in the output file.
  
  [libopus @ 0000029938e24d80] Queue input is backward in timeitrate= 192.0kbits/s dup=0 drop=5 speed=0.308x
  
  [webm @ 0000029938e09b00] Non-monotonous DTS in output stream 0:1; previous: 15667, current: 15627; changing to 15667. This may result in incorrect timestamps in the output file.
  
  [webm @ 0000029938e09b00] Non-monotonous DTS in output stream 0:1; previous: 15667, current: 15647; changing to 15667. This may result in incorrect timestamps in the output file.
  
  [libopus @ 0000029938e24d80] Queue input is backward in time

  I tried running the same inputs through another transcoding service and those outputs worked a lot better - video was still stuttering but the audio played smoothly, which is more important to the use case of my application.

  To my knowledge, this have so far only occurred for users on Mac OSX machines.

  Questions

  1. Is there anything I can do to have the files work better ? Or is this entirely a consequence of how encoding of video and audio in Google Chrome works ?

  2. One step in the right direction would be to just be able to detect when the video is corrupt. How can I do that ?

• Patent skullduggery : Tandberg rips off x264 algorithm

  25 novembre 2010, par Dark Shikari — patents, ripoffs, x264

  Update : Tandberg claims they came up with the algorithm independently : to be fair, I can actually believe this to some extent, as I think the algorithm is way too obvious to be patented. Of course, they also claim that the algorithm isn’t actually identical, since they don’t want to lose their patent application.

  I still don’t trust them, but it’s possible it’s merely bad research (and thus being unaware of prior art) as opposed to anything malicious. Furthermore, word from within their office suggests they’re quite possibly being honest : supposedly the development team does not read x264 code at all. So this might just all be very bad luck.

  Regardless, the patent is still complete tripe, and should never have been filed.
  

  Most importantly, stop harassing the guy whose name is on the patent (Lars) : he’s just a programmer, not the management or lawyers responsible for filing the patent. This is stupid and unnecessary. I’ve removed the original post because of this ; it can be found here for those who want to read it.

  Appendix : the details of the patent :

  I figure I’ll go over the exact correspondence between the patent and my code here.

  1. A method for calculating run and level representations of quantized transform coefficients representing pixel values included in a block of a video picture, the method comprising :

  Translation : It’s a run-level coder.

  packing, at a video processing apparatus, each quantized transform coefficients in a value interval [Max, Min] by setting all quantized transform coefficients greater than Max equal to Max, and all quantized transform coefficients less than Min equal to Min

  The quantized coefficients are clipped to a certain valid range to allow them to be packed into bytes (they start as 16-bit values).

  reordering, at the video processing apparatus, the quantized transform ID coefficients according to a predefined order depending on respective positions in the block resulting in an array C of reordered quantized transform coefficients

  This is the zigzag pattern used in H.264 (and most formats) for reordering DCT coefficients. In x264, this is done before the run-level coder ste.

  masking, at the video processing apparatus, C by generating an array M containing ones in positions corresponding to positions of C having non-zero values, and zeros in positions corresponding to positions of C having zero values

  This is creating a bitmask based on the coefficient values, the pmovmskb step.

  is generating, at the video processing apparatus, for each position containing a one in M, a run and a level representation by setting the level value equal to an occurring value in a corresponding position of C ; and setting, at the video processing apparatus, for each position containing a one in M₅ the run value equal to the number of proceeding positions relative to a current position in M since a previous occurrence of one in M.

  This is the process of creating run/level values from the bitmask.

  Now into the detailed claims :

  2. The method according to Claim 1, wherein the masking further includes, creating an array C from C where positions corresponding to positions of nonzero values in C are filled with ones, and positions corresponding to positions of zero values in C are filled with zeros, and creating M from C by extracting the most significant bit from values in respective position of C and inserting the bits in corresponding positions in M.

  They’re extracting the most significant bit of the values to create a bitmask. This is exactly what the pmovmskb in my algorithm does.

  3. The method according to Claim 2, wherein the creating of the array C is executed by a C++ function PCMPGTB, and the creating of M from C is executed by a C++ function PMOVMSKB.

  And here they use pcmpgtb (they call it a C++ function for some reason, but it’s a SSE instruction) to do the clipping of the input values. This is exactly the same method I used in decimate_score. They also use pmovmskb as mentioned.

  4. The method according to Claim 1 , wherein the generating of the run and level representation further includes determining positions containing non-zero values in C by corresponding positions containing ones in M.

  5. The method according to Claim 4, wherein the determining of positions containing non-zero values in C is executed by a C++ function BSF.

  Here they iterate over the bitmask of transform coefficients using a “BSF” function to find runs, which is exactly what I did. Of course, BSF isn’t a function, it’s an x86 instruction.

  6. The method according to Claim 1 , wherein Max is 256 and Min is 0.

  This is almost surely a typo or mistake of some sort. They mean the Max should be 255, not 256 : 256 doesn’t fit in a uint8_t.

  7. The method according to Claim 1 , wherein the predefined order follows a zigzag path of transform coefficient positions in the block starting in an upper left corner heading towards a lower right corner.

  This is a description of the typical DCT zigzag pattern (like in H.264, MPEG-2, Theora, etc).

  Everything after this part is just repeating itself with the phrase “an apparatus” added in order to make the USPTO listen to them.

• Video from images to mp4 in nvidia GPU

  16 août 2019, par M.y

  I am trying to encode a h264 .mp4 video created from .jpg images using a 1070ti nvidia cuda power, having a a crossfade transition between each image.
  I am able to render the video in GPU using the flags -c:v h264_nvenc, I see a short peak in the GPU encoding, but with a long period of computer CPU hight load, I guess preparing the transitioning images. But the image preparation it happens on cpu/ram due the -filter_complex and is quite slow.
  This works :

  ffmpeg.exe, -y, -loop, 1, -t, 2.5, -i, 1565957420594_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565957453659_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565957487743_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565957525280_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565957587308_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565957644898_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565957859119_labeled.jpg, -loop, 1, -t, 2.5, -i,1565959133561_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565959412948_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565959501884_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565959755432_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565959882380_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565960023185_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565960157174_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565960683303_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565961151548_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565961230278_labeled.jpg, -loop, 1, -t, 2.5, -i, 1565961671766_labeled.jpg, -loop, 1, -t, 2.5, -i, final.jpg, -loop, 1, -t, 2.5, -i, final.jpg, -c:v, h264_nvenc, -preset, fast, -filter_complex, [1]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+0.5/TB[f0];[2]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+1.0/TB[f1];[3]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+1.5/TB[f2];[4]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+2.0/TB[f3];[5]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+2.5/TB[f4];[6]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+3.0/TB[f5];[7]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+3.5/TB[f6];[8]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+4.0/TB[f7];[9]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+4.5/TB[f8];[10]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+5.0/TB[f9];[11]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+5.5/TB[f10];[12]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+6.0/TB[f11];[13]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+6.5/TB[f12];[14]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+7.0/TB[f13];[15]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+7.5/TB[f14];[16]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+8.0/TB[f15];[17]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+8.5/TB[f16];[18]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+9.0/TB[f17];[19]fade=d=0.5:t=in:alpha=1,setpts=PTS-STARTPTS+9.5/TB[f18];[0][f0]overlay[bg1];[bg1][f1]overlay[bg2];[bg2][f2]overlay[bg3];[bg3][f3]overlay[bg4];[bg4][f4]overlay[bg5];[bg5][f5]overlay[bg6];[bg6][f6]overlay[bg7];[bg7][f7]overlay[bg8];[bg8][f8]overlay[bg9];[bg9][f9]overlay[bg10];[bg10][f10]overlay[bg11];[bg11][f11]overlay[bg12];[bg12][f12]overlay[bg13];[bg13][f13]overlay[bg14];[bg14][f14]overlay[bg15];[bg15][f15]overlay[bg16];[bg16][f16]overlay[bg17];[bg17][f17]overlay[bg18];[bg18][f18]overlay[v], -map, [v], -movflags, +faststart, output.mp4

  I am trying to do all work in the GPU, theoretically I can encode all images in GPU memory using in each -i the flags "-hwaccel cuvid -c:v mjpeg_cuvid" I receive the following error :

  [mjpeg_cuvid @ 00000000024ef980] ignoring invalid SAR: 0/0
  
  Impossible to convert between the formats supported by the filter 'graph 0 input from stream 1:0' and the filter 'auto_scaler_0'
  
  Error reinitializing filters!
  
  Failed to inject frame into filter network: Function not implemented
  
  Error while processing the decoded data for stream #0:0

  Is there a way to load images in the GPU with the "fade" flag applied ?

  Thanks in advance !