Recherche avancée

Sur d’autres sites (11745)

• Combining multiple image files into a video while using filter_complex to apply a watermark

  14 décembre 2017, par Geuis

  I’m trying to combine two ffmpeg operations into a single one.

  Currently I have two sets of ffmpeg commands that first generate a video from existing images, then runs that video through ffmpeg again to apply a watermark.

  I’d like to see if its possible to combine these into a single operation.

  # Create the source video
  
  ffmpeg -y \
  
  -framerate 1/1 \
  
  -i layer-%d.png \
  
  -r 30 -vcodec libx264 -preset ultrafast -crf 23 -pix_fmt yuv420p \
  
  output.mp4
  
  
  
  # Apply the watermark and render the final output
  
  ffmpeg -y \
  
  -i output.mp4 \
  
  -i logo.png \
  
  -filter_complex "[1:v][0:v]scale2ref=40:40[a][b];[b][a]overlay=(80):(main_h-200-80)" \
  
  final.mp4

• Beware the builtins

  14 janvier 2010, par Mans — Compilers

  GCC includes a large number of builtin functions allegedly providing optimised code for common operations not easily expressed directly in C. Rather than taking such claims at face value (this is GCC after all), I decided to conduct a small investigation to see how well a few of these functions are actually implemented for various targets.

  For my test, I selected the following functions :

  • __builtin_bswap32 : Byte-swap a 32-bit word.
  • __builtin_bswap64 : Byte-swap a 64-bit word.
  • __builtin_clz : Count leading zeros in a word.
  • __builtin_ctz : Count trailing zeros in a word.
  • __builtin_prefetch : Prefetch data into cache.

  To test the quality of these builtins, I wrapped each in a normal function, then compiled the code for these targets :

  • ARMv7
  • AVR32
  • MIPS
  • MIPS64
  • PowerPC
  • PowerPC64
  • x86
  • x86_64

  In all cases I used compiler flags were -O3 -fomit-frame-pointer plus any flags required to select a modern CPU model.
  

  ARM

  Both __builtin_clz and __builtin_prefetch generate the expected CLZ and PLD instructions respectively. The code for __builtin_ctz is reasonable for ARMv6 and earlier :

  rsb     r3, r0, #0
  and     r0, r3, r0
  clz     r0, r0
  rsb     r0, r0, #31
  

  For ARMv7 (in fact v6T2), however, using the new bit-reversal instruction would have been better :

  rbit    r0, r0
  clz     r0, r0
  

  I suspect this is simply a matter of the function not yet having been updated for ARMv7, which is perhaps even excusable given the relatively rare use cases for it.

  The byte-reversal functions are where it gets shocking. Rather than use the REV instruction found from ARMv6 on, both of them generate external calls to __bswapsi2 and __bswapdi2 in libgcc, which is plain C code :

  SItype
  __bswapsi2 (SItype u)
  
    return ((((u) & 0xff000000) >> 24)
            | (((u) & 0x00ff0000) >>  8)
            | (((u) & 0x0000ff00) <<  8)
            | (((u) & 0x000000ff) << 24)) ;
  
  DItype
  
  __bswapdi2 (DItype u)
  
  
  
     return ((((u) & 0xff00000000000000ull) >> 56)
  
            | (((u) & 0x00ff000000000000ull) >> 40)
  
            | (((u) & 0x0000ff0000000000ull) >> 24)
  
            | (((u) & 0x000000ff00000000ull) >>  8)
  
            | (((u) & 0x00000000ff000000ull) <<  8)
  
            | (((u) & 0x0000000000ff0000ull) << 24)
  
            | (((u) & 0x000000000000ff00ull) << 40)
  
            | (((u) & 0x00000000000000ffull) << 56)) ;
  
  
  

  While the 32-bit version compiles to a reasonable-looking shift/mask/or job, the 64-bit one is a real WTF. Brace yourselves :

  push    r4, r5, r6, r7, r8, r9, sl, fp
  mov     r5, #0
  mov     r6, #65280 ; 0xff00
  sub     sp, sp, #40 ; 0x28
  and     r7, r0, r5
  and     r8, r1, r6
  str     r7, [sp, #8]
  str     r8, [sp, #12]
  mov     r9, #0
  mov     r4, r1
  and     r5, r0, r9
  mov     sl, #255 ; 0xff
  ldr     r9, [sp, #8]
  and     r6, r4, sl
  mov     ip, #16711680 ; 0xff0000
  str     r5, [sp, #16]
  str     r6, [sp, #20]
  lsl     r2, r0, #24
  and     ip, ip, r1
  lsr     r7, r4, #24
  mov     r1, #0
  lsr     r5, r9, #24
  mov     sl, #0
  mov     r9, #-16777216 ; 0xff000000
  and     fp, r0, r9
  lsr     r6, ip, #8
  orr     r9, r7, r1
  and     ip, r4, sl
  orr     sl, r1, r2
  str     r6, [sp]
  str     r9, [sp, #32]
  str     sl, [sp, #36] ; 0x24
  add     r8, sp, #32
  ldm     r8, r7, r8
  str     r1, [sp, #4]
  ldm     sp, r9, sl
  orr     r7, r7, r9
  orr     r8, r8, sl
  str     r7, [sp, #32]
  str     r8, [sp, #36] ; 0x24
  mov     r3, r0
  mov     r7, #16711680 ; 0xff0000
  mov     r8, #0
  and     r9, r3, r7
  and     sl, r4, r8
  ldr     r0, [sp, #16]
  str     fp, [sp, #24]
  str     ip, [sp, #28]
  stm     sp, r9, sl
  ldr     r7, [sp, #20]
  ldr     sl, [sp, #12]
  ldr     fp, [sp, #12]
  ldr     r8, [sp, #28]
  lsr     r0, r0, #8
  orr     r7, r0, r7, lsl #24
  lsr     r6, sl, #24
  orr     r5, r5, fp, lsl #8
  lsl     sl, r8, #8
  mov     fp, r7
  add     r8, sp, #32
  ldm     r8, r7, r8
  orr     r6, r6, r8
  ldr     r8, [sp, #20]
  ldr     r0, [sp, #24]
  orr     r5, r5, r7
  lsr     r8, r8, #8
  orr     sl, sl, r0, lsr #24
  mov     ip, r8
  ldr     r0, [sp, #4]
  orr     fp, fp, r5
  ldr     r5, [sp, #24]
  orr     ip, ip, r6
  ldr     r6, [sp]
  lsl     r9, r5, #8
  lsl     r8, r0, #24
  orr     fp, fp, r9
  lsl     r3, r3, #8
  orr     r8, r8, r6, lsr #8
  orr     ip, ip, sl
  lsl     r7, r6, #24
  and     r5, r3, #16711680 ; 0xff0000
  orr     r7, r7, fp
  orr     r8, r8, ip
  orr     r4, r1, r7
  orr     r5, r5, r8
  mov     r9, r6
  mov     r1, r5
  mov     r0, r4
  add     sp, sp, #40 ; 0x28
  pop     r4, r5, r6, r7, r8, r9, sl, fp
  bx      lr
  

  That’s right, 91 instructions to move 8 bytes around a bit. GCC definitely has a problem with 64-bit numbers. It is perhaps worth noting that the bswap_64 macro in glibc splits the 64-bit value into 32-bit halves which are then reversed independently, thus side-stepping this weakness of gcc.

  As a side note, ARM RVCT (armcc) compiles those functions perfectly into one and two REV instructions, respectively.

  AVR32

  There is not much to report here. The latest gcc version available is 4.2.4, which doesn’t appear to have the bswap functions. The other three are handled nicely, even using a bit-reverse for __builtin_ctz.

  MIPS / MIPS64

  The situation MIPS is similar to ARM. Both bswap builtins result in external libgcc calls, the rest giving sensible code.

  PowerPC

  I scarcely believe my eyes, but this one is actually not bad. The PowerPC has no byte-reversal instructions, yet someone seems to have taken the time to teach gcc a good instruction sequence for this operation. The PowerPC does have some powerful rotate-and-mask instructions which come in handy here. First the 32-bit version :

  rotlwi  r0,r3,8
  rlwimi  r0,r3,24,0,7
  rlwimi  r0,r3,24,16,23
  mr      r3,r0
  blr
  

  The 64-bit byte-reversal simply applies the above code on each half of the value :

  rotlwi  r0,r3,8
  rlwimi  r0,r3,24,0,7
  rlwimi  r0,r3,24,16,23
  rotlwi  r3,r4,8
  rlwimi  r3,r4,24,0,7
  rlwimi  r3,r4,24,16,23
  mr      r4,r0
  blr
  

  Although I haven’t analysed that code carefully, it looks pretty good.

  PowerPC64

  Doing 64-bit operations is easier on a 64-bit CPU, right ? For you and me perhaps, but not for gcc. Here __builtin_bswap64 gives us the now familiar __bswapdi2 call, and while not as bad as the ARM version, it is not pretty :

  rldicr  r0,r3,8,55
  rldicr  r10,r3,56,7
  rldicr  r0,r0,56,15
  rldicl  r11,r3,8,56
  rldicr  r9,r3,16,47
  or      r11,r10,r11
  rldicr  r9,r9,48,23
  rldicl  r10,r0,24,40
  rldicr  r0,r3,24,39
  or      r11,r11,r10
  rldicl  r9,r9,40,24
  rldicr  r0,r0,40,31
  or      r9,r11,r9
  rlwinm  r10,r3,0,0,7
  rldicl  r0,r0,56,8
  or      r0,r9,r0
  rldicr  r10,r10,8,55
  rlwinm  r11,r3,0,8,15
  or      r0,r0,r10
  rldicr  r11,r11,24,39
  rlwinm  r3,r3,0,16,23
  or      r0,r0,r11
  rldicr  r3,r3,40,23
  or      r3,r0,r3
  blr
  

  That is 6 times longer than the (presumably) hand-written 32-bit version.

  x86 / x86_64

  As one might expect, results on x86 are good. All the tested functions use the available special instructions. One word of caution though : the bit-counting instructions are very slow on some implementations, specifically the Atom, AMD chips, and the notoriously slow Pentium4E.

  Conclusion

  In conclusion, I would say gcc builtins can be useful to avoid fragile inline assembler. Before using them, however, one should make sure they are not in fact harmful on the required targets. Not even those builtins mapping directly to CPU instructions can be trusted.

• Decoding audio w/ ffmpeg error on Android

  14 août 2012, par stranded

  Well, I knew I was going out of my comfort zone when I decided to try and decode audio using ffmpeg on Android but now I will have to admit that I'm stranded.
  It took me many days to just build ffmpeg for Android. Roman's10 guide did not work for me but finally things started looking up, thanks to this tutorial. So because of Dmitry's help I managed to build the armeabi version (not armeabi-v7) for my phone (LG P500) and everything basic works.

  But when I try to use avcodec_decode_audio3() things go downhill :( Never before have I felt so close to making things work (after all it seems to be only one line that is troublesome)
  but unable to though. I've read many questions here on SO that have brought me closer to the goal. Googling, on the other hand, has had limited results - making questions here the only fruit.

  Yes, I know ! I ramble. But I can't help it, I'm only trying to explain in detail where I'm stuck and how I got there. So without further ado I bring you the code :

  NATIVE CODE :

  #include 
  
  #include <android></android>log.h>
  
  
  
  #include "libavcodec/avcodec.h"
  
  #include "libavformat/avformat.h"
  
  
  
  #define LOG_TAG "mylib"
  
  #define LOGI(...)  __android_log_print(ANDROID_LOG_INFO, LOG_TAG, __VA_ARGS__)
  
  #define LOGE(...)  __android_log_print(ANDROID_LOG_ERROR, LOG_TAG, __VA_ARGS__)
  
  
  
  #define INBUFF_SIZE 4096
  
  #define AUDIO_INBUFF 20480
  
  #define AUDIO_REFILL_SIZE 4096
  
  
  
  jint Java_com_nothingworks_for_me_MainActivity_decode(JNIEnv * env, jobject this, jstring jfilename){
  
      const char *filename = (*env)->GetStringUTFChars(env, jfilename, NULL);
  
      AVCodec *codec;
  
      AVCodecContext *c= NULL;
  
      int audioStream;
  
      int out_size, len, i;
  
      FILE *f, *outfile;
  
      uint8_t *outbuf;
  
      uint8_t inbuf[AUDIO_INBUFF + FF_INPUT_BUFFER_PADDING_SIZE];
  
      AVPacket avpkt;
  
      AVFormatContext *pFormatCtx;
  
  
  
      av_register_all();
  
      avcodec_init();
  
      av_init_packet(&amp;avpkt);
  
  
  
      if(av_open_input_file(&amp;pFormatCtx, filename, NULL, 0, NULL)!=0)
  
      {
  
          LOGE("Can&#39;t open file &#39;%s&#39;\n", filename);
  
          return 1;
  
      }
  
      else
  
      {
  
          LOGI("File was opened\n");
  
          LOGI("File &#39;%s&#39;, Codec %s",
  
                  pFormatCtx->filename,
  
                  pFormatCtx->iformat->name
  
          );
  
      }
  
  
  
      if (av_find_stream_info(pFormatCtx) &lt; 0){
  
          LOGE("Can&#39;t find stream info");
  
      }
  
      audioStream = -1;
  
      for (i = 0; i &lt; pFormatCtx->nb_streams; i++) {
  
          if (pFormatCtx->streams[i]->codec->codec_type==AVMEDIA_TYPE_AUDIO) {
  
              audioStream = i;
  
              break;
  
          }
  
      }
  
  
  
      if (audioStream == -1) {
  
          LOGE("Didn&#39;t find stream!");
  
      }
  
      c = pFormatCtx->streams[audioStream]->codec;
  
  
  
      codec = avcodec_find_decoder(c->codec_id);
  
      if (!codec) {
  
          LOGE("Unsupported Codec!");
  
      }
  
  
  
      c= avcodec_alloc_context();
  
  
  
      /* open it */
  
      if (avcodec_open(c, codec) &lt; 0) {
  
          LOGE("Can&#39;t open codec");
  
          exit(1);
  
      }
  
  
  
      outbuf = av_malloc(AVCODEC_MAX_AUDIO_FRAME_SIZE * 2);
  
  
  
      f = fopen(filename, "rb");
  
      if (!f) {
  
          LOGE("Can&#39;t open file");
  
          exit(1);
  
      }
  
  
  
      /* decode until eof */
  
      avpkt.data = inbuf;
  
      avpkt.size = fread(inbuf, 1, AUDIO_INBUFF, f);
  
      LOGI("avpkt.size %d", avpkt.size);
  
  
  
      while (avpkt.size > 0) {
  
          out_size = AVCODEC_MAX_AUDIO_FRAME_SIZE * 2; 

  THINGS GO WRONG HERE ! avcodec_decode_audio3() The code continues from ▲ to ▼ :

          len = avcodec_decode_audio3(c, (int16_t *)outbuf, &out_size, &avpkt);
  
          LOGI("data_size %d len %d", out_size, len);
  
          if (len < 0) {
  
              LOGE("Error while decoding");
  
              exit(1);
  
          }
  
          if (out_size > 0) {
  
  
  
          }
  
          avpkt.size -= len;
  
          avpkt.data += len;
  
          if (avpkt.size < AUDIO_REFILL_SIZE) {
  
              /* Refill the input buffer, to avoid trying to decode
  
               * incomplete frames. Instead of this, one could also use
  
               * a parser, or use a proper container format through
  
               * libavformat. */
  
              memmove(inbuf, avpkt.data, avpkt.size);
  
              avpkt.data = inbuf;
  
              len = fread(avpkt.data + avpkt.size, 1,
  
                      AUDIO_INBUFF - avpkt.size, f);
  
              if (len > 0)
  
                  avpkt.size += len;
  
          }
  
      }
  
      fclose(f);
  
      free(outbuf);
  
  
  
      avcodec_close(c);
  
      av_free(c);
  
      return 0;
  
  }

  ---

  What happens is that avcodec_decode_audio3() returns -1 and that's pretty much it :(
  I have no idea what to do next. I can't find much info about this and I only started fiddling with C less than two weeks ago so your guidance is my only hope now [play dramatic sound]. Hope someone can shed a little light on this mystery.

  Ohh ! And the native code is some kind of a hybrid between what I have found here on SO, like this and this, and the ffmpeg example. On the java side I only have a call to this native method and pass it string which is the path to a MP3 song on my droid. I don't use AudioTrack or anything else in my java code yet 'cause I'm only trying to get the decoding to work for now.

  -Drama Queen OUT !