Recherche avancée

Sur d’autres sites (11036)

• Build FFmpeg for All Android architecture correctly

  26 octobre 2017, par fthopkins

  I built ffmpeg libraries for all android architecture. But some details are confusing my mind. First of all let me explain step by step how i built.

  • I downloaded latest version of libx264 from
    https://www.videolan.org/developers/x264.html
  • Created and run build_all.sh script (in x264 folder) for creating libs for (almost) all android platforms. Then i got android folder in x264 folder. e.g android/arm android/arm64 android/x86 android/x86_64 in x264 folder.
  • I download the ffmpeg3.3.4 from https://ffmpeg.org/download.html
  • I created 4 copies of ffmpeg3.3.4 folder. Named them with android architectures names.
    Like ffmpeg3.3.4-arm64 ffmpeg3.3.4-arm ffmpeg3.3.4-x86 ffmpeg3.3.4-x86_64
  • Created build_(architecture-name).sh for each platforms of android. Than move each one of 'named'.sh files to the named ffmpeg folders according to their architecture names.
    E.g build_android_x86.sh => ffmpeg3.3.4-x86

  build_android_arm64.sh example

  #!/bin/bash
  
  #Change NDK to your Android NDK location
  
  NDK=$HOME/Android/Sdk/ndk-bundle
  
  PLATFORM=$NDK/platforms/android-21/arch-arm64/
  
  PREBUILT=$NDK/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64
  
  
  
  GENERAL="\
  
  --enable-small \
  
  --enable-cross-compile \
  
  --extra-libs="-lgcc" \
  
  --arch=aarch64 \
  
  --cc=$PREBUILT/bin/aarch64-linux-android-gcc \
  
  --cross-prefix=$PREBUILT/bin/aarch64-linux-android- \
  
  --nm=$PREBUILT/bin/aarch64-linux-android-nm \
  
  --extra-cflags="-I../x264/android/arm64/include" \
  
  --extra-ldflags="-L../x264/android/arm64/lib" "
  
  
  
  MODULES="\
  
  --enable-gpl \
  
  --enable-libx264"
  
  
  
  
  
  
  
  function build_arm64
  
  {
  
    ./configure \
  
    --logfile=conflog.txt \
  
    --target-os=linux \
  
    --prefix=./android/arm64-v8a \
  
    ${GENERAL} \
  
    --sysroot=$PLATFORM \
  
    --extra-cflags="" \
  
    --extra-ldflags="-lx264 -Wl,-rpath-link=$PLATFORM/usr/lib -L$PLATFORM/usr/lib -nostdlib -lc -lm -ldl -llog" \
  
    --enable-shared \
  
    --disable-static \
  
    --disable-doc \
  
    --enable-zlib \
  
    ${MODULES}
  
  
  
    make clean
  
    make
  
    make install
  
  }
  
  
  
  build_arm64
  
  
  
  
  
  echo Android ARM64 builds finished

  • Moved all ffmpeg3.3.4-arm64 ffmpeg3.3.4-arm ffmpeg3.3.4-x86 ffmpeg3.3.4-x86_64 folders to the NDK_PATH (C:\Users\MyName\AppData\Local\Android\sdk\ndk-bundle)
  • Put x264 folder at same level of hierarchy with ffmpeg folders
    in (C:\Users\MyName\AppData\Local\Android\sdk\ndk-bundle)
  • Run each of build_android_x86.sh, build_android_x86_64.sh.. scripts one by one in their own ffmpeg folder. Then i got android folders in ffmpeg folders. (created android folders have library folders)
    e.g ffmpeg3.3.4-x86/android/x86/(bin,inlude,lib,share)

  • Created Android.mk files for all created android folders and moved each one of them to android/armName folder. e.g Android.mk => ffmpeg3.3.4-x86/android/x86/

    LOCAL_PATH:= $(call my-dir)
    
    
    
    include $(CLEAR_VARS)
    
    LOCAL_MODULE:= libavcodec
    
    LOCAL_SRC_FILES:= lib/libavcodec-57.so
    
    LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include
    
    include $(PREBUILT_SHARED_LIBRARY)
    
    
    
    include $(CLEAR_VARS)
    
    LOCAL_MODULE:= libavformat
    
    LOCAL_SRC_FILES:= lib/libavformat-57.so
    
    LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include
    
    include $(PREBUILT_SHARED_LIBRARY)
    
    
    
    include $(CLEAR_VARS)
    
    LOCAL_MODULE:= libswscale
    
    LOCAL_SRC_FILES:= lib/libswscale-4.so
    
    LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include
    
    include $(PREBUILT_SHARED_LIBRARY)
    
    
    
    include $(CLEAR_VARS)
    
    LOCAL_MODULE:= libavutil
    
    LOCAL_SRC_FILES:= lib/libavutil-55.so
    
    LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include
    
    include $(PREBUILT_SHARED_LIBRARY)
    
    
    
    include $(CLEAR_VARS)
    
    LOCAL_MODULE:= libavfilter
    
    LOCAL_SRC_FILES:= lib/libavfilter-6.so
    
    LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include
    
    include $(PREBUILT_SHARED_LIBRARY)
    
    
    
    include $(CLEAR_VARS)
    
    LOCAL_MODULE:= libswresample
    
    LOCAL_SRC_FILES:= lib/libswresample-2.so
    
    LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include
    
    include $(PREBUILT_SHARED_LIBRARY)
    
    
    
    include $(CLEAR_VARS)
    
    LOCAL_MODULE := postproc
    
    LOCAL_SRC_FILES := lib/libpostproc-54.so
    
    LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include
    
    include $(PREBUILT_SHARED_LIBRARY)
    
    
    
    include $(CLEAR_VARS)
    
    LOCAL_MODULE := avdevice
    
    LOCAL_SRC_FILES := lib/libavdevice-57.so
    
    LOCAL_EXPORT_C_INCLUDES := $(LOCAL_PATH)/include
    
    include $(PREBUILT_SHARED_LIBRARY)

  • Create "arm named" folders e.g x86 x86_4 arm64 arm7 in jni folder of my project (myproject/app/jni/x86, myproject/app/jni/x86_64).

  • Go to ndk path open named ffmpeg folders one by one and copy ffmpeg.c ffmpeg_filter.c ffmpeg_opt.c cmdutils.c files from ffmpeg folders to the "arm named" folders. e.g ffmpeg.c => myproject/app/jni/x86

  • Created Android.mk file in my project.

    LOCAL_PATH := $(call my-dir)
    
    #$(warning $(LOCAL_PATH))
    
    
    
    include $(CLEAR_VARS)
    
    LOCAL_MODULE    := videoEdit
    
    LOCAL_LDLIBS := -llog -ljnigraphics -lz -landroid
    
    LOCAL_CFLAGS := -Wdeprecated-declarations
    
    ANDROID_LIB := -landroid
    
    
    
    $(info $(TARGET_ARCH_ABI))
    
    
    
    ifeq ($(TARGET_ARCH_ABI),armeabi-v7a)
    
        LOCAL_C_INCLUDES:=/Users/MyName/AppData/Local/Android/sdk/ndk-bundle/sources/ffmpeg-3.3.4-armeabi
    
        LOCAL_SRC_FILES :=  videoEdit.c arm7/ffmpeg.c arm7/ffmpeg_filter.c arm7/ffmpeg_opt.c arm7/cmdutils.c
    
        LOCAL_CFLAGS += -lx264 -Wl,--no-merge-exidx-entries
    
    else
    
    
    
        ifeq ($(TARGET_ARCH_ABI),arm64-v8a)
    
                LOCAL_C_INCLUDES:=/Users/MyName/AppData/Local/Android/sdk/ndk-bundle/sources/ffmpeg-3.3.4-arm64-v8a
    
                LOCAL_SRC_FILES :=  videoEdit.c arm64/ffmpeg.c arm64/ffmpeg_filter.c arm64/ffmpeg_opt.c arm64/cmdutils.c
    
                LOCAL_CFLAGS += -funwind-tables -Wl,--no-merge-exidx-entries
    
        else
    
    
    
            ifeq ($(TARGET_ARCH_ABI),x86_64)
    
                   LOCAL_C_INCLUDES:=/Users/MyName/AppData/Local/Android/sdk/ndk-bundle/sources/ffmpeg3.3.4-x86_64
    
                   LOCAL_SRC_FILES :=  videoEdit.c x86_64/ffmpeg.c x86_64/ffmpeg_filter.c x86_64/ffmpeg_opt.c x86_64/cmdutils.c
    
            else
    
                    LOCAL_C_INCLUDES:=/Users/MyName/AppData/Local/Android/sdk/ndk-bundle/sources/ffmpeg3.3.4-x86
    
                    LOCAL_SRC_FILES :=  videoEdit.c x86/ffmpeg.c x86/ffmpeg_filter.c x86/ffmpeg_opt.c x86/cmdutils.c
    
            endif
    
    
    
        endif
    
    
    
    endif
    
    
    
    LOCAL_SHARED_LIBRARIES := libavformat libavcodec libswscale libavutil libswresample libavfilter libavdevice libpostproc
    
    
    
    include $(BUILD_SHARED_LIBRARY)
    
    
    
    ifeq ($(TARGET_ARCH_ABI),armeabi-v7a)
    
    
    
        $(call import-module,ffmpeg-3.3.4-armeabi/android/armeabi-v7a)
    
    
    
    else
    
    
    
        ifeq ($(TARGET_ARCH_ABI),arm64-v8a)
    
                $(call import-module,ffmpeg-3.3.4-arm64-v8a/android/arm64-v8a)
    
        else
    
    
    
            ifeq ($(TARGET_ARCH_ABI),x86_64)
    
                   $(call import-module,ffmpeg3.3.4-x86_64/android/x86_64)
    
            else
    
                $(call import-module,ffmpeg3.3.4-x86/android/i686-diasm)
    
            endif
    
    
    
        endif
    
    
    
    endif

  • Created Application.mk file in my project.

    APP_ABI := armeabi-v7a, arm64-v8a, x86_64, x86
    
    APP_PLATFORM := android-14

  • Created videoEdit.c than run ndk-build build command.

  Built with no error. I can run ffmpeg commands but it takes too much time i think. E.g when i try to run hflip it flips the video with 15 seconds.

  Code example :

  JNIEXPORT jint JNICALL Java_com_name_app_library_VideoEditer_natives_VideoEditer_flipHorizontally
  
  (JNIEnv *env, jclass someclass, jstring inputFile, jstring outFile) {
  
  
  
      int numberOfArgs = 14;
  
  
  
      char** arguments = calloc(numberOfArgs, sizeof(char*));
  
      const char *in, *out;
  
  
  
      in = (*env)->GetStringUTFChars(env, inputFile, 0);
  
      out = (*env)->GetStringUTFChars(env, outFile, 0);
  
  
  
      arguments[0] = "ffmpeg";
  
      arguments[1] = "-i";
  
      arguments[2] = in;
  
      arguments[3] = "-c:v";
  
      arguments[4] = "libx264";
  
      arguments[5] = "-preset";
  
      arguments[6] = "ultrafast";
  
      arguments[7] = "-threads";
  
      arguments[8] = "5";
  
      arguments[9] = "-c:a";
  
      arguments[10] = "copy";
  
      arguments[11] = "-vf";
  
      arguments[12] = "hflip";
  
      arguments[13] = out;
  
  
  
      int i;
  
      for (i = 0; i < numberOfArgs; i++) {
  
          log_message(arguments[i]);
  
      }
  
      log_message("Printed all");
  
  
  
      main(numberOfArgs, arguments);
  
      free(arguments);
  
      (*env)->ReleaseStringUTFChars(env, inputFile, in);
  
      (*env)->ReleaseStringUTFChars(env, outFile, out);
  
  
  
      return 0;
  
  }

  FFmpeg Commands that runs on android, working very fast on emaulator but very slow on real device.

  Also another thing to confusing my mind is output of ./configure command.
  When i open the (e.g) ffmpeg-3.3.4-arm64-v8a folder and run ./configure command in terminal. Output says ARCH   x86 (generic). For all platform output is the same ARCH   x86 (generic).

  Output of ./configure command in ffmpeg-3.3.4-arm64-v8a folder

  install prefix            /usr/local
  
  source path               .
  
  C compiler                gcc
  
  C library                 glibc
  
  ARCH                      x86 (generic)
  
  big-endian                no
  
  runtime cpu detection     yes
  
  yasm                      yes
  
  MMX enabled               yes
  
  MMXEXT enabled            yes
  
  3DNow! enabled            yes
  
  3DNow! extended enabled   yes
  
  SSE enabled               yes
  
  SSSE3 enabled             yes
  
  AESNI enabled             yes
  
  AVX enabled               yes
  
  XOP enabled               yes
  
  FMA3 enabled              yes
  
  FMA4 enabled              yes
  
  i686 features enabled     yes
  
  CMOV is fast              yes
  
  EBX available             yes
  
  EBP available             yes
  
  debug symbols             yes
  
  strip symbols             yes
  
  optimize for size         no
  
  optimizations             yes
  
  static                    yes
  
  shared                    no
  
  postprocessing support    no
  
  network support           yes
  
  threading support         pthreads
  
  safe bitstream reader     yes
  
  texi2html enabled         no
  
  perl enabled              yes
  
  pod2man enabled           yes
  
  makeinfo enabled          no
  
  makeinfo supports HTML    no
  
  
  
  External libraries:
  
  iconv            xlib

  EDIT all of these steps made on ubuntu then moved all folders to the windows. So dont be confuse about paths.

  With all of these steps i want to ask is, is there anything wrong on my steps. Should i follow any other way or not ? Why the ffmpeg is running slower on real device. I am wondering one thing badly, how retrica, snapchat, instagram recording video as mirrored flipping ? Are they flipping after recorded (if they are how are they doing this in a second) or are they recording a video as flipped in run time ? I guess building ffmpeg for android is bit messy and making wrong things is very easy. If you look my steps, questions and give me an advice, i will be much appreciated.

• Revision 31972 : Et maintenant on peut aussi importer les fichiers d’export. On peut aussi ...

  8 octobre 2009, par rastapopoulos@… — Log

  Et maintenant on peut aussi importer les fichiers d’export.
  On peut aussi supprimer entièrement un menu, ce qui n’était bizarement pas possible avant.
  Pour la peine, on hausse la version, parce que c’est une nouvelle fonctionnalité d’ajoutée.

• Anatomy of an optimization : H.264 deblocking

  26 mai 2010, par Dark Shikari — H.264, assembly, development, speed, x264

  As mentioned in the previous post, H.264 has an adaptive deblocking filter. But what exactly does that mean — and more importantly, what does it mean for performance ? And how can we make it as fast as possible ? In this post I’ll try to answer these questions, particularly in relation to my recent deblocking optimizations in x264.

  H.264′s deblocking filter has two steps : strength calculation and the actual filter. The first step calculates the parameters for the second step. The filter runs on all the edges in each macroblock. That’s 4 vertical edges of length 16 pixels and 4 horizontal edges of length 16 pixels. The vertical edges are filtered first, from left to right, then the horizontal edges, from top to bottom (order matters !). The leftmost edge is the one between the current macroblock and the left macroblock, while the topmost edge is the one between the current macroblock and the top macroblock.

  Here’s the formula for the strength calculation in progressive mode. The highest strength that applies is always selected.

  If we’re on the edge between an intra macroblock and any other macroblock : Strength 4
  If we’re on an internal edge of an intra macroblock : Strength 3
  If either side of a 4-pixel-long edge has residual data : Strength 2
  If the motion vectors on opposite sides of a 4-pixel-long edge are at least a pixel apart (in either x or y direction) or the reference frames aren’t the same : Strength 1
  Otherwise : Strength 0 (no deblocking)

  These values are then thrown into a lookup table depending on the quantizer : higher quantizers have stronger deblocking. Then the actual filter is run with the appropriate parameters. Note that Strength 4 is actually a special deblocking mode that performs a much stronger filter and affects more pixels.

  One can see somewhat intuitively why these strengths are chosen. The deblocker exists to get rid of sharp edges caused by the block-based nature of H.264, and so the strength depends on what exists that might cause such sharp edges. The strength calculation is a way to use existing data from the video stream to make better decisions during the deblocking process, improving compression and quality.

  Both the strength calculation and the actual filter (not described here) are very complex if naively implemented. The latter can be SIMD’d with not too much difficulty ; no H.264 decoder can get away with reasonable performance without such a thing. But what about optimizing the strength calculation ? A quick analysis shows that this can be beneficial as well.

  Since we have to check both horizontal and vertical edges, we have to check up to 32 pairs of coefficient counts (for residual), 16 pairs of reference frame indices, and 128 motion vector values (counting x and y as separate values). This is a lot of calculation ; a naive implementation can take 500-1000 clock cycles on a modern CPU. Of course, there’s a lot of shortcuts we can take. Here’s some examples :

  • If the macroblock uses the 8×8 transform, we only need to check 2 edges in each direction instead of 4, because we don’t deblock inside of the 8×8 blocks.
  • If the macroblock is a P-skip, we only have to check the first edge in each direction, since there’s guaranteed to be no motion vector differences, reference frame differences, or residual inside of the macroblock.
  • If the macroblock has no residual at all, we can skip that check.
  • If we know the partition type of the macroblock, we can do motion vector checks only along the edges of the partitions.
  • If the effective quantizer is so low that no deblocking would be performed no matter what, don’t bother calculating the strength.

  But even all of this doesn’t save us from ourselves. We still have to iterate over a ton of edges, checking each one. Stuff like the partition-checking logic greatly complicates the code and adds overhead even as it reduces the number of checks. And in many cases decoupling the checks to add such logic will make it slower : if the checks are coupled, we can avoid doing a motion vector check if there’s residual, since Strength 2 overrides Strength 1.

  But wait. What if we could do this in SIMD, just like the actual loopfilter itself ? Sure, it seems more of a problem for C code than assembly, but there aren’t any obvious things in the way. Many years ago, Loren Merritt (pengvado) wrote the first SIMD implementation that I know of (for ffmpeg’s decoder) ; it is quite fast, so I decided to work on porting the idea to x264 to see if we could eke out a bit more speed here as well.

  Before I go over what I had to do to make this change, let me first describe how deblocking is implemented in x264. Since the filter is a loopfilter, it acts “in loop” and must be done in both the encoder and decoder — hence why x264 has it too, not just decoders. At the end of encoding one row of macroblocks, x264 goes back and deblocks the row, then performs half-pixel interpolation for use in encoding the next frame.

  We do it per-row for reasons of cache coherency : deblocking accesses a lot of pixels and a lot of code that wouldn’t otherwise be used, so it’s more efficient to do it in a single pass as opposed to deblocking each macroblock immediately after encoding. Then half-pixel interpolation can immediately re-use the resulting data.

  Now to the change. First, I modified deblocking to implement a subset of the macroblock_cache_load function : spend an extra bit of effort loading the necessary data into a data structure which is much simpler to address — as an assembly implementation would need (x264_macroblock_cache_load_deblock). Then I massively cleaned up deblocking to move all of the core strength-calculation logic into a single, small function that could be converted to assembly (deblock_strength_c). Finally, I wrote the assembly functions and worked with Loren to optimize them. Here’s the result.

  And the timings for the resulting assembly function on my Core i7, in cycles :

  deblock_strength_c : 309
  deblock_strength_mmx : 79
  deblock_strength_sse2 : 37
  deblock_strength_ssse3 : 33

  Now that is a seriously nice improvement. 33 cycles on average to perform that many comparisons–that’s absurdly low, especially considering the SIMD takes no branchy shortcuts : it always checks every single edge ! I walked over to my performance chart and happily crossed off a box.

  But I had a hunch that I could do better. Remember, as mentioned earlier, we’re reloading all that data back into our data structures in order to address it. This isn’t that slow, but takes enough time to significantly cut down on the gain of the assembly code. And worse, less than a row ago, all this data was in the correct place to be used (when we just finished encoding the macroblock) ! But if we did the deblocking right after encoding each macroblock, the cache issues would make it too slow to be worth it (yes, I tested this). So I went back to other things, a bit annoyed that I couldn’t get the full benefit of the changes.

  Then, yesterday, I was talking with Pascal, a former Xvid dev and current video hacker over at Google, about various possible x264 optimizations. He had seen my deblocking changes and we discussed that a bit as well. Then two lines hit me like a pile of bricks :

  <_skal_> tried computing the strength at least ?
  <_skal_> while it’s fresh

  Why hadn’t I thought of that ? Do the strength calculation immediately after encoding each macroblock, save the result, and then go pick it up later for the main deblocking filter. Then we can use the data right there and then for strength calculation, but we don’t have to do the whole deblock process until later.

  I went and implemented it and, after working my way through a horde of bugs, eventually got a working implementation. A big catch was that of slices : deblocking normally acts between slices even though normal encoding does not, so I had to perform extra munging to get that to work. By midday today I was able to go cross yet another box off on the performance chart. And now it’s committed.

  Sometimes chatting for 10 minutes with another developer is enough to spot the idea that your brain somehow managed to miss for nearly a straight week.

  NB : the performance chart is on a specific test clip at a specific set of settings (super fast settings) relevant to the company I work at, so it isn’t accurate nor complete for, say, default settings.

  Update : Here’s a higher resolution version of the current chart, as requested in the comments.
  

  http://git.videolan.org/?p=x264.git ;a=commit ;h=b9ec3ea27812288485329ac7771143021cd4917b