Sur d’autres sites (9844)

• How to use ffmpeg in JavaScript to decode H.264 frames into RGB frames

  17 juin 2020, par noel

  I'm trying to compile ffmpeg into javascript so that I can decode H.264 video streams using node. The streams are H.264 frames packed into RTP NALUs so any solution has to be able to accept H.264 frames rather than a whole file name. These frames can't be in a container like MP4 or AVI because then the demuxer needs to needs the timestamp of every frame before demuxing can occur, but I'm dealing with a real time stream, no containers.

  



  Streaming H.264 over RTP

  



  Below is the basic code I'm using to listen on a udp socket. Inside the 'message' callback the data packet is an RTP datagram. The data portion of the data gram is an H.264 frame (P-frames and I-frames).

  



  var PORT = 33333;
var HOST = '127.0.0.1';

var dgram = require('dgram');
var server = dgram.createSocket('udp4');

server.on('listening', function () {
    var address = server.address();
    console.log('UDP Server listening on ' + address.address + ":" + address.port);
});

server.on('message', function (message, remote) {
    console.log(remote.address + ':' + remote.port +' - ' + message);
    frame = parse_rtp(message);

    rgb_frame = some_library.decode_h264(frame); // This is what I need.

});

server.bind(PORT, HOST);  


  



  I found the Broadway.js library, but I couldn't get it working and it doesn't handle P-frames which I need. I also found ffmpeg.js, but could get that to work and it needs a whole file not a stream. Likewise, fluent-ffmpeg doesn't appear to support file streams ; all of the examples show a filename being passed to the constructor. So I decided to write my own API.

  



  My current solution attempt

  



  I have been able to compile ffmpeg into one big js file, but I can't use it like that. I want to write an API around ffmpeg and then expose those functions to JS. So it seems to me like I need to do the following :

  



  

  1. Compile ffmpeg components (avcodec, avutil, etc.) into llvm bitcode.

  


  2. Write a C wrapper that exposes the decoding functionality and uses EMSCRIPTEN_KEEPALIVE.

  


  3. Use emcc to compile the wrapper and link it to the bitcode created in step 1.

  


  



  I found WASM+ffmpeg, but it's in Chinese and some of the steps aren't clear. In particular there is this step :

  



  emcc web.c process.c ../lib/libavformat.bc ../lib/libavcodec.bc ../lib/libswscale.bc ../lib/libswresample.bc ../lib/libavutil.bc \


  



   :( Where I think I'm stuck

  



  I don't understand how all the ffmpeg components get compiled into separate *.bc files. I followed the emmake commands in that article and I end up with one big .bc file.

  



  2 questions

  



  1. Does anyone know the steps to compile ffmpeg using emscripten so that I can expose some API to javascript ?
  
 2. Is there a better way (with decent documentation/examples) to decode h264 video streams using node ?

  


• FFmpeg cross compile for Android fails with on C compiler test for x86_64

  5 novembre 2018, par nLL

  I am trying to cross compile FFmpeg for Android with NDK r16b
  I have manged to compile without any issues for all platforms but x86_64.

  Here is my compile script

          #!/bin/bash
  
  
  
  
  
  
  
      function build_one
  
      {
  
      ./configure \
  
      --prefix=$PREFIX \
  
      --enable-shared \
  
      --disable-static \
  
      --enable-pic \
  
      --enable-small \
  
      --disable-programs \
  
      --disable-symver \
  
      --target-os=android \
  
      --enable-cross-compile \
  
      --cross-prefix=$CROSS_PREFIX \
  
      --extra-cflags="-Os -fpic $ADDI_CFLAGS" \
  
      --extra-ldflags="$ADDI_LDFLAGS" \
  
      --sysroot=$TOOLCHAIN/sysroot $ADDITIONAL_CONFIG_FLAG \
  
      --disable-all --disable-autodetect --disable-network --enable-pthreads \
  
      --enable-protocol=file --enable-protocol=pipe --enable-protocol=concat \
  
      --disable-all --disable-autodetect --disable-network --enable-pthreads \
  
      --enable-avcodec --enable-avformat --enable-swresample --enable-avfilter --enable-filter=aresample \
  
      --enable-parser=aac --enable-parser=aac_latm --enable-parser=flac --enable-parser=mpegaudio --enable-parser=vorbis \
  
      --enable-muxer=mp3,wav --enable-encoder=pcm*,libmp3lame --enable-nonfree --enable-gpl \
  
      --enable-jni --enable-mediacodec --enable-libmp3lame
  
  
  
  
  
      make -j4
  
      make install
  
      make distclean
  
      }
  
  
  
      HOME_PATH=/home/nll/Desktop
  
      CPU=x86_64
  
      TOOLCHAIN=$HOME_PATH/my_toolchains/x86_64
  
      CROSS_PREFIX=$TOOLCHAIN/bin/x86_64-linux-android-
  
      mkdir -p $(pwd)/android/$CPU
  
      PREFIX=$(pwd)/android/$CPU
  
      ADDI_CFLAGS="-march=x86-64 -I$INCLUDE_PATH"
  
      ADDI_LDFLAGS="-shared -L$LIBS_PATH"
  
      ADDITIONAL_CONFIG_FLAG="--arch=x86_64 --enable-x86asm"
  
      build_one

  Above script fails with

          WARNING: /home/nll/Desktop/my_toolchains/x86_64/bin/x86_64-linux-android-pkg-config not found, library detection may fail.
  
      mktemp -u XXXXXX
  
      5ZpQya
  
      test_ld cc
  
      test_cc
  
      BEGIN /tmp/ffconf.KWqViHoN/test.c
  
          1   int main(void){ return 0; }
  
      END /tmp/ffconf.KWqViHoN/test.c
  
      /home/nll/Desktop/my_toolchains/x86_64/bin/x86_64-linux-android-gcc --sysroot=/home/nll/Desktop/my_toolchains/x86_64/sysroot -Os -fpic -march=x86-64 -I -c -o /tmp/ffconf.KWqViHoN/test.o /tmp/ffconf.KWqViHoN/test.c
  
      /home/nll/Desktop/my_toolchains/x86_64/bin/x86_64-linux-android-gcc -shared -L --sysroot=/home/nll/Desktop/my_toolchains/x86_64/sysroot -o /tmp/ffconf.KWqViHoN/test /tmp/ffconf.KWqViHoN/test.o
  
      /home/nll/Desktop/my_toolchains/x86_64/bin/../lib/gcc/x86_64-linux-android/4.9.x/../../../../x86_64-linux-android/bin/ld: error: /tmp/ffconf.KWqViHoN/test.o: unsupported ELF file type 2
  
      collect2: error: ld returned 1 exit status
  
      C compiler test failed.

  So, error is "unsupported ELF file type 2" which means 64 bit. But, I am trying to build 64 bit library anyway.

  I do not understand what is the issue here. Can anyone give me some direction ?

• errors while building ffmpeg with ndk16b standalone toolchain

  11 octobre 2018, par soni sahu

  I am trying to build ffmpeg with android ndk18b toolchain. My configuration looks like this.

  TOOLCHAIN_PREFIX=/home/git/ndks/stand_alone_toolchain_ndk16b
  
  NDK_SYSROOT=$TOOLCHAIN_PREFIX/sysroot
  
  NDK_ARCH='arm'
  
  NDK_ABIARCH='arm-linux-androideabi'
  
  target_host=arm-linux-androideabi
  
  export CPP="${CROSS_PREFIX}clang++"
  
  export PATH=${TOOLCHAIN_PREFIX}/bin:$PATH
  
  export CROSS_PREFIX=${TOOLCHAIN_PREFIX}/bin/${NDK_ABIARCH}-
  
  export CC="${CROSS_PREFIX}clang"
  
  export CXX="${CROSS_PREFIX}clang++"
  
  export AS="${CROSS_PREFIX}clang"
  
  export AR="${CROSS_PREFIX}ar"
  
  export LD="${CROSS_PREFIX}ld"
  
  export RANLIB="${CROSS_PREFIX}ranlib"
  
  export STRIP="${CROSS_PREFIX}strip"
  
  export OBJDUMP="${CROSS_PREFIX}objdump"
  
  export CPP="${CROSS_PREFIX}cpp"
  
  export GCONV="${CROSS_PREFIX}gconv"
  
  export NM="${CROSS_PREFIX}nm"
  
  export SIZE="${CROSS_PREFIX}size"
  
  
  
  # Tell configure what flags Android requires.
  
  export CFLAGS="-fPIE -fPIC"
  
  export LDFLAGS="-pie"
  
  
  
  ./configure \
  
      --cpu="armv7-a" \
  
      --enable-pic \
  
      --disable-runtime-cpudetect \
  
      --enable-pthreads \
  
      --enable-hardcoded-tables \
  
      --prefix=$PREFIX \
  
      --disable-doc \
  
      --disable-ffplay \
  
      --disable-ffprobe \
  
      --disable-ffserver \
  
      --disable-doc \
  
      --disable-network \
  
      --enable-libmp3lame \
  
      --enable-libx264 \
  
      --enable-gpl \
  
      --extra-ldflags="-latomic -L${NDK_SYSROOT}/usr/lib -L$TOOLCHAIN_PREFIX/arm-linux-androideabi/lib -L$TOOLCHAIN_PREFIX/lib -Llibmp3lame/lib -Lx264/android/armeabi-v7a/lib $LDFLAGS -v -lc -lm -ldl -llog -march=armv7-a"\
  
      --extra-cflags="-ffast-math -funroll-loops -mfloat-abi=softfp -mfpu=vfpv3-d16 -Ilibmp3lame/include -Ix264/android/armeabi-v7a/include $CFLAGS -march=armv7-a -marm -mfloat-abi=softfp -mfpu=neon -mtune=cortex-a8 -mthumb -D__thumb__ -fno-exceptions -fno-rtti -march=armv7-a -Wl,--fix-cortex-a8" \
  
      --arch=arm\
  
      --target-os=linux \
  
      --enable-cross-compile \
  
      --cross-prefix=$TOOLCHAIN_PREFIX/bin/arm-linux-androideabi- \
  
      --nm=${NM} \
  
      --cc=${CC} \
  
      --cxx=${CXX} \
  
      --ld=${LD} \
  
      --ar=${AR} \
  
      --as=${AS} \
  
      --strip=${STRIP}

  I have build libmp3lame using this http://developer.samsung.com/android/technical-docs/Porting-and-using-LAME-MP3-on-Android-with-JNI and placed inside libmp3lame/lib folder.

  I am getting these undefined errors in config.log.

  /home/git/ndks/stand_alone_toolchain_ndk18b/bin/arm-linux-androideabi-clang
  -D_ISOC99_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Dstrtod=avpriv_strtod -DPIC -fPIE -fPIC -ffast-math -funroll-loops -mfloat-abi=softfp -mfpu=vfpv3-d16 -Ilibmp3lame/include -Ix264/android/armeabi-v7a/include -fPIE -fPIC -march=armv7-a -marm -mfloat-abi=softfp -mfpu=neon -mtune=cortex-a8 -mthumb -D__thumb__ -fno-exceptions -fno-rtti -march=armv7-a -Wl,—fix-cortex-a8 -march=armv7-a -std=c11 -fomit-frame-pointer -fPIC -mthumb -c -o /tmp/ffconf.PT1BY4UI.o /tmp/ffconf.5z69vrhC.c clang70 : warning :
  -Wl,—fix-cortex-a8 : ’linker’ input unused [-Wunused-command-line-argument]
  /home/git/ndks/stand_alone_toolchain_ndk18b/bin/arm-linux-androideabi-ld
  -pie -latomic -L/home/git/ndks/stand_alone_toolchain_ndk18b/sysroot/usr/lib -L/home/git/ndks/stand_alone_toolchain_ndk18b/arm-linux-androideabi/lib
  -L/home/git/ndks/stand_alone_toolchain_ndk18b/lib -Llibmp3lame/lib -Lx264/android/armeabi-v7a/lib -pie -v -lc -lm -ldl -llog -march=armv7-a -o /tmp/ffconf.o9IsP7bS /tmp/ffconf.PT1BY4UI.o -lmp3lame -lm -lz libmp3lame/lib/libmp3lame.a(set_get.o)(.ARM.exidx.text.lame_set_num_samples+0x0) :
  error : undefined reference to ’__aeabi_unwind_cpp_pr0’
  libmp3lame/lib/libmp3lame.a(set_get.o)(.ARM.exidx.text.lame_set_ogg+0x0) :
  error : undefined reference to ’__aeabi_unwind_cpp_pr0’
  libmp3lame/lib/libmp3lame.a(set_get.o)(.ARM.exidx.text.lame_get_ogg+0x0) :
  error : undefined reference to ’__aeabi_unwind_cpp_pr0’
  libmp3lame/lib/libmp3lame.a(set_get.o)(.ARM.exidx.text.lame_set_quality+0x0) :
  error : undefined reference to ’__aeabi_unwind_cpp_pr0’
  D :/work_dir/battefield3/chimpoon/proj.android/app/jni/../../../../lame/lame-3.100/./libmp3lame/set_get.c:2161 :
  error : undefined reference to ’__aeabi_uidiv’
  D :/work_dir/battefield3/chimpoon/proj.android/app/jni/../../../../lame/lame-3.100/./libmp3lame/set_get.c:2165 :
  error : undefined reference to ’__aeabi_uidivmod’
  D :/work_dir/battefield3/chimpoon/proj.android/app/jni/../../../../lame/lame-3.100/./libmp3lame/set_get.c:2170 :
  error : undefined reference to ’__aeabi_uidiv’

  Please help me to resolve this issue. Thanks.