Sur d’autres sites (12333)

• FFmpeg library is detecting pixel format of VP9 video stream not correctly

  6 avril 2023, par Pikachu

  I am using a C code to detect pixel format of VP9 video stream in a WebM container. FFmpeg version 6.0, full shared library build, downloaded from official website. Operating system is Windows 10. I feed the library with a VP9 video encoded with alpha channel, pixel format is YUVA420p. It detects pixel format as YUV420p.

  


  I have found a similar question on StackOverflow.com, Is there a way to force FFMPEG to decode a video stream with alpha from ​a WebM video encoded with libvpx-vp9 ?, but it does not actually help.

  


  When I override the decoder with a libvpx, it continues to detect the pixel format as YUV420p instead of YUVA420p.

  


  C code is following. Note that error handling in code is omitted here for StackOverflow question to be shorter.

  


  AVFormatContext *fmt_ctx = NULL;
int err = avformat_open_input(&fmt_ctx, infp, NULL, NULL);
err = avformat_find_stream_info(fmt_ctx, NULL);
int stream = av_find_best_stream(fmt_ctx, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
AVCodecParameters *codecpar = fmt_ctx->streams[stream]->codecpar;

const AVCodec *codec = NULL;
if (codecpar->codec_id == AV_CODEC_ID_VP9) {
    codec = avcodec_find_decoder_by_name(CODEC_LIBVPX_VP9);
} else {
    codec = avcodec_find_decoder(codecpar->codec_id);
}

AVCodecContext *ctx = avcodec_alloc_context3(codec);
err = avcodec_parameters_to_context(ctx, codecpar);
av_log(NULL, AV_LOG_DEBUG, "Pixel format: %d.\n", ctx->pix_fmt); //TODO:DEBUG.
err = avcodec_open2(ctx, codec, NULL);


  


  The program tells Pixel format: 0., which means AV_PIX_FMT_YUV420P, not the AV_PIX_FMT_YUVA420P !

  


  If I override pixel format manually, I am able to decode video with alpha channel and to see the transparent background, but it breaks the logics, because when a real YUV420p pixel format comes in and gets overridden by YUVA420p, this will be a problem.

  


  if (codecpar->codec_id == AV_CODEC_ID_VP9) {
    if (strcmp(codec->name, CODEC_LIBVPX_VP9) == 0) {
        if (ctx->pix_fmt == AV_PIX_FMT_YUV420P) {
            ctx->pix_fmt = AV_PIX_FMT_YUVA420P;
        }
    }
}


  


  At the same time ffmpeg tool started from command line with libvpx decoder tells that my video has YUVA420p pixel format. Output is following.

  


  D:\Temp\4>ffmpeg -c:v libvpx-vp9 -i yuva.webm
ffmpeg version 6.0-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-shared --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
[libvpx-vp9 @ 000001ecdf6002c0] v1.13.0-71-g45dc0d34d
    Last message repeated 1 times
Input #0, matroska,webm, from 'yuva.webm':
  Metadata:
    ENCODER         : Lavf60.3.100
  Duration: 00:00:05.55, start: 0.000000, bitrate: 227 kb/s
  Stream #0:0: Video: vp9 (Profile 0), yuva420p(tv, progressive), 1920x1080, SAR 1:1 DAR 16:9, 60 fps, 60 tbr, 1k tbn
    Metadata:
      alpha_mode      : 1
      ENCODER         : Lavc60.3.100 libvpx-vp9
      DURATION        : 00:00:05.550000000
At least one output file must be specified


  


  Here is my YUVA420p in the first video stream at the end of the console output :

  


  Stream #0:0: Video: vp9 (Profile 0), yuva420p(tv, progressive), 1920x1080, SAR 1:1 DAR 16:9, 60 fps, 60 tbr, 1k tbn


  


  


  The questions are following.

  


  

  1. How to detect real pixel format of VP9 video with FFmpeg library in C code reliably ?

  


  2. Why is the C code not detecting the actual pixel format even with codec overriden to libvpx ?

  


  


  Thank you.

  


• Unable to port FFmpeg C library into android

  13 décembre 2017, par Navjot Bedi

  What I exactly want to-do : Access the ffmpeg.c file to modify the int main(int argc, char **argv) function to JNI and passing the command of ffmpeg as string.

  I have tried to port ffmpeg C library to android(ARM processor). I followed following different ways to do this.

  1st Try : using official ffmpeg installation documentation. Steps as follows

  a) git clone git ://source.ffmpeg.org/ffmpeg.git ffmpeg

  b) Read the INSTALL file.

  c) Download x264 library and build by using build_x264.sh which build successfully.

      NDK=/home/nav/Work/android/ndk
  
      PREBUILT=$NDK/toolchains/arm-linux-androideabi-4.4.3/prebuilt
  
      PLATFORM=$NDK/platforms/android-8/arch-arm
  
      PREFIX=/home/nav/28ffmpeg/android-ffmpeg                
  
      ./configure --prefix=$PREFIX --enable-static --enable-pic --disable-asm --disable-cli --host=arm-linux --cross-prefix=$PREBUILT/linux-x86/bin/arm-linux-androideabi- --sysroot=$PLATFORM
  
    make
  
    sudo make install
  
    sudo ldconfig

  d) Then I download ffmpeg library from (http://ffmpeg.org/releases/ffmpeg-snapshot.tar.bz2) build it by using build_android.sh

  #!/bin/bash
  
  
  
  NDK=/home/nav/Work/android/ndk
  
  PLATFORM=$NDK/platforms/android-8/arch-arm
  
  PREBUILT=$NDK/toolchains/arm-linux-androideabi-4.4.3/prebuilt/linux-x86
  
  PREFIX=/home/nav/28ffmpeg/android-ffmpeg
  
  
  
  function build_one
  
                  {
  
                  ./configure --target-os=linux --prefix=$PREFIX \
  
                  --enable-cross-compile \
  
                  --enable-runtime-cpudetect \
  
                  --disable-asm \
  
                  --arch=arm \
  
                  --cc=$PREBUILT/bin/arm-linux-androideabi-gcc \
  
                  --cross-prefix=$PREBUILT/bin/arm-linux-androideabi- \
  
                  --disable-stripping \
  
                  --nm=$PREBUILT/bin/arm-linux-androideabi-nm \
  
                  --sysroot=$PLATFORM \
  
                  --enable-nonfree \
  
                  --enable-version3 \
  
                  --disable-everything \
  
                  --enable-gpl \
  
                  --disable-doc \
  
                  --enable-avresample \
  
                  --enable-demuxer=rtsp \
  
                  --enable-muxer=rtsp \
  
                  --disable-ffplay \
  
                  --disable-ffserver \
  
                  --enable-ffmpeg \
  
                  --disable-ffprobe \
  
                  --enable-libx264 \
  
                  --enable-encoder=libx264 \
  
                  --enable-decoder=h264 \
  
                  --enable-protocol=rtp \
  
                  --enable-hwaccels \
  
                  --enable-zlib \
  
                  --disable-devices \
  
                  --disable-avdevice \
  
  --extra-cflags="-I/home/android-ffmpeg/include -fPIC -DANDROID -D__thumb__ -mthumb -Wfatal-errors -Wno-deprecated -mfloat-abi=softfp -mfpu=vfpv3-d16 -marm -march=armv7-a" \
  
                  --extra-ldflags="-L/home/android-ffmpeg/lib"
  
                  make -j4 install
  
  
  
  $PREBUILT/bin/arm-linux-androideabi-ar d libavcodec/libavcodec.a inverse.o
  
  $PREBUILT/bin/arm-linux-androideabi-ld -rpath-link=$PLATFORM/usr/lib -L$PLATFORM/usr/lib -L$PREFIX/lib  -soname libffmpeg.so -shared -nostdlib  -z,noexecstack -Bsymbolic --whole-archive --no-undefined -o $PREFIX/libffmpeg.so libavcodec/libavcodec.a libavfilter/libavfilter.a libavresample/libavresample.a libavformat/libavformat.a libavutil/libavutil.a libswscale/libswscale.a -lc -lm -lz -ldl -llog -lx264 --warn-once --dynamic-linker=/system/bin/linker $PREBUILT/lib/gcc/arm-linux-androideabi/4.4.3/libgcc.a
  
                  }
  
  
  
                  build_one

  e) Initially it works perfectly. But after that error comes :

  libavformat/libavformat.a(log2_tab.o):(.rodata+0x0): multiple definition of `ff_log2_tab'
  
  libavcodec/libavcodec.a(log2_tab.o):(.rodata+0x0): first defined here
  
  libavutil/libavutil.a(log2_tab.o):(.rodata+0x0): multiple definition of `ff_log2_tab'
  
  libavcodec/libavcodec.a(log2_tab.o):(.rodata+0x0): first defined here
  
  build_android.sh: 48: build_one: not found

  Result : libffmpeg.so not found.

  2nd Try : Then I follow steps in http://dl.dropbox.com/u/22605641/ffmpeg_android/main.html-> Builds

  a) I downloaded Pre-Build libffmpeg.so from above link.

  b) Add libffmpeg.so to libs/armeabi/ .

  c) Make Android.mk

  include $(CLEAR_VARS)
  
  LOCAL_MODULE := ffmpeg
  
  LOCAL_SRC_FILES := libffmpeg.so
  
  include $(PREBUILT_SHARED_LIBRARY)
  
  
  
  LOCAL_PATH := $(call my-dir)
  
  
  
  include $(CLEAR_VARS)
  
  LOCAL_MODULE := main
  
  LOCAL_STATIC_LIBRARIES += ffmpeg
  
  LOCAL_SRC_FILES := ffmpeg-test.c
  
  include $(BUILD_SHARED_LIBRARY)

  d) Then I do all ndk set up and all.Copy ffmpeg.c from library to ffmpeg-test.c by changing its int main function to my JNI functin and include all necessary include files.

  Error :

  Console :

    /home/nav/Work/android/ndk/ndk-build all 
  
  Prebuilt       : libffmpeg.so <= jni/
  
  Install        : libffmpeg.so => libs/armeabi/libffmpeg.so
  
  Compile thumb  : main <= ffmpeg-test.c
  
  jni/ffmpeg-test.c: In function 'print_report':
  
  jni/ffmpeg-test.c:1139:94: warning: incompatible implicit declaration of built-in function 'log2' [enabled by default]
  
  SharedLibrary  : libmain.so
  
  /home/nav/Work/android/ndk/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-x86/bin/../lib/gcc/arm-linux-androideabi/4.6/../../../../arm-linux-androideabi/bin/ld: ./obj/local/armeabi/objs/main/ffmpeg-test.o: in function check_keyboard_interaction:jni/ffmpeg-test.c:2496: error: undefined reference to 'qp_hist'

  and many other undefined references.

• ffmpeg for android, out of memory on encoding h264

  15 juin 2018, par Rafael Lima

  I’m getting an weird error when enconding a video with ffmpeg on android

  Failed to inject frame into filter network: Out of memory

  Althoug is a pretty objective error, I do believe this is some "generic message" for another problem, because the video I used for testing is 17MB and less than 11 seconds length and I’ve sucessfully tested with videos bigger than 200MB and longer than 5min

  This problem happens to any video vertical videos recorded on my phone (the videos were recorded and play fine on my phone but i cant reencode them... other video h264 were encoded fine on same phone)

  this is ffmpeg output

  ffmpeg version 4.0 Copyright (c) 2000-2018 the FFmpeg developers
  
    built with Android (4691093 based on r316199) clang version 6.0.2 (https://android.googlesource.com/toolchain/clang 183abd29fc496f55536e7d904e0abae47888fc7f) (https://android.googlesource.com/toolchain/llvm 34361f192e41ed6e4e8f9aca80a4ea7e9856f327) (based on LLVM 6.0.2svn)
  
    configuration: --prefix=/home/rafa/Desktop/m4/build --target-os=android --arch=i686 --cpu=i686 --cross-prefix=/home/rafa/Desktop/m4/ndk/toolchain/i686/bin/i686-linux-android- --enable-cross-compile --cc=/home/rafa/Desktop/m4/ndk/toolchain/i686/bin/clang --cxx=/home/rafa/Desktop/m4/ndk/toolchain/i686/bin/clang++ --pkg-config=/usr/bin/pkg-config --pkg-config-flags=--static --enable-pic --enable-gpl --enable-nonfree --enable-static --disable-shared --enable-ffmpeg --disable-ffplay --disable-ffprobe --disable-doc --disable-devices --enable-libx264 --enable-libfdk-aac --enable-bsf=aac_adtstoasc --disable-asm
  
    libavutil      56. 14.100 / 56. 14.100
  
    libavcodec     58. 18.100 / 58. 18.100
  
    libavformat    58. 12.100 / 58. 12.100
  
    libavdevice    58.  3.100 / 58.  3.100
  
    libavfilter     7. 16.100 /  7. 16.100
  
    libswscale      5.  1.100 /  5.  1.100
  
    libswresample   3.  1.100 /  3.  1.100
  
    libpostproc    55.  1.100 / 55.  1.100
  
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/data/data/com.tomatedigital.toolsforinstagram/qqq.mp4':
  
    Metadata:  
  
      major_brand     : mp42
  
      minor_version   : 0
  
      compatible_brands: isommp42
  
      creation_time   : 2018-06-09T04:28:55.000000Z
  
      location        : -08.0638-034.8723/
  
      location-eng    : -08.0638-034.8723/
  
      com.android.version: 7.0
  
    Duration: 00:00:09.06, start: 0.000000, bitrate: 17349 kb/s
  
      Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080, 17088 kb/s, SAR 1:1 DAR 16:9, 30.04 fps, 30 tbr, 90k tbn, 180k tbc (default)
  
      Metadata:
  
        rotate          : 90
  
        creation_time   : 2018-06-09T04:28:55.000000Z
  
        handler_name    : VideoHandle
  
      Side data:
  
        displaymatrix: rotation of -90.00 degrees
  
      Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 255 kb/s (default)
  
      Metadata:
  
        creation_time   : 2018-06-09T04:28:55.000000Z
  
        handler_name    : SoundHandle
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  
    Stream #0:1 -> #0:1 (copy)
  
  Press [q] to stop, [?] for help
  
  Error reinitializing filters!
  
  Failed to inject frame into filter network: Out of memory
  
  Error while processing the decoded data for stream #0:0
  
  Conversion failed!

  =====================UPDATE===================

  I re-run the command with -v 9 -loglevel 99 and got paste the log file at :

  https://pastebin.com/ihawsPvZ

  If someone have real good expertise at ffmpeg i would apreciate very much the help

  =====================UPDATE 2=================

  I made more tests and the problem happens only when i record the video with my phone at vertical (other videos in vertical work and videos from my phone in horizontal also work)

  ==================UPDATE 3====================
  I detected the problem happens to ANY video with the

  Metadata:
  
        rotate          : 90 (or any value)

  It was suggested to use -noautorotate as workaround and it WORKDED, but this implies in tons of other complications ex :

  -if i decide to apply any filter to the video, it will be applied to the horizontal version but the video will be played at vertical producing a totally different effect

  ex : https://i.stack.imgur.com/R3jXJ.jpg (at this photo the drawtext filter should be placed at bottom line with horizontal text, but as ffmpeg can’t handle the orientation it is missplaced and rotated at runtime producing this vertical message)

  -I checked and if I set -metadata:s:v:0 rotate= while enconding it has no effect ex :

  ffmpeg -i input.mp4 -c copy -metadata:s:v:0 rotate= output.mp4
  will remove the rotate metadata and produce a non rotated video

  BUT
  ffmpeg -i input.mp4 -c:v libx264 -metadata:s:v:0 rotate= output.mp4
  will keep the metadata and the produced video will be played rotated (this behavior was also noticed on the official ffmpeg 4.0 binaries for windows available at ffmpeg website)

  I’M STILL LOOKING FOR A BETTER SOLUTION, THANKS FOR THE HELP