Recherche avancée

Sur d’autres sites (7148)

• Sending per frame metadata with H264 encoded frames

  21 septembre 2013, par user2459280

  We're looking for a way to send per frame metadata (for example an ID) with H264 encoded frames from a server to a client.

  We're currently developing a remote rendering application, where both client and server side are actively involved.
  The server renders a high quality image with all effects, lighting etc.
  The client also has model-informations and renders a diffuse image that is used when the bandwidth is too low or the images have to be warped in order to avoid stuttering .

  So far we're encoding the frames on the server side with ffmpeg and streaming them with live555 to the client, who receives an rtsp-stream and decodes the frames again using ffmpeg.

  For our application, we now need to send per frame metadata.
  We want the client to tell the server where the camera is right now.
  Ideally we'd be able to send the client's view matrix to the server, render the corresponding frame and send it back to the client together with its view matrix. So when the client receives a frame, we need to know exactly at what camera position the frame was rendered.

  Alternatively we could also tag each view matrix with an ID, send it to the server, render the frame and tag it with the same ID and send it back. In this case we'd have to assign the right matrix to the frame again on the client side.

  After several attempts to realize the above intent with ffmpeg we came to the conclusion that ffmpeg does not provide the required functionality. ffmpeg only provides a fix, predefined set of fields for metadata, that either cannot store a matrix or can only be set for every key frame, which is not frequently enough for our purpose.

  Now we're considering using live555. So far we have an on demand Server, witch gets a VideoSubsession with a H264VideoStreamDiscreteFramer to contain our own FramedSource class. In this class we load the encoded AVPacket (from ffmpeg) and send its data-buffer over the network. Now we need a way to send some kind of metadata with every frame to the client.

  Do you have any ideas how to solve this metadata problem with live555 oder another library ?

  Thanks for your help !

• Simulate packet loss in internet video streaming

  31 décembre 2015, par james

  I have encoded H.264 video using FFmpeg. I want to apply packet loss model like Gilbert Eliot model to depict video transmission over unreliable network.Like in this thread TV noise is simulated. But noise on the internet is different. So is there a way to simulate packet loss/ noise for internet video streaming using FFmpeg or MATLAB ?

• Unable to integrate ffmpeg as a native dependency for an Android NDK project

  24 avril 2016, par mmark

  I have an Android NDK project which consists entirely of C/C++ code, and it basically processes images without using any external libraries.

  I’m using Android Studio + Gradle NDK Experimental plugin (0.7.0-alpha1).

  Now, I need to integrate ffmpeg as a native library to use it from the C/C++ code in order to decode a H.264 video frame.

  These are the questions I’ve found here regarding this issue :

  Android - Integrating ffmpeg and android-ndk-r9c

  Android NDK w/ ffmpeg library - error running project

  Using FFmpeg native libraries with Android-NDK

  Can not build with prebuilt static libraries using gradle-experimental

  Here is my build.gradle file :

  apply plugin: 'com.android.model.application'
  
  
  
  def ffmpeg_path = file(project(':ffmpeg').projectDir).absolutePath + "/ffmpeg-android"
  
  
  
  model {
  
  
  
      repositories {
  
          libs(PrebuiltLibraries) {
  
              libavcodec {
  
                  headers.srcDir "${ffmpeg_path}/include"
  
                  binaries.withType(StaticLibraryBinary) {
  
                      staticLibraryFile = file("${ffmpeg_path}/${targetPlatform.getName()}/lib/libavcodec.a")
  
                  }
  
              }
  
              libavutil {
  
                  headers.srcDir "${ffmpeg_path}/include"
  
                  binaries.withType(StaticLibraryBinary) {
  
                      staticLibraryFile = file("${ffmpeg_path}/${targetPlatform.getName()}/lib/libavutil.a")
  
                  }
  
              }
  
              libswresample {
  
                  headers.srcDir "${ffmpeg_path}/include"
  
                  binaries.withType(StaticLibraryBinary) {
  
                      staticLibraryFile = file("${ffmpeg_path}/${targetPlatform.getName()}/lib/libswresample.a")
  
                  }
  
              }
  
              libswscale {
  
                  headers.srcDir "${ffmpeg_path}/include"
  
                  binaries.withType(StaticLibraryBinary) {
  
                      staticLibraryFile = file("${ffmpeg_path}/${targetPlatform.getName()}/lib/libswscale.a")
  
                  }
  
              }
  
              libavformat {
  
                  headers.srcDir "${ffmpeg_path}/include"
  
                  binaries.withType(StaticLibraryBinary) {
  
                      staticLibraryFile = file("${ffmpeg_path}/${targetPlatform.getName()}/lib/libavformat.a")
  
                  }
  
              }
  
              libavdevice {
  
                  headers.srcDir "${ffmpeg_path}/include"
  
                  binaries.withType(StaticLibraryBinary) {
  
                      staticLibraryFile = file("${ffmpeg_path}/${targetPlatform.getName()}/lib/libavdevice.a")
  
                  }
  
              }
  
              libavfilter {
  
                  headers.srcDir "${ffmpeg_path}/include"
  
                  binaries.withType(StaticLibraryBinary) {
  
                      staticLibraryFile = file("${ffmpeg_path}/${targetPlatform.getName()}/lib/libavfilter.a")
  
                  }
  
              }
  
              libpostproc {
  
                  headers.srcDir "${ffmpeg_path}/include"
  
                  binaries.withType(StaticLibraryBinary) {
  
                      staticLibraryFile = file("${ffmpeg_path}/${targetPlatform.getName()}/lib/libpostproc.a")
  
                  }
  
              }
  
          }
  
      }
  
  
  
      android {
  
          compileSdkVersion = 23
  
          buildToolsVersion = "23.0.2"
  
  
  
          defaultConfig.with {
  
              applicationId = "com.example.hellojni"
  
              minSdkVersion.apiLevel = 4
  
              targetSdkVersion.apiLevel = 23
  
          }
  
      }
  
  
  
      /*
  
       * native build settings
  
       */
  
      android.ndk {
  
          moduleName = "hello-jni"
  
          platformVersion = 9 //same as minSdkVersion.apiLevel for better compatibility
  
          stl    = "c++_static"
  
          abiFilters.addAll(["armeabi", "armeabi-v7a", "x86"]) //filtering ABIs on the ones Google Play Games library is compatible with.
  
      }
  
      android.buildTypes {
  
          release {
  
              minifyEnabled = false
  
              proguardFiles.add(file('proguard-rules.txt'))
  
          }
  
      }
  
  
  
      android.sources {
  
          main {
  
              jni {
  
                  dependencies {
  
                      library "libavcodec" linkage "static"
  
                      library "libavutil" linkage "static"
  
                      library "libswresample" linkage "static"
  
                      library "libswscale" linkage "static"
  
                      library "libavformat" linkage "static"
  
                      library "libavdevice" linkage "static"
  
                      library "libavfilter" linkage "static"
  
                      library "libpostproc" linkage "static"
  
                  }
  
              }
  
          }
  
      }
  
  
  
  }

  I based on this sample from Google which integrates an external native library.

  This is the ffmpeg project I compiled before importing it into mine :
  https://github.com/WritingMinds/ffmpeg-android

  I’ve added all the .a and .h files and I’m able to import the headers in my C/C++ code, but when I try to compile it I get the following error :

  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function try_decode_frame: error: undefined reference to 'avpriv_h264_has_num_reorder_frames'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function compute_pkt_fields: error: undefined reference to 'avpriv_h264_has_num_reorder_frames'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function parse_packet: error: undefined reference to 'av_parser_parse2'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function parse_packet: error: undefined reference to 'av_parser_close'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function read_frame_internal: error: undefined reference to 'av_parser_init'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function avformat_find_stream_info: error: undefined reference to 'av_parser_init'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function avformat_find_stream_info: error: undefined reference to 'avcodec_pix_fmt_to_codec_tag'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function avformat_find_stream_info: error: undefined reference to 'avpriv_get_raw_pix_fmt_tags'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function ff_stream_add_bitstream_filter: error: undefined reference to 'av_bitstream_filter_init'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function ff_stream_add_bitstream_filter: error: undefined reference to 'av_bitstream_filter_close'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(utils.o):utils.c:function av_apply_bitstream_filters: error: undefined reference to 'av_bitstream_filter_filter'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(aadec.o):aadec.c:function aa_read_packet: error: undefined reference to 'av_tea_init'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(aadec.o):aadec.c:function aa_read_packet: error: undefined reference to 'av_tea_crypt'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(aadec.o):aadec.c:function aa_read_header: error: undefined reference to 'av_tea_alloc'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(aadec.o):aadec.c:function aa_read_header: error: undefined reference to 'av_tea_init'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(aadec.o):aadec.c:function aa_read_header: error: undefined reference to 'av_tea_crypt'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(ac3dec.o):ac3dec.c:function ac3_eac3_probe.isra.0: error: undefined reference to 'avpriv_ac3_parse_header'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(adtsenc.o):adtsenc.c:function adts_write_header: error: undefined reference to 'avpriv_mpeg4audio_get_config'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(adtsenc.o):adtsenc.c:function adts_write_header: error: undefined reference to 'avpriv_copy_pce_data'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(asfcrypt.o):asfcrypt.c:function ff_asfcrypt_dec: error: undefined reference to 'av_rc4_alloc'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(asfcrypt.o):asfcrypt.c:function ff_asfcrypt_dec: error: undefined reference to 'av_rc4_init'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(asfcrypt.o):asfcrypt.c:function ff_asfcrypt_dec: error: undefined reference to 'av_rc4_crypt'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(asfcrypt.o):asfcrypt.c:function ff_asfcrypt_dec: error: undefined reference to 'av_rc4_init'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(asfcrypt.o):asfcrypt.c:function ff_asfcrypt_dec: error: undefined reference to 'av_rc4_crypt'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(matroska.o):matroska.c:function ff_mkv_stereo3d_conv: error: undefined reference to 'av_stereo3d_alloc'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(nut.o):nut.c:function ff_nut_add_sp: error: undefined reference to 'av_tree_node_alloc'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(nut.o):nut.c:function ff_nut_add_sp: error: undefined reference to 'av_tree_insert'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(nut.o):nut.c:function ff_nut_free_sp: error: undefined reference to 'av_tree_enumerate'
  
  /Users/marcos/Documents/Android/fdecoder/ffmpeg/ffmpeg-android/armeabi-v7a/lib/libavformat.a(nut.o):nut.c:function ff_nut_free_sp: error: undefined reference to 'av_tree_destroy'
  
  Error:error: ld returned 1 exit status
  
  Error:Execution failed for task ':app:linkHello-jniArmeabi-v7aDebugSharedLibrary'.
  
  > A build operation failed.
  
        Linker failed while linking libhello-jni.so.
  
    See the complete log at: file:///Users/marcos/Documents/Android/fdecoder/app/build/tmp/linkHello-jniArmeabi-v7aDebugSharedLibrary/output.txt
  
  Information:BUILD FAILED
  
  Information:Total time: 14.993 secs
  
  Information:2 errors
  
  Information:0 warnings
  
  Information:See complete output in console

  There’s obviously something I’m not importing properly, but can’t figure exactly what’s missing.