Recherche avancée

Sur d’autres sites (11756)

• How to send encoded video (or audio) data from server to client in a way that's decodable by webcodecs API using minimal latency and data overhead

  11 janvier 2023, par Tiger Yang

  My question (read entire post for context) :

  


  Given the unique circumstance of only ever decoding data from a specifically-configured encoder, what is the best way I can send the encoded bitstream along with the bare minimum extra bytes required to properly configure the decoder on the client's end (including only things that change per stream, and omitting things that don't, such as resolution) ? I'm a sucker for zero compromises, and I think I am willing to design my own minimal container format to accomplish this.

  


  Context and problem :

  


  I'm working on a remote desktop implementation that consists of a server that captures and encodes the display and speakers using FFmpeg and forwards it via pipe to a go (language) program which sends it on two unidirectional webtransport streams to my client, which I plan to decode using the webcodecs API. According to MDN, the video decoder needs to be fed via .configure() an object containing the following : https://developer.mozilla.org/en-US/docs/Web/API/VideoDecoder/configure before it's able to decode anything.

  


  same goes for the audio decoder : https://developer.mozilla.org/en-US/docs/Web/API/AudioDecoder/configure

  


  What I've tried so far :

  


  Because this remote desktop will be for my personal use only, it would only ever receive streams from a specific encoder configured in a specific way encoding video at a specific resolution, framerate, color space, etc.. Therefore, I took my video capture FFmpeg command...

  


  videoString := []string{
        "ffmpeg",
        "-init_hw_device", "d3d11va",
        "-filter_complex", "ddagrab=video_size=1920x1080:framerate=60",
        "-vcodec", "hevc_nvenc",
        "-tune", "ll",
        "-preset", "p7",
        "-spatial_aq", "1",
        "-temporal_aq", "1",
        "-forced-idr", "1",
        "-rc", "cbr",
        "-b:v", "500K",
        "-no-scenecut", "1",
        "-g", "216000",
        "-f", "hevc", "-",
    }


  


  ...and instructed it to write to an mp4 file instead of outputting to pipe, and then I had this webcodecs demo https://w3c.github.io/webcodecs/samples/video-decode-display/ demux it using mp4box.js. Knowing that the demo outputs a proper .configure() object, I blindly copied it and had my client configure using that every time. Sadly, it didn't work, and I since noticed that the "description" part of the configure object changes despite the encoder and parameters being the same.

  


  I knew that mp4 files worked via mp4box, but they can't be streamed with low latency over a network, and additionally, ffmpeg's -f parameters specifies the muxer to use, but there are so many different types.

  


  At this point, I think I'm completely out of my depth, so :

  


  Given the unique circumstance of only ever decoding data from a specifically-configured encoder, what is the best way I can send the encoded bitstream along with the bare minimum extra bytes required to properly configure the decoder on the client's end (including only things that change per stream, and omitting things that don't, such as resolution) ? I'm a sucker for zero compromises, and I think I am willing to design my own minimal container format to accomplish this. (copied above)

  


• Revision 527fc5caf6 : Fix loopfilter bug In the rare case were 4x4 interior filtering was called for

  9 juillet 2013, par John Koleszar

  Changed Paths :
   Modify /test/test-data.sha1

  
   Modify /test/test.mk

  
   Modify /test/test_vector_test.cc

  
   Modify /vp9/common/vp9_loopfilter.c

  
  
  Fix loopfilter bug

  In the rare case were 4x4 interior filtering was called for but no
  8x8 or larger filtering takes place, the previous code was skipping
  the filtering. This patch fixes the issue by including the interior
  mask in the overall mask for the filter application loops.

  Change-Id : I4a0b65056c64f97478827c2ff41e0914fc7779d0

• unable to find a suitable output format for 'ffmpeg'

  14 avril 2020, par KDeogharkar

  I am using ffmpeg library for android.

  



  I am passing one ffmpeg command for convert video format from .mp4 to .avi

  



  ffmpeg -r 20 -i /storage/emulated/0/WisperMedia/WisperVideos/sample.mp4 -b:v 20M /storage/emulated/0/WisperMedia/WisperVideos/output.avi


  



  But I am getting error like

  



      09-23 09:29:41.525: E/Home(31343): FAILED with output : ffmpeg version n2.4.2 Copyright (c) 2000-2014 the FFmpeg developers
09-23 09:29:41.525: E/Home(31343):   built on Oct  7 2014 15:08:46 with gcc 4.8 (GCC)
09-23 09:29:41.525: E/Home(31343):   configuration: --target-os=linux --cross-prefix=/home/sb/Source-Code/ffmpeg-android/toolchain-android/bin/arm-linux-androideabi- --arch=arm --cpu=cortex-a8 --enable-runtime-cpudetect --sysroot=/home/sb/Source-Code/ffmpeg-android/toolchain-android/sysroot --enable-pic --enable-libx264 --enable-libass --enable-libfreetype --enable-libfribidi --enable-fontconfig --enable-pthreads --disable-debug --disable-ffserver --enable-version3 --enable-hardcoded-tables --disable-ffplay --disable-ffprobe --enable-gpl --enable-yasm --disable-doc --disable-shared --enable-static --pkg-config=/home/sb/Source-Code/ffmpeg-android/ffmpeg-pkg-config --prefix=/home/sb/Source-Code/ffmpeg-android/build/armeabi-v7a-neon --extra-cflags='-I/home/sb/Source-Code/ffmpeg-android/toolchain-android/include -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-strict-overflow -fstack-protector-all -mfpu=neon' --extra-ldflags='-L/home/sb/Source-Code/ffmpeg-android/toolchain-android/lib -Wl,-z,relro -Wl,-z,now -pie' --extra-libs='-lpng -lexpat -lm' --extra-cxxflags=
09-23 09:29:41.525: E/Home(31343):   libavutil      54.  7.100 / 54.  7.100
09-23 09:29:41.525: E/Home(31343):   libavcodec     56.  1.100 / 56.  1.100
09-23 09:29:41.525: E/Home(31343):   libavformat    56.  4.101 / 56.  4.101
09-23 09:29:41.525: E/Home(31343):   libavdevice    56.  0.100 / 56.  0.100
09-23 09:29:41.525: E/Home(31343):   libavfilter     5.  1.100 /  5.  1.100
09-23 09:29:41.525: E/Home(31343):   libswscale      3.  0.100 /  3.  0.100
09-23 09:29:41.525: E/Home(31343):   libswresample   1.  1.100 /  1.  1.100
09-23 09:29:41.525: E/Home(31343):   libpostproc    53.  0.100 / 53.  0.100
09-23 09:29:41.525: E/Home(31343): Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/WisperMedia/WisperVideos/sample.mp4':
09-23 09:29:41.525: E/Home(31343):   Metadata:
09-23 09:29:41.525: E/Home(31343):     major_brand     : isom
09-23 09:29:41.525: E/Home(31343):     minor_version   : 0
09-23 09:29:41.525: E/Home(31343):     compatible_brands: isom3gp4
09-23 09:29:41.525: E/Home(31343):     creation_time   : 2015-09-23 03:50:16
09-23 09:29:41.525: E/Home(31343):   Duration: 00:00:01.56, start: 0.000000, bitrate: 14737 kb/s
09-23 09:29:41.525: E/Home(31343):     Stream #0:0(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1280x720, 11819 kb/s, SAR 65536:65536 DAR 16:9, 29.79 fps, 29.92 tbr, 90k tbn, 180k tbc (default)
09-23 09:29:41.525: E/Home(31343):     Metadata:
09-23 09:29:41.525: E/Home(31343):       rotate          : 90
09-23 09:29:41.525: E/Home(31343):       creation_time   : 2015-09-23 03:50:16
09-23 09:29:41.525: E/Home(31343):       handler_name    : VideoHandle
09-23 09:29:41.525: E/Home(31343):     Side data:
09-23 09:29:41.525: E/Home(31343):       displaymatrix: rotation of -90.00 degrees
09-23 09:29:41.525: E/Home(31343):     Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 128 kb/s (default)
09-23 09:29:41.525: E/Home(31343):     Metadata:
09-23 09:29:41.525: E/Home(31343):       creation_time   : 2015-09-23 03:50:16
09-23 09:29:41.525: E/Home(31343):       handler_name    : SoundHandle
09-23 09:29:41.525: E/Home(31343): [NULL @ 0x41f7f420] Unable to find a suitable output format for 'ffmpeg'
09-23 09:29:41.525: E/Home(31343): ffmpeg: Invalid argument