Recherche avancée

Sur d’autres sites (5840)

• How to send encoded video (or audio) data from server to client in a way that's decodable by webcodecs API using minimal latency and data overhead

  11 janvier 2023, par Tiger Yang

  My question (read entire post for context) :

  


  Given the unique circumstance of only ever decoding data from a specifically-configured encoder, what is the best way I can send the encoded bitstream along with the bare minimum extra bytes required to properly configure the decoder on the client's end (including only things that change per stream, and omitting things that don't, such as resolution) ? I'm a sucker for zero compromises, and I think I am willing to design my own minimal container format to accomplish this.

  


  Context and problem :

  


  I'm working on a remote desktop implementation that consists of a server that captures and encodes the display and speakers using FFmpeg and forwards it via pipe to a go (language) program which sends it on two unidirectional webtransport streams to my client, which I plan to decode using the webcodecs API. According to MDN, the video decoder needs to be fed via .configure() an object containing the following : https://developer.mozilla.org/en-US/docs/Web/API/VideoDecoder/configure before it's able to decode anything.

  


  same goes for the audio decoder : https://developer.mozilla.org/en-US/docs/Web/API/AudioDecoder/configure

  


  What I've tried so far :

  


  Because this remote desktop will be for my personal use only, it would only ever receive streams from a specific encoder configured in a specific way encoding video at a specific resolution, framerate, color space, etc.. Therefore, I took my video capture FFmpeg command...

  


  videoString := []string{
        "ffmpeg",
        "-init_hw_device", "d3d11va",
        "-filter_complex", "ddagrab=video_size=1920x1080:framerate=60",
        "-vcodec", "hevc_nvenc",
        "-tune", "ll",
        "-preset", "p7",
        "-spatial_aq", "1",
        "-temporal_aq", "1",
        "-forced-idr", "1",
        "-rc", "cbr",
        "-b:v", "500K",
        "-no-scenecut", "1",
        "-g", "216000",
        "-f", "hevc", "-",
    }


  


  ...and instructed it to write to an mp4 file instead of outputting to pipe, and then I had this webcodecs demo https://w3c.github.io/webcodecs/samples/video-decode-display/ demux it using mp4box.js. Knowing that the demo outputs a proper .configure() object, I blindly copied it and had my client configure using that every time. Sadly, it didn't work, and I since noticed that the "description" part of the configure object changes despite the encoder and parameters being the same.

  


  I knew that mp4 files worked via mp4box, but they can't be streamed with low latency over a network, and additionally, ffmpeg's -f parameters specifies the muxer to use, but there are so many different types.

  


  At this point, I think I'm completely out of my depth, so :

  


  Given the unique circumstance of only ever decoding data from a specifically-configured encoder, what is the best way I can send the encoded bitstream along with the bare minimum extra bytes required to properly configure the decoder on the client's end (including only things that change per stream, and omitting things that don't, such as resolution) ? I'm a sucker for zero compromises, and I think I am willing to design my own minimal container format to accomplish this. (copied above)

  


• Revision 872b207b78 : Moves transform type defines to vp9_common Moves transform type defines to vp9_

  30 septembre 2014, par Deb Mukherjee

  Changed Paths :
   Modify /vp9/common/vp9_common.h

  
   Modify /vp9/common/vp9_idct.h

  
   Modify /vp9/common/vp9_rtcd_defs.pl

  
  
  Moves transform type defines to vp9_common

  Moves transform type defines to vp9_common.h from vp9_idct.h
  so that they can be included in vp9_rtcd_defs.pl safely.

  Change-Id : Id5106227bee5934f7ce8b06f2eb9fa8a9a2e0ddb

• H264 to MP4 with Bframe play back and forth on google chrome[ffmpeg]

  2 mai 2017, par Ravi Agola

  I want to generate a mp4 container with h264 encoded file.

  H264 file contains [I P B B][P B B][P B B][P B B] frames.

  when I generate a mp4 file with FFmpeg, It works well with FFplay as well as VLC but on google-chrome mp4 file plays frame back and forth.

  ffmpeg -i input.h264 -vcodec copy output.mp4

  when i use internal codec library(libx264) it works well on (ffplay,vlc and google-chrome)

  ffmpeg -i input.h264 -vcodec h264 output.mp4

  as above command transcode h264(native) to h264(x264), I don’t want to transcode file as I will be using it with ffmpeg library.

  when I use h264 without B frame it works well in both cases.

  I have tried some experiment with sample test file available here.

  direct conversion(MKV to MP4) works well with chrome as below

  ffmpeg -i jellyfish-3-mbps-hd-h264.mkv -vcodec copy output.mp4

  (MKV to H264 and H264 to MP4) plays back and forth on chrome

  ffmpeg -i jellyfish-3-mbps-hd-h264.mkv -vcodec copy output.h264
  ffmpeg -i output.h264 -vcodec copy output.mp4

  I get message in this case
  [mp4 @ 0xb6f8b20] Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly
  [mp4 @ 0xb6f8b20] pts has no value"

  what can be a reason behind this behavior ?

  Thanks.