Recherche avancée

Sur d’autres sites (13479)

• How to send encoded video (or audio) data from server to client in a way that's decodable by webcodecs API using minimal latency and data overhead

  11 janvier 2023, par Tiger Yang

  My question (read entire post for context) :

  


  Given the unique circumstance of only ever decoding data from a specifically-configured encoder, what is the best way I can send the encoded bitstream along with the bare minimum extra bytes required to properly configure the decoder on the client's end (including only things that change per stream, and omitting things that don't, such as resolution) ? I'm a sucker for zero compromises, and I think I am willing to design my own minimal container format to accomplish this.

  


  Context and problem :

  


  I'm working on a remote desktop implementation that consists of a server that captures and encodes the display and speakers using FFmpeg and forwards it via pipe to a go (language) program which sends it on two unidirectional webtransport streams to my client, which I plan to decode using the webcodecs API. According to MDN, the video decoder needs to be fed via .configure() an object containing the following : https://developer.mozilla.org/en-US/docs/Web/API/VideoDecoder/configure before it's able to decode anything.

  


  same goes for the audio decoder : https://developer.mozilla.org/en-US/docs/Web/API/AudioDecoder/configure

  


  What I've tried so far :

  


  Because this remote desktop will be for my personal use only, it would only ever receive streams from a specific encoder configured in a specific way encoding video at a specific resolution, framerate, color space, etc.. Therefore, I took my video capture FFmpeg command...

  


  videoString := []string{
        "ffmpeg",
        "-init_hw_device", "d3d11va",
        "-filter_complex", "ddagrab=video_size=1920x1080:framerate=60",
        "-vcodec", "hevc_nvenc",
        "-tune", "ll",
        "-preset", "p7",
        "-spatial_aq", "1",
        "-temporal_aq", "1",
        "-forced-idr", "1",
        "-rc", "cbr",
        "-b:v", "500K",
        "-no-scenecut", "1",
        "-g", "216000",
        "-f", "hevc", "-",
    }


  


  ...and instructed it to write to an mp4 file instead of outputting to pipe, and then I had this webcodecs demo https://w3c.github.io/webcodecs/samples/video-decode-display/ demux it using mp4box.js. Knowing that the demo outputs a proper .configure() object, I blindly copied it and had my client configure using that every time. Sadly, it didn't work, and I since noticed that the "description" part of the configure object changes despite the encoder and parameters being the same.

  


  I knew that mp4 files worked via mp4box, but they can't be streamed with low latency over a network, and additionally, ffmpeg's -f parameters specifies the muxer to use, but there are so many different types.

  


  At this point, I think I'm completely out of my depth, so :

  


  Given the unique circumstance of only ever decoding data from a specifically-configured encoder, what is the best way I can send the encoded bitstream along with the bare minimum extra bytes required to properly configure the decoder on the client's end (including only things that change per stream, and omitting things that don't, such as resolution) ? I'm a sucker for zero compromises, and I think I am willing to design my own minimal container format to accomplish this. (copied above)

  


• avcodec/hevcdec : fix stat_coeff save/load for persistent_rice_adaptation_enabled_flag

  15 novembre 2020, par Xu Guangxin

  avcodec/hevcdec : fix stat_coeff save/load for persistent_rice_adaptation_enabled_flag
  It's required by the 9.3.1 TableStatCoeff* section.
  Following clips have this feature :
  
  WPP_HIGH_TP_444_8BIT_RExt_Apple_2.bit
  
  Bitdepth_A_RExt_Sony_1.bin
  
  Bitdepth_B_RExt_Sony_1.bin
  
  EXTPREC_HIGHTHROUGHPUT_444_16_INTRA_10BIT_RExt_Sony_1.bit
  
  EXTPREC_HIGHTHROUGHPUT_444_16_INTRA_12BIT_RExt_Sony_1.bit
  
  EXTPREC_HIGHTHROUGHPUT_444_16_INTRA_8BIT_RExt_Sony_1.bit
  
  EXTPREC_MAIN_444_16_INTRA_10BIT_RExt_Sony_1.bit
  
  EXTPREC_MAIN_444_16_INTRA_12BIT_RExt_Sony_1.bit
  
  EXTPREC_MAIN_444_16_INTRA_8BIT_RExt_Sony_1.bit
  
  WPP_AND_TILE_10Bit422Test_HIGH_TP_444_10BIT_RExt_Apple_2.bit
  
  WPP_AND_TILE_AND_CABAC_BYPASS_ALIGN_0_HIGH_TP_444_14BIT_RExt_Apple_2.bit
  
  WPP_AND_TILE_AND_CABAC_BYPASS_ALIGN_1_HIGH_TP_444_14BIT_RExt_Apple_2.bit
  
  WPP_AND_TILE_HIGH_TP_444_8BIT_RExt_Apple_2.bit
  you can download them from :
  
  https://www.itu.int/wftp3/av-arch/jctvc-site/bitstream_exchange/draft_conformance/RExt/
  Signed-off-by : Xu Guangxin <oddstone@gmail.com>
  
  Signed-off-by : Linjie Fu <linjie.justin.fu@gmail.com>
  

  • [DH] libavcodec/hevc_cabac.c
  • [DH] libavcodec/hevcdec.c
  • [DH] libavcodec/hevcdec.h

• I am trying to save frames from a webcam via Python and ffmpeg, but the video becomes way to fast

  10 mai 2022, par Matthias

  I get a stream of cv2 images from my webcam and want to save it to a video file. After playing a bit with cv2.VideoWriter() it turned out that using ffmpeg would provide more options and - apparently, following a few threads here on SO - lead to better results. So I gave the VidGear Python library a try, and it seems to work fine.

  


  There is one catch though : My webcam provides a variable framerate, most of the time between 10 and 30 FPS. When saving these frames the video file becomes way too fast, like watching in fast-forward. One real-time minute becomes only a few seconds in the video.

  


  I tried to play with various combinations of the ffmpeg's -framerate and/or -r parameters, but without luck. Here is the command I am using right now :

  


  ffmpeg -y -f rawvideo -vcodec rawvideo -s 1920x1080 -pix_fmt bgra -framerate 25.0 -i - -vcodec libx265 -crf 25 -r 25 -preset fast 


  


  For the records, I am creating the WriteGear class from the VidGear library like this :

  


  video_params = {
    "-vcodec": "libx265",
    "-crf": 25,
    "-input_framerate": 25,
    "-r": 25,
}
WriteGear(output_filename=video_file, logging=True, **video_params)


  


  Any ideas what I am doing wrong here and how I need to call ffmpeg ?