Recherche avancée

Sur d’autres sites (4946)

• Can I know which byte range to read from a remote mp4 file for FFMpeg to decode a keyframe ?

  12 octobre 2023, par db9117

  I need to decode a of keyframe of a video file (mp4, h264 encoded). I know the timestamp of the keyframe I want to extract/decode. I want to minimize amount of data being read in memory. For this, I need to know beforehand exactly the minimal byte range I would require that encompasses this keyframe. How do I know what is the minimal byte range in the whole mp4 byte stream I need to read in order to be able to decode the keyframe ?

  


  I currently find the appropriate keyframe in the index_entries contained in the header. I get its byte position (pos attribute) and timestamp (timestamp attribute). I calculate the range as follows :

  


  startBytes : minimum of :

  


  

  1. the pos of the keyframe

  


  2. the pos of the nearest index entry in the audio stream happening at or before the keyframe's timestamp.

  


  


  This way when it's decoding the frame, if it also needs the audio content for demuxing, it would have it.

  


  endBytes : maximum of :

  


  

  1. the pos of the next frame in the video stream's index, after the keyframe

  


  2. the pos of the next frame in the audio stream's index after the timestamp of the wished keyframe.

  


  


  This way I know that I have everything up until the next frame in the index, which theoretically should be enough to decode the keyframe only.

  


  I then read the appropriate byte range.

  


  When I try to decode the frame, I run in a loop until I succeed :

  


  

  • avcodec_read_frame

  


  • avcodec_send_packet

  


  • avcodec_receive_frame

  


  


  I ignore AVERROR(EAGAIN) errors.

  


  avcodec_receive_frame fails multiple times with error AVERROR(EAGAIN) which I ignore, until it fails saying that the memory it wants to read isn't available (wants to read after endBytes). I explicitly tell it to fail if it wants to read more than it has already read.

  


  Note : for other keyframes at other positions in other videos, it sometimes succeeds (probably because the range is big enough by chance), but it fails more often than not.

  


  My question is : Why is the end of the range not enough to be able to decode only the one keyframe ? Is there any way to more precisely calculate the exact range in bytes I would need in order to decode a particular keyframe ?

  


• D3D11 hardware screen recording with ffmpeg using Intel H264 QSV hardware encoding

  13 mai 2024, par Cactus

  I'm trying to screen record using ffmpeg and having it all done via hardware on my Intel integrated graphics.

  


  Specs

  


  For reference, here are my specs :

  


  

  • Windows 11 Pro Version 10.0.22631 Build 22631

  


  • Model : Dell Inc. OptiPlex 3090

  


  • Processor : 11th Gen Intel(R) Core(TM) i3-1115G4 @ 3.00GHz, 2995 Mhz, 2 Core(s), 4 Logical Processor(s)

  


  • Terminal : Windows Terminal using Windows Powershell

  


  


  In this wiki, it looked like I could use the "Windows 8+ Desktop Duplication API" to be "directly encoded by a compatible hardware encoder" — so I thought that I could use h264_qsv to encode, allowing me to record without much performance overhead costs.

  


  Attempt 1

  


  I've tried copying one of the command examples, but it didn't work :

  


  ffmpeg -init_hw_device d3d11va -filter_complex ddagrab=0 


  


  Output :

  


  ffmpeg version 2024-05-02-git-71669f2ad5-full_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libxevd --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxeve --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-dxva2 --enable-d3d11va --enable-d3d12va --enable-ffnvcodec --enable-libvpl --enable-nvdec --enable-nvenc --enable-vaapi --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      59. 16.101 / 59. 16.101
  libavcodec     61.  5.103 / 61.  5.103
  libavformat    61.  3.103 / 61.  3.103
  libavdevice    61.  2.100 / 61.  2.100
  libavfilter    10.  2.101 / 10.  2.101
  libswscale      8.  2.100 /  8.  2.100
  libswresample   5.  2.100 /  5.  2.100
  libpostproc    58.  2.100 / 58.  2.100
Stream mapping:
  ddagrab:default -> Stream #0:0 (h264_qsv)
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'Parsed_ddagrab_0' and the filter 'auto_scale_0'
[fc#0 @ 000001e4ed4393c0] Error configuring filter graph: Function not implemented
[fc#0 @ 000001e4ed4393c0] Task finished with error code: -40 (Function not implemented)
[fc#0 @ 000001e4ed4393c0] Terminating thread with return code -40 (Function not implemented)
[vost#0:0/h264_qsv @ 000001e4ed454000] Could not open encoder before EOF
[vost#0:0/h264_qsv @ 000001e4ed454000] Task finished with error code: -22 (Invalid argument)
[vost#0:0/h264_qsv @ 000001e4ed454000] Terminating thread with return code -22 (Invalid argument)
[out#0/matroska @ 000001e4ed453280] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0KiB time=N/A bitrate=N/A speed=N/A
Conversion failed!


  


  Attempt 2

  


  I then tried copying one of the command examples in the linked ddagrab wiki page, but also no luck.

  


  ffmpeg -f lavfi -i ddagrab -c:v h264_qsv output.mkv


  


  Output :

  


  ffmpeg version 2024-05-02-git-71669f2ad5-full_build-www.gyan.dev Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 13.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libxevd --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxeve --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-dxva2 --enable-d3d11va --enable-d3d12va --enable-ffnvcodec --enable-libvpl --enable-nvdec --enable-nvenc --enable-vaapi --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      59. 16.101 / 59. 16.101
  libavcodec     61.  5.103 / 61.  5.103
  libavformat    61.  3.103 / 61.  3.103
  libavdevice    61.  2.100 / 61.  2.100
  libavfilter    10.  2.101 / 10.  2.101
  libswscale      8.  2.100 /  8.  2.100
  libswresample   5.  2.100 /  5.  2.100
  libpostproc    58.  2.100 / 58.  2.100
Input #0, lavfi, from 'ddagrab':
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: wrapped_avframe, d3d11, 1920x1080 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 1000k tbn
File 'output.mkv' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (wrapped_avframe (native) -> h264 (h264_qsv))
Press [q] to stop, [?] for help
Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scale_0'
[vf#0:0 @ 00000115e105bd40] Error reinitializing filters!
[vf#0:0 @ 00000115e105bd40] Task finished with error code: -40 (Function not implemented)
[vf#0:0 @ 00000115e105bd40] Terminating thread with return code -40 (Function not implemented)
[vost#0:0/h264_qsv @ 00000115e105aa00] Could not open encoder before EOF
[vost#0:0/h264_qsv @ 00000115e105aa00] Task finished with error code: -22 (Invalid argument)
[vost#0:0/h264_qsv @ 00000115e105aa00] Terminating thread with return code -22 (Invalid argument)
[out#0/matroska @ 00000115e1054b80] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0KiB time=N/A bitrate=N/A speed=N/A
Conversion failed!


  


  My main problem is that I don't know how to use really any of these encoders and the examples I've found above don't work with me. What command should I use to screen record all with hardware acceleration on my Intel system ? I don't care about any presets or other options at the moment.

  


  Is there also a way to record audio ? When I tried to use -f dshow -i audio="Stereo Mix (Realtek(R) Audio)" by itself, which I made sure was turned on in Windows' settings and named correctly, it didn't work (is it because I'm using Bluetooth headphones that isn't connected to Realtek's system ?). Should this question be made separately than this one ?

  


• I have an application in flask where it streams a camera using ffmpeg, the problem is that I can not display the video from the camera using the GPU [closed]

  17 janvier, par Ruben

  I'll put you in context, I am using flask (python) to display a camera in the browser to stream it, for this I use the following Python code :

  


  command = [
     'ffmpeg',
     '-loglevel', 'warning',
     '-rtsp_transport', 'tcp',
     '-i', self.config['url'],
     '-map', '0:v:0', # fuerzo que solo procese el video
     '-vf', f'fps={self.config["fps"]},scale=640:360:force_original_aspect_ratio=decrease',
     '-c:v', 'h264_nvenc', # especificamos que queremos tirar de la gpu de nvidia
     '-preset', 'p7', # ajusta para la maxima calidad/velocidad (p1 mas rapida pero peor calidad - p7 más lento pero mejor calidad)
     '-qp', self.config['quality'], # control de calidad del codificador (0 [mejor calidad] - 51 [peor calidad])
     '-pix_fmt', 'yuv444p', # se mete explicitamente el formato de pixeles
     '-color_range', 'pc',
     '-an', # desactiva el audio
     '-f', 'image2pipe',
     'pipe:1'
] 

self.process = subprocess.Popen(
   command,
   stdout=subprocess.PIPE,
   stderr=subprocess.PIPE,
   bufsize=10**8
)


  


  The problem is that it does not display the video streaming, but it connects correctly to the camera.

  


  On the other hand, It show me the following warnings, which may have something to do with the display, it's probably the second warning that has to do with the pixel format :

  


  DEBUG :main:FFmpeg [camera1] : Guessed Channel Layout for Input Stream #0.1 : mono
DEBUG :main:FFmpeg [camera1] : [swscaler @ 0x560f70b78680] deprecated pixel format used, make sure you did set range correctly

  


  The server has different encodes installed :

  


  DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders : h264 h264_v4l2m2m h264_qsv h264_cuvid ) (encoders : libx264 libx264rgb h264_nvenc h264_omx h264_qsv h264_v4l2m2m h264_vaapi nvenc nvenc_h264 )

  


  Y uso el h264_nvenc, tambien el servidor tiene soporte de aceleración de hardware con :

  


  libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Hardware acceleration methods :
vdpau
cuda
vaapi
qsv
drm
opencl

  


  Between them h264_nvenc uses cuda

  


  I expand a little on the information it provides me when using h264_nvenc :

  


  Encoder h264_nvenc [NVIDIA NVENC H.264 encoder]:&#xA;    General capabilities: dr1 delay hardware&#xA;    Threading capabilities: none&#xA;    Supported hardware devices: cuda cuda&#xA;    Supported pixel formats: yuv420p nv12 p010le yuv444p p016le yuv444p16le bgr0 rgb0 cuda&#xA;h264_nvenc AVOptions:&#xA;  -preset            <int>        E..V....... Set the encoding preset (from 0 to 18) (default p4)&#xA;     default         0            E..V.......&#xA;     slow            1            E..V....... hq 2 passes&#xA;     medium          2            E..V....... hq 1 pass&#xA;     fast            3            E..V....... hp 1 pass&#xA;     hp              4            E..V.......&#xA;     hq              5            E..V.......&#xA;     bd              6            E..V.......&#xA;     ll              7            E..V....... low latency&#xA;     llhq            8            E..V....... low latency hq&#xA;     llhp            9            E..V....... low latency hp&#xA;     lossless        10           E..V.......&#xA;     losslesshp      11           E..V.......&#xA;     p1              12           E..V....... fastest (lowest quality)&#xA;     p2              13           E..V....... faster (lower quality)&#xA;     p3              14           E..V....... fast (low quality)&#xA;     p4              15           E..V....... medium (default)&#xA;     p5              16           E..V....... slow (good quality)&#xA;     p6              17           E..V....... slower (better quality)&#xA;     p7              18           E..V....... slowest (best quality)&#xA;  -tune              <int>        E..V....... Set the encoding tuning info (from 1 to 4) (default hq)&#xA;     hq              1            E..V....... High quality&#xA;     ll              2            E..V....... Low latency&#xA;     ull             3            E..V....... Ultra low latency&#xA;     lossless        4            E..V....... Lossless&#xA;  -profile           <int>        E..V....... Set the encoding profile (from 0 to 3) (default main)&#xA;     baseline        0            E..V.......&#xA;     main            1            E..V.......&#xA;     high            2            E..V.......&#xA;     high444p        3            E..V.......&#xA;  -level             <int>        E..V....... Set the encoding level restriction (from 0 to 62) (default auto)&#xA;     auto            0            E..V.......&#xA;     1               10           E..V.......&#xA;     1.0             10           E..V.......&#xA;     1b              9            E..V.......&#xA;     1.0b            9            E..V.......&#xA;     1.1             11           E..V.......&#xA;     1.2             12           E..V.......&#xA;     1.3             13           E..V.......&#xA;     2               20           E..V.......&#xA;     2.0             20           E..V.......&#xA;     2.1             21           E..V.......&#xA;     2.2             22           E..V.......&#xA;     3               30           E..V.......&#xA;     3.0             30           E..V.......&#xA;     3.1             31           E..V.......&#xA;     3.2             32           E..V.......&#xA;     4               40           E..V.......&#xA;     4.0             40           E..V.......&#xA;     4.1             41           E..V.......&#xA;     4.2             42           E..V.......&#xA;     5               50           E..V.......&#xA;     5.0             50           E..V.......&#xA;     5.1             51           E..V.......&#xA;     5.2             52           E..V.......&#xA;     6.0             60           E..V.......&#xA;     6.1             61           E..V.......&#xA;     6.2             62           E..V.......&#xA;  -rc                <int>        E..V....... Override the preset rate-control (from -1 to INT_MAX) (default -1)&#xA;     constqp         0            E..V....... Constant QP mode&#xA;     vbr             1            E..V....... Variable bitrate mode&#xA;     cbr             2            E..V....... Constant bitrate mode&#xA;     vbr_minqp       8388612      E..V....... Variable bitrate mode with MinQP (deprecated)&#xA;     ll_2pass_quality 8388616      E..V....... Multi-pass optimized for image quality (deprecated)&#xA;     ll_2pass_size   8388624      E..V....... Multi-pass optimized for constant frame size (deprecated)&#xA;     vbr_2pass       8388640      E..V....... Multi-pass variable bitrate mode (deprecated)&#xA;     cbr_ld_hq       8388616      E..V....... Constant bitrate low delay high quality mode&#xA;     cbr_hq          8388624      E..V....... Constant bitrate high quality mode&#xA;     vbr_hq          8388640      E..V....... Variable bitrate high quality mode&#xA;  -rc-lookahead      <int>        E..V....... Number of frames to look ahead for rate-control (from 0 to INT_MAX) (default 0)&#xA;  -surfaces          <int>        E..V....... Number of concurrent surfaces (from 0 to 64) (default 0)&#xA;  -cbr               <boolean>    E..V....... Use cbr encoding mode (default false)&#xA;  -2pass             <boolean>    E..V....... Use 2pass encoding mode (default auto)&#xA;  -gpu               <int>        E..V....... Selects which NVENC capable GPU to use. First GPU is 0, second is 1, and so on. (from -2 to INT_MAX) (default any)&#xA;     any             -1           E..V....... Pick the first device available&#xA;     list            -2           E..V....... List the available devices&#xA;  -delay             <int>        E..V....... Delay frame output by the given amount of frames (from 0 to INT_MAX) (default INT_MAX)&#xA;  -no-scenecut       <boolean>    E..V....... When lookahead is enabled, set this to 1 to disable adaptive I-frame insertion at scene cuts (default false)&#xA;  -forced-idr        <boolean>    E..V....... If forcing keyframes, force them as IDR frames. (default false)&#xA;  -b_adapt           <boolean>    E..V....... When lookahead is enabled, set this to 0 to disable adaptive B-frame decision (default true)&#xA;  -spatial-aq        <boolean>    E..V....... set to 1 to enable Spatial AQ (default false)&#xA;  -spatial_aq        <boolean>    E..V....... set to 1 to enable Spatial AQ (default false)&#xA;  -temporal-aq       <boolean>    E..V....... set to 1 to enable Temporal AQ (default false)&#xA;  -temporal_aq       <boolean>    E..V....... set to 1 to enable Temporal AQ (default false)&#xA;  -zerolatency       <boolean>    E..V....... Set 1 to indicate zero latency operation (no reordering delay) (default false)&#xA;  -nonref_p          <boolean>    E..V....... Set this to 1 to enable automatic insertion of non-reference P-frames (default false)&#xA;  -strict_gop        <boolean>    E..V....... Set 1 to minimize GOP-to-GOP rate fluctuations (default false)&#xA;  -aq-strength       <int>        E..V....... When Spatial AQ is enabled, this field is used to specify AQ strength. AQ strength scale is from 1 (low) - 15 (aggressive) (from 1 to 15) (default 8)&#xA;  -cq                <float>      E..V....... Set target quality level (0 to 51, 0 means automatic) for constant quality mode in VBR rate control (from 0 to 51) (default 0)&#xA;  -aud               <boolean>    E..V....... Use access unit delimiters (default false)&#xA;  -bluray-compat     <boolean>    E..V....... Bluray compatibility workarounds (default false)&#xA;  -init_qpP          <int>        E..V....... Initial QP value for P frame (from -1 to 51) (default -1)&#xA;  -init_qpB          <int>        E..V....... Initial QP value for B frame (from -1 to 51) (default -1)&#xA;  -init_qpI          <int>        E..V....... Initial QP value for I frame (from -1 to 51) (default -1)&#xA;  -qp                <int>        E..V....... Constant quantization parameter rate control method (from -1 to 51) (default -1)&#xA;  -weighted_pred     <int>        E..V....... Set 1 to enable weighted prediction (from 0 to 1) (default 0)&#xA;  -coder             <int>        E..V....... Coder type (from -1 to 2) (default default)&#xA;     default         -1           E..V.......&#xA;     auto            0            E..V.......&#xA;     cabac           1            E..V.......&#xA;     cavlc           2            E..V.......&#xA;     ac              1            E..V.......&#xA;     vlc             2            E..V.......&#xA;  -b_ref_mode        <int>        E..V....... Use B frames as references (from 0 to 2) (default disabled)&#xA;     disabled        0            E..V....... B frames will not be used for reference&#xA;     each            1            E..V....... Each B frame will be used for reference&#xA;     middle          2            E..V....... Only (number of B frames)/2 will be used for reference&#xA;  -a53cc             <boolean>    E..V....... Use A53 Closed Captions (if available) (default true)&#xA;  -dpb_size          <int>        E..V....... Specifies the DPB size used for encoding (0 means automatic) (from 0 to INT_MAX) (default 0)&#xA;  -multipass         <int>        E..V....... Set the multipass encoding (from 0 to 2) (default disabled)&#xA;     disabled        0            E..V....... Single Pass&#xA;     qres            1            E..V....... Two Pass encoding is enabled where first Pass is quarter resolution&#xA;     fullres         2            E..V....... Two Pass encoding is enabled where first Pass is full resolution&#xA;  -ldkfs             <int>        E..V....... Low delay key frame scale; Specifies the Scene Change frame size increase allowed in case of single frame VBV and CBR (from 0 to 255) (default 0)&#xA;</int></int></int></boolean></int></int></int></int></int></int></int></boolean></boolean></float></int></boolean></boolean></boolean></boolean></boolean></boolean></boolean></boolean></boolean></boolean></int></int></boolean></boolean></int></int></int></int></int></int></int>

  


  If anyone has some idea or needs more information to help me, I would appreciate it.