Recherche avancée

Sur d’autres sites (6010)

• Extracting each individual frame from an H264 stream for real-time analysis with OpenCV

  5 mai 2017, par exclmtnpt

  Problem Outline

  I have an h264 real-time video stream (I’ll call this "the stream") being captured in Process1. My goal is to extract each frame from the stream as it comes through and use Process2 to analyze it with OpenCV. (Process1 is nodejs, Process2 is Python)

  Things I’ve tried, and their failure modes :

  • Send the stream directly from one Process1 to Process2 over a named fifo pipe :

  I succeeded in directing the stream from Process1 into the pipe. However, in Process2 (which is Python) I could not (a) extract individual frames from the stream, and (b) convert any extracted data from h264 into an OpenCV format (e.g. JPEG, numpy array).

  I had hoped to use OpenCV’s VideoCapture() method, but it does not allow you to pass a FIFO pipe as an input. I was able to use VideoCapture by saving the h264 stream to a .h264 file, and then passing that as the file path. This doesn’t help me, because I need to do my analysis in real time (i.e. I can’t save the stream to a file before reading it in to OpenCV).

  • Pipe the stream from Process1 to FFMPEG, use FFMPEG to change the stream format from h264 to MJPEG, then pipe the output to Process2 :

  I attempted this using the command :

  cat pipeFromProcess1.fifo | ffmpeg -i pipe:0 -f h264 -f mjpeg pipe:1 | cat > pipeToProcess2.fifo

  The biggest issue with this approach is that FFMPEG takes inputs from Process1 until Process1 is killed, and only then does Process2 begin to receive the data.

  Additionally, on the Process2 side, I still don’t understand how to extract individual frames from the data coming over the pipe. I open the pipe for reading (as "f") and then execute data = f.readline(). The size of data varies drastically (some reads have length on the order of 100, others length on the order of 1,000). When I use f.read() instead of f.readline(), the length is much larger, on the order of 100,000.

  If I were to know that I was getting the correct size chunk of data, I would still not know how to transform it into an OpenCV-compatible array because I don’t understand the format it’s coming over in. It’s a string, but when I print it out it looks like this :

  ��_M 0A0����tQ,\%��e���f/�H�#Y�p�f#�Kus�} F����ʳa�G������+$x�%V�� }[����Wo �1’̶A���c����*�&=Z^�o’��Ͽ� SX-ԁ涶V&H|��$
   ��<�E�� ��>�����u���7�����cR� �f�=�9 ��fs�q�ڄߧ�9v�]�Ӷ���& gr]�n�IRܜ�檯����
  � ����+ �I��w�}� ��9�o��� �w��M�m���IJ ��� �m�=�Soՙ}S �>j �,�ƙ�’���tad =i ��WY�FeC֓z �2�g� ;EXX��S��Ҁ*, ���w� _|�&�y��H��=��)� ���Ɗ3@ �h���Ѻ�Ɋ��ZzR`��)�y�� c�ڋ.��v� !u���� �S�I#�$9R�Ԯ0py z ��8 #��A�q�� �͕� ijc �bp=��۹ c SqH

  Converting from base64 doesn’t seem to help. I also tried :

  array = np.fromstring(data, dtype=np.uint8)

  which does convert to an array, but not one of a size that makes sense based on the 640x368x3 dimensions of the frames I’m trying to decode.

  • Using decoders such as Broadway.js to convert the h264 stream :

  These seem to be focused on streaming to a website, and I did not have success trying to re-purpose them for my goal.

  Clarification about what I’m NOT trying to do :

  I’ve found many related questions about streaming h264 video to a website. This is a solved problem, but none of the solutions help me extract individual frames and put them in an OpenCV-compatible format.

  Also, I need to use the extracted frames in real time on a continual basis. So saving each frame as a .jpg is not helpful.

  System Specs

  Raspberry Pi 3 running Raspian Jessie

  Additional Detail

  I’ve tried to generalize the problem I’m having in my question. If it’s useful to know, Process1 is using the node-bebop package to pull down the h264 stream (using drone.getVideoStream()) from a Parrot Bebop 2.0. I tried using the other video stream available through node-bebop (getMjpegStream()). This worked, but was not nearly real-time ; I was getting very intermittent data streams. I’ve entered that specific problem as an Issue in the node-bebop repository.

  Thanks for reading ; I really appreciate any help anyone can give !

• Transcoding HEVC 4K HDR using ffmpeg and vaapi

  13 mai 2021, par P. Brand

  I am trying to transcode a 4K HEVC HDR video to a lower bitrate using ffmpeg and VAAPI hardware acceleration for my LG OLED television.

  &#xA;&#xA;

  I use the following file :&#xA;Video

  &#xA;&#xA;

  Using the following command :

  &#xA;&#xA;

  ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device va -filter_hw_device va -i LG\ 4K\ HDR\ Demo\ -\ Daylight.mkv -map 0:0 -t 00:00:20 -c:v hevc_vaapi -sei hdr -qp:v 21 lg_vaapi.mkv


  &#xA;&#xA;

  But this file is not recognized as HDR on my TV.

  &#xA;&#xA;

  When using the libx265 encoder (which is off course much slower) it works fine :

  &#xA;&#xA;

  ffmpeg -i LG\ 4K\ HDR\ Demo\ -\ Daylight.mkv -t 00:00:20 -map 0:0 -c:v libx265 -preset medium -crf 18 -x265-params "colorprim=bt2020:colormatrix=bt2020nc:transfer=smpte2084:colormatrix=bt2020nc:hdr=1:info=1:repeat-headers=1:master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(11000000,40)" lg_libx265.mkv


  &#xA;&#xA;

  When comparing the output of mediainfo on both files I can see the following properties are not set for the video track :

  &#xA;&#xA;

  &#xA;
  • Color range

  &#xA;

  • Color primaries

  &#xA;

  • Transfer characteristics

  &#xA;

  &#xA;&#xA;

  mediainfo for vaapi encode :

  &#xA;&#xA;

  General
Unique ID                                : 168011494166392912924249315217763643529 
(0x7E65D636029A5354FE73998A5ED6B089)
Complete name                            : lg_vaapi.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 118 MiB
Duration                                 : 20 s 4 ms
Overall bit rate                         : 49.6 Mb/s
Writing application                      : Lavf58.20.100
Writing library                          : Lavf58.20.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@Main
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 20 s 4 ms
Bit rate                                 : 48.6 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 (60000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.098
Stream size                              : 116 MiB (98%)
Writing library                          : Lavc58.35.100 hevc_vaapi
Default                                  : Yes
Forced                                   : No
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0040 cd/m2, max: 1100 cd/m2


  &#xA;&#xA;

  mediainfo for the libx265 encode :

  &#xA;&#xA;

  General
Unique ID                                : 261605749278874425324960669956061374077 (0xC4CF6F42FE952004772A02D5819DA67D)
Complete name                            : lg_libx265.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 155 MiB
Duration                                 : 47 s 715 ms
Overall bit rate                         : 27.3 Mb/s
Writing application                      : Lavf58.20.100
Writing library                          : Lavf58.20.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@Main
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 47 s 715 ms
Bit rate                                 : 26.8 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 (60000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.054
Stream size                              : 152 MiB (98%)
Writing library                          : x265 2.8:[Linux][GCC 9.1.0][64 bit] 10bit
Encoding settings                        : cpuid=1111039 / frame-threads=3 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=8 / scenecut=40 / radl=0 / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=2 / limit-refs=3 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=3 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=0 / display-window=0 / master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,50) / max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-mv-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0040 cd/m2, max: 1100 cd/m2


  &#xA;&#xA;

  I suspect Color range, Color primaries and Transfer characteristics are necessary to make the file recognized as HDR by my TV. &#xA;How can I add these properties to the VAAPI encoded file ?

  &#xA;&#xA;

  Edit :&#xA;This patch fixes the problem :&#xA;Patch

  &#xA;

• lavc/amrwbdec : Remove a dead comparison.

  11 avril 2020, par Carl Eugen Hoyos

  lavc/amrwbdec : Remove a dead comparison.
  Modes > MODE_SID are already filtered out at this point.
  
  Simplifies the following change.
  

  • [DH] libavcodec/amrwbdec.c