Recherche avancée

Sur d’autres sites (9073)

• How to send ffmpeg AVPacket through WebRTC (using libdatachannel)

  14 novembre 2022, par mike

  I'm encoding a video frame with the ffmpeg libraries, generating an AVPacket with compressed data.

  


  Thanks to some recent advice here on S/O, I am trying to send that frame over a network using the WebRTC library libdatachannel, specifically by adapting the example here :

  


  https://github.com/paullouisageneau/libdatachannel/tree/master/examples/streamer

  


  I am seeing problems inside h264rtppacketizer.cpp (part of the library, not the example) which are almost certainly to do with how I'm providing the sample data.
(I don't think that this is anything to do with libdatachannel specifically, it will be an issue with what I'm sending)

  


  The example code reads each encoded frame from a file, and populates a sample by setting the content of the file to the contents of the file :

  


  sample = *reinterpret_cast *>(&fileContents);

  


  sample is just a std::vector<byte>;</byte>

  


  I have naively copied the contents of an AVPacket->data pointer into the sample vector :

  


  sample.resize(pkt->size);
memcpy(sample.data(), pkt->data, pkt->size * sizeof(std::byte));    


  


  but the packetizer is falling over when trying to get length values out of that data.
Specifically, in the following code, the first iteration gets a length of 1, but the second, looking up index 5, gives 1119887324. This is way too big for my data, which is only 3526 bytes (the whole frame is a single colour so likely to be small once encoded) :

  


  while (index &lt; message->size()) {&#xA;assert(index &#x2B; 4 &lt; message->size());&#xA;auto lengthPtr = (uint32_t *)(message->data() &#x2B; index);&#xA;uint32_t length = ntohl(*lengthPtr);&#xA;auto naluStartIndex = index &#x2B; 4;&#xA;auto naluEndIndex = naluStartIndex &#x2B; length;&#xA;assert(naluEndIndex &lt;= message->size());    &#xA;        &#xA;auto begin = message->begin() &#x2B; naluStartIndex;&#xA;auto end = message->begin() &#x2B; naluEndIndex;&#xA;nalus->push_back(std::make_shared<nalunit>(begin, end));&#xA;index = naluEndIndex;&#xA;}&#xA;</nalunit>

  


  Here is a dump of

  


  uint32_t length = ntohl(*lengthPtr);


  


  for the first few elements of the message (*lengthPtr in parentheses) :

  


  [2022-03-29 15:12:01.182] [info] index 0: 1  (16777216)
[2022-03-29 15:12:01.183] [info] index 1: 359  (1728118784)
[2022-03-29 15:12:01.184] [info] index 2: 91970  (1114046720)
[2022-03-29 15:12:01.186] [info] index 3: 23544512  (3225577217)
[2022-03-29 15:12:01.186] [info] index 4: 1732427807  (532693607)
[2022-03-29 15:12:01.187] [info] index 5: 1119887324  (3693068354)
[2022-03-29 15:12:01.188] [info] index 6: 3223313413  (98312128)
[2022-03-29 15:12:01.188] [info] index 7: 534512896  (384031)
[2022-03-29 15:12:01.188] [info] index 8: 3691315291  (1526728156)
[2022-03-29 15:12:01.189] [info] index 9: 83909537  (2707095557)
[2022-03-29 15:12:01.189] [info] index 10: 6004992  (10574592)
[2022-03-29 15:12:01.190] [info] index 11: 1537277952  (41307)
[2022-03-29 15:12:01.190] [info] index 12: 2701131779  (50331809)
[2022-03-29 15:12:01.192] [info] index 13: 768  (196608)


  


  (I know I should post a complete sample, I am working on it)

  


  


  • I am fairly sure I am just missing something basic. E.g. am I supposed to do something with the AVPacket side_data, does AVPacket have or miss some header info ?

    


  


  • If I just fwrite the pkt->data for a single frame to disk, I can read the codec information with ffprobe :

    


  


  


  Input #0, h264, from 'encodedOut.h264':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 1280x720, 30 tbr, 1200k tbn


  


  Update : This issue is solved by changing the H264RtpPacketizer separator setting from H264RtpPacketizer::Separator::Length to H264RtpPacketizer::Separator::LongStartSequence, many thanks to author of libdatachannel paullouisageneau (see answer below)

  


  I have issues related to settings for the libx264 encoder, but can happily encode with h264_nvenc and h264_mf

  


• How correctly show video with transparency in Qt with OpenCV + FFMpeg

  11 avril 2022, par TheEnigmist

  I'm trying to show a video with transparency in a Qt6 application using OpenCV + FFMPEG.
Actually those are tool versions :

  


  

  • Win 11

  


  • Qt 6.3.0

  


  • OpenCV 4.5.5 (built with CMake)

  


  • FFMPEG 2022-04-03-git-1291568c98-full_build-www.gyan.dev

  


  


  I've used a base .mov video with transparency as test (link provided below).
First of all I've converted .mov video to .webm video (VP9) and I see in output text that alpha channel remains

  


  


  ffmpeg -i '.\Retro Bars.mov' -c:v libvpx-vp9 -crf 30 -b:v 0 output.webm

  


  


  Input #0, mov,mp4,m4a,3gp,3g2,mj2,
    ...
    Stream #0:0[0x1](eng): Video: qtrle (rle  / 0x20656C72), argb(progressive),
    ...

Output #0, webm, 
   ...
   Stream #0:0(eng): Video: vp9, yuva420p(tv, progressive),
   ...


  


  But when I show info of output file with ffmpeg it loses alpha channel :

  


  


  ffmpeg -i .\output.webm

  


  


  Input #0, matroska,webm,
    ...
    Stream #0:0(eng): Video: vp9 (Profile 0), yuv420p(tv, progressive),
    ...


  


  If I open output.webm with OBS it is shown correctly without a background, as shown in picture :


  


  If I try to open it with OpenCV + FFMPEG it shows a black background under bars, as shown in picture :


  


  This is how I load video in Qt :

  


  cv::VideoCapture capture;
capture.open(filename, cv::CAP_FFMPEG);
capture.set(cv::CAP_PROP_CONVERT_RGB, false); // try forcing load alpha channel
... //in a thread
while (capture.read(frame)) {
    qDebug() << "c" << frame.channels() << "t" <<  frame.type() << "d" <<  frame.depth(); // output: c 3 t 16 d 0
    cv::cvtColor(frame, frame, cv::COLOR_BGR2RGBA); //useless since no alpha channel is detected
    img = QImage(frame.data, frame.cols, frame.rows, QImage::Format_RGBA8888);
    emit processedImage(img); // to show image in a QLabel with QPixmap::fromImage(img)
}


  


  I think the problem is when I load the video with OpenCV, it doens't detect alpha channel, since I can load correctly in other player (obs, html5, etc.)

  


  What I'm wrong with all process to show this video in Qt with transparency ?

  


  EDIT : Added dropbox link with test video + ffmpeg outputs :
sample items

  


• FFMpeg crop a portrait (vertical) video square 1:1

  1er avril 2022, par hugger

  SOLVED... It was a UI issue... Not an FFpeg issue.

  


  I am new to FFMpeg. I am stuck on cropping a portrait video taken from a portrait device square.

  


  I would like my video output to be 1080x1080.

  


  First, I tried this.

  


  FFmpegKit.execute(`-y -i ${media.path} -vf "crop=1080:1080:exact=1" ${path}`)


  


  As I hoped, this worked for photos. (but strangely rotates the photo) - looking into that...

  


  BUT, for videos it does not work. instead it turns the video landscape.

  


  I then tried to add scale :

  


  FFmpegKit.execute(`-y -i ${media.path} -vf "crop=1080:1080:exact=1, scale=1080:1080" ${path}`)


  


  But this left me with the same result.

  


  Here are the console logs for some more information :

  


   LOG  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file:///private/var/mobile/Containers/Data/Application/71E462FC-4824-41FE-B28D-57AF7B6078C3/tmp/ReactNative/329ACC6F-42B6-4B12-A289-889DADE1BC3A.mov':
 LOG    Metadata:
 LOG      major_brand     :
 LOG  qt
 LOG  
 LOG      minor_version   :
 LOG  0
 LOG  
 LOG      compatible_brands:
 LOG  qt
 LOG  
 LOG      creation_time   :
 LOG  2022-04-01T03:41:12.000000Z
 LOG  
 LOG    Duration:
 LOG  00:00:02.35
 LOG  , start:
 LOG  0.000000
 LOG  , bitrate:
 LOG  21200 kb/s
 LOG  
 LOG    Stream #0:0
 LOG  [0x1]
 LOG  (und)
 LOG  : Video: hevc (hvc1 / 0x31637668), yuv420p(tv, bt709), 1080x1920, 21140 kb/s
 LOG  ,
 LOG  59.96 fps,
 LOG  59.94 tbr,
 LOG  600 tbn
 LOG   (default)
 LOG  
 LOG      Metadata:
 LOG        creation_time   :
 LOG  2022-04-01T03:41:13.000000Z
 LOG  
 LOG        handler_name    :
 LOG  Core Media Video
 LOG  
 LOG        vendor_id       :
 LOG  [0][0][0][0]
 LOG  
 LOG        encoder         :
 LOG  HEVC
 LOG  
 LOG    Stream #0:1
 LOG  [0x2]
 LOG  (und)
 LOG  : Audio: aac (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 61 kb/s
 LOG   (default)
 LOG  
 LOG      Metadata:
 LOG        creation_time   :
 LOG  2022-04-01T03:41:13.000000Z
 LOG  
 LOG        handler_name    :
 LOG  Core Media Audio
 LOG  
 LOG        vendor_id       :
 LOG  [0][0][0][0]
 LOG  
 LOG  [hevc @ 0x118ce07d0] The "sub_text_format" option is deprecated: Deprecated, does nothing
 LOG  [aac @ 0x10d30e190] The "sub_text_format" option is deprecated: Deprecated, does nothing
 LOG  Stream mapping:
 LOG    Stream #0:0 -> #0:0
 LOG   (hevc (native) -> mpeg4 (native))
 LOG  
 LOG    Stream #0:1 -> #0:1
 LOG   (aac (native) -> aac (native))
 LOG  
 LOG  Press [q] to stop, [?] for help
 LOG  Output #0, mp4, to '/var/mobile/Containers/Data/Application/71E462FC-4824-41FE-B28D-57AF7B6078C3/Documents/after.mp4':
 LOG    Metadata:
 LOG      major_brand     :
 LOG  qt
 LOG  
 LOG      minor_version   :
 LOG  0
 LOG  
 LOG      compatible_brands:
 LOG  qt
 LOG  
 LOG      encoder         :
 LOG  Lavf59.10.100
 LOG  
 LOG    Stream #0:0
 LOG  (und)
 LOG  : Video: mpeg4 (mp4v / 0x7634706D), yuv420p(tv, bt709, progressive), 1080x1080, q=2-31, 10000 kb/s
 LOG  ,
 LOG  59.94 fps,
 LOG  60k tbn
 LOG   (default)
 LOG  
 LOG      Metadata:
 LOG        creation_time   :
 LOG  2022-04-01T03:41:13.000000Z
 LOG  
 LOG        handler_name    :
 LOG  Core Media Video
 LOG  
 LOG        vendor_id       :
 LOG  [0][0][0][0]
 LOG  
 LOG        encoder         :
 LOG  Lavc59.15.102 mpeg4
 LOG  
 LOG      Side data:
 LOG  
 LOG  cpb:
 LOG  bitrate max/min/avg: 0/0/10000000 buffer size: 0
 LOG  vbv_delay: N/A
 LOG  
 LOG    Stream #0:1
 LOG  (und)
 LOG  : Audio: aac (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 69 kb/s
 LOG   (default)
 LOG  
 LOG      Metadata:
 LOG        creation_time   :
 LOG  2022-04-01T03:41:13.000000Z
 LOG  
 LOG        handler_name    :
 LOG  Core Media Audio
 LOG  
 LOG        vendor_id       :
 LOG  [0][0][0][0]
 LOG  
 LOG        encoder         :
 LOG  Lavc59.15.102 aac
 LOG  
 LOG  frame=    1 fps=0.0 q=3.6 size=       0kB time=00:00:01.06 bitrate=   0.3kbits/s speed=9.23x
 LOG  frame=   47 fps=0.0 q=2.0 size=     768kB time=00:00:01.85 bitrate=3390.0kbits/s speed=3.01x
 LOG  frame=   95 fps= 81 q=2.2 size=    1792kB time=00:00:02.32 bitrate=6313.3kbits/s speed=1.99x
 LOG  frame=  129 fps= 77 q=2.5 size=    2560kB time=00:00:02.32 bitrate=9018.9kbits/s speed=1.39x
 LOG  frame=  139 fps= 78 q=2.6 Lsize=    2953kB time=00:00:02.38 bitrate=10124.6kbits/s speed=1.34x
 LOG  video:2929kB audio:20kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead:
 LOG  0.136336%
 LOG  
 LOG  [aac @ 0x112820f50] Qavg: 113.412
 LOG  COMPLETED
 LOG  {}


  


  This is the result, when i replace
-i {media} with
-f lavfi -i smptebars=r=60000/1001:s=1080x1920:d=1

  


  

  


  ALMOST but not 1:1...

  


  I have not had any luck online trying to find a solution for this so I have decided to post on here.

  


  I hope I can get some guidance here !

  


  Cheers.