Recherche avancée

Sur d’autres sites (6308)

• Latency and DAF in RTP transmissions

  24 février 2023, par jfernandz

  I'm trying to perform some tests for audio RTP transmissions to know their technical limitations. The idea is to prevent DAF effect in this kind of transmissions, I'm assuming a latency lower than 50ms will prevent it. But there is another handicap in my analysis, the RTP transmission must be over WiFi.

  


  For this tests I'm trying to transmit raw audio (not sure if skipping the encoding stage will improve latency) through ffmpeg between two different laptops, so I'm running ffmpeg in the first laptop (172.20.1.2) as :

  


  $ ffmpeg -f pulse -i 56 -c copy -f rtp rtp://172.20.1.5:10000

  


  which produces the following output :

  


  ffmpeg version n5.1.2 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.2.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-version3 --enable-vulkan
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, pulse, from '56':
  Duration: N/A, start: 1677234050.938677, bitrate: 1536 kb/s
  Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
Output #0, rtp, to 'rtp://172.20.1.5:10000':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
SDP:
v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 172.20.1.5
t=0 0
a=tool:libavformat LIBAVFORMAT_VERSION
m=audio 10000 RTP/AVP 97
b=AS:1536

Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
size=     322kB time=00:00:01.67 bitrate=1573.6kbits/s speed=1.06x


  


  I'm assuming the shown SDP is a valid one :

  


  v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 172.20.1.5
t=0 0
a=tool:libavformat LIBAVFORMAT_VERSION
m=audio 10000 RTP/AVP 97
b=AS:1536


  


  So I saved it in a file called ccopy.sdp on the second laptop (172.20.1.5). However, when I run ffplay in this other laptop as :

  


  $ ffplay -protocol_whitelist file,rtp,udp -i ccopy.sdp

  


  I can see there is problems with this SDP :

  


  ffplay version n5.1.2 Copyright (c) 2003-2022 the FFmpeg developers
  built with gcc 12.2.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-opencl --enable-opengl --enable-shared --enable-version3 --enable-vulkan
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
[sdp @ 0x7f8eec000c80] Could not find codec parameters for stream 0 (Audio: none, 0 channels): unknown codec
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, sdp, from 'ccopy.sdp':
  Metadata:
    title           : No Name
  Duration: N/A, bitrate: N/A
  Stream #0:0: Audio: none, 0 channels
Failed to open file 'ccopy.sdp' or configure filtergraph
    nan    :  0.000 fd=   0 aq=    0KB vq=    0KB sq=    0B f=0/0   


  


  Not sure if I'm doing something wrong or this is because of I cannot actually use pcm_s16le for an RTP transmission. Moreover ... Is there some argument for ffmpeg that I can use to improve this RTP transmission and reduce latency under 50ms.

  


  Thank you all :-)

  


  PS : When I don't use -c copy argument for ffmpeg and therefore I have this SDP

  


  v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 172.20.1.5
t=0 0
a=tool:libavformat LIBAVFORMAT_VERSION
m=audio 10000 RTP/AVP 97
b=AS:768
a=rtpmap:97 PCMU/48000/2


  


  The RTP transmission works as I expect, but with a significant DAF.

  


• ffmpeg - when cutting video, last frame is green [on hold]

  4 juillet 2016, par Mister Fresh

  In a php web app, the user can upload a video and cut it in samples. Php launches a shell command with shell_exec() to start ffmpeg and cut the video.

  It works except that the last frame shows a green splash in the cut sample.

  Server is on linux. In dev environment (mac) with latest ffmpeg, the problem cannot be reproduced.

  Command is the following :

  ffmpeg -i /var/www/user_dir/web/projects/127472/cuts/1146 -ss 00:00:45 -t 15 -vcodec libx264 -s 640x360 -strict experimental -v quiet -y /var/www/user_dir/web/projects/127472/cuts/sample1954_small.mp4

  Console output :

  ffmpeg version 0.8.17-6:0.8.17-1, 
  
  
  
  Copyright (c) 2000-2014 the Libav developers
  
  
  
  built on Mar 15 2015 17:00:31 with gcc 4.7.2
  
  
  
  configuration: --arch=amd64 --enable-pthreads --enable-runtime-cpudetect --extra-version='6:0.8.17-1' --libdir=/usr/lib/x86_64-linux-gnu --prefix=/usr --enable-bzlib --enable-libdc1394 --enable-libdirac --enable-libfreetype --enable-frei0r --enable-gnutls --enable-libgsm --enable-libmp3lame --enable-librtmp --enable-libopencv --enable-libopenjpeg --enable-libpulse --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-vaapi --enable-vdpau --enable-libvorbis --enable-libvpx --enable-zlib --enable-gpl --enable-postproc --enable-swscale --enable-libcdio --enable-x11grab --enable-libx264 --enable-libxvid --shlibdir=/usr/lib/x86_64-linux-gnu --enable-shared --disable-static
  
  
  
  libavutil    51. 22. 3 / 51. 22. 3
  
  
  
  libavcodec   53. 35. 0 / 53. 35. 0
  
  
  
  libavformat  53. 21. 1 / 53. 21. 1
  
  
  
  libavdevice  53.  2. 0 / 53.  2. 0
  
  
  
  libavfilter   2. 15. 0 /  2. 15. 0
  
  
  
  libswscale    2.  1. 0 /  2.  1. 0
  
  
  
  libpostproc  52.  0. 0 / 52.  0. 0
  
  
  
  The ffmpeg program is only provided for script compatibility and will be removed
  
  in a future release. It has been deprecated in the Libav project to allow for
  
  incompatible command line syntax improvements in its replacement called avconv
  
  (see Changelog for details). Please use avconv instead.
  
  
  
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/var/www/user_dir/web/projects/127472/cuts/1146':
  
  
  
  Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      encoder         : Lavf56.15.103
  
      genre           : Blues
  
  
  
  Duration: 00:01:29.47, start: 0.036281, bitrate: 808 kb/s
  
  
  
  Stream #0.0(und): Video: h264 (High), yuv420p, 1920x1080 [PAR 1:1 DAR 16:9], 675 kb/s, 25 fps, 25 tbr, 12800 tbn, 50 tbc
  
  
  
  Stream #0.1(und): Audio: aac, 44100 Hz, stereo, s16, 128 kb/s
  
  [buffer @ 0x30a2ca0] w:1920 h:1080 pixfmt:yuv420p
  
  [scale @ 0x309bf00] w:1920 h:1080 fmt:yuv420p -> w:640 h:360 fmt:yuv420p flags:0x4
  
  [libx264 @ 0x30a3200] using SAR=1/1
  
  [libx264 @ 0x30a3200] using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
  
  [libx264 @ 0x30a3200] profile Main, level 3.0
  
  [libx264 @ 0x30a3200] 264 - core 123 r2189 35cf912 - H.264/MPEG-4 AVC codec - 
  
  
  
      Copyleft 2003-2012 - http://www.videolan.org/x264.html 
  
  - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=0 b_adapt=1 b_bias=0 direct=1 weightb=0 open_gop=1 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.25 aq=1:1.00
  
  
  
  Output #0, mp4, to '/var/www/user_dir/web/projects/127472/cuts/sample1954_small.mp4':
  
  
  
  Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      genre           : Blues
  
      encoder         : Lavf53.21.1
  
  
  
  Stream #0.0(und): Video: libx264, yuv420p, 640x360 [PAR 1:1 DAR 16:9], q=-1--1, 25 tbn, 25 tbc
  
  
  
  Stream #0.1(und): Audio: aac, 44100 Hz, stereo, s16, 200 kb/s
  
  Stream mapping:
  
    Stream #0.0 -> #0.0
  
    Stream #0.1 -> #0.1
  
  
  
  Press ctrl-c to stop encoding
  
  
  
  [buffer @ 0x30a2ca0] Buffering several frames is not supported. Please consume all available frames before adding a new one.
  
  
  
  Last message repeated 1124 times 565kB time=12.80 bitrate= 361.7kbits/s    ts/s    
  
  frame=  375 fps= 17 q=28.0 Lsize=     742kB time=14.96 bitrate= 406.4kbits/s    
  
  video:360kB audio:371kB global headers:0kB muxing overhead 1.515374%
  
  frame I:3     Avg QP:16.97  size:  6922
  
  
  
  [libx264 @ 0x30a3200] frame P:158   Avg QP:24.40  size:  1858
  
  
  
  [libx264 @ 0x30a3200] frame B:214   Avg QP:28.69  size:   253
  
  
  
  [libx264 @ 0x30a3200] consecutive B-frames: 13.9% 28.3%  5.6% 52.3%
  
  
  
  [libx264 @ 0x30a3200] mb I  I16..4: 70.8%  0.0% 29.2%
  
  
  
  [libx264 @ 0x30a3200] mb P  I16..4:  4.8%  0.0%  1.6%  P16..4: 16.0%  3.9%  2.0%  0.0%  0.0%    skip:71.9%
  
  
  
  [libx264 @ 0x30a3200] mb B  I16..4:  0.5%  0.0%  0.1%  B16..8: 11.1%  0.7%  0.1%  direct: 0.1%  skip:87.4%  L0:44.2% L1:44.7% BI:11.1%
  
  
  
  [libx264 @ 0x30a3200] coded y,uvDC,uvAC intra: 19.3% 30.9% 23.5% inter: 2.3% 2.3% 1.3%
  
  
  
  [libx264 @ 0x30a3200] i16 v,h,dc,p: 39% 47%  2% 12%
  
  
  
  [libx264 @ 0x30a3200] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 25% 32%  5%  5%  3%  4%  3%  3%
  
  
  
  [libx264 @ 0x30a3200] i8c dc,h,v,p: 51% 40%  7%  2%
  
  
  
  [libx264 @ 0x30a3200] Weighted P-Frames: Y:0.0% UV:0.0%
  
  
  
  [libx264 @ 0x30a3200] ref P L0: 63.7%  4.9% 19.3% 12.1%
  
  
  
  [libx264 @ 0x30a3200] ref B L0: 67.6% 32.4%
  
  
  
  [libx264 @ 0x30a3200] kb/s:196.48

• RoQ on Dreamcast

  18 mars 2011, par Multimedia Mike — Sega Dreamcast

  I have been working on that challenge to play back video on the Sega Dreamcast. To review, I asserted that the RoQ format would be a good fit for the Sega Dreamcast hardware. The goal was to play 640x480 video at 30 frames/second. Short version : I have determined that it is possible to decode such video in real time. However, I ran into certain data rate caveats.

  First off : Have you ever wondered if the Dreamcast can read an 80mm optical disc ? It can ! I discovered this when I only had 60 MB of RoQ samples to burn on a disc and a spindle full of these 210MB-capacity 80mm CD-Rs that I never have occasion to use.

  
  
  

  New RoQ Library
  There are open source RoQ decoders out there but I decided to write a new one. A few reasons : 1) RoQ is so simple that I didn’t think it would take too long ; 2) it would be nice to have a RoQ library that is license-compatible (BSD-like) with the rest of the KallistiOS distribution ; 3) the idroq.tar.gz distribution, while license-compatible, has enough issues that I didn’t want to correct it.

  Thankfully, I was correct about the task not being too difficult : I put together a new RoQ decoder in short order. I’m a bit embarrassed to admit that the part I had the most trouble with was properly converting YUV -> RGB.
  
  About the approach I took : While the original idroq.tar.gz decoder maintains YUV 4:2:0 codebooks (which led to chroma bugs during motion compensation) and FFmpeg’s decoder maintains YUV 4:4:4 codebooks, this decoder is built to convert the YUV 4:2:0 vectors into RGB565 vectors during the vector unpacking phase. Thus, the entire frame is rendered in RGB565 — no lengthy YUV -> RGB conversion after decoding — and all pixels are shuffled around as 16-bit units (minor speedup vs. shuffling everything as bytes).

  I also entertained the idea of maintaining YUYV codebooks (since the DC supports that colorspace as a texture format). But I scrapped that idea when I remembered it would lead to the same chroma bleeding problem seen in the original idroq.tar.gz decoder.

  Onto The Dreamcast
  I developed the library on a Linux computer, allowing it to output a series of PNM files for visual verification and debugging. Dropping it into a basic DC/KOS-compatible program was trivial and the first order of business was profiling.

  At first, I profiled the entire decode operation : open file, then read and decode each chunk while tossing away the results. I was roundly disappointed to see that, e.g., an 8.5-second RoQ sample needed a little more than 20 seconds to complete. Not real time. I performed a series of optimizations on the decoding library that netted notable performance gains when profiling on Linux. When I brought these same optimizations over to the DC, decoding time didn’t improve at all. This was my first suspicion that perhaps my assumptions regarding the DC’s optical drive’s data rate were not correct.

  Dreamcast Data Rate Profiling
  Let’s start with some definitions : In terms of data rate, an ’X’, i.e., 1X is the minimum data rate needed to read CD quality audio from a disc. At that speed, a drive should be able to stream 75 sectors each second. When reading mode 1/form 1 CD-ROM data, each sector has 2048 bytes (2 kbytes), so a single-speed data rate should achieve 150 kbytes/sec.

  The Dreamcast is supposed to possess a 12X optical drive. This would imply a maximum data rate of 150 kbytes/sec * 12 = 1800 kbytes/sec.

  Rigging up a trivial experiment using the RoQ samples burned on a few different CD-R discs, the best data rate I can see is about 500-525 kbytes/sec, or around 3.5X.

  Where’s the discrepancy ? My first theory has to do with the fact that not all optical media is created equal. This is why optical drives often advertise a slew of numbers which refer to the best theoretical speed for reading a CD vs. writing a CD-R vs. writing a CD-RW, etc. Perhaps the DC drive can’t read CD-Rs very quickly. To test this theory, I tried streaming a large file from a conventionally mastered CD-ROM. This worked well for the closest CD-ROM I had on hand : I was able to stream data at a rate that works out to about 6.5X.

  I smell a science project for another evening : Profiling read speeds from a mastered CD-ROM, burned CD-R, and also a mastered GD-ROM, on each of the 3 Dreamcast consoles I possess (I’ve heard that there’s variance between optical drives depending on manufacturing run).

  The Good News
  I added a little finer-grained code to profile just the video decoding functions. The good news is that the decoder meets my real time goals : That 8.5-second RoQ sample encoded at 640x480x30fps makes its way through the video decoding functions on the DC in a little less than 5 seconds. If the optical drive can supply the data fast enough, the video decoder can take care of the rest.

  The RoQ encoder included with FFmpeg does not honor any bitrate parameters. Instead, I encoded the same file at 320x240. It reportedly decoded in real time and can be streamed in real time as well.

  I say "reportedly" because I’m simply working from textual output at this point ; the next phase is to hook the decoder up to the display hardware.