Recherche avancée

Sur d’autres sites (14321)

• WebRTC books – a brief review

  1er janvier 2014, par silvia

  I just finished reading Rob Manson’s awesome book “Getting Started with WebRTC” and I can highly recommend it for any Web developer who is interested in WebRTC.

  Rob explains very clearly how to create your first video, audio or data peer-connection using WebRTC in current Google Chrome or Firefox (I think it also now applies to Opera, though that wasn’t the case when his book was published). He makes available example code, so you can replicate it in your own Web application easily, including the setup of a signalling server. He also points out that you need a ICE (STUN/TURN) server to punch through firewalls and gives recommendations for what software is available, but stops short of explaining how to set them up.

  Rob’s focus is very much on the features required in a typical Web application :

  • video calls
  • audio calls
  • text chats
  • file sharing

  In fact, he provides the most in-depth demo of how to set up a good file sharing interface I have come across.

  Rob then also extends his introduction to WebRTC to two key application areas : education and team communication. His recommendations are spot on and required reading for anyone developing applications in these spaces.

  —

  Before Rob’s book, I have also read Alan Johnson and Dan Burnett’s “WebRTC” book on APIs and RTCWEB protocols of the HTML5 Real-Time Web.

  Alan and Dan’s book was written more than a year ago and explains that state of standardisation at that time. It’s probably a little out-dated now, but it still gives you good foundations on why some decisions were made the way they are and what are contentious issues (some of which still remain). If you really want to understand what happens behind the scenes when you call certain functions in the WebRTC APIs of browsers, then this is for you.

  Alan and Dan’s book explains in more details than Rob’s book how IP addresses of communication partners are found, how firewall holepunching works, how sessions get negotiated, and how the standards process works. It’s probably less useful to a Web developer who just wants to implement video call functionality into their Web application, though if something goes wrong you may find yourself digging into the details of SDP, SRTP, DTLS, and other cryptic abbreviations of protocols that all need to work together to get a WebRTC call working.

  —

  Overall, both books are worthwhile and cover different aspects of WebRTC that you will stumble across if you are directly dealing with WebRTC code.

• Texai Remote Presence System Using VP8 Video

  19 novembre 2010, par noreply@blogger.com (John Luther)

  Guest blogger Josh Tyler is a member of the Texai team at Willow Garage.

  Willow Garage is busy building the next version of its Texai remote presence platform with VP8, the video codec used in WebM.

  In short, Texai is a two-way videoconferencing app on a tele-operated robotic platform (for more details, see the piece about Texai in the New York Times). Video and audio quality are critical to providing the best user experience on Texai. We’ve evaluated several video codecs and found VP8’s image quality, low latency and tolerance to packet loss far better than anything else we tested.

  The video below shows VP8 running on one of our systems.

  (If you have a WebM-enabled browser and are enrolled in the YouTube HTML5 beta the video will play in WebM HTML5, otherwise it will play in Flash Player.)

  We’re also looking for help ! If you’re interested in helping us create an incredible, high-fidelity user experience, either by joining our team, partnering on development, or by being added to our early customer interest list, please email us at texai-info@willowgarage.com.

  

• Overlaying a transparent webm on an MP4 freezes at the start sometimes

  23 janvier 2021, par joshlat94

  I'm currently using the command
 ffmpeg -i mainvideo.mp4 -vcodec libvpx -i myoverlay.webm -filter_complex "overlay = 5:5" done.mp4
To overlay a transparent webm on top of my main mp4. But for some reason the output video is frozen for the first 1 second but the audio is completely fine.

  


  I've tried this command with other videos and it works sometimes but most of the time the first few frames seem frozen

  


  Edit.
I have two main mp4 files, broken_source.mp4 and working_source.mp4. Putting the overlay on broken_source results in the output having the first 1 second of video frozen (audio completely fine) but putting the same overlay on working_source.mp4 is perfect and has no issues

  


  Logs for putting the overlay on working_source :

  


  ffmpeg -i working_source.mp4 -vcodec libvpx -i chat.webm -filter_complex "overlay = 0:0" working_complete.mp4
ffmpeg version 4.3.1-2021-01-01-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'working_source.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.51.101
  Duration: 00:00:31.02, start: 0.000000, bitrate: 6176 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 6035 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[libvpx @ 0000016fb5074a80] v1.9.0-128-g3a38edea2
    Last message repeated 1 times
Input #1, matroska,webm, from 'chat.webm':
  Metadata:
    ENCODER         : Lavf58.51.101
  Duration: 00:00:31.00, start: 0.000000, bitrate: 449 kb/s
    Stream #1:0: Video: vp8, yuva420p(tv, progressive), 500x300, SAR 1:1 DAR 5:3, 60 fps, 60 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      alpha_mode      : 1
      ENCODER         : Lavc58.101.101 libvpx
      DURATION        : 00:00:31.000000000
[libvpx @ 0000016fb50761c0] v1.9.0-128-g3a38edea2
Stream mapping:
  Stream #0:0 (h264) -> overlay:main (graph 0)
  Stream #1:0 (libvpx) -> overlay:overlay (graph 0)
  overlay (graph 0) -> Stream #0:0 (libx264)
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libvpx @ 0000016fb50761c0] v1.9.0-128-g3a38edea2
[libx264 @ 0000016fb5072040] using SAR=1/1
[libx264 @ 0000016fb5072040] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000016fb5072040] profile High, level 4.2, 4:2:0, 8-bit
[libx264 @ 0000016fb5072040] 264 - core 161 r3027 4121277 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'working_complete.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 60 fps, 15360 tbn, 60 tbc (default)
    Metadata:
      encoder         : Lavc58.91.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.91.100 aac
frame= 1861 fps= 44 q=-1.0 Lsize=   20013kB time=00:00:31.01 bitrate=5285.3kbits/s speed=0.729x
video:19477kB audio:486kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.244893%
[libx264 @ 0000016fb5072040] frame I:8     Avg QP:19.04  size:240534
[libx264 @ 0000016fb5072040] frame P:538   Avg QP:23.76  size: 23836
[libx264 @ 0000016fb5072040] frame B:1315  Avg QP:31.66  size:  3951
[libx264 @ 0000016fb5072040] consecutive B-frames:  3.0%  3.5% 14.8% 78.7%
[libx264 @ 0000016fb5072040] mb I  I16..4:  6.4% 47.4% 46.2%
[libx264 @ 0000016fb5072040] mb P  I16..4:  1.2%  4.2%  1.9%  P16..4: 21.4%  5.3%  2.9%  0.0%  0.0%    skip:63.1%
[libx264 @ 0000016fb5072040] mb B  I16..4:  0.1%  0.2%  0.1%  B16..8: 12.6%  1.5%  0.5%  direct: 0.4%  skip:84.6%  L0:35.9% L1:60.0% BI: 4.1%
[libx264 @ 0000016fb5072040] 8x8 transform intra:55.5% inter:60.0%
[libx264 @ 0000016fb5072040] coded y,uvDC,uvAC intra: 63.5% 77.4% 40.4% inter: 3.7% 4.6% 0.7%
[libx264 @ 0000016fb5072040] i16 v,h,dc,p: 19% 37% 15% 29%
[libx264 @ 0000016fb5072040] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 18% 23%  7%  7%  7%  7%  6%  8%
[libx264 @ 0000016fb5072040] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 20% 22% 17%  7%  8%  7%  7%  6%  6%
[libx264 @ 0000016fb5072040] i8c dc,h,v,p: 45% 27% 18% 11%
[libx264 @ 0000016fb5072040] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000016fb5072040] ref P L0: 74.5% 13.9%  9.1%  2.5%
[libx264 @ 0000016fb5072040] ref B L0: 91.9%  7.4%  0.7%
[libx264 @ 0000016fb5072040] ref B L1: 96.6%  3.4%
[libx264 @ 0000016fb5072040] kb/s:5144.05
[aac @ 0000016fb5026a80] Qavg: 193.230 


  


  Logs for putting the overlay on broken_source :

  


  ffmpeg -i broken_source.mp4 -vcodec libvpx -i chat.webm -filter_complex "overlay = 0:0" broken_complete.mp4
ffmpeg version 4.3.1-2021-01-01-full_build-www.gyan.dev Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 10.2.0 (Rev5, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-lzma --enable-libsnappy --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libzvbi --enable-librav1e --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'broken_source.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.26.101
  Duration: 00:00:45.02, start: 0.000000, bitrate: 5962 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 5952 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[libvpx @ 0000021071f90280] v1.9.0-128-g3a38edea2
    Last message repeated 1 times
Input #1, matroska,webm, from 'chat.webm':
  Metadata:
    ENCODER         : Lavf58.51.101
  Duration: 00:00:31.00, start: 0.000000, bitrate: 449 kb/s
    Stream #1:0: Video: vp8, yuva420p(tv, progressive), 500x300, SAR 1:1 DAR 5:3, 60 fps, 60 tbr, 1k tbn, 1k tbc (default)
    Metadata:
      alpha_mode      : 1
      ENCODER         : Lavc58.101.101 libvpx
      DURATION        : 00:00:31.000000000
[libvpx @ 0000021072020dc0] v1.9.0-128-g3a38edea2
  Stream #0:0 (h264) -> overlay:main (graph 0)
  Stream #1:0 (libvpx) -> overlay:overlay (graph 0)
  overlay (graph 0) -> Stream #0:0 (libx264)
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libvpx @ 0000021072020dc0] v1.9.0-128-g3a38edea2
[libx264 @ 0000021072022980] using SAR=1/1
[libx264 @ 0000021072022980] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000021072022980] profile High, level 4.2, 4:2:0, 8-bit
[libx264 @ 0000021072022980] 264 - core 161 r3027 4121277 - H.264/MPEG-4 AVC codec - Copyleft 2003-2020 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'broken_complete.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 60 fps, 15360 tbn, 60 tbc (default)
    Metadata:
      encoder         : Lavc58.91.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.91.100 aac
frame= 2701 fps= 47 q=-1.0 Lsize=   23020kB time=00:00:45.01 bitrate=4189.5kbits/s dup=60 drop=0 speed=0.789x
video:22242kB audio:708kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.309473%
[libx264 @ 0000021072022980] frame I:11    Avg QP:18.82  size:210832
[libx264 @ 0000021072022980] frame P:742   Avg QP:23.14  size: 18106
[libx264 @ 0000021072022980] frame B:1948  Avg QP:31.16  size:  3604
[libx264 @ 0000021072022980] consecutive B-frames:  1.7%  4.2%  6.2% 87.8%
[libx264 @ 0000021072022980] mb I  I16..4: 10.0% 46.2% 43.7%
[libx264 @ 0000021072022980] mb P  I16..4:  1.6%  4.5%  1.5%  P16..4: 16.5%  4.2%  2.4%  0.0%  0.0%    skip:69.3%
[libx264 @ 0000021072022980] mb B  I16..4:  0.1%  0.3%  0.1%  B16..8: 11.3%  1.4%  0.4%  direct: 0.4%  skip:86.0%  L0:36.5% L1:59.3% BI: 4.3%
[libx264 @ 0000021072022980] 8x8 transform intra:57.4% inter:64.8%
[libx264 @ 0000021072022980] coded y,uvDC,uvAC intra: 55.4% 42.1% 14.8% inter: 3.3% 2.1% 0.3%
[libx264 @ 0000021072022980] i16 v,h,dc,p: 21% 37% 15% 27%
[libx264 @ 0000021072022980] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 18% 18% 24%  6%  7%  6%  7%  6%  7%
[libx264 @ 0000021072022980] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 22% 16%  7%  7%  7%  7%  6%  6%
[libx264 @ 0000021072022980] i8c dc,h,v,p: 58% 22% 15%  5%
[libx264 @ 0000021072022980] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000021072022980] ref P L0: 68.9% 16.6% 10.8%  3.6%
[libx264 @ 0000021072022980] ref B L0: 91.6%  7.2%  1.2%
[libx264 @ 0000021072022980] ref B L1: 95.9%  4.1%
[libx264 @ 0000021072022980] kb/s:4047.35
[aac @ 0000021072024840] Qavg: 189.405