Recherche avancée

Sur d’autres sites (11586)

• Live video encoding using...?

  27 décembre 2013, par Basic

  I'm attempting to write a fairly simplistic application that will stream video/audio from a webcam to someone else across the internet (ala Skypebut with more control).

  There seems to be very little useful/relevant information on the subjectand what I can find is largely outdated. From my research so far, x264 seems to be the way to go as it offers an ultrafast option which is designed for this situation

  I'm able to turn on the webcam and receive a stream of images. I can also listen on an audio device and get samples.

  Where I'm failing is encoding that information in such a way as to be able to stream with a minimum of latency (from what I've read, 200ms delay is the goal for no obvious lag, including network latency - so let's aim for 100-150ms)

  Things I've tried

  ffmpeg

  This seems to be the most widely used option for encoding. I've had two real issues using it. Firstly, even using x264 with no look-aheads and the bare minimum buffers for stability, the delay seems to be on the order of 700ms using image2pipe. Secondly, it requires ffmpeg to be installed - being able to do this without an external dependency would be nice.

  VLC

  As with ffmpeg this requires an external program which is a negative. Even worse, I can't seem to get a latency of under 2 seconds which seems to increase over time. I've also only been able to get VLC to capture the camera itself rather than take a stream of images which means I don't get a chance to pre-process them.

  DirectShow

  I've seen a number of sites recommending using the windows direct show encoders but I haven't been able to find one that works at anything like real time. In fact, the only one I've managed to get going reliably is a Windows Media codec that has a massive latency and fairly large size.

  Other considerations

  None of the above address the problem of adding an audio stream to the video. I'm not sure if I should attempt to encode them together or send a separate stream alongside the video.

  In short, I've been Googling for a week or so now and haven't found a decent way to do this. Can someone please point me at a decent example/guide ?

• Confusion about PTS in video files and media streams

  11 novembre 2014, par user2452253

  Is it possible that the PTS of a particular frame in a file is different with the PTS of the same frame in the same file while it is being streamed ?

  When I read a frame using av_read_frame I store the video stream in an AVStream. After I decode the frame with avcodec_decode_video2, I store the time stamp of that frame in an int64_t using av_frame_get_best_effort_timestamp. Now if the program is getting its input from a file I get a different timestamp from when I stream the input (from the same file) to the program.

  To change the input type I simply change the argv argument from "/path/to/file.mp4" to something like "udp ://localhost:1234", then I stream the file with ffmpeg in command line : "ffmpeg -re -i /path/to/file.mp4 -f mpegts udp ://localhost:1234". Can it be because the "-f mpegts" arguments change some characteristics of the media ?

  Below is my code (simplified). By reading the ffmpeg mailing list archives I realized that the time_base that I’m looking for is in the AVStream and not the AVCodecContext. Instead of using av_frame_get_best_effort_timestamp I have also tried using the packet.pts but the results don’t change.
  I need the time stamps to have a notion of frame number in a streaming video that is being received.
  I would really appreciate any sort of help.

  //..
  
  //argv[1]="/file.mp4";
  
  argv[1]="udp://localhost:7777";
  
  // define AVFormatContext, AVFrame, etc.
  
  // register av, avcodec, avformat_network_init(), etc.
  
  avformat_open_input(&pFormatCtx, argv, NULL, NULL);
  
  avformat_find_stream_info(pFormatCtx, NULL);
  
  // find the video stream...
  
  // pointer to the codec context...
  
  // open codec...
  
  pFrame=av_frame_alloc();
  
  while(av_read_frame(pFormatCtx, &packet)>=0) {
  
          AVStream *strem = pFormatCtx->streams[videoStream];
  
          if(packet.stream_index==videoStream) {
  
              avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packet);
  
              if(frameFinished) {
  
                  int64_t perts = av_frame_get_best_effort_timestamp(pFrame);
  
                  if (isMyFrame(pFrame)){
  
                       cout << perts*av_q2d(strem->time_base) << "\n";
  
                  }
  
               }
  
  }
  
  //free allocated space
  
  }
  
  //..

• FFMPEG = I tried resizing a video, but got different resolution than I wanted [closed]

  10 janvier 2024, par wakanasakai

  I downloaded a video that had some black bars (left & right), so I used the following command in FFmpeg to make various changes to it. I tested it on a 10 second clip to see what the result would look like.

  


  -ss 00:04:44 -to 00:04:54 -vf "crop=1870:20:20:0","scale=640x480:flags=lanczos","eq=gamma=1.5:saturation=1.3:contrast=1.2"

  


  The original video is an mp4, with a resolution of 1920 x 1080. Besides trying to crop it & adjust the gamma, saturation, & contrast, I also tried to resize it to 640 x 480. Instead, it's resulting resolution is 44880 x 480 ! I have a link to it for anybody who wants to examine it directly. (It's only 487 kb.)
text

  


  I've tried using FFmpeg before, & it never did anything so insane. (It cropped it, & adjusted the gamma a saturation (I didn't test the contrast until THIS time), but it did not resize it at all.)

  


  Here is FFmpeg's log file for it. Guesses as to the cause of the insane result, & advice on how to achieve the DESIRED result (in 1 pass, if possible) are requested.

  


  ffmpeg -hwaccel auto -y -i "/storage/emulated/0/bluetooth/Barbie & the Rockers=1080-Out of this world (1987).mp4" -ss 00:04:44 -to 00:04:54 -vf "crop=1870:20:20:0","scale=640x480:flags=lanczos","eq=gamma=1.5:saturation=1.3:contrast=1.2" "/storage/emulated/0/Movies/Barbie.mp4"

ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 4.9.x (GCC) 20150123 (prerelease)
  configuration: --enable-version3 --enable-gpl --enable-nonfree --disable-indev=v4l2 --enable-libmp3lame --enable-libx264 --enable-libx265 --enable-libvpx --enable-libvorbis --enable-libtheora --enable-libopus --enable-libfdk-aac --enable-libfreetype --enable-libass --enable-libfribidi --enable-fontconfig --enable-pthreads --enable-libxvid --enable-filters --enable-openssl --enable-librtmp --disable-protocol='udp,udplite' --enable-libopencore-amrwb --enable-libopencore-amrnb --enable-libvo-amrwbenc --enable-libspeex --enable-libsoxr --enable-libwebp --enable-libxml2 --enable-libopenh264 --enable-jni --prefix=/home/silentlexx/AndroidstudioProjects/ffmpeg/ffmpeg/build/arm-api18-r13b --sysroot=/home/silentlexx/Android/android-ndk-r13b/platforms/android-18/arch-arm --arch=arm --disable-shared --enable-static --enable-pic --enable-ffmpeg --disable-ffplay --disable-ffprobe --disable-ffnvcodec --disable-avdevice --disable-debug --disable-doc --disable-htmlpages --disable-manpages --disable-podpages --disable-txtpages --disable-symver --cross-prefix=/home/silentlexx/Android/android-ndk-r13b/toolchains/arm-linux-androideabi-4.9/prebuilt/linux-x86_64/bin/arm-linux-androideabi- --target-os=android --enable-cross-compile --pkg-config-flags=--static --extra-libs='-lgnustl_static -lm -lpng -l:libz.so -lpthread' --enable-asm --enable-neon --enable-small
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/storage/emulated/0/bluetooth/Barbie & the Rockers=1080-Out of this world (1987).mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: mp41isomiso2
    creation_time   : 2024-01-04T01:46:07.000000Z
  Duration: 00:45:33.10, start: 0.000000, bitrate: 3404 kb/s
  Stream #0:0[0x1](und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 3272 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
    Metadata:
      creation_time   : 2023-06-25T13:25:03.000000Z
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2023-06-25T13:25:03.000000Z
      vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0xf38cd180] using SAR=561/8
[libx264 @ 0xf38cd180] using cpu capabilities: ARMv6 NEON
[libx264 @ 0xf38cd180] profile High, level 3.0, 4:2:0, 8-bit
[libx264 @ 0xf38cd180] 264 - core 158 r2984 3759fcb - H.264/MPEG-4 AVC codec - Copyleft 2003-2019 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to '/storage/emulated/0/Movies/Barbie.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: mp41isomiso2
    encoder         : Lavf60.3.100
  Stream #0:0(und): Video: h264 (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 640x480 [SAR 561:8 DAR 187:2], q=2-31, 30 fps, 15360 tbn (default)
    Metadata:
      creation_time   : 2023-06-25T13:25:03.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.3.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2023-06-25T13:25:03.000000Z
      vendor_id       : [0][0][0][0]
      encoder         : Lavc60.3.100 aac
frame=    0 fps=0.0 q=0.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A    
frame=    0 fps=0.0 q=0.0 size=       0kB time=00:00:00.16 bitrate=   2.4kbits/s speed=0.00197x    
frame=    0 fps=0.0 q=0.0 size=       0kB time=00:00:00.71 bitrate=   0.5kbits/s speed=0.00867x    
frame=   13 fps=0.2 q=29.0 size=       0kB time=00:00:01.48 bitrate=   0.3kbits/s speed=0.0178x    
frame=   45 fps=0.5 q=29.0 size=       0kB time=00:00:02.55 bitrate=   0.2kbits/s speed=0.0304x    
frame=   78 fps=0.9 q=29.0 size=       0kB time=00:00:03.66 bitrate=   0.1kbits/s speed=0.0434x    
frame=  114 fps=1.3 q=29.0 size=       0kB time=00:00:04.85 bitrate=   0.1kbits/s speed=0.057x    
frame=  146 fps=1.7 q=29.0 size=       0kB time=00:00:05.92 bitrate=   0.1kbits/s speed=0.0692x    
frame=  178 fps=2.1 q=29.0 size=       0kB time=00:00:07.03 bitrate=   0.1kbits/s speed=0.0817x    
frame=  209 fps=2.4 q=29.0 size=     256kB time=00:00:08.03 bitrate= 261.1kbits/s speed=0.0928x    
frame=  240 fps=2.8 q=29.0 size=     256kB time=00:00:09.07 bitrate= 231.0kbits/s speed=0.104x    
frame=  300 fps=3.4 q=-1.0 Lsize=     445kB time=00:00:09.98 bitrate= 365.2kbits/s speed=0.114x    
video:275kB audio:159kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.692692%
[libx264 @ 0xf38cd180] frame I:10    Avg QP:20.34  size:  2434
[libx264 @ 0xf38cd180] frame P:129   Avg QP:21.89  size:  1292
[libx264 @ 0xf38cd180] frame B:161   Avg QP:21.69  size:   555
[libx264 @ 0xf38cd180] consecutive B-frames: 20.0% 18.7% 20.0% 41.3%
[libx264 @ 0xf38cd180] mb I  I16..4: 30.2% 66.5%  3.2%
[libx264 @ 0xf38cd180] mb P  I16..4: 14.3% 17.7%  0.2%  P16..4: 12.7%  2.7%  0.4%  0.0%  0.0%    skip:52.1%
[libx264 @ 0xf38cd180] mb B  I16..4:  2.1%  1.1%  0.0%  B16..8: 21.9%  1.7%  0.0%  direct: 1.5%  skip:71.6%  L0:46.0% L1:53.0% BI: 1.0%
[libx264 @ 0xf38cd180] 8x8 transform intra:54.9% inter:98.2%
[libx264 @ 0xf38cd180] coded y,uvDC,uvAC intra: 10.3% 14.9% 1.5% inter: 2.2% 5.4% 0.0%
[libx264 @ 0xf38cd180] i16 v,h,dc,p: 93%  2%  2%  4%
[libx264 @ 0xf38cd180] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 69%  1% 28%  0%  0%  1%  0%  0%  0%
[libx264 @ 0xf38cd180] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 76%  3% 17%  1%  1%  2%  0%  1%  0%
[libx264 @ 0xf38cd180] i8c dc,h,v,p: 45%  2% 53%  1%
[libx264 @ 0xf38cd180] Weighted P-Frames: Y:0.8% UV:0.8%
[libx264 @ 0xf38cd180] ref P L0: 57.0%  8.7% 24.0% 10.4%
[libx264 @ 0xf38cd180] ref B L0: 79.7% 17.3%  3.0%
[libx264 @ 0xf38cd180] ref B L1: 95.6%  4.4%
[libx264 @ 0xf38cd180] kb/s:224.32
[aac @ 0xf38cd880] Qavg: 457.489