Sur d’autres sites (7065)

• Why does use of H264 in sender/receiver pipelines introduce just HUGE delay ?

  24 janvier 2012, par Serguey Zefirov

  When I try to create pipeline that uses H264 to transmit video, I get some enormous delay, up to 10 seconds to transmit video from my machine to... my machine ! This is unacceptable for my goals and I'd like to consult StackOverflow over what I (or someone else) do wrong.

  I took pipelines from gstrtpbin documentation page and slightly modified them to use Speex :

  This is sender pipeline :
  # !/bin/sh

  gst-launch -v gstrtpbin name=rtpbin \
  
          v4l2src ! ffmpegcolorspace ! ffenc_h263 ! rtph263ppay ! rtpbin.send_rtp_sink_0 \
  
                    rtpbin.send_rtp_src_0 ! udpsink host=127.0.0.1 port=5000                            \
  
                    rtpbin.send_rtcp_src_0 ! udpsink host=127.0.0.1 port=5001 sync=false async=false    \
  
                    udpsrc port=5005 ! rtpbin.recv_rtcp_sink_0                           \
  
          pulsesrc ! audioconvert ! audioresample  ! audio/x-raw-int,rate=16000 !    \
  
                    speexenc bitrate=16000 ! rtpspeexpay ! rtpbin.send_rtp_sink_1                   \
  
                    rtpbin.send_rtp_src_1 ! udpsink host=127.0.0.1 port=5002                            \
  
                    rtpbin.send_rtcp_src_1 ! udpsink host=127.0.0.1 port=5003 sync=false async=false    \
  
                    udpsrc port=5007 ! rtpbin.recv_rtcp_sink_1

  Receiver pipeline :

   !/bin/sh

  gst-launch -v\
  
      gstrtpbin name=rtpbin                                          \
  
      udpsrc caps="application/x-rtp,media=(string)video, clock-rate=(int)90000, encoding-name=(string)H263-1998" \
  
              port=5000 ! rtpbin.recv_rtp_sink_0                                \
  
          rtpbin. ! rtph263pdepay ! ffdec_h263 ! xvimagesink                    \
  
       udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0                               \
  
       rtpbin.send_rtcp_src_0 ! udpsink port=5005 sync=false async=false        \
  
      udpsrc caps="application/x-rtp,media=(string)audio, clock-rate=(int)16000, encoding-name=(string)SPEEX, encoding-params=(string)1, payload=(int)110" \
  
              port=5002 ! rtpbin.recv_rtp_sink_1                                \
  
          rtpbin. ! rtpspeexdepay ! speexdec ! audioresample ! audioconvert ! alsasink \
  
       udpsrc port=5003 ! rtpbin.recv_rtcp_sink_1                               \
  
       rtpbin.send_rtcp_src_1 ! udpsink host=127.0.0.1 port=5007 sync=false async=false

  Those pipelines, a combination of H263 and Speex, work fine enough. I snap my fingers near camera and micropohne and then I see movement and hear sound at the same time.

  Then I changed pipelines to use H264 along the video path.

  The sender becomes :
  # !/bin/sh

  gst-launch -v gstrtpbin name=rtpbin \
  
          v4l2src ! ffmpegcolorspace ! x264enc bitrate=300 ! rtph264pay ! rtpbin.send_rtp_sink_0 \
  
                    rtpbin.send_rtp_src_0 ! udpsink host=127.0.0.1 port=5000                            \
  
                    rtpbin.send_rtcp_src_0 ! udpsink host=127.0.0.1 port=5001 sync=false async=false    \
  
                    udpsrc port=5005 ! rtpbin.recv_rtcp_sink_0                           \
  
          pulsesrc ! audioconvert ! audioresample  ! audio/x-raw-int,rate=16000 !    \
  
                    speexenc bitrate=16000 ! rtpspeexpay ! rtpbin.send_rtp_sink_1                   \
  
                    rtpbin.send_rtp_src_1 ! udpsink host=127.0.0.1 port=5002                            \
  
                    rtpbin.send_rtcp_src_1 ! udpsink host=127.0.0.1 port=5003 sync=false async=false    \
  
                    udpsrc port=5007 ! rtpbin.recv_rtcp_sink_1

  And receiver becomes :
  # !/bin/sh

  gst-launch -v\
  
      gstrtpbin name=rtpbin                                          \
  
      udpsrc caps="application/x-rtp,media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264" \
  
              port=5000 ! rtpbin.recv_rtp_sink_0                                \
  
          rtpbin. ! rtph264depay ! ffdec_h264 ! xvimagesink                    \
  
       udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0                               \
  
       rtpbin.send_rtcp_src_0 ! udpsink port=5005 sync=false async=false        \
  
      udpsrc caps="application/x-rtp,media=(string)audio, clock-rate=(int)16000, encoding-name=(string)SPEEX, encoding-params=(string)1, payload=(int)110" \
  
              port=5002 ! rtpbin.recv_rtp_sink_1                                \
  
          rtpbin. ! rtpspeexdepay ! speexdec ! audioresample ! audioconvert ! alsasink \
  
       udpsrc port=5003 ! rtpbin.recv_rtcp_sink_1                               \
  
       rtpbin.send_rtcp_src_1 ! udpsink host=127.0.0.1 port=5007 sync=false async=false

  This is what happen under Ubuntu 10.04. I didn't noticed such huge delays on Ubuntu 9.04 - the delays there was in range 2-3 seconds, AFAIR.

• FFMpeg and WebM/VP8

  25 novembre 2011, par Anand Suresh

  I am trying to use ffmpeg and ffserver to stream VP8 video.

  I am using the following command to start FFMpeg :

  ffmpeg -v 9 -loglevel 99 -f x11grab -s 1440x900 -r2 -i :0.0 -f webm http://localhost:8090/feed1.ffm

  The above command abruptly terminates generating the following error :

  > FFmpeg version 0.6.2-4:0.6.2-1ubuntu1.1, Copyright (c) 2000-2010 the Libav developers
  
    built on Sep 16 2011 16:57:46 with gcc 4.5.2
  
    configuration: --extra-version=4:0.6.2-1ubuntu1.1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --enable-shared --disable-static
  
    WARNING: library configuration mismatch
  
    libavutil   configuration: --extra-version=4:0.6.2-1ubuntu1.1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --shlibdir=/usr/lib/i686/cmov --cpu=i686 --enable-shared --disable-static --disable-ffmpeg --disable-ffplay
  
    libavcodec  configuration: --extra-version=4:0.6.2-1ubuntu1.1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --shlibdir=/usr/lib/i686/cmov --cpu=i686 --enable-shared --disable-static --disable-ffmpeg --disable-ffplay
  
    libavformat configuration: --extra-version=4:0.6.2-1ubuntu1.1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --shlibdir=/usr/lib/i686/cmov --cpu=i686 --enable-shared --disable-static --disable-ffmpeg --disable-ffplay
  
    libavdevice configuration: --extra-version=4:0.6.2-1ubuntu1.1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --shlibdir=/usr/lib/i686/cmov --cpu=i686 --enable-shared --disable-static --disable-ffmpeg --disable-ffplay
  
    libavfilter configuration: --extra-version=4:0.6.2-1ubuntu1.1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --shlibdir=/usr/lib/i686/cmov --cpu=i686 --enable-shared --disable-static --disable-ffmpeg --disable-ffplay
  
    libswscale  configuration: --extra-version=4:0.6.2-1ubuntu1.1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --shlibdir=/usr/lib/i686/cmov --cpu=i686 --enable-shared --disable-static --disable-ffmpeg --disable-ffplay
  
    libpostproc configuration: --extra-version=4:0.6.2-1ubuntu1.1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --shlibdir=/usr/lib/i686/cmov --cpu=i686 --enable-shared --disable-static --disable-ffmpeg --disable-ffplay
  
    libavutil     50.15. 1 / 50.15. 1
  
    libavcodec    52.72. 2 / 52.72. 2
  
    libavformat   52.64. 2 / 52.64. 2
  
    libavdevice   52. 2. 0 / 52. 2. 0
  
    libavfilter    1.19. 0 /  1.19. 0
  
    libswscale     0.11. 0 /  0.11. 0
  
    libpostproc   51. 2. 0 / 51. 2. 0
  
  [x11grab @ 0x9869420]device: :0.0 -> display: :0.0 x: 0 y: 0 width: 1440 height: 900
  
  [x11grab @ 0x9869420]shared memory extension  found
  
  [x11grab @ 0x9869420]Probe buffer size limit 5000000 reached
  
  [x11grab @ 0x9869420]Estimating duration from bitrate, this may be inaccurate
  
  Input #0, x11grab, from ':0.0':
  
    Duration: N/A, start: 1322253753.374957, bitrate: 41472 kb/s
  
      Stream #0.0, 1, 1/1000000: Video: rawvideo, bgra, 1440x900, 1/1, 41472 kb/s, 1 tbr, 1000k tbn, 1 tbc
  
  [libvpx @ 0x9876540]v0.9.6
  
  [libvpx @ 0x9876540]--enable-pic --enable-shared --disable-install-bins --disable-install-srcs --target=x86-linux-gcc
  
  [libvpx @ 0x9876540]vpx_codec_enc_cfg
  
  [libvpx @ 0x9876540]generic settings
  
    g_usage:                      0
  
    g_threads:                    0
  
    g_profile:                    0
  
    g_w:                          320
  
    g_h:                          240
  
    g_timebase:                   {1/30}
  
    g_error_resilient:            0
  
    g_pass:                       0
  
    g_lag_in_frames:              0
  
  [libvpx @ 0x9876540]rate control settings
  
    rc_dropframe_thresh:          0
  
    rc_resize_allowed:            0
  
    rc_resize_up_thresh:          60
  
    rc_resize_down_thresh:        30
  
    rc_end_usage:                 0
  
    rc_twopass_stats_in:          (nil)(0)
  
    rc_target_bitrate:            256
  
  [libvpx @ 0x9876540]quantizer settings
  
    rc_min_quantizer:             4
  
    rc_max_quantizer:             63
  
  [libvpx @ 0x9876540]bitrate tolerance
  
    rc_undershoot_pct:            95
  
    rc_overshoot_pct:             200
  
  [libvpx @ 0x9876540]decoder buffer model
  
    rc_buf_sz:                    6000
  
    rc_buf_initial_sz:            4000
  
    rc_buf_optimal_sz:            5000
  
  [libvpx @ 0x9876540]2 pass rate control settings
  
    rc_2pass_vbr_bias_pct:        50
  
    rc_2pass_vbr_minsection_pct:  0
  
    rc_2pass_vbr_maxsection_pct:  400
  
  [libvpx @ 0x9876540]keyframing settings
  
    kf_mode:                      1
  
    kf_min_dist:                  0
  
    kf_max_dist:                  9999
  
  [libvpx @ 0x9876540]
  
  [libvpx @ 0x9876540]vpx_codec_enc_cfg
  
  [libvpx @ 0x9876540]generic settings
  
    g_usage:                      0
  
    g_threads:                    1
  
    g_profile:                    0
  
    g_w:                          1440
  
    g_h:                          900
  
    g_timebase:                   {1/1}
  
    g_error_resilient:            0
  
    g_pass:                       0
  
    g_lag_in_frames:              0
  
  [libvpx @ 0x9876540]rate control settings
  
    rc_dropframe_thresh:          0
  
    rc_resize_allowed:            0
  
    rc_resize_up_thresh:          60
  
    rc_resize_down_thresh:        30
  
    rc_end_usage:                 0
  
    rc_twopass_stats_in:          (nil)(0)
  
    rc_target_bitrate:            200
  
  [libvpx @ 0x9876540]quantizer settings
  
    rc_min_quantizer:             1
  
    rc_max_quantizer:             38
  
  [libvpx @ 0x9876540]bitrate tolerance
  
    rc_undershoot_pct:            95
  
    rc_overshoot_pct:             200
  
  [libvpx @ 0x9876540]decoder buffer model
  
    rc_buf_sz:                    6000
  
    rc_buf_initial_sz:            4000
  
    rc_buf_optimal_sz:            5000
  
  [libvpx @ 0x9876540]2 pass rate control settings
  
    rc_2pass_vbr_bias_pct:        50
  
    rc_2pass_vbr_minsection_pct:  0
  
    rc_2pass_vbr_maxsection_pct:  400
  
  [libvpx @ 0x9876540]keyframing settings
  
    kf_mode:                      1
  
    kf_min_dist:                  0
  
    kf_max_dist:                  12
  
  [libvpx @ 0x9876540]
  
  [libvpx @ 0x9876540]vpx_codec_control
  
  [libvpx @ 0x9876540]  VP8E_SET_CPUUSED:             3
  
  [libvpx @ 0x9876540]  VP8E_SET_NOISE_SENSITIVITY:   0
  
  Output #0, webm, to 'http://127.0.0.1:8090/feed1.ffm':
  
    Metadata:
  
      encoder         : Lavf52.64.2
  
      Stream #0.0, 0, 1/1000: Video: libvpx, yuv420p, 1440x900, 1/1, q=2-31, 200 kb/s, 1k tbn, 1 tbc
  
  Stream mapping:
  
    Stream #0.0 -> #0.0
  
  Press [q] to stop encoding
  
  [webm @ 0x98753b0]Writing block at offset 15, size 158658, pts 0, dts 0, duration 1000, flags 128
  
  [webm @ 0x98753b0]Starting new cluster at offset 158681 bytes, pts 0

  Can anyone point out what I am doing wrong here ? Why does ffmpeg die everytime it starts a new cluster ?

  Thanks

• Python vlc stream from outside NAT

  7 octobre 2020, par xKedar

  I'm trying to stream, using FFmpeg, my webcam and audio from PC1 to PC2 in another LAN.

  


  PC1 : Public IP address with port forwarding so I can reach it

  


  PC2 : In a different NAT from PC1

  


  I basically run a server on PC1 in order to acquire IP and port from PC2 and reply on the same address

  


      import socket

    localPort   = 1234
    bufferSize  = 1024

    UDPServerSocket = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM)
    UDPServerSocket.bind(("", localPort)) # Bind to address and port

    while(True):
        bytesAddressPair = UDPServerSocket.recvfrom(bufferSize)
        message = bytesAddressPair[0].decode("utf-8")
        address = bytesAddressPair[1]
        # Sending a reply to client
        UDPServerSocket.sendto(str.encode("Hello"), address)
        break

    UDPServerSocket.close()


  


  Then I try to send the stream with the same port number both for the server(localPort) and the client(the one I acquired from address)

  


      import re
    from threading import Thread
    from subprocess import Popen, PIPE

    def detect_devices():
            list_cmd = 'ffmpeg -list_devices true -f dshow -i dummy'.split()
            p = Popen(list_cmd, stderr=PIPE)
            flagcam = flagmic = False
            for line in iter(p.stderr.readline,''):
                if flagcam:
                    cam = re.search('".*"',line.decode(encoding='UTF-8')).group(0)
                    cam = cam if cam else ''
                    flagcam = False
                if flagmic:
                    mic = re.search('".*"',line.decode(encoding='UTF-8')).group(0)
                    mic = mic if mic else ''
                    flagmic = False
                elif 'DirectShow video devices'.encode(encoding='UTF-8') in line:
                    flagcam = True
                elif 'DirectShow audio devices'.encode(encoding='UTF-8') in line:
                    flagmic = True
                elif 'Immediate exit requested'.encode(encoding='UTF-8') in line:
                    break
            return cam, mic   


    class ffmpegThread (Thread):
        def __init__(self, address):
            Thread.__init__(self)
            self.address = address

        def run(self):
            cam, mic = detect_devices()
            command = 'ffmpeg -f dshow -i video='+cam+':audio='+mic+' -profile:v high -pix_fmt yuvj420p -level:v 4.1 -preset ultrafast -tune zerolatency -vcodec libx264 -r 10 -b:v 512k -s 240x160 -acodec aac -ac 2 -ab 32k -ar 44100 -f mpegts -flush_packets 0 -t 40 udp://'+self.address+'?pkt_size=1316?localport='+str(localPort)
            p = Popen(command , stderr=PIPE)
            for line in iter(p.stderr.readline,''):
                if len(line) <5: break
            p.terminate()

    thread1 = ffmpegThread(address[0]+":"+str(address[1]))
    thread1.start()


  


  While on the other side(PC2) I have :

  


      from threading import Thread
    import tkinter as tk
    import vlc

    class myframe(tk.Frame):
        def __init__(self, width=240, height=160):
            self.root = tk.Tk()
            super(myframe, self).__init__(self.root)
            self.root.geometry("%dx%d" % (width, height))
            self.root.wm_attributes("-topmost", 1)
            self.grid()
            self.frame = tk.Frame(self, width=240, height=160)
            self.frame.configure(bg="black")
            self.frame.grid(row=0, column=0, columnspan=2)
            self.play()
            self.root.mainloop()

        def play(self):
            self.player = vlc.Instance().media_player_new()
            self.player.set_mrl('udp://@0.0.0.0:5000')
            self.player.set_hwnd(self.frame.winfo_id())
            self.player.play()

    class guiThread (Thread):
        def __init__(self, nome):
            Thread.__init__(self)
            self.nome = nome

        def run(self):
            app = myframe()


  


  and :

  


      import socket

    msgFromClient       = "Hello UDP Server"
    bytesToSend         = str.encode(msgFromClient)
    serverAddressPort   = ("MYglobal_IPaddress", 1234)
    bufferSize          = 1024
    localPort   = 5000

    # Create a UDP socket at client side
    UDPClientSocket = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM) 
    UDPClientSocket.bind(("", localPort))

    UDPClientSocket.sendto(bytesToSend, serverAddressPort)

    msgFromServer = UDPClientSocket.recvfrom(bufferSize)
    msg = msgFromServer[0].decode("utf-8")
    print(msg)
    UDPClientSocket.close()
    gui = guiThread("ThreadGUI")
    gui.start()


  


  Where I basically try to reach the server both to send my IP:Port and to punch a hole in the NAT in order to be able to get the packages sent from PC1 despite being behind a NAT.

  


  I think it is not working because I can not reach PC2 but I really can not figure out how to fix that because I was expecting that the first part, where I reach PC1 from PC2 was enough in order to establish a connection