Recherche avancée

Sur d’autres sites (11989)

• Decoding a h264 (High) stream with OpenCV's ffmpeg on Ubuntu

  5 juin 2018, par arvids

  I am working with a video stream from an ip camera on Ubuntu 14.04. Everything was going great with a camera that has these parameters (from FFMPEG) :

      Stream #0:0: Video: h264 (Main), yuv420p(progressive), 352x192, 29.97 tbr, 90k tbn, 180k tbc

  But then i changed to a newer camera, which has these parameters :

      Stream #0:0: Video: h264 (High), yuvj420p(pc, bt709, progressive), 1280x720, 25 fps, 25 tbr, 90k tbn, 50 tbc

  My C++ program uses OpenCV3 to process the stream. By default OpenCV uses ffmpeg to decode and display the stream with function VideoCapture.

  VideoCapture vc;
  
  vc.open(input_stream);
  
  while ((vc >> frame), !frame.empty()) {
  
     *do work*
  
  }

  With the new camera stream i get errors like these (from ffmpeg) :

  [h264 @ 0x7c6980] cabac decode of qscale diff failed at 41 38
  
  [h264 @ 0x7c6980] error while decoding MB 41 38, bytestream (3572)
  
  [h264 @ 0x7c6980] left block unavailable for requested intra mode at 0 44
  
  [h264 @ 0x7bc2c0] SEI type 25 truncated at 208

  The image sometimes is glitched, sometimes completely frozen. However on vlc it plays perfectly. I installed the newest version (3.2.2) of ffmpeg player with

  ./configure --enable-gpl --enable-libx264

  Now playing directly with ffplay (instead of launching from source code with OpenCV function VideoCapture), the stream plays better, but sometimes still displays warnings :

  [NULL @ 0x7f834c008c00] SEI type 25 size 896 truncated at 320=1/1   
  
  [h264 @ 0x7f834c0d5d20] SEI type 25 size 896 truncated at 319=1/1   
  
  [rtsp @ 0x7f834c0008c0] max delay reached. need to consume packet   
  
  [rtsp @ 0x7f834c0008c0] RTP: missed 1 packets
  
  [h264 @ 0x7f834c094740] concealing 675 DC, 675 AC, 675 MV errors in P frame

  Changing the camera hardware is not an option. The camera can be set to encode to h265 or mjpeg. When encoding to mjpeg it can output 5 fps, which is not enough. Decoding to a static video is not an option either, because i need to display real time results about the stream. Here is a list of API backends that can be used in function VideoCapture. Maybe i should switch to some other decoder and player ?
  From my research i conclude that i have these options :

  • Somehow get OpenCV to use libVlc instead of ffmpeg

  One example of switching to vlc is here, but i don’t understand it well enough to say if that is what i need. Or maybe i should be parsing the stream in code ?

  • Use vlc to preprocess the stream, as suggested here.

  This is probably slow, which again is bad for real time results.
  Any suggestions and coments will be appreciated.

• MKV video playback shows a black picture and audio plays good

  24 mars 2023, par gbriones.gdl

  I have an MKV video file that has something wrong, every time I start my player, the video shows full black picture and the audio starts playing good, if I use the seek bar to position the playback anywhere in the time, then the video shows correct picture but audio stops playing, eventually (it takes several seconds) audio starts playing again in sync with video.

  


  The player I am using is the one in Kodi 20.1.0, I have tried enabling/disabling HW acceleration without success.

  


  I also tried VLC 3.0.16, which has a simmilar behavior with the exception that video starts showing from the very begining when opening the file, just when seeking the audio stops for several seconds.

  


  I also tried using ffmpeg to reencode the video with different codecs without success, I have tried multiple video and audio codec combinations, and also tried changing the pixel format.

  


  Then eventually I managed to fix the video by extracting streams into separate files, then merge them again, without reencoding, I thought that maybe something could be wrong with the metadata of the video, so I copied the same metadata and final output file still works fine. My question is how can I fix my video without having to split and merge streams ?

  


  Here is the info about my input file :

  


  $ ffmpeg -i S01E08.mkv 
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, matroska,webm, from 'S01E08.mkv':
  Metadata:
    ENCODER         : Lavf59.27.100
  Duration: 00:25:02.21, start: 0.000000, bitrate: 1382 kb/s
  Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt470bg/unknown/unknown, progressive), 1280x960, 59.94 fps, 59.94 tbr, 1k tbn, 59.94 tbc
    Metadata:
      ENCODER         : Lavc59.37.100 libx265
      DURATION        : 00:25:00.032000000
  Stream #0:1(spa): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp (default)
    Metadata:
      title           : Stereo
      DURATION        : 00:25:02.207000000
  Stream #0:2(jpn): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp
    Metadata:
      title           : Stereo
      DURATION        : 00:25:00.116000000
  Stream #0:3(spa): Subtitle: ass
    Metadata:
      ENCODER         : Lavc59.37.100 ssa
      DURATION        : 00:24:56.585000000
At least one output file must be specified


  


  Here are the steps I followed to fix the video, I had to use mkv format for the outputs because other formats were no good at writing metadata :

  


  ffmpeg -i S01E08.mkv -map 0:v:0 -c:v copy -map_metadata 0 video.mkv
ffmpeg -i S01E08.mkv -map 0:a:0 -c:a copy -map_metadata 0 audio1.mkv
ffmpeg -i S01E08.mkv -map 0:a:1 -c:a copy -map_metadata 0 audio2.mkv
ffmpeg -i S01E08.mkv -map 0:s -c:s copy -map_metadata 0 subs.mkv
ffmpeg -i video.mkv -i audio1.mkv -i audio2.mkv -i subs.mkv -map 0:v -map 1:a -map 2:a -map 3:s -c:v copy -c:a copy -c:s copy -map_metadata 0 -map_metadata 1 -map_metadata 2 -map_metadata 3 -disposition:s:0 0 -default_mode infer_no_subs video_audio.mkv


  


  I had to compare both input and output files :

  


  $ ffmpeg -i video_audio.mkv
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, matroska,webm, from 'video_audio.mkv':
  Metadata:
    ENCODER         : Lavf58.76.100
  Duration: 00:25:02.21, start: 0.000000, bitrate: 1382 kb/s
  Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt470bg/unknown/unknown, progressive), 1280x960, 59.94 fps, 59.94 tbr, 1k tbn, 59.94 tbc (default)
    Metadata:
      ENCODER         : Lavc59.37.100 libx265
      DURATION        : 00:25:00.031000000
  Stream #0:1(spa): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp (default)
    Metadata:
      title           : Stereo
      DURATION        : 00:25:02.207000000
  Stream #0:2(jpn): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp
    Metadata:
      title           : Stereo
      DURATION        : 00:25:00.116000000
  Stream #0:3(spa): Subtitle: ass
    Metadata:
      ENCODER         : Lavc59.37.100 ssa
      DURATION        : 00:24:56.585000000
At least one output file must be specified

$ ffprobe -show_streams S01E08.mkv > probe_bad.txt
$ ffprobe -show_streams video_audio.mkv > probe_good.txt
$ diff probe_bad.txt probe_good.txt 
32,33c32,33
< duration_ts=1502207
< duration=1502.207000
---
> duration_ts=N/A
> duration=N/A
40c40
< DISPOSITION:default=0
---
> DISPOSITION:default=1
53c53
< TAG:DURATION=00:25:00.032000000
---
> TAG:DURATION=00:25:00.031000000


  


  Then I thought that maybe the disposition flag in the video stream was causing the error, so I ran ffmpeg to copy all streams from my input video S01E08.mkv and added the disposition flag with the following command :

  


  ffmpeg -i S01E08.mkv -map 0:v -map 0:a:0 -map 0:a:1 -map 0:s -c:v copy -c:a copy -c:s copy -map_metadata 0 -disposition:v:0 default -disposition:s:0 0 -default_mode infer_no_subs test.mkv


  


  Still no luck, test.mkv shows no video when opening the file, I had to double check that disposition flag was changed :

  


  $ ffprobe -show_streams test.mkv > probe_bad2.txt
$ diff probe_bad.txt probe_bad2.txt 
40c40
< DISPOSITION:default=0
---
> DISPOSITION:default=1
53c53
< TAG:DURATION=00:25:00.032000000
---
> TAG:DURATION=00:25:00.031000000


  


  In other words I have to focibly split and merge to make the file work correctly. May be I need to set the duration and duration_ts of the video stream to N/A but I don't know how to do this.

  


• Video streaming FFMPEG - VLC video starts black

  7 novembre 2020, par xKedar

  I am trying to stream my webcam when an event occurs to another machine.
Giving the fact that FFMPEG needs around 2 seconds since I call it to when it starts streaming I'm running it in background sending everything to a local socket that consumes it until the event happens and then I reverse the data to the other machine

  


  Here I capture the camera

  


  class ffmpegThread (Thread):
    def __init__(self):
        Thread.__init__(self)
     
    def run(self):
        cam, mic = detect_devices()
        command = 'ffmpeg -f dshow -i video='+cam+':audio='+mic+' -profile:v high -pix_fmt yuvj420p -level:v 4.1 -preset ultrafast -tune zerolatency -vcodec libx264 -r 14 -b:v 512k -s 240x160 -acodec aac -ac 2 -ab 32k -ar 44100 -f mpegts -flush_packets 0 udp://127.0.0.1:'+str(config.ffmpeg_port)+'?pkt_size=1316'
        p = Popen(command , stderr=PIPE)
        for line in iter(p.stderr.readline,''):
           if config.end: break
        p.terminate()
        return 0


  


  This is where I receive and forward the video

  


  
class sendVideoThread (Thread):
    def __init__(self, UDPsenderSocket):
        Thread.__init__(self)
        self.UDPsenderSocket = UDPsenderSocket
 
    def run(self):
        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        sock.bind(("127.0.0.1", config.ffmpeg_port))
        bufferSize  = 1348
        firstData = sock.recvfrom(bufferSize)
        while True:
            data = sock.recvfrom(bufferSize)
            if config.event == True:
                startTime = time.time()
                self.UDPsenderSocket.sendto(firstData[0], config.address)
                while time.time() - startTime < 14:
                    self.UDPsenderSocket.sendto(data[0], config.address) 
                    data = sock.recvfrom(bufferSize)                    
                config.event= False
            if config.end: sys.exit()



  


  And those are the vlc options on the other machine(not the whole code)

  


      def play(self):
        self.player = vlc.Instance(["--file-caching=0 --network-caching=0"]).media_player_new()
        self.player.set_mrl('udp://@0.0.0.0:'+str(config.vlcPlayer_port))
        self.player.set_hwnd(self.frame.winfo_id())
        self.player.audio_set_mute(False)
        self.player.play()



  


  The video reaches the other machine but it starts black with audio-only for some seconds then it starts working fine. I guess that by sending the video from a random moment(frame) when the event happens it may happen that vlc will miss some information about the video and needs the next frame with information to come in order to show the video, I was looking to something in order to have more frames with that info but I was not able to find it.