Advanced search

On other websites (12209)

• Video streaming FFMPEG - VLC video starts black

  7 November 2020, by xKedar

  I am trying to stream my webcam when an event occurs to another machine.
Giving the fact that FFMPEG needs around 2 seconds since I call it to when it starts streaming I'm running it in background sending everything to a local socket that consumes it until the event happens and then I reverse the data to the other machine

  


  Here I capture the camera

  


  class ffmpegThread (Thread):
    def __init__(self):
        Thread.__init__(self)
     
    def run(self):
        cam, mic = detect_devices()
        command = 'ffmpeg -f dshow -i video='+cam+':audio='+mic+' -profile:v high -pix_fmt yuvj420p -level:v 4.1 -preset ultrafast -tune zerolatency -vcodec libx264 -r 14 -b:v 512k -s 240x160 -acodec aac -ac 2 -ab 32k -ar 44100 -f mpegts -flush_packets 0 udp://127.0.0.1:'+str(config.ffmpeg_port)+'?pkt_size=1316'
        p = Popen(command , stderr=PIPE)
        for line in iter(p.stderr.readline,''):
           if config.end: break
        p.terminate()
        return 0


  


  This is where I receive and forward the video

  


  
class sendVideoThread (Thread):
    def __init__(self, UDPsenderSocket):
        Thread.__init__(self)
        self.UDPsenderSocket = UDPsenderSocket
 
    def run(self):
        sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
        sock.bind(("127.0.0.1", config.ffmpeg_port))
        bufferSize  = 1348
        firstData = sock.recvfrom(bufferSize)
        while True:
            data = sock.recvfrom(bufferSize)
            if config.event == True:
                startTime = time.time()
                self.UDPsenderSocket.sendto(firstData[0], config.address)
                while time.time() - startTime < 14:
                    self.UDPsenderSocket.sendto(data[0], config.address) 
                    data = sock.recvfrom(bufferSize)                    
                config.event= False
            if config.end: sys.exit()



  


  And those are the vlc options on the other machine(not the whole code)

  


      def play(self):
        self.player = vlc.Instance(["--file-caching=0 --network-caching=0"]).media_player_new()
        self.player.set_mrl('udp://@0.0.0.0:'+str(config.vlcPlayer_port))
        self.player.set_hwnd(self.frame.winfo_id())
        self.player.audio_set_mute(False)
        self.player.play()



  


  The video reaches the other machine but it starts black with audio-only for some seconds then it starts working fine. I guess that by sending the video from a random moment(frame) when the event happens it may happen that vlc will miss some information about the video and needs the next frame with information to come in order to show the video, I was looking to something in order to have more frames with that info but I was not able to find it.

  


• MKV video playback shows a black picture and audio plays good

  24 March 2023, by gbriones.gdl

  I have an MKV video file that has something wrong, every time I start my player, the video shows full black picture and the audio starts playing good, if I use the seek bar to position the playback anywhere in the time, then the video shows correct picture but audio stops playing, eventually (it takes several seconds) audio starts playing again in sync with video.

  


  The player I am using is the one in Kodi 20.1.0, I have tried enabling/disabling HW acceleration without success.

  


  I also tried VLC 3.0.16, which has a simmilar behavior with the exception that video starts showing from the very begining when opening the file, just when seeking the audio stops for several seconds.

  


  I also tried using ffmpeg to reencode the video with different codecs without success, I have tried multiple video and audio codec combinations, and also tried changing the pixel format.

  


  Then eventually I managed to fix the video by extracting streams into separate files, then merge them again, without reencoding, I thought that maybe something could be wrong with the metadata of the video, so I copied the same metadata and final output file still works fine. My question is how can I fix my video without having to split and merge streams?

  


  Here is the info about my input file:

  


  $ ffmpeg -i S01E08.mkv 
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, matroska,webm, from 'S01E08.mkv':
  Metadata:
    ENCODER         : Lavf59.27.100
  Duration: 00:25:02.21, start: 0.000000, bitrate: 1382 kb/s
  Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt470bg/unknown/unknown, progressive), 1280x960, 59.94 fps, 59.94 tbr, 1k tbn, 59.94 tbc
    Metadata:
      ENCODER         : Lavc59.37.100 libx265
      DURATION        : 00:25:00.032000000
  Stream #0:1(spa): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp (default)
    Metadata:
      title           : Stereo
      DURATION        : 00:25:02.207000000
  Stream #0:2(jpn): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp
    Metadata:
      title           : Stereo
      DURATION        : 00:25:00.116000000
  Stream #0:3(spa): Subtitle: ass
    Metadata:
      ENCODER         : Lavc59.37.100 ssa
      DURATION        : 00:24:56.585000000
At least one output file must be specified


  


  Here are the steps I followed to fix the video, I had to use mkv format for the outputs because other formats were no good at writing metadata:

  


  ffmpeg -i S01E08.mkv -map 0:v:0 -c:v copy -map_metadata 0 video.mkv
ffmpeg -i S01E08.mkv -map 0:a:0 -c:a copy -map_metadata 0 audio1.mkv
ffmpeg -i S01E08.mkv -map 0:a:1 -c:a copy -map_metadata 0 audio2.mkv
ffmpeg -i S01E08.mkv -map 0:s -c:s copy -map_metadata 0 subs.mkv
ffmpeg -i video.mkv -i audio1.mkv -i audio2.mkv -i subs.mkv -map 0:v -map 1:a -map 2:a -map 3:s -c:v copy -c:a copy -c:s copy -map_metadata 0 -map_metadata 1 -map_metadata 2 -map_metadata 3 -disposition:s:0 0 -default_mode infer_no_subs video_audio.mkv


  


  I had to compare both input and output files:

  


  $ ffmpeg -i video_audio.mkv
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, matroska,webm, from 'video_audio.mkv':
  Metadata:
    ENCODER         : Lavf58.76.100
  Duration: 00:25:02.21, start: 0.000000, bitrate: 1382 kb/s
  Stream #0:0: Video: hevc (Main 10), yuv420p10le(tv, bt470bg/unknown/unknown, progressive), 1280x960, 59.94 fps, 59.94 tbr, 1k tbn, 59.94 tbc (default)
    Metadata:
      ENCODER         : Lavc59.37.100 libx265
      DURATION        : 00:25:00.031000000
  Stream #0:1(spa): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp (default)
    Metadata:
      title           : Stereo
      DURATION        : 00:25:02.207000000
  Stream #0:2(jpn): Audio: aac (HE-AAC), 48000 Hz, stereo, fltp
    Metadata:
      title           : Stereo
      DURATION        : 00:25:00.116000000
  Stream #0:3(spa): Subtitle: ass
    Metadata:
      ENCODER         : Lavc59.37.100 ssa
      DURATION        : 00:24:56.585000000
At least one output file must be specified

$ ffprobe -show_streams S01E08.mkv > probe_bad.txt
$ ffprobe -show_streams video_audio.mkv > probe_good.txt
$ diff probe_bad.txt probe_good.txt 
32,33c32,33
< duration_ts=1502207
< duration=1502.207000
---
> duration_ts=N/A
> duration=N/A
40c40
< DISPOSITION:default=0
---
> DISPOSITION:default=1
53c53
< TAG:DURATION=00:25:00.032000000
---
> TAG:DURATION=00:25:00.031000000


  


  Then I thought that maybe the disposition flag in the video stream was causing the error, so I ran ffmpeg to copy all streams from my input video S01E08.mkv and added the disposition flag with the following command:

  


  ffmpeg -i S01E08.mkv -map 0:v -map 0:a:0 -map 0:a:1 -map 0:s -c:v copy -c:a copy -c:s copy -map_metadata 0 -disposition:v:0 default -disposition:s:0 0 -default_mode infer_no_subs test.mkv


  


  Still no luck, test.mkv shows no video when opening the file, I had to double check that disposition flag was changed:

  


  $ ffprobe -show_streams test.mkv > probe_bad2.txt
$ diff probe_bad.txt probe_bad2.txt 
40c40
< DISPOSITION:default=0
---
> DISPOSITION:default=1
53c53
< TAG:DURATION=00:25:00.032000000
---
> TAG:DURATION=00:25:00.031000000


  


  In other words I have to focibly split and merge to make the file work correctly. May be I need to set the duration and duration_ts of the video stream to N/A but I don't know how to do this.

  


• How can I read System.Drawing.Bitmaps from a video file using FFMpegCore?

  18 April 2021, by David Sackstein

  I am using https://github.com/rosenbjerg/FFMpegCore to write System.Drawing.Bitmaps to a video file and to read them back as bitmaps.

  


  Based on the examples I was able to implement the first step as so:

  


  public static void WriteToH264(System.Drawing.Bitmap[] bitmaps, string fileName)&#xA;{&#xA;    Directory.CreateDirectory(Path.GetDirectoryName(fileName)!);&#xA;&#xA;    WriteToH264(&#xA;        bitmaps.Select(bitmap => new BitmapVideoFrameWrapper(bitmap)),  &#xA;        4,            &#xA;        fileName);&#xA;}&#xA;&#xA;private static void WriteToH264(IEnumerable<ivideoframe> bitmaps, int frameRate, string fileName)&#xA;{&#xA;    var videoFramesSource = new RawVideoPipeSource(bitmaps)&#xA;    {&#xA;        FrameRate = frameRate&#xA;    };&#xA;  &#xA;    FFMpegArguments&#xA;        .FromPipeInput(videoFramesSource)&#xA;        .OutputToFile(fileName, true, options => options&#xA;            .WithVideoCodec(VideoCodec.LibX264))&#xA;        .ProcessSynchronously();&#xA;}&#xA;</ivideoframe>

  


  For the second part I need to implement the following function using the same library:

  


  public static System.Drawing.Bitmap[] ReadFromH264(string fileName)


  


  But I have not find an samples of how to do this.

  


  How should I implement this function?