Recherche avancée

Sur d’autres sites (3281)

• codification of whammy.js fails

  17 décembre 2015, par DevStarlight

  I’ve been developing a mediastream application capturing video and audio using mediastreamrecorder and whammy.js.

  In this process I have found a issue and I suspect it is related to the way whammy.js render and store into a blob var.

  I realiced about this because I was trying to encode audio (wav) and video (webm) into mp4 with ffmpeg.
  For any reason I don’t really know at all, some of the videos were encoded correctly but others were not. When It wasn’t encoded correctly, the video was not fluent but the audio was.

  I’ve been using this ffmpeg sentence :

  'ffmpeg -i path/to/file.wav -i path/to/file.webm -vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" -map 0:0 -map 1:0 -c:v libx264 -preset ultrafast -qp 0 -c:a libvo_aacenc -b:a 128k path/to/file.mp4'

  Then checking with ffprobe on a good conversion and a bad conversion into mp4 with the same camera I got what it comes :

  For the good one :

  ffprobe version N-77003-g64f7db5 Copyright (c) 2007-2015 the FFmpeg developers
  
    built with gcc 5.2.1 (Ubuntu 5.2.1-22ubuntu2) 20151010
  
    configuration: --prefix=/home/devstarlight/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/devstarlight/ffmpeg_build/include --extra-ldflags=-L/home/devstarlight/ffmpeg_build/lib --bindir=/home/devstarlight/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree
  
    libavutil      55.  9.100 / 55.  9.100
  
    libavcodec     57. 16.101 / 57. 16.101
  
    libavformat    57. 19.100 / 57. 19.100
  
    libavdevice    57.  0.100 / 57.  0.100
  
    libavfilter     6. 19.100 /  6. 19.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/devstarlight/Escritorio/178.mp4':
  
    Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      encoder         : Lavf56.36.100
  
    Duration: 00:00:04.50, start: 0.036281, bitrate: 5821 kb/s
  
      Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 129 kb/s (default)
  
      Metadata:
  
        handler_name    : SoundHandler
  
      Stream #0:1(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p, 400x400 [SAR 1:1 DAR 1:1], 5841 kb/s, 16 fps, 16 tbr, 16384 tbn, 32 tbc (default)
  
      Metadata:
  
        handler_name    : VideoHandler

  For the wrong conversion :

  ffprobe version N-77003-g64f7db5 Copyright (c) 2007-2015 the FFmpeg developers
  
    built with gcc 5.2.1 (Ubuntu 5.2.1-22ubuntu2) 20151010
  
    configuration: --prefix=/home/devstarlight/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/devstarlight/ffmpeg_build/include --extra-ldflags=-L/home/devstarlight/ffmpeg_build/lib --bindir=/home/devstarlight/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree
  
    libavutil      55.  9.100 / 55.  9.100
  
    libavcodec     57. 16.101 / 57. 16.101
  
    libavformat    57. 19.100 / 57. 19.100
  
    libavdevice    57.  0.100 / 57.  0.100
  
    libavfilter     6. 19.100 /  6. 19.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/home/devstarlight/Escritorio/179.mp4':
  
    Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      encoder         : Lavf56.36.100
  
    Duration: 00:00:04.50, start: 0.036281, bitrate: 12815 kb/s
  
      Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
  
      Metadata:
  
        handler_name    : SoundHandler
  
      Stream #0:1(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p, 400x400 [SAR 1:1 DAR 1:1], 13381 kb/s, 1000.24 fps, 1k tbr, 16k tbn, 2k tbc (default)
  
      Metadata:
  
        handler_name    : VideoHandler

  Then I looked into the webm files directly and I realiced that there was no fps defined in the bad file :

  The good record :

  ffprobe version N-77003-g64f7db5 Copyright (c) 2007-2015 the FFmpeg developers
  
    built with gcc 5.2.1 (Ubuntu 5.2.1-22ubuntu2) 20151010
  
    configuration: --prefix=/home/devstarlight/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/devstarlight/ffmpeg_build/include --extra-ldflags=-L/home/devstarlight/ffmpeg_build/lib --bindir=/home/devstarlight/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree
  
    libavutil      55.  9.100 / 55.  9.100
  
    libavcodec     57. 16.101 / 57. 16.101
  
    libavformat    57. 19.100 / 57. 19.100
  
    libavdevice    57.  0.100 / 57.  0.100
  
    libavfilter     6. 19.100 /  6. 19.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  Input #0, matroska,webm, from '/home/devstarlight/Escritorio/179.webm':
  
    Metadata:
  
      encoder         : whammy
  
    Duration: 00:00:04.25, start: 0.000000, bitrate: 1859 kb/s
  
      Stream #0:0: Video: vp8, yuv420p, 400x400, SAR 1:1 DAR 1:1, 1k tbr, 1k tbn, 1k tbc (default)

  The bad record :

  ffprobe version N-77003-g64f7db5 Copyright (c) 2007-2015 the FFmpeg developers
  
    built with gcc 5.2.1 (Ubuntu 5.2.1-22ubuntu2) 20151010
  
    configuration: --prefix=/home/devstarlight/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/devstarlight/ffmpeg_build/include --extra-ldflags=-L/home/devstarlight/ffmpeg_build/lib --bindir=/home/devstarlight/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree
  
    libavutil      55.  9.100 / 55.  9.100
  
    libavcodec     57. 16.101 / 57. 16.101
  
    libavformat    57. 19.100 / 57. 19.100
  
    libavdevice    57.  0.100 / 57.  0.100
  
    libavfilter     6. 19.100 /  6. 19.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  Input #0, matroska,webm, from '/home/devstarlight/Escritorio/178.webm':
  
    Metadata:
  
      encoder         : whammy
  
    Duration: 00:00:04.36, start: 0.000000, bitrate: 1461 kb/s
  
      Stream #0:0: Video: vp8, yuv420p, 400x400, SAR 1:1 DAR 1:1, 16 fps, 16 tbr, 1k tbn, 1k tbc (default)

  ffmpeg output rate option (-r) is empty and rate is forced to 25 fps (because in fact in the webm it doesn’t exist).

  Is that issue registered at any point or someone know how to solve it ?

  Thanks in advice.

• TS video copied to MP4, missing 3 first frames when programmatically read (ffmpeg bug)

  3 septembre 2023, par Vasilis Lemonidis

  Running :

  


  ffmpeg -i test.ts -fflags +genpts -c copy -y test.mp4


  


  for this test.ts, which has 30 frames, readable by opencv, I end up with 28 frames, out of which 27 are readable by opencv. More specifically :

  


  ffprobe -v error -select_streams v:0 -count_packets  -show_entries stream=nb_read_packets -of csv=p=0 tmp.ts 


  


  returns 30.

  


  ffprobe -v error -select_streams v:0 -count_packets     -show_entries stream=nb_read_packets -of csv=p=0 tmp.mp4


  


  returns 28.

  


  Using OpenCV in that manner

  


  cap = cv2.VideoCapture(tmp_path)
readMat = []
while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break
        readMat.append(frame)


  


  I get for the ts file 30 frames, while for the mp4 27 frames.

  


  Could someone explain why the discrepancies ? I get no error during the transformation from ts to mp4 :

  


  ffmpeg version N-111746-gd53acf452f Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 11.3.0 (GCC)
  configuration: --ld=g++ --bindir=/bin --extra-libs='-lpthread -lm' --pkg-config-flags=--static --enable-static --enable-gpl --enable-libaom --enable-libass --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-cuda-nvcc --enable-cuvid --enable-nvenc --enable-libnpp 
  libavutil      58. 16.101 / 58. 16.101
  libavcodec     60. 23.100 / 60. 23.100
  libavformat    60. 10.100 / 60. 10.100
  libavdevice    60.  2.101 / 60.  2.101
  libavfilter     9. 10.100 /  9. 10.100
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
[mpegts @ 0x4237240] DTS discontinuity in stream 0: packet 5 with DTS 306003, packet 6 with DTS 396001
Input #0, mpegts, from 'tmp.ts':
  Duration: 00:00:21.33, start: 3.400000, bitrate: 15 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 300x300, 1 fps, 3 tbr, 90k tbn
Output #0, mp4, to 'test.mp4':
  Metadata:
    encoder         : Lavf60.10.100
  Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 300x300, q=2-31, 1 fps, 3 tbr, 90k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[out#0/mp4 @ 0x423e280] video:25kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.192123%
frame=   30 fps=0.0 q=-1.0 Lsize=      26kB time=00:00:21.00 bitrate=  10.3kbits/s speed=1e+04x 


  


  Additional information

  


  The origin of the video I am processing comes from a continuous stitching operation of still images ts videos, produced by this class update method :

  


  import cv2
import os
import subprocess
from tempfile import NamedTemporaryFile
class VideoUpdater:
    def __init__(
        self, video_path: str, framerate: int, timePerFrame: Optional[int] = None
    ):
        """
        Video updater takes in a video path, and updates it using a supplied frame, based on a given framerate.
        Args:
            video_path: str: Specify the path to the video file
            framerate: int: Set the frame rate of the video
        """
        if not video_path.endswith(".mp4"):
            LOGGER.warning(
                f"File type {os.path.splitext(video_path)[1]} not supported for streaming, switching to ts"
            )
            video_path = os.path.splitext(video_path)[0] + ".mp4"

        self._ps = None
        self.env = {
            
        }
        self.ffmpeg = "/usr/bin/ffmpeg "

        self.video_path = video_path
        self.ts_path = video_path.replace(".mp4", ".ts")
        self.tfile = None
        self.framerate = framerate
        self._video = None
        self.last_frame = None
        self.curr_frame = None


    def update(self, frame: np.ndarray):
        if len(frame.shape) == 2:
            frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
        else:
            frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
        self.writeFrame(frame)

    def writeFrame(self, frame: np.ndarray):
        """
        The writeFrame function takes a frame and writes it to the video file.
        Args:
            frame: np.ndarray: Write the frame to a temporary file
        """


        tImLFrame = NamedTemporaryFile(suffix=".png")
        tVidLFrame = NamedTemporaryFile(suffix=".ts")

        cv2.imwrite(tImLFrame.name, frame)
        ps = subprocess.Popen(
            self.ffmpeg
            + rf"-loop 1 -r {self.framerate} -i {tImLFrame.name} -t {self.framerate} -vcodec libx264 -pix_fmt yuv420p -y {tVidLFrame.name}",
            env=self.env,
            shell=True,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
        )
        ps.communicate()
        if os.path.isfile(self.ts_path):
            # this does not work to watch, as timestamps are not updated
            ps = subprocess.Popen(
                self.ffmpeg
                + rf'-i "concat:{self.ts_path}|{tVidLFrame.name}" -c copy -y {self.ts_path.replace(".ts", ".bak.ts")}',
                env=self.env,
                shell=True,
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE,
            )
            ps.communicate()
            shutil.move(self.ts_path.replace(".ts", ".bak.ts"), self.ts_path)

        else:
            shutil.copyfile(tVidLFrame.name, self.ts_path)
        # fixing timestamps, we dont have to wait for this operation
        ps = subprocess.Popen(
            self.ffmpeg
            + rf"-i {self.ts_path} -fflags +genpts -c copy -y {self.video_path}",
            env=self.env,
            shell=True,
            # stdout=subprocess.PIPE,
            # stderr=subprocess.PIPE,
        )
        tImLFrame.close()
        tVidLFrame.close()


  


• FFMPEG : TS video copied to MP4, missing 3 first frames [closed]

  21 août 2023, par Vasilis Lemonidis

  Running :

  


  ffmpeg -i test.ts -fflags +genpts -c copy -y test.mp4


  


  for this test.ts, which has 30 frames, readable by opencv, I end up with 28 frames, out of which 27 are readable by opencv. More specifically :

  


  ffprobe -v error -select_streams v:0 -count_packets  -show_entries stream=nb_read_packets -of csv=p=0 tmp.ts 


  


  returns 30.

  


  ffprobe -v error -select_streams v:0 -count_packets     -show_entries stream=nb_read_packets -of csv=p=0 tmp.mp4


  


  returns 28.

  


  Using OpenCV in that manner

  


  cap = cv2.VideoCapture(tmp_path)
readMat = []
while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break
        readMat.append(frame)


  


  I get for the ts file 30 frames, while for the mp4 27 frames.

  


  Could someone explain why the discrepancies ? I get no error during the transformation from ts to mp4 :

  


  ffmpeg version N-111746-gd53acf452f Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 11.3.0 (GCC)
  configuration: --ld=g++ --bindir=/bin --extra-libs='-lpthread -lm' --pkg-config-flags=--static --enable-static --enable-gpl --enable-libaom --enable-libass --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libsvtav1 --enable-libdav1d --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-cuda-nvcc --enable-cuvid --enable-nvenc --enable-libnpp 
  libavutil      58. 16.101 / 58. 16.101
  libavcodec     60. 23.100 / 60. 23.100
  libavformat    60. 10.100 / 60. 10.100
  libavdevice    60.  2.101 / 60.  2.101
  libavfilter     9. 10.100 /  9. 10.100
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
[mpegts @ 0x4237240] DTS discontinuity in stream 0: packet 5 with DTS 306003, packet 6 with DTS 396001
Input #0, mpegts, from 'tmp.ts':
  Duration: 00:00:21.33, start: 3.400000, bitrate: 15 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 300x300, 1 fps, 3 tbr, 90k tbn
Output #0, mp4, to 'test.mp4':
  Metadata:
    encoder         : Lavf60.10.100
  Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 300x300, q=2-31, 1 fps, 3 tbr, 90k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[out#0/mp4 @ 0x423e280] video:25kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 4.192123%
frame=   30 fps=0.0 q=-1.0 Lsize=      26kB time=00:00:21.00 bitrate=  10.3kbits/s speed=1e+04x 


  


  Additional information

  


  The origin of the video I am processing comes from a continuous stitching operation of still images ts videos, produced by this class update method :

  


  import cv2
import os
import subprocess
from tempfile import NamedTemporaryFile
class VideoUpdater:
    def __init__(
        self, video_path: str, framerate: int, timePerFrame: Optional[int] = None
    ):
        """
        Video updater takes in a video path, and updates it using a supplied frame, based on a given framerate.
        Args:
            video_path: str: Specify the path to the video file
            framerate: int: Set the frame rate of the video
        """
        if not video_path.endswith(".mp4"):
            LOGGER.warning(
                f"File type {os.path.splitext(video_path)[1]} not supported for streaming, switching to ts"
            )
            video_path = os.path.splitext(video_path)[0] + ".mp4"

        self._ps = None
        self.env = {
            
        }
        self.ffmpeg = "/usr/bin/ffmpeg "

        self.video_path = video_path
        self.ts_path = video_path.replace(".mp4", ".ts")
        self.tfile = None
        self.framerate = framerate
        self._video = None
        self.last_frame = None
        self.curr_frame = None


    def update(self, frame: np.ndarray):
        if len(frame.shape) == 2:
            frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
        else:
            frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
        self.writeFrame(frame)

    def writeFrame(self, frame: np.ndarray):
        """
        The writeFrame function takes a frame and writes it to the video file.
        Args:
            frame: np.ndarray: Write the frame to a temporary file
        """


        tImLFrame = NamedTemporaryFile(suffix=".png")
        tVidLFrame = NamedTemporaryFile(suffix=".ts")

        cv2.imwrite(tImLFrame.name, frame)
        ps = subprocess.Popen(
            self.ffmpeg
            + rf"-loop 1 -r {self.framerate} -i {tImLFrame.name} -t {self.framerate} -vcodec libx264 -pix_fmt yuv420p -y {tVidLFrame.name}",
            env=self.env,
            shell=True,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
        )
        ps.communicate()
        if os.path.isfile(self.ts_path):
            # this does not work to watch, as timestamps are not updated
            ps = subprocess.Popen(
                self.ffmpeg
                + rf'-i "concat:{self.ts_path}|{tVidLFrame.name}" -c copy -y {self.ts_path.replace(".ts", ".bak.ts")}',
                env=self.env,
                shell=True,
                stdout=subprocess.PIPE,
                stderr=subprocess.PIPE,
            )
            ps.communicate()
            shutil.move(self.ts_path.replace(".ts", ".bak.ts"), self.ts_path)

        else:
            shutil.copyfile(tVidLFrame.name, self.ts_path)
        # fixing timestamps, we dont have to wait for this operation
        ps = subprocess.Popen(
            self.ffmpeg
            + rf"-i {self.ts_path} -fflags +genpts -c copy -y {self.video_path}",
            env=self.env,
            shell=True,
            # stdout=subprocess.PIPE,
            # stderr=subprocess.PIPE,
        )
        tImLFrame.close()
        tVidLFrame.close()