Recherche avancée

Médias (1)

Mot : - Tags -/illustrator

Autres articles (54)

  • Publier sur MédiaSpip

    13 juin 2013

    Puis-je poster des contenus à partir d’une tablette Ipad ?
    Oui, si votre Médiaspip installé est à la version 0.2 ou supérieure. Contacter au besoin l’administrateur de votre MédiaSpip pour le savoir

  • Submit bugs and patches

    13 avril 2011

    Unfortunately a software is never perfect.
    If you think you have found a bug, report it using our ticket system. Please to help us to fix it by providing the following information : the browser you are using, including the exact version as precise an explanation as possible of the problem if possible, the steps taken resulting in the problem a link to the site / page in question
    If you think you have solved the bug, fill in a ticket and attach to it a corrective patch.
    You may also (...)

  • Librairies et binaires spécifiques au traitement vidéo et sonore

    31 janvier 2010, par

    Les logiciels et librairies suivantes sont utilisées par SPIPmotion d’une manière ou d’une autre.
    Binaires obligatoires FFMpeg : encodeur principal, permet de transcoder presque tous les types de fichiers vidéo et sonores dans les formats lisibles sur Internet. CF ce tutoriel pour son installation ; Oggz-tools : outils d’inspection de fichiers ogg ; Mediainfo : récupération d’informations depuis la plupart des formats vidéos et sonores ;
    Binaires complémentaires et facultatifs flvtool2 : (...)

Sur d’autres sites (6371)

  • extracting video and data streams from MPEG2 TS over RTP in real-time

    10 janvier 2024, par Tejal Barnwal

    I have H264 video stream and KLV meta data encapsulated inside MPEG2 TS container which are sent over an RTP over UDP from a camera.
I intend to do the following :

    


      

    1. Extract both video and data streams from RTP

      2. 


    2. Process video feed using opencv in a seperate thread

      3. 


    3. process klv metadata in a seperate thread

      4. 


    


    My problem what exact arguments should I provide to ffmpeg so as to read h264 video stream and show the images frame by frame using opencv ?

    


    With the help of some previous posts like Simultaneously map video and data streams to one subprocess pipeline in real-time, I was able to get some idea about how could I proceed to procees the stream over RTP.

    


    I started out by using the following script :

    


    #!/usr/bin/env python3
from asyncio import streams
from logging.handlers import QueueListener
import klvdata
import subprocess as sp
import shlex
import threading
import numpy as np
import cv2
import time
from io import BytesIO

# Video reader thread.
def video_reader(pipe):
    cols, rows = 1280, 720  # Assume we know frame size is 1280x720

    counter = 0
    while True:
        print("read image")
        raw_image = pipe.read(cols*rows*3)  # Read raw video frame

        # Break the loop when length is too small
        if len(raw_image) < cols*rows*3:
            break

        if (counter % 10) == 0:
            # Show video frame evey 60 frames
            image = np.frombuffer(raw_image, np.uint8).reshape([rows, cols, 3])
            cv2.imshow('Video', image) # Show video image for testing
            cv2.waitKey(1)
        counter += 1
        print("image showed on window")
        time.sleep(0.25)



# https://github.com/paretech/klvdata/tree/master/klvdata
def bytes_to_int(value, signed=False):
    """Return integer given bytes."""
    return int.from_bytes(bytes(value), byteorder='big', signed=signed)


# Data reader thread (read KLV data).
def data_reader(pipe):
    key_length = 16  # Assume key length is 16 bytes.

    f = open('data.bin', 'wb')  # For testing - store the KLV data to data.bin (binary file)

    while True:
        # https://en.wikipedia.org/wiki/KLV
        # The first few bytes are the Key, much like a key in a standard hash table data structure.
        # Keys can be 1, 2, 4, or 16 bytes in length.
        # Presumably in a separate specification document you would agree on a key length for a given application.
        key = pipe.read(key_length)  # Read the key
        
        if len(key) < key_length:
            break  # Break the loop when length is too small
        f.write(key)  # Write data to binary file for testing

        # https://github.com/paretech/klvdata/tree/master/klvdata
        # Length field
        len_byte = pipe.read(1)

        if len(len_byte) < 1:
            break  # Break the loop when length is too small
        f.write(len_byte)  # Write data to binary file for testing

        byte_length = bytes_to_int(len_byte)

        # https://github.com/paretech/klvdata/tree/master/klvdata                                                
        if byte_length < 128:
            # BER Short Form
            length = byte_length
            ber_len_bytes = b''
        else:
            # BER Long Form
            ber_len = byte_length - 128
            ber_len_bytes = pipe.read(ber_len)

            if len(ber_len_bytes) < ber_len:
                break  # Break the loop when length is too small
            f.write(ber_len_bytes)  # Write ber_len_bytes to binary file for testing

            length = bytes_to_int(ber_len_bytes)

        # Read the value (length bytes)
        value = pipe.read(length)
        if len(value) < length:
            break  # Break the loop when length is too small
        f.write(value)  # Write data to binary file for testing

        klv_data = key + len_byte + ber_len_bytes + value  # Concatenate key length and data
        klv_data_as_bytes_io = BytesIO(klv_data)  # Wrap klv_data with BytesIO (before parsing)

        # Parse the KLV data
        for packet in klvdata.StreamParser(klv_data_as_bytes_io): 
            metadata = packet.MetadataList()
            for key, value in metadata.items():
                print(key, value)
                
            print("\n") # New line

# Execute FFmpeg as sub-process
# Map the video to stderr and map the data to stdout
process = sp.Popen(shlex.split('ffmpeg -hide_banner -loglevel quiet '                        # Set loglevel to quiet for disabling the prints ot stderr
                               '-i "rtp://192.168.0.141:11024" '                                        # Input video "Day Flight.mpg"
                               '-map 0:v -c:v rawvideo -pix_fmt bgr24 -f:v rawvideo pipe:2 ' # rawvideo format is mapped to stderr pipe (raw video codec with bgr24 pixel format)
                               '-map 0:d -c copy -copy_unknown -f:d data pipe:1 '            # Copy the data without ddecoding.
                               '-report'),                                                   # Create a log file (because we can't the statuses that are usually printed to stderr).
                                stdout=sp.PIPE, stderr=sp.PIPE)


# Start video reader thread (pass stderr pipe as argument).
video_thread = threading.Thread(target=video_reader, args=(process.stderr,))
video_thread.start()

# Start data reader thread (pass stdout pipe as argument).
data_thread = threading.Thread(target=data_reader, args=(process.stdout,))
data_thread.start()


# Wait for threads (and process) to finish.
video_thread.join()
data_thread.join()
process.wait()



    


    With the above script, I was facing two issues :

    


      

    1. The second thread resulted in an attribute error

      2. 


    


    Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "video_data_extraction.py", line 97, in data_reader
    print(packet.MetadataList())
AttributeError: 'UnknownElement' object has no attribute 'MetadataList'



    


      

    1. With this though I continuously able to see following output on the terminal regarding reading the images

      2. 


    


    read image
image showed on window
read image
image showed on window
read image
image showed on window
read image
image showed on window
read image
image showed on window
read image
image showed on window


    


    The imshow windows wasnt updating properly ! It seemed stuck after a few frames.

    


    Further diving into the lane with the help of following command, I concluded that the video stream that I am reading has H264 encoding

    


    ffprobe -i rtp://192.168.0.141:11024 -show_streams -show_formats


    


    Output of the above command :

    


    ffprobe version 4.2.7-0ubuntu0.1 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[rtp @ 0xaaaac81ecce0] PES packet size mismatch
    Last message repeated 62 times
[NULL @ 0xaaaac81f09b0] non-existing PPS 0 referenced
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[rtp @ 0xaaaac81ecce0] PES packet size mismatch
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[rtp @ 0xaaaac81ecce0] PES packet size mismatch
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[rtp @ 0xaaaac81ecce0] PES packet size mismatch
    Last message repeated 187 times
Input #0, rtp, from 'rtp://192.168.0.141:11024':
  Duration: N/A, start: 1317.040656, bitrate: N/A
  Program 1 
    Stream #0:1: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720, 25 fps, 25 tbr, 90k tbn
    Stream #0:0: Data: klv (KLVA / 0x41564C4B)
Unsupported codec with id 100356 for input stream 0
[STREAM]
index=0
codec_name=klv
codec_long_name=SMPTE 336M Key-Length-Value (KLV) metadata
profile=unknown
codec_type=data
codec_tag_string=KLVA
codec_tag=0x41564c4b
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/90000
start_pts=118533659
start_time=1317.040656
duration_ts=N/A
duration=N/A
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=N/A
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=0
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
[/STREAM]
[STREAM]
index=1
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=Constrained Baseline
codec_type=video
codec_time_base=1/50
codec_tag_string=[27][0][0][0]
codec_tag=0x001b
width=1280
height=720
coded_width=1280
coded_height=720
has_b_frames=0
sample_aspect_ratio=N/A
display_aspect_ratio=N/A
pix_fmt=yuv420p
level=31
color_range=unknown
color_space=unknown
color_transfer=unknown
color_primaries=unknown
chroma_location=left
field_order=progressive
timecode=N/A
refs=1
is_avc=false
nal_length_size=0
id=N/A
r_frame_rate=25/1
avg_frame_rate=25/1
time_base=1/90000
start_pts=118533659
start_time=1317.040656
duration_ts=N/A
duration=N/A
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=0
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
[/STREAM]
[FORMAT]
filename=rtp://192.168.0.141:11024
nb_streams=2
nb_programs=1
format_name=rtp
format_long_name=RTP input
start_time=1317.040656
duration=N/A
size=N/A
bit_rate=N/A
probe_score=100
[/FORMAT]


    


    Further, in the log output, I see a lot of statements in regard to missed packets and PES packet mismatch

    


    [rtp @ 0xaaaaf31896c0] max delay reached. need to consume packet
[rtp @ 0xaaaaf31896c0] RTP: missed 98 packets
[rtp @ 0xaaaaf31896c0] Continuity check failed for pid 40 expected 14 got 10
[rtp @ 0xaaaaf31896c0] PES packet size mismatch
rtp://192.168.0.141:11024: corrupt input packet in stream 0
frame=  124 fps=2.6 q=-0.0 size=  334800kB time=00:00:05.32 bitrate=515406.0kbits/s dup=97 drop=0 speed=0.111x 


    


    What arguments do I provide to ffmpeg and in what order because my stream 0 is metadata and stream 1 is video so as to display image frame by frame with opencv ?
I would be grateful for any help that you could provide.

    


    Further, I also have a query regarding how does ffmpeg know to that it has to first convert the rtp packets into mpeg2 TS packets before segregating video stream and data stream ?

    


  • How to record frames with ffmpeg and finish the recording

    20 février 2024, par Jorge Augusto Wilchen

    In the following code, i trying to create a class to record frames from an IP camera (RTSP), save frames on a .avi file and finish the record, but, when i kill the operation, the video file may be corrupted. Have any other more safely way to stop the ffmpeg recording ?

    


    .cpp file :

    


    #include "videorecorder.h"


VideoRecorder::VideoRecorder(const std::string& rtspUrl) :
    url(rtspUrl),
    recording(false)
{

}

VideoRecorder::~VideoRecorder()
{
    end_record();
}

bool VideoRecorder::start_record(const std::string &fileName)
{
    if (recording) {
        std::cerr << "Already recording." << std::endl;
        return false;
    }

    std::string command = "ffmpeg -rtsp_transport udp -i " + url
                          + " -c:v mjpeg -preset fast -qp 0 " + fileName;

    videoWriter = popen(command.c_str(), "w");
    if (!videoWriter) {
        std::cerr << "Error opening ffmpeg process." << std::endl;
        return false;
    }

    recording = true;
    ffmpegProcessId = getpid();
    std::cout << "Recording started." << std::endl;
    return true;
}

bool VideoRecorder::end_record()
{
    if (recording) {
        if (videoWriter) {
            pid_t ffmpegPID = fileno(videoWriter);

            if (kill(ffmpegPID, SIGTERM) == 0) {
                std::cout << "Recording terminated successfully." << std::endl;
            } else {
                std::cerr << "Error terminating recording." << std::endl;
                return false;
            }

            int status = pclose(videoWriter);

            if (status == 0) {
                std::cout << "Recording ended successfully." << std::endl;
            } else {
                std::cerr << "Error ending recording. pclose status: " << status << std::endl;
                return false;
            }
        } else {
            std::cerr << "Error ending recording. videoWriter is nullptr." << std::endl;
            return false;
        }

        recording = false;
        return true;
    }

    return false;
}


    


    .h file :

    


    #ifndef VIDEORECORDER_H&#xA;#define VIDEORECORDER_H&#xA;&#xA;#include <string>&#xA;#include <iostream>&#xA;#include <fstream>&#xA;#include <cstdlib>&#xA;#include <csignal>&#xA;#include <sys></sys>wait.h>&#xA;&#xA;extern "C" {&#xA;#include <libavcodec></libavcodec>avcodec.h>&#xA;#include <libavformat></libavformat>avformat.h>&#xA;#include <libavutil></libavutil>avutil.h>&#xA;#include <libavutil></libavutil>opt.h>&#xA;#include <libswscale></libswscale>swscale.h>&#xA;#include &#xA;}&#xA;&#xA;#include <linux></linux>videodev2.h>&#xA;&#xA;#include <opencv2></opencv2>opencv.hpp>&#xA;#include <opencv2></opencv2>videoio.hpp>&#xA;#include <opencv2></opencv2>highgui/highgui.hpp>&#xA;&#xA;&#xA;class VideoRecorder&#xA;{&#xA;public:&#xA;    VideoRecorder(const std::string&amp; rtspUrl);&#xA;    ~VideoRecorder();&#xA;    bool start_record(const std::string&amp; fileName);&#xA;    bool end_record();&#xA;&#xA;private:&#xA;    std::string url;&#xA;    AVFormatContext *formatContext;&#xA;    AVStream *videoStream;&#xA;    AVCodecContext *codecContext;&#xA;    AVCodec *codec;&#xA;    SwsContext *swsContext;&#xA;    AVFrame *frame;&#xA;    AVPacket packet;&#xA;    bool recording;&#xA;    pid_t ffmpegProcessId;&#xA;    FILE* videoWriter;&#xA;};&#xA;&#xA;#endif // VIDEORECORDER_H&#xA;</csignal></cstdlib></fstream></iostream></string>

    &#xA;

    I'm using the ffmpeg lib becouse i need max speed on frames recording, and OpenCV and AV Lib is much slowness than ffmpeg.

    &#xA;

    This my terminal output after recording during 10 seconds (generated a file with 23 seconds duration) :

    &#xA;

    Recording started.&#xA;ffmpeg version 4.3.6-0&#x2B;deb11u1&#x2B;rpt5 Copyright (c) 2000-2023 the FFmpeg developers&#xA;  built with gcc 10 (Debian 10.2.1-6)&#xA;  configuration: --prefix=/usr --extra-version=0&#x2B;deb11u1&#x2B;rpt5 --toolchain=hardened --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-mmal --enable-neon --enable-v4l2-request --enable-libudev --enable-epoxy --enable-sand --libdir=/usr/lib/aarch64-linux-gnu --arch=arm64 --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-vout-drm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared&#xA;  libavutil      56. 51.100 / 56. 51.100&#xA;  libavcodec     58. 91.100 / 58. 91.100&#xA;  libavformat    58. 45.100 / 58. 45.100&#xA;  libavdevice    58. 10.100 / 58. 10.100&#xA;  libavfilter     7. 85.100 /  7. 85.100&#xA;  libavresample   4.  0.  0 /  4.  0.  0&#xA;  libswscale      5.  7.100 /  5.  7.100&#xA;  libswresample   3.  7.100 /  3.  7.100&#xA;  libpostproc    55.  7.100 / 55.  7.100&#xA;Input #0, rtsp, from &#x27;rtsp://admin:[password]@[ip]:[port]/live/0/MAIN&#x27;:&#xA;  Metadata:&#xA;    title           : RTSP Server&#xA;  Duration: N/A, start: 0.280000, bitrate: N/A&#xA;    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1920x1080, 25 fps, 25 tbr, 90k tbn, 50 tbc&#xA;Codec AVOption preset (Configuration preset) specified for output file #0 (/home/guardian-tech/Pictures/output_frame.avi) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.&#xA;Codec AVOption qp (Constant quantization parameter rate control method) specified for output file #0 (/home/guardian-tech/Pictures/output_frame.avi) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.&#xA;Stream mapping:&#xA;  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))&#xA;Press [q] to stop, [?] for help&#xA;Output #0, avi, to &#x27;/home/guardian-tech/Pictures/output_frame.avi&#x27;:&#xA;  Metadata:&#xA;    INAM            : RTSP Server&#xA;    ISFT            : Lavf58.45.100&#xA;    Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), yuvj420p(pc), 1920x1080, q=2-31, 200 kb/s, 25 fps, 25 tbn, 25 tbc&#xA;    Metadata:&#xA;      encoder         : Lavc58.91.100 mjpeg&#xA;    Side data:&#xA;      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A&#xA;[rtsp @ 0x5592e7bb00] max delay reached. need to consume packet&#xA;[rtsp @ 0x5592e7bb00] RTP: missed 212 packets&#xA;[h264 @ 0x5592ebb790] concealing 2192 DC, 2192 AC, 2192 MV errors in I frame&#xA;rtsp://admin:[password]@[ip]:[port]/live/0/MAIN: corrupt decoded frame in stream 0&#xA;[rtsp @ 0x5592e7bb00] max delay reached. need to consume packet&#xA;[rtsp @ 0x5592e7bb00] RTP: missed 6 packets&#xA;[rtsp @ 0x5592e7bb00] max delay reached. need to consume packet&#xA;[rtsp @ 0x5592e7bb00] RTP: missed 14 packets&#xA;[h264 @ 0x5592f1bd30] cabac decode of qscale diff failed at 42 29&#xA;[h264 @ 0x5592f1bd30] error while decoding MB 42 29, bytestream 0&#xA;[h264 @ 0x5592f1bd30] concealing 4687 DC, 4687 AC, 4687 MV errors in I frame&#xA;rtsp://admin:[password]@[ip]:[port]/live/0/MAIN: corrupt decoded frame in stream 0&#xA;Error terminating recording.&#xA;

    &#xA;

  • Processing h264 video/audio stream

    1er mars 2024, par Dinamo

    I have separate audio and video streams from a camera that I'm trying to merge to a single file.

    &#xA;

    Video processing

    &#xA;

    I have JVT NAL sequence, H.264 video, main @ L 31 data, I had the frames separated and I appended (concat) the frames to one file.

    &#xA;

    here is the ffprobe output of the data :

    &#xA;

    ffprobe version 4.2.7-0ubuntu0.1 Copyright (c) 2007-2022 the FFmpeg developers&#xA;  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)&#xA;  configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared&#xA;  libavutil      56. 31.100 / 56. 31.100&#xA;  libavcodec     58. 54.100 / 58. 54.100&#xA;  libavformat    58. 29.100 / 58. 29.100&#xA;  libavdevice    58.  8.100 / 58.  8.100&#xA;  libavfilter     7. 57.100 /  7. 57.100&#xA;  libavresample   4.  0.  0 /  4.  0.  0&#xA;  libswscale      5.  5.100 /  5.  5.100&#xA;  libswresample   3.  5.100 /  3.  5.100&#xA;  libpostproc    55.  5.100 / 55.  5.100&#xA;Input #0, h264, from &#x27;frame_data.h264&#x27;:&#xA;  Duration: N/A, bitrate: N/A&#xA;    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1280x720, 25 fps, 25 tbr, 1200k tbn, 50 tbc&#xA;

    &#xA;

    the file is being played on vlc, however the video is played like it is on fast forward (every 2 seconds of the video should be about 1 second)

    &#xA;

    The video should run at 15fps (not sure about that because of the mentioned avi file)

    &#xA;

    here is ffprobe output of working avi file that contain the video stream :

    &#xA;

    Input #0, avi, from &#x27;t.avi&#x27;:&#xA;  Duration: 00:03:29.00, start: 0.000000, bitrate: 1116 kb/s&#xA;    Stream #0:0: Video: h264 (Main) (H264 / 0x34363248), yuvj420p(pc, bt709, progressive), 1280x720, 849 kb/s, 30 fps, 15 tbr, 30 tbn, 60 tbc&#xA;

    &#xA;

    I tried this command ffmpeg -i frame_data.h264 -filter:v fps=15 test.h264 but the video is still playing like its on fast forward

    &#xA;

    vlc data before changing fps to 15

    &#xA;

    enter image description here&#xA;enter image description here

    &#xA;

    vlc data after changing fps to 15

    &#xA;

    enter image description here&#xA;enter image description here

    &#xA;

    frames info (raw)

    &#xA;

    [FRAME]&#xA;media_type=video&#xA;stream_index=0&#xA;key_frame=1&#xA;pkt_pts=N/A&#xA;pkt_pts_time=N/A&#xA;-> pkt_dts=N/A&#xA;-> pkt_dts_time=N/A&#xA;best_effort_timestamp=N/A&#xA;best_effort_timestamp_time=N/A&#xA;-> pkt_duration=48000&#xA;-> pkt_duration_time=0.040000&#xA;pkt_pos=1476573&#xA;pkt_size=57677&#xA;width=1280&#xA;height=720&#xA;pix_fmt=yuvj420p&#xA;sample_aspect_ratio=N/A&#xA;pict_type=I&#xA;coded_picture_number=189&#xA;display_picture_number=0&#xA;interlaced_frame=0&#xA;top_field_first=0&#xA;repeat_pict=0&#xA;color_range=pc&#xA;color_space=bt709&#xA;color_primaries=bt709&#xA;color_transfer=bt709&#xA;chroma_location=left&#xA;[/FRAME]&#xA;[FRAME]&#xA;media_type=video&#xA;stream_index=0&#xA;key_frame=0&#xA;pkt_pts=N/A&#xA;pkt_pts_time=N/A&#xA;-> pkt_dts=N/A&#xA;-> pkt_dts_time=N/A&#xA;best_effort_timestamp=N/A&#xA;best_effort_timestamp_time=N/A&#xA;-> pkt_duration=48000&#xA;-> pkt_duration_time=0.040000&#xA;pkt_pos=1534250&#xA;pkt_size=3928&#xA;width=1280&#xA;height=720&#xA;pix_fmt=yuvj420p&#xA;sample_aspect_ratio=N/A&#xA;pict_type=P&#xA;coded_picture_number=190&#xA;display_picture_number=0&#xA;interlaced_frame=0&#xA;top_field_first=0&#xA;repeat_pict=0&#xA;color_range=pc&#xA;color_space=bt709&#xA;color_primaries=bt709&#xA;color_transfer=bt709&#xA;chroma_location=left&#xA;[/FRAME]&#xA;

    &#xA;

    frame info from working (processed) avi

    &#xA;

    [FRAME]&#xA;media_type=video&#xA;stream_index=0&#xA;key_frame=0&#xA;pkt_pts=N/A&#xA;pkt_pts_time=N/A&#xA;-> pkt_dts=500&#xA;-> pkt_dts_time=16.666667&#xA;best_effort_timestamp=500&#xA;best_effort_timestamp_time=16.666667&#xA;-> pkt_duration=1&#xA;-> pkt_duration_time=0.033333&#xA;pkt_pos=1772182&#xA;pkt_size=3070&#xA;width=1280&#xA;height=720&#xA;pix_fmt=yuvj420p&#xA;sample_aspect_ratio=N/A&#xA;pict_type=P&#xA;coded_picture_number=191&#xA;display_picture_number=0&#xA;interlaced_frame=0&#xA;top_field_first=0&#xA;repeat_pict=0&#xA;color_range=pc&#xA;color_space=bt709&#xA;color_primaries=bt709&#xA;color_transfer=bt709&#xA;chroma_location=left&#xA;[/FRAME]&#xA;

    &#xA;

    pkt_pts and pkt_dts are missing in the raw data, maybe that is the reason the video plays on fast forward ? in the working (processed) avi pkt_dts, pkt_dts_time, pkt_duration_time are not missing.&#xA;pkt_duration_time is always 0.033333&#xA;pkt_dts always jumps by 2 (there is no odd pkt_dts)

    &#xA;

    Audio processing

    &#xA;

    I have raw pcm audio of the video :

    &#xA;

    here is an output of ffplay -f s16le -ar 16k -ac 1 frame_data2.h264 on the raw data

    &#xA;

    enter image description here

    &#xA;

    The raw audio does not playing well using this command.

    &#xA;

    I guessed the parameters (incorrectly ?) from working avi file that contains the pcm data :

    &#xA;

    Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 16000 Hz, 1 channels, s16, 256 kb/s&#xA;

    &#xA;

    here is frame info from working (processed) avi file

    &#xA;

    [FRAME]&#xA;media_type=audio&#xA;stream_index=1&#xA;key_frame=1&#xA;pkt_pts=4320&#xA;pkt_pts_time=0.270000&#xA;pkt_dts=4320&#xA;pkt_dts_time=0.270000&#xA;best_effort_timestamp=4320&#xA;best_effort_timestamp_time=0.270000&#xA;pkt_duration=480&#xA;pkt_duration_time=0.030000&#xA;pkt_pos=11670&#xA;pkt_size=960&#xA;sample_fmt=s16&#xA;nb_samples=480&#xA;channels=1&#xA;channel_layout=unknown&#xA;[/FRAME]&#xA;

    &#xA;

    I'm trying to understand how should I process the video and the audio so that they will play well (merge them myself with the right av processing to avi/mp4 file)

    &#xA;

    I also find this extra data that was included for each frame :&#xA;I suspect the first column is a sequence number maybe related to dts somehow, second column is a timestamp (milliseconds ?) (maybe you can help me parse it)

    &#xA;

    600 0xd96533 (audio)&#xA;601 0xd9e1dd (audio)&#xA;602 0xda4f52 (audio)&#xA;603 0xda5a63 (video)&#xA;604 0xdacc4b (audio)&#xA;605 0xdb39a3 (audio)&#xA;606 0xdb5ee9 (video)&#xA;607 0xdbb6d8 (audio)&#xA;608 0xdc23fe (audio)&#xA;609 0xdcb255 (audio)&#xA;610 0xdd0e69 (audio)&#xA;611 0xdd8b96 (audio)&#xA;612 0xdd67d0 (video)&#xA;613 0xddf8bd (audio)&#xA;614 0xde75fe (audio)&#xA;615 0xde6c28 (video)&#xA;616 0xdee328 (audio)&#xA;617 0xdf6012 (audio)&#xA;618 0xdfcd9d (audio)&#xA;619 0xe04a7d (audio)&#xA;620 0xe07523 (video)&#xA;621 0xe0b80c (audio)&#xA;622 0xe134db (audio)&#xA;623 0xe1798b (video)&#xA;624 0xe1a251 (audio)&#xA;625 0xe21f5c (audio)&#xA;626 0xe28ce1 (audio)&#xA;627 0xe27df7 (video)&#xA;628 0xe309a7 (audio)&#xA;629 0xe376fe (audio)&#xA;630 0xe3826e (video)&#xA;631 0xe3f3fd (audio)&#xA;632 0xe46154 (audio)&#xA;633 0xe486e9 (video)&#xA;634 0xe4de79 (audio)&#xA;635 0xe54bbe (audio)&#xA;636 0xe58b79 (video)&#xA;637 0xe5c8cb (audio)&#xA;638 0xe6361e (audio)&#xA;639 0xe6b35b (audio)&#xA;640 0xe68fb7 (video)&#xA;641 0xe72093 (audio)&#xA;642 0xe79db1 (audio)&#xA;643 0xe7942f (video)&#xA;644 0xe80b09 (audio)&#xA;645 0xe887ed (audio)&#xA;646 0xe8f546 (audio)&#xA;647 0xe97256 (audio)&#xA;648 0xe99d2c (video)&#xA;649 0xe9dfbd (audio)&#xA;650 0xea5ca8 (audio)&#xA;

    &#xA;

    Thanks.

    &#xA;

Navigation