Recherche avancée
Autres articles (47)
-
Configurer la prise en compte des langues
15 novembre 2010, parAccéder à la configuration et ajouter des langues prises en compte
Afin de configurer la prise en compte de nouvelles langues, il est nécessaire de se rendre dans la partie "Administrer" du site.
De là, dans le menu de navigation, vous pouvez accéder à une partie "Gestion des langues" permettant d’activer la prise en compte de nouvelles langues.
Chaque nouvelle langue ajoutée reste désactivable tant qu’aucun objet n’est créé dans cette langue. Dans ce cas, elle devient grisée dans la configuration et (...) -
Emballe médias : à quoi cela sert ?
4 février 2011, parCe plugin vise à gérer des sites de mise en ligne de documents de tous types.
Il crée des "médias", à savoir : un "média" est un article au sens SPIP créé automatiquement lors du téléversement d’un document qu’il soit audio, vidéo, image ou textuel ; un seul document ne peut être lié à un article dit "média" ; -
Encoding and processing into web-friendly formats
13 avril 2011, parMediaSPIP automatically converts uploaded files to internet-compatible formats.
Video files are encoded in MP4, Ogv and WebM (supported by HTML5) and MP4 (supported by Flash).
Audio files are encoded in MP3 and Ogg (supported by HTML5) and MP3 (supported by Flash).
Where possible, text is analyzed in order to retrieve the data needed for search engine detection, and then exported as a series of image files.
All uploaded files are stored online in their original format, so you can (...)
Sur d’autres sites (5311)
-
Still images to video for storage - But back to still images for viewing
29 mai 2012, par michaelUsing ffmpeg I can take a number of still images and turn them into a video. I would like to do this to decrease the total size of all my timelapse photos. But I would also like to extract the still images for use at a later date.
In order to use this method :
- I will need to correlate the original still image against a frame number in the video.
- And I will need to extract a thumbnail of a given frame number in a
video.But before I go down this rabbit hole, I want to know if the requirements are possible using ffmpeg, and if so any hints on how to accomplish the task.
note : The still images are timelapse from a single camera over a day, so temporal compression will be measurable compared to a stack of jpegs.
-
extracting video and data streams from MPEG2 TS over RTP in real-time
10 janvier 2024, par Tejal BarnwalI have H264 video stream and KLV meta data encapsulated inside MPEG2 TS container which are sent over an RTP over UDP from a camera.
I intend to do the following :


- 

- Extract both video and data streams from RTP
- Process video feed using opencv in a seperate thread
- process klv metadata in a seperate thread








My problem what exact arguments should I provide to ffmpeg so as to read h264 video stream and show the images frame by frame using opencv ?


With the help of some previous posts like Simultaneously map video and data streams to one subprocess pipeline in real-time, I was able to get some idea about how could I proceed to procees the stream over RTP.


I started out by using the following script :


#!/usr/bin/env python3
from asyncio import streams
from logging.handlers import QueueListener
import klvdata
import subprocess as sp
import shlex
import threading
import numpy as np
import cv2
import time
from io import BytesIO

# Video reader thread.
def video_reader(pipe):
 cols, rows = 1280, 720 # Assume we know frame size is 1280x720

 counter = 0
 while True:
 print("read image")
 raw_image = pipe.read(cols*rows*3) # Read raw video frame

 # Break the loop when length is too small
 if len(raw_image) < cols*rows*3:
 break

 if (counter % 10) == 0:
 # Show video frame evey 60 frames
 image = np.frombuffer(raw_image, np.uint8).reshape([rows, cols, 3])
 cv2.imshow('Video', image) # Show video image for testing
 cv2.waitKey(1)
 counter += 1
 print("image showed on window")
 time.sleep(0.25)



# https://github.com/paretech/klvdata/tree/master/klvdata
def bytes_to_int(value, signed=False):
 """Return integer given bytes."""
 return int.from_bytes(bytes(value), byteorder='big', signed=signed)


# Data reader thread (read KLV data).
def data_reader(pipe):
 key_length = 16 # Assume key length is 16 bytes.

 f = open('data.bin', 'wb') # For testing - store the KLV data to data.bin (binary file)

 while True:
 # https://en.wikipedia.org/wiki/KLV
 # The first few bytes are the Key, much like a key in a standard hash table data structure.
 # Keys can be 1, 2, 4, or 16 bytes in length.
 # Presumably in a separate specification document you would agree on a key length for a given application.
 key = pipe.read(key_length) # Read the key
 
 if len(key) < key_length:
 break # Break the loop when length is too small
 f.write(key) # Write data to binary file for testing

 # https://github.com/paretech/klvdata/tree/master/klvdata
 # Length field
 len_byte = pipe.read(1)

 if len(len_byte) < 1:
 break # Break the loop when length is too small
 f.write(len_byte) # Write data to binary file for testing

 byte_length = bytes_to_int(len_byte)

 # https://github.com/paretech/klvdata/tree/master/klvdata 
 if byte_length < 128:
 # BER Short Form
 length = byte_length
 ber_len_bytes = b''
 else:
 # BER Long Form
 ber_len = byte_length - 128
 ber_len_bytes = pipe.read(ber_len)

 if len(ber_len_bytes) < ber_len:
 break # Break the loop when length is too small
 f.write(ber_len_bytes) # Write ber_len_bytes to binary file for testing

 length = bytes_to_int(ber_len_bytes)

 # Read the value (length bytes)
 value = pipe.read(length)
 if len(value) < length:
 break # Break the loop when length is too small
 f.write(value) # Write data to binary file for testing

 klv_data = key + len_byte + ber_len_bytes + value # Concatenate key length and data
 klv_data_as_bytes_io = BytesIO(klv_data) # Wrap klv_data with BytesIO (before parsing)

 # Parse the KLV data
 for packet in klvdata.StreamParser(klv_data_as_bytes_io): 
 metadata = packet.MetadataList()
 for key, value in metadata.items():
 print(key, value)
 
 print("\n") # New line

# Execute FFmpeg as sub-process
# Map the video to stderr and map the data to stdout
process = sp.Popen(shlex.split('ffmpeg -hide_banner -loglevel quiet ' # Set loglevel to quiet for disabling the prints ot stderr
 '-i "rtp://192.168.0.141:11024" ' # Input video "Day Flight.mpg"
 '-map 0:v -c:v rawvideo -pix_fmt bgr24 -f:v rawvideo pipe:2 ' # rawvideo format is mapped to stderr pipe (raw video codec with bgr24 pixel format)
 '-map 0:d -c copy -copy_unknown -f:d data pipe:1 ' # Copy the data without ddecoding.
 '-report'), # Create a log file (because we can't the statuses that are usually printed to stderr).
 stdout=sp.PIPE, stderr=sp.PIPE)


# Start video reader thread (pass stderr pipe as argument).
video_thread = threading.Thread(target=video_reader, args=(process.stderr,))
video_thread.start()

# Start data reader thread (pass stdout pipe as argument).
data_thread = threading.Thread(target=data_reader, args=(process.stdout,))
data_thread.start()


# Wait for threads (and process) to finish.
video_thread.join()
data_thread.join()
process.wait()



With the above script, I was facing two issues :


- 

- The second thread resulted in an attribute error




Exception in thread Thread-2:
Traceback (most recent call last):
 File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
 self.run()
 File "/usr/lib/python3.8/threading.py", line 870, in run
 self._target(*self._args, **self._kwargs)
 File "video_data_extraction.py", line 97, in data_reader
 print(packet.MetadataList())
AttributeError: 'UnknownElement' object has no attribute 'MetadataList'



- 

- With this though I continuously able to see following output on the terminal regarding reading the images




read image
image showed on window
read image
image showed on window
read image
image showed on window
read image
image showed on window
read image
image showed on window
read image
image showed on window


The imshow windows wasnt updating properly ! It seemed stuck after a few frames.


Further diving into the lane with the help of following command, I concluded that the video stream that I am reading has H264 encoding


ffprobe -i rtp://192.168.0.141:11024 -show_streams -show_formats


Output of the above command :


ffprobe version 4.2.7-0ubuntu0.1 Copyright (c) 2007-2022 the FFmpeg developers
 built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
 configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
 libavutil 56. 31.100 / 56. 31.100
 libavcodec 58. 54.100 / 58. 54.100
 libavformat 58. 29.100 / 58. 29.100
 libavdevice 58. 8.100 / 58. 8.100
 libavfilter 7. 57.100 / 7. 57.100
 libavresample 4. 0. 0 / 4. 0. 0
 libswscale 5. 5.100 / 5. 5.100
 libswresample 3. 5.100 / 3. 5.100
 libpostproc 55. 5.100 / 55. 5.100
[rtp @ 0xaaaac81ecce0] PES packet size mismatch
 Last message repeated 62 times
[NULL @ 0xaaaac81f09b0] non-existing PPS 0 referenced
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[rtp @ 0xaaaac81ecce0] PES packet size mismatch
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[rtp @ 0xaaaac81ecce0] PES packet size mismatch
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced
 Last message repeated 1 times
[h264 @ 0xaaaac81f09b0] decode_slice_header error
[h264 @ 0xaaaac81f09b0] no frame!
[rtp @ 0xaaaac81ecce0] PES packet size mismatch
 Last message repeated 187 times
Input #0, rtp, from 'rtp://192.168.0.141:11024':
 Duration: N/A, start: 1317.040656, bitrate: N/A
 Program 1 
 Stream #0:1: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720, 25 fps, 25 tbr, 90k tbn
 Stream #0:0: Data: klv (KLVA / 0x41564C4B)
Unsupported codec with id 100356 for input stream 0
[STREAM]
index=0
codec_name=klv
codec_long_name=SMPTE 336M Key-Length-Value (KLV) metadata
profile=unknown
codec_type=data
codec_tag_string=KLVA
codec_tag=0x41564c4b
id=N/A
r_frame_rate=0/0
avg_frame_rate=0/0
time_base=1/90000
start_pts=118533659
start_time=1317.040656
duration_ts=N/A
duration=N/A
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=N/A
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=0
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
[/STREAM]
[STREAM]
index=1
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=Constrained Baseline
codec_type=video
codec_time_base=1/50
codec_tag_string=[27][0][0][0]
codec_tag=0x001b
width=1280
height=720
coded_width=1280
coded_height=720
has_b_frames=0
sample_aspect_ratio=N/A
display_aspect_ratio=N/A
pix_fmt=yuv420p
level=31
color_range=unknown
color_space=unknown
color_transfer=unknown
color_primaries=unknown
chroma_location=left
field_order=progressive
timecode=N/A
refs=1
is_avc=false
nal_length_size=0
id=N/A
r_frame_rate=25/1
avg_frame_rate=25/1
time_base=1/90000
start_pts=118533659
start_time=1317.040656
duration_ts=N/A
duration=N/A
bit_rate=N/A
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=N/A
nb_read_frames=N/A
nb_read_packets=N/A
DISPOSITION:default=0
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
[/STREAM]
[FORMAT]
filename=rtp://192.168.0.141:11024
nb_streams=2
nb_programs=1
format_name=rtp
format_long_name=RTP input
start_time=1317.040656
duration=N/A
size=N/A
bit_rate=N/A
probe_score=100
[/FORMAT]


Further, in the log output, I see a lot of statements in regard to missed packets and PES packet mismatch


[rtp @ 0xaaaaf31896c0] max delay reached. need to consume packet
[rtp @ 0xaaaaf31896c0] RTP: missed 98 packets
[rtp @ 0xaaaaf31896c0] Continuity check failed for pid 40 expected 14 got 10
[rtp @ 0xaaaaf31896c0] PES packet size mismatch
rtp://192.168.0.141:11024: corrupt input packet in stream 0
frame= 124 fps=2.6 q=-0.0 size= 334800kB time=00:00:05.32 bitrate=515406.0kbits/s dup=97 drop=0 speed=0.111x 


What arguments do I provide to ffmpeg and in what order because my stream 0 is metadata and stream 1 is video so as to display image frame by frame with opencv ?
I would be grateful for any help that you could provide.


Further, I also have a query regarding how does ffmpeg know to that it has to first convert the rtp packets into mpeg2 TS packets before segregating video stream and data stream ?


-
Capture and encode desktop with libav in real time not giving corect images
3 septembre 2022, par thoxeyAs part of a larger project I want to be able to capture and encode the desktop frame by frame in real time. I have the following test code to reproduce the issue shown in the screenshot :


#include 
#include 
#include <iostream>
#include <fstream>
#include <string>
#include 
#include 

extern "C"
{
#include "libavdevice/avdevice.h"
#include "libavutil/channel_layout.h"
#include "libavutil/mathematics.h"
#include "libavutil/opt.h"
#include "libavformat/avformat.h"
#include "libswscale/swscale.h"
}


/* 5 seconds stream duration */
#define STREAM_DURATION 5.0
#define STREAM_FRAME_RATE 25 /* 25 images/s */
#define STREAM_NB_FRAMES ((int)(STREAM_DURATION * STREAM_FRAME_RATE))
#define STREAM_PIX_FMT AV_PIX_FMT_YUV420P /* default pix_fmt */

int videoStreamIndx;
int framerate = 30;

int width = 1920;
int height = 1080;

int encPacketCounter;

AVFormatContext* ifmtCtx;
AVCodecContext* avcodecContx;
AVFormatContext* ofmtCtx;
AVStream* videoStream;
AVCodecContext* avCntxOut;
AVPacket* avPkt;
AVFrame* avFrame;
AVFrame* outFrame;
SwsContext* swsCtx;

std::ofstream fs;


AVDictionary* ConfigureScreenCapture()
{

 AVDictionary* options = NULL;
 //Try adding "-rtbufsize 100M" as in https://stackoverflow.com/questions/6766333/capture-windows-screen-with-ffmpeg
 av_dict_set(&options, "rtbufsize", "100M", 0);
 av_dict_set(&options, "framerate", std::to_string(framerate).c_str(), 0);
 char buffer[16];
 sprintf(buffer, "%dx%d", width, height);
 av_dict_set(&options, "video_size", buffer, 0);
 return options;
}

AVCodecParameters* ConfigureAvCodec()
{
 AVCodecParameters* av_codec_par_out = avcodec_parameters_alloc();
 av_codec_par_out->width = width;
 av_codec_par_out->height = height;
 av_codec_par_out->bit_rate = 40000;
 av_codec_par_out->codec_id = AV_CODEC_ID_H264; //AV_CODEC_ID_MPEG4; //Try H.264 instead of MPEG4
 av_codec_par_out->codec_type = AVMEDIA_TYPE_VIDEO;
 av_codec_par_out->format = 0;
 return av_codec_par_out;
}

int GetVideoStreamIndex()
{
 int VideoStreamIndx = -1;
 avformat_find_stream_info(ifmtCtx, NULL);
 /* find the first video stream index . Also there is an API available to do the below operations */
 for (int i = 0; i < (int)ifmtCtx->nb_streams; i++) // find video stream position/index.
 {
 if (ifmtCtx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO)
 {
 VideoStreamIndx = i;
 break;
 }
 }

 if (VideoStreamIndx == -1)
 {
 }

 return VideoStreamIndx;
}

void CreateFrames(AVCodecParameters* av_codec_par_in, AVCodecParameters* av_codec_par_out)
{

 avFrame = av_frame_alloc();
 avFrame->width = avcodecContx->width;
 avFrame->height = avcodecContx->height;
 avFrame->format = av_codec_par_in->format;
 av_frame_get_buffer(avFrame, 0);

 outFrame = av_frame_alloc();
 outFrame->width = avCntxOut->width;
 outFrame->height = avCntxOut->height;
 outFrame->format = av_codec_par_out->format;
 av_frame_get_buffer(outFrame, 0);
}

bool Init()
{
 AVCodecParameters* avCodecParOut = ConfigureAvCodec();

 AVDictionary* options = ConfigureScreenCapture();

 AVInputFormat* ifmt = av_find_input_format("gdigrab");
 auto ifmtCtxLocal = avformat_alloc_context();
 if (avformat_open_input(&ifmtCtxLocal, "desktop", ifmt, &options) < 0)
 {
 return false;
 }
 ifmtCtx = ifmtCtxLocal;

 videoStreamIndx = GetVideoStreamIndex();

 AVCodecParameters* avCodecParIn = avcodec_parameters_alloc();
 avCodecParIn = ifmtCtx->streams[videoStreamIndx]->codecpar;

 AVCodec* avCodec = avcodec_find_decoder(avCodecParIn->codec_id);
 if (avCodec == NULL)
 {
 return false;
 }

 avcodecContx = avcodec_alloc_context3(avCodec);
 if (avcodec_parameters_to_context(avcodecContx, avCodecParIn) < 0)
 {
 return false;
 }

 //av_dict_set
 int value = avcodec_open2(avcodecContx, avCodec, NULL); //Initialize the AVCodecContext to use the given AVCodec.
 if (value < 0)
 {
 return false;
 }

 AVOutputFormat* ofmt = av_guess_format("h264", NULL, NULL);

 if (ofmt == NULL)
 {
 return false;
 }

 auto ofmtCtxLocal = avformat_alloc_context();
 avformat_alloc_output_context2(&ofmtCtxLocal, ofmt, NULL, NULL);
 if (ofmtCtxLocal == NULL)
 {
 return false;
 }
 ofmtCtx = ofmtCtxLocal;

 AVCodec* avCodecOut = avcodec_find_encoder(avCodecParOut->codec_id);
 if (avCodecOut == NULL)
 {
 return false;
 }

 videoStream = avformat_new_stream(ofmtCtx, avCodecOut);
 if (videoStream == NULL)
 {
 return false;
 }

 avCntxOut = avcodec_alloc_context3(avCodecOut);
 if (avCntxOut == NULL)
 {
 return false;
 }

 if (avcodec_parameters_copy(videoStream->codecpar, avCodecParOut) < 0)
 {
 return false;
 }

 if (avcodec_parameters_to_context(avCntxOut, avCodecParOut) < 0)
 {
 return false;
 }

 avCntxOut->gop_size = 30; //3; //Use I-Frame frame every 30 frames.
 avCntxOut->max_b_frames = 0;
 avCntxOut->time_base.num = 1;
 avCntxOut->time_base.den = framerate;

 //avio_open(&ofmtCtx->pb, "", AVIO_FLAG_READ_WRITE);

 if (avformat_write_header(ofmtCtx, NULL) < 0)
 {
 return false;
 }

 value = avcodec_open2(avCntxOut, avCodecOut, NULL); //Initialize the AVCodecContext to use the given AVCodec.
 if (value < 0)
 {
 return false;
 }

 if (avcodecContx->codec_id == AV_CODEC_ID_H264)
 {
 av_opt_set(avCntxOut->priv_data, "preset", "ultrafast", 0);
 av_opt_set(avCntxOut->priv_data, "zerolatency", "1", 0);
 av_opt_set(avCntxOut->priv_data, "tune", "ull", 0);
 }

 if ((ofmtCtx->oformat->flags & AVFMT_GLOBALHEADER) != 0)
 {
 avCntxOut->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
 }

 CreateFrames(avCodecParIn, avCodecParOut);

 swsCtx = sws_alloc_context();
 if (sws_init_context(swsCtx, NULL, NULL) < 0)
 {
 return false;
 }

 swsCtx = sws_getContext(avcodecContx->width, avcodecContx->height, avcodecContx->pix_fmt,
 avCntxOut->width, avCntxOut->height, avCntxOut->pix_fmt, SWS_FAST_BILINEAR,
 NULL, NULL, NULL);
 if (swsCtx == NULL)
 {
 return false;
 }

 return true;
}

void Encode(AVCodecContext* enc_ctx, AVFrame* frame, AVPacket* pkt)
{
 int ret;

 /* send the frame to the encoder */
 ret = avcodec_send_frame(enc_ctx, frame);
 if (ret < 0)
 {
 return;
 }

 while (ret >= 0)
 {
 ret = avcodec_receive_packet(enc_ctx, pkt);
 if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
 return;
 if (ret < 0)
 {
 return;
 }

 fs.write((char*)pkt->data, pkt->size);
 av_packet_unref(pkt);
 }
}

void EncodeFrames(int noFrames)
{
 int frameCount = 0;
 avPkt = av_packet_alloc();
 AVPacket* outPacket = av_packet_alloc();
 encPacketCounter = 0;

 while (av_read_frame(ifmtCtx, avPkt) >= 0)
 {
 if (frameCount++ == noFrames)
 break;
 if (avPkt->stream_index != videoStreamIndx) continue;

 avcodec_send_packet(avcodecContx, avPkt);

 if (avcodec_receive_frame(avcodecContx, avFrame) >= 0) // Frame successfully decoded :)
 {
 outPacket->data = NULL; // packet data will be allocated by the encoder
 outPacket->size = 0;

 outPacket->pts = av_rescale_q(encPacketCounter, avCntxOut->time_base, videoStream->time_base);
 if (outPacket->dts != AV_NOPTS_VALUE)
 outPacket->dts = av_rescale_q(encPacketCounter, avCntxOut->time_base, videoStream->time_base);

 outPacket->dts = av_rescale_q(encPacketCounter, avCntxOut->time_base, videoStream->time_base);
 outPacket->duration = av_rescale_q(1, avCntxOut->time_base, videoStream->time_base);

 outFrame->pts = av_rescale_q(encPacketCounter, avCntxOut->time_base, videoStream->time_base);
 outFrame->pkt_duration = av_rescale_q(encPacketCounter, avCntxOut->time_base, videoStream->time_base);
 encPacketCounter++;

 int sts = sws_scale(swsCtx,
 avFrame->data, avFrame->linesize, 0, avFrame->height,
 outFrame->data, outFrame->linesize);

 /* make sure the frame data is writable */
 auto ret = av_frame_make_writable(outFrame);
 if (ret < 0)
 break;
 Encode(avCntxOut, outFrame, outPacket);
 }
 av_frame_unref(avFrame);
 av_packet_unref(avPkt);
 }
}

void Dispose()
{
 fs.close();

 auto ifmtCtxLocal = ifmtCtx;
 avformat_close_input(&ifmtCtx);
 avformat_free_context(ifmtCtx);
 avcodec_free_context(&avcodecContx);

}

int main(int argc, char** argv)
{
 avdevice_register_all();

 fs.open("out.h264");

 if (Init())
 {
 EncodeFrames(300);
 }
 else
 {
 std::cout << "Failed to Init \n";
 } 

 Dispose();

 return 0;
}
</string></fstream></iostream>

As far as I can tell the setup of the encoding process is correct as it is largely unchanged from how the example given in the official documentation is working : https://libav.org/documentation/doxygen/master/encode__video_8c_source.html


However there is limited documentation around the desktop capture online so I am not sure if I have set that up correctly.




Syndiquer tout le site
