Recherche avancée

Médias (1)

Mot : - Tags -/Christian Nold

Autres articles (62)

  • Les formats acceptés

    28 janvier 2010, par

    Les commandes suivantes permettent d’avoir des informations sur les formats et codecs gérés par l’installation local de ffmpeg :
    ffmpeg -codecs ffmpeg -formats
    Les format videos acceptés en entrée
    Cette liste est non exhaustive, elle met en exergue les principaux formats utilisés : h264 : H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 m4v : raw MPEG-4 video format flv : Flash Video (FLV) / Sorenson Spark / Sorenson H.263 Theora wmv :
    Les formats vidéos de sortie possibles
    Dans un premier temps on (...)

  • Ajouter notes et légendes aux images

    7 février 2011, par

    Pour pouvoir ajouter notes et légendes aux images, la première étape est d’installer le plugin "Légendes".
    Une fois le plugin activé, vous pouvez le configurer dans l’espace de configuration afin de modifier les droits de création / modification et de suppression des notes. Par défaut seuls les administrateurs du site peuvent ajouter des notes aux images.
    Modification lors de l’ajout d’un média
    Lors de l’ajout d’un média de type "image" un nouveau bouton apparait au dessus de la prévisualisation (...)

  • Demande de création d’un canal

    12 mars 2010, par

    En fonction de la configuration de la plateforme, l’utilisateur peu avoir à sa disposition deux méthodes différentes de demande de création de canal. La première est au moment de son inscription, la seconde, après son inscription en remplissant un formulaire de demande.
    Les deux manières demandent les mêmes choses fonctionnent à peu près de la même manière, le futur utilisateur doit remplir une série de champ de formulaire permettant tout d’abord aux administrateurs d’avoir des informations quant à (...)

Sur d’autres sites (9742)

  • Getting and decoding video by RTP H264

    29 novembre 2023, par AlekseiKraev

    Parsing RTP H264.
WINAPI C. A queue from C++ has been applied, I repent.

    


    RTP is generated using FFMPEG with the following command :
ffmpeg.exe -f gdigrab -framerate 25 -i desktop -s 853x480 -b:v 120000 -c:v libx264 -f rtp rtp ://127.0.0.1:8080

    


    Parsing the incoming RTP/h264 stream and converting it to an RGB matrix.

    


    video_H264_decode.h

    


    #pragma once
#ifndef _VIDEO_H264_DECODE_SEND_H // Блокируем повторное включение этого модуля
#define _VIDEO_H264_DECODE_SEND_H
//******************************************************************************
// Section include
//******************************************************************************
#include "main.h"
#include 
//******************************************************************************
// Constants
//******************************************************************************

//******************************************************************************
// Type
//******************************************************************************
typedef struct {
    unsigned char* data;
    int size;
}RTPData_DType;

typedef struct 
{
    union
    {
        struct
        {
            char V:2;               //Версия
            char P:1;               //заполнение
            char X:1;               //расширение
            char CC:4;              //количество CSRC

            char M:1;               //маркер (флаг последнего пакета AU),
            char PT:7;              //полезная нагрузка (тип данных носителя полезной нагрузки RTP, H264 = 96)

            short sequence_number;  //Порядковый номер: порядковый номер пакета RTP, увеличенный на 1.
            int time_stamp;         //временная метка выборки медиа. 
            int SSRC;               //Пакет данных имеет одинаковое происхождение.
        };
        unsigned char data[12];
    };
}RTPHeader_DType;
//******************************************************************************
// Global var
//******************************************************************************

//******************************************************************************
// Local function prototype
//******************************************************************************
UCHAR rtp_H264_recive_init(void);
UCHAR RTPStop(void);
//******************************************************************************
// Macros
//******************************************************************************
#define BYTE2_SWAP(X)   ((((short)(X) & 0xff00) >> 8) |(((short)(X) & 0x00ff) << 8))
#endif
//******************************************************************************
// ENF OF FILE
//******************************************************************************


    


    video_H264_decode.c

    


    //******************************************************************************&#xA;//include&#xA;//******************************************************************************&#xA;#include "main.h"&#xA;/*#include "video_H264_decode.h"&#xA;#include &#xA;#include &#xA;#include <chrono>&#xA;#include &#xA;&#xA;#pragma comment(lib, "ws2_32.lib")&#xA;#include */&#xA;#include <iostream>&#xA;#include <queue>&#xA;&#xA;extern "C" {&#xA;#include "libavformat/avformat.h"&#xA;#include "libavfilter/avfilter.h"&#xA;#include "libavdevice/avdevice.h"&#xA;#include "libswscale/swscale.h"&#xA;#include "libswresample/swresample.h"&#xA;#include "libpostproc/postprocess.h"&#xA;#include "libavcodec/avcodec.h"&#xA;}&#xA;&#xA;#pragma comment(lib,"avcodec.lib")&#xA;#pragma comment(lib,"avdevice.lib")&#xA;#pragma comment(lib,"avfilter.lib")&#xA;#pragma comment(lib,"avformat.lib")&#xA;#pragma comment(lib,"avutil.lib")&#xA;#pragma comment(lib,"postproc.lib")&#xA;#pragma comment(lib,"swresample.lib")&#xA;#pragma comment(lib,"swscale.lib")&#xA;&#xA;#pragma warning(disable: 4996)&#xA;//******************************************************************************&#xA;// Section for determining the variables used in the module&#xA;//******************************************************************************&#xA;//------------------------------------------------------------------------------&#xA;// Global&#xA;//------------------------------------------------------------------------------&#xA;&#xA;//------------------------------------------------------------------------------&#xA;// Local&#xA;//------------------------------------------------------------------------------&#xA;const int inteval = 0x01000000;&#xA;BOOL FlagRTPActive = TRUE;&#xA;&#xA;HANDLE hMutexRTPRecive;&#xA;HANDLE hSemaphoreRTP;&#xA;HANDLE hTreadRTPRecive;&#xA;HANDLE hTreadRTPDecode;&#xA;&#xA;&#xA;SOCKET RTPSocket; //socket UDP RTP&#xA;RTPData_DType packet;&#xA;RTPData_DType FU_buffer = { 0 };&#xA;&#xA;std::queue q;&#xA;std::set<int> seq;&#xA;&#xA;AVFormatContext* pAVFormatContext;&#xA;AVCodecContext* pAVCodecContext;&#xA;const AVCodec* pAVCodec;&#xA;AVFrame* pAVFrame;&#xA;AVFrame* AVFrameRGG;&#xA;SwsContext* pSwsContext;&#xA;AVPacket *pAVPacket;&#xA;AVCodecParserContext* pAVCodecParserContext;&#xA;&#xA;UINT port;&#xA;//******************************************************************************&#xA;// Section of prototypes of local functions&#xA;//******************************************************************************&#xA;DWORD WINAPI rtp_H264_recive_Procedure(CONST LPVOID lpParam);&#xA;DWORD WINAPI rtp_decode_Procedure(CONST LPVOID lpParam);&#xA;char RTPSocketInit(void);&#xA;void RTPPacketParser(void);&#xA;char RTPSocketRecive(void);&#xA;void Decode_NaluToAVFrameRGG();&#xA;//******************************************************************************&#xA;// Section of the description of functions&#xA;//******************************************************************************&#xA;UCHAR rtp_H264_recive_init(void)&#xA;{&#xA;    hSemaphoreRTP = CreateSemaphore(&#xA;        NULL,           // default security attributes&#xA;        0,              // initial count&#xA;        1,              // maximum count&#xA;        NULL);          // unnamed semaphore&#xA;&#xA;    hMutexRTPRecive = CreateMutex(&#xA;        NULL,              // default security attributes&#xA;        FALSE,             // initially not owned&#xA;        NULL);             // unnamed mutex&#xA;&#xA;    hTreadRTPRecive = CreateThread(NULL, NULL, rtp_H264_recive_Procedure, NULL, NULL, NULL);&#xA;    hTreadRTPDecode = CreateThread(NULL, NULL, rtp_decode_Procedure, NULL, NULL, NULL);&#xA;&#xA;    return 0;&#xA;}&#xA;//------------------------------------------------------------------------------&#xA;UCHAR RTPStop(void)&#xA;{&#xA;    FlagRTPActive = FALSE;&#xA;&#xA;    if (hSemaphoreRTP) CloseHandle(hSemaphoreRTP);&#xA;    if (hMutexRTPRecive) CloseHandle(hMutexRTPRecive);&#xA;    if (hTreadRTPRecive) CloseHandle(hTreadRTPRecive);&#xA;&#xA;    closesocket(RTPSocket);  &#xA;&#xA;    return 0;&#xA;}&#xA;//------------------------------------------------------------------------------&#xA;DWORD WINAPI rtp_H264_recive_Procedure(CONST LPVOID lpParam)&#xA;{&#xA;    while (RTPSocketInit() == 0)&#xA;        Sleep(2000);&#xA;       &#xA;    while (1)&#xA;    {&#xA;        RTPSocketRecive();&#xA;        RTPPacketParser();&#xA;        ReleaseSemaphore(hSemaphoreRTP, 1, NULL);&#xA;    }&#xA;}&#xA;//------------------------------------------------------------------------------&#xA;DWORD WINAPI rtp_decode_Procedure(CONST LPVOID lpParam)&#xA;{&#xA;    port = param.Option.VideoPort;&#xA;&#xA;    pAVPacket = av_packet_alloc();&#xA;    if (!pAVPacket)&#xA;    {&#xA;        MessageBox(NULL, L"ERROR Could not allocate pAVPacket", L"Init decoder error", MB_OK | MB_ICONERROR);&#xA;        exit(1);&#xA;    }&#xA;    av_init_packet(pAVPacket);&#xA;&#xA;    /* find the MPEG-1 video decoder */&#xA;    pAVCodec = avcodec_find_decoder(AV_CODEC_ID_H264);&#xA;    if (!pAVCodec)&#xA;    {&#xA;        MessageBox(NULL, L"ERROR Codec not found", L"Init decoder error", MB_OK | MB_ICONERROR);&#xA;        exit(1);&#xA;    }&#xA;&#xA;    pAVCodecParserContext = av_parser_init(pAVCodec->id);&#xA;    if (!pAVCodecParserContext)&#xA;    {&#xA;        MessageBox(NULL, L"ERROR Parser not found", L"Init decoder error", MB_OK | MB_ICONERROR);&#xA;        exit(1);&#xA;    }&#xA;&#xA;    pAVCodecContext = avcodec_alloc_context3(pAVCodec);&#xA;    if (!pAVCodecContext) &#xA;    {&#xA;        MessageBox(NULL, L"ERROR Could not allocate video codec context", L"Init decoder error", MB_OK | MB_ICONERROR);&#xA;        exit(1);&#xA;    }&#xA;               &#xA;    if (avcodec_open2(pAVCodecContext, pAVCodec, NULL) &lt; 0)&#xA;    {&#xA;        MessageBox(NULL, L"ERROR Could not open codec", L"Init decoder error", MB_OK | MB_ICONERROR);&#xA;        exit(1);&#xA;    }&#xA;&#xA;    pAVFrame = av_frame_alloc();&#xA;    if (!pAVFrame) &#xA;    {&#xA;        MessageBox(NULL, L"ERROR Could not allocate video frame", L"Init decoder error", MB_OK | MB_ICONERROR);&#xA;        exit(1);&#xA;    }    &#xA;    &#xA;    while (FlagRTPActive)&#xA;    {&#xA;        if(port != param.Option.VideoPort)&#xA;            closesocket(RTPSocket);&#xA;&#xA;        WaitForSingleObject(hSemaphoreRTP, 500);        &#xA;        Decode_NaluToAVFrameRGG();&#xA;    }&#xA;&#xA;    avformat_free_context(pAVFormatContext);&#xA;    av_frame_free(&amp;pAVFrame);&#xA;    avcodec_close(pAVCodecContext);&#xA;    av_packet_free(&amp;pAVPacket);&#xA;&#xA;    &#xA;    if (hTreadRTPDecode) CloseHandle(hTreadRTPDecode);&#xA;&#xA;    return 0;&#xA;}&#xA;&#xA;//------------------------------------------------------------------------------&#xA;char RTPSocketInit(void)&#xA;{    &#xA;    sockaddr_in RTPSocketAddr;&#xA;    &#xA;    RTPSocketAddr.sin_family = AF_INET;&#xA;    RTPSocketAddr.sin_addr.s_addr = htonl(INADDR_ANY);&#xA;    RTPSocketAddr.sin_port = htons(param.Option.VideoPort);&#xA;&#xA;    RTPSocket = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);&#xA;    if (RTPSocket == INVALID_SOCKET)&#xA;    {&#xA;        MessageBox(NULL, L"ERROR Invalid RTP Socket", L"UDP Socket", MB_OK | MB_ICONERROR);&#xA;        return 0;&#xA;    }&#xA;&#xA;    int option = 12000;&#xA;    if (setsockopt(RTPSocket, SOL_SOCKET, SO_RCVBUF, (char*)&amp;option, sizeof(option)) &lt; 0)&#xA;    {&#xA;        printf("setsockopt failed\n");&#xA;&#xA;    }&#xA;&#xA;    /*option = TRUE;&#xA;    if (setsockopt(RTPSocket, SOL_SOCKET, SO_CONDITIONAL_ACCEPT, (char*)&amp;option, sizeof(option)) &lt; 0)&#xA;    {&#xA;        printf("setsockopt failed\n");&#xA;&#xA;    }*/&#xA;&#xA;    if (bind(RTPSocket, (sockaddr*)&amp;RTPSocketAddr, sizeof(RTPSocketAddr)) == SOCKET_ERROR)&#xA;    {&#xA;        MessageBox(NULL, L"ERROR bind", L"UDP Socket", MB_OK | MB_ICONERROR);&#xA;        closesocket(RTPSocket);&#xA;        return 0;&#xA;    }&#xA;    return 1;&#xA;}&#xA;&#xA;//------------------------------------------------------------------------------&#xA;char RTPSocketRecive(void)&#xA;{&#xA;    static char RecvBuf[2000];&#xA;    static int BufLen = 2000;&#xA;&#xA;    int iResult = 0;&#xA;    struct sockaddr_in SenderAddr;&#xA;    int SenderAddrSize = sizeof(SenderAddr);&#xA;    TCHAR szErrorMsg[100];&#xA;&#xA;    iResult = recvfrom(RTPSocket, RecvBuf, BufLen, 0, NULL, NULL);&#xA;    if (iResult == SOCKET_ERROR &amp;&amp; FlagRTPActive == TRUE)&#xA;    {&#xA;        StringCchPrintf(szErrorMsg, 100, L"ERROR recvfrom Ошибка:%d", WSAGetLastError());&#xA;        MessageBox(NULL, szErrorMsg, L"UDP Socket", MB_OK | MB_ICONERROR);&#xA;        return -1;&#xA;    }&#xA;&#xA;    packet.data = (unsigned char*)RecvBuf;&#xA;    packet.size = iResult;&#xA;&#xA;    return 1;&#xA;}&#xA;&#xA;//------------------------------------------------------------------------------&#xA;void RTPPacketParser(void)&#xA;{&#xA;    RTPHeader_DType RTPHeader;&#xA;    RTPData_DType NALU;&#xA;    unsigned char* buffer = packet.data;&#xA;    int pos = 0;&#xA;    static int count = 0;&#xA;    static short lastSequenceNumber = 0xFFFF;&#xA;    short type;&#xA;    char payload_header;&#xA;&#xA;    //read rtp header&#xA;    memcpy(&amp;RTPHeader, buffer, sizeof(RTPHeader));&#xA;    RTPHeader.sequence_number = BYTE2_SWAP(RTPHeader.sequence_number);&#xA;    pos &#x2B;= 12;    &#xA;  &#xA;    if (RTPHeader.X) {&#xA;        //profile extension&#xA;        short define;&#xA;        short length;&#xA;        length = buffer[pos &#x2B; 3];//suppose not so long extension&#xA;        pos &#x2B;= 4;&#xA;        pos &#x2B;= (length * 4);&#xA;    }&#xA;&#xA;    payload_header = buffer[pos];&#xA;    type = payload_header &amp; 0x1f; //Тип полезной нагрузки RTP&#xA;    pos&#x2B;&#x2B;;&#xA;    &#xA;    //STAP-A&#xA;    if (type == 24)&#xA;    {        &#xA;        while (pos &lt; packet.size)&#xA;        {&#xA;            unsigned short NALU_size;&#xA;            memcpy(&amp;NALU_size, buffer &#x2B; pos, 2);&#xA;            NALU_size = BYTE2_SWAP(NALU_size);&#xA;            pos &#x2B;= 2;&#xA;            char NAL_header = buffer[pos];&#xA;            short NAL_type = NAL_header &amp; 0x1f;&#xA;&#xA;            if (NAL_type == 7) &#xA;            {&#xA;                count&#x2B;&#x2B;;&#xA;                //cout&lt;&lt;"SPS, sequence number: "&lt;/cout&lt;&lt;"PPS, sequence number: "&lt;/cout&lt;&lt;"end of sequence, sequence number: "&lt; 0) &#xA;            {&#xA;                NALU.data = (unsigned char*) malloc(NALU_size &#x2B; 4);&#xA;                NALU.size = NALU_size &#x2B; 4;&#xA;                memcpy(NALU.data, &amp;inteval, 4);&#xA;                memcpy(NALU.data &#x2B; 4, &amp;buffer[pos], NALU_size);&#xA;&#xA;                WaitForSingleObject(hMutexRTPRecive, INFINITE);&#xA;                q.push(NALU);&#xA;                ReleaseMutex(hMutexRTPRecive);&#xA;            }&#xA;&#xA;            pos &#x2B;= NALU_size;&#xA;        }&#xA;    }&#xA;    //FU-A Fragmentation unit&#xA;    else if (type == 28)&#xA;    {        &#xA;        //FU header&#xA;        char FU_header = buffer[pos];&#xA;        bool fStart = FU_header &amp; 0x80;&#xA;        bool fEnd = FU_header &amp; 0x40;&#xA;&#xA;        //NAL header&#xA;        char NAL_header = (payload_header &amp; 0xe0) | (FU_header &amp; 0x1f);&#xA;        short NAL_type = FU_header &amp; 0x1f;&#xA;        if (NAL_type == 7) &#xA;        {&#xA;            count&#x2B;&#x2B;;&#xA;            //SPS&#xA;        }&#xA;        else if (NAL_type == 8) &#xA;        {&#xA;            //PPS&#xA;        }&#xA;        else if (NAL_type == 10) &#xA;        {&#xA;            //end of sequence&#xA;        }&#xA;        pos&#x2B;&#x2B;;&#xA;&#xA;        int size = packet.size - pos;&#xA;        &#xA;        if (count > 0)&#xA;        {&#xA;            if (fStart) &#xA;            {&#xA;                if (FU_buffer.size != 0)&#xA;                {&#xA;                    free(FU_buffer.data);&#xA;                    FU_buffer.size = 0;&#xA;                }&#xA;                FU_buffer.data = (unsigned char*)malloc(size &#x2B; 5);&#xA;                if (FU_buffer.data == NULL)&#xA;                    return;&#xA;                FU_buffer.size = size &#x2B; 5;&#xA;                memcpy(FU_buffer.data, &amp;inteval, 4);&#xA;                memcpy(FU_buffer.data &#x2B; 4, &amp;NAL_header, 1);&#xA;                memcpy(FU_buffer.data &#x2B; 5, buffer &#x2B; pos, size);&#xA;            }&#xA;            else&#xA;            {&#xA;                unsigned char* temp = (unsigned char*)malloc(FU_buffer.size &#x2B; size);&#xA;                memcpy(temp, FU_buffer.data, FU_buffer.size);&#xA;                memcpy(temp &#x2B; FU_buffer.size, buffer &#x2B; pos, size);&#xA;                if (FU_buffer.size != 0) free(FU_buffer.data);&#xA;                FU_buffer.data = temp;&#xA;                FU_buffer.size &#x2B;= size;&#xA;            }&#xA;&#xA;            if (fEnd)&#xA;            {&#xA;                NALU.data = (unsigned char*)malloc(FU_buffer.size);&#xA;                NALU.size = FU_buffer.size;&#xA;                memcpy(NALU.data, FU_buffer.data, FU_buffer.size);&#xA;&#xA;                WaitForSingleObject(hMutexRTPRecive, INFINITE);&#xA;                q.push(NALU);&#xA;                ReleaseMutex(hMutexRTPRecive);&#xA;&#xA;                free(FU_buffer.data);&#xA;                FU_buffer.size = 0;&#xA;            }&#xA;        }&#xA;        &#xA;    }&#xA;    else &#xA;    {&#xA;        //other type&#xA;        short NAL_type = type;&#xA;        if (NAL_type == 7) &#xA;        {&#xA;            count&#x2B;&#x2B;;&#xA;            //SPS&#xA;        }&#xA;        else if (NAL_type == 8) &#xA;        {&#xA;            //PPS&#xA;        }&#xA;        else if (NAL_type == 10) &#xA;        {&#xA;            //end of sequence&#xA;        }&#xA;&#xA;        int size = packet.size - pos &#x2B; 1;&#xA;        if (count > 0)&#xA;        {&#xA;            NALU.data = (unsigned char*)malloc(size&#x2B;4);&#xA;            NALU.size = size &#x2B; 4;&#xA;            memcpy(NALU.data, &amp;inteval, 4);&#xA;            memcpy(NALU.data &#x2B; 4, &amp;buffer[12], size);&#xA;&#xA;            WaitForSingleObject(hMutexRTPRecive, INFINITE);&#xA;            q.push(NALU);&#xA;            ReleaseMutex(hMutexRTPRecive);&#xA;        }&#xA;    }&#xA;}&#xA;&#xA;//------------------------------------------------------------------------------&#xA;void Decode_NaluToAVFrameRGG()&#xA;{&#xA;    unsigned char cntNALU;&#xA;    int len = 0;&#xA;    static int size = 0;&#xA;    unsigned char* data = NULL;&#xA;    int ret;&#xA;    RTPData_DType NALU;&#xA;&#xA;    av_frame_unref(pAVFrame);&#xA;    av_packet_unref(pAVPacket);&#xA;&#xA;    while(1)&#xA;    {&#xA;        WaitForSingleObject(hMutexRTPRecive, INFINITE);&#xA;        if (q.empty())&#xA;        {&#xA;            ReleaseMutex(hMutexRTPRecive);&#xA;            break;&#xA;        }&#xA;        NALU = q.front();&#xA;        q.pop();&#xA;        ReleaseMutex(hMutexRTPRecive);&#xA;&#xA;        data = NALU.data;&#xA;        size = NALU.size;&#xA;&#xA;        while(size)&#xA;        {            &#xA;            len = av_parser_parse2(pAVCodecParserContext, pAVCodecContext, &amp;pAVPacket->data, &amp;pAVPacket->size,&#xA;                (uint8_t*)data, size,&#xA;                AV_NOPTS_VALUE, AV_NOPTS_VALUE, AV_NOPTS_VALUE);&#xA;&#xA;            data = len ? data &#x2B; len : data;&#xA;            size -= len;&#xA;            &#xA;            if (pAVPacket->size)&#xA;            {&#xA;                ret = avcodec_send_packet(pAVCodecContext, pAVPacket);&#xA;&#xA;                if(ret == AVERROR_EOF)&#xA;                    MessageBox(NULL, L"the codec has been fully flushed, and there will be no more output frames", L"avcodec_send_packet", MB_OK | MB_ICONERROR);&#xA;&#xA;                if (ret &lt; 0)&#xA;                {&#xA;                    MessageBox(NULL, L"ERROR sending a packet for decoding", L"Decode", MB_OK | MB_ICONERROR);&#xA;                    //exit(1);&#xA;                }&#xA;&#xA;                while (ret >= 0) &#xA;                {&#xA;                    ret = avcodec_receive_frame(pAVCodecContext, pAVFrame);&#xA;                    if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)&#xA;                        break;&#xA;                    else if (ret &lt; 0) {&#xA;                        MessageBox(NULL, L"ERROR during decoding", L"Decode", MB_OK | MB_ICONERROR);&#xA;                        //exit(1);&#xA;                    }&#xA;                    &#xA;                    AVFrameRGG = av_frame_alloc();                                       &#xA;                    pSwsContext = sws_getContext(pAVCodecContext->width, pAVCodecContext->height, pAVCodecContext->pix_fmt,&#xA;                        pAVCodecContext->width, pAVCodecContext->height, AV_PIX_FMT_RGB24, SWS_SPLINE,&#xA;                        0, 0, 0);&#xA;&#xA;                    sws_scale_frame(pSwsContext, AVFrameRGG, pAVFrame);&#xA;                                        &#xA;                    ImgBufferSet(AVFrameRGG->data[0], pAVCodecContext->height, pAVCodecContext->width, AVFrameRGG->linesize[0]);&#xA;&#xA;                    sws_freeContext(pSwsContext);&#xA;                    av_frame_free(&amp;AVFrameRGG);&#xA;                }&#xA;            }&#xA;        }&#xA;        free(NALU.data);&#xA;        NALU.size = 0;&#xA;    }    &#xA;}&#xA;&#xA;</int></queue></iostream></chrono>

    &#xA;

    It's DECIDED

    &#xA;

  • extracting video and data streams from MPEG2 TS over RTP in real-time

    10 janvier 2024, par Tejal Barnwal

    I have H264 video stream and KLV meta data encapsulated inside MPEG2 TS container which are sent over an RTP over UDP from a camera.&#xA;I intend to do the following :

    &#xA;

      &#xA;
    1. Extract both video and data streams from RTP
    2. &#xA;

    3. Process video feed using opencv in a seperate thread
    4. &#xA;

    5. process klv metadata in a seperate thread
    6. &#xA;

    &#xA;

    My problem what exact arguments should I provide to ffmpeg so as to read h264 video stream and show the images frame by frame using opencv ?

    &#xA;

    With the help of some previous posts like Simultaneously map video and data streams to one subprocess pipeline in real-time, I was able to get some idea about how could I proceed to procees the stream over RTP.

    &#xA;

    I started out by using the following script :

    &#xA;

    #!/usr/bin/env python3&#xA;from asyncio import streams&#xA;from logging.handlers import QueueListener&#xA;import klvdata&#xA;import subprocess as sp&#xA;import shlex&#xA;import threading&#xA;import numpy as np&#xA;import cv2&#xA;import time&#xA;from io import BytesIO&#xA;&#xA;# Video reader thread.&#xA;def video_reader(pipe):&#xA;    cols, rows = 1280, 720  # Assume we know frame size is 1280x720&#xA;&#xA;    counter = 0&#xA;    while True:&#xA;        print("read image")&#xA;        raw_image = pipe.read(cols*rows*3)  # Read raw video frame&#xA;&#xA;        # Break the loop when length is too small&#xA;        if len(raw_image) &lt; cols*rows*3:&#xA;            break&#xA;&#xA;        if (counter % 10) == 0:&#xA;            # Show video frame evey 60 frames&#xA;            image = np.frombuffer(raw_image, np.uint8).reshape([rows, cols, 3])&#xA;            cv2.imshow(&#x27;Video&#x27;, image) # Show video image for testing&#xA;            cv2.waitKey(1)&#xA;        counter &#x2B;= 1&#xA;        print("image showed on window")&#xA;        time.sleep(0.25)&#xA;&#xA;&#xA;&#xA;# https://github.com/paretech/klvdata/tree/master/klvdata&#xA;def bytes_to_int(value, signed=False):&#xA;    """Return integer given bytes."""&#xA;    return int.from_bytes(bytes(value), byteorder=&#x27;big&#x27;, signed=signed)&#xA;&#xA;&#xA;# Data reader thread (read KLV data).&#xA;def data_reader(pipe):&#xA;    key_length = 16  # Assume key length is 16 bytes.&#xA;&#xA;    f = open(&#x27;data.bin&#x27;, &#x27;wb&#x27;)  # For testing - store the KLV data to data.bin (binary file)&#xA;&#xA;    while True:&#xA;        # https://en.wikipedia.org/wiki/KLV&#xA;        # The first few bytes are the Key, much like a key in a standard hash table data structure.&#xA;        # Keys can be 1, 2, 4, or 16 bytes in length.&#xA;        # Presumably in a separate specification document you would agree on a key length for a given application.&#xA;        key = pipe.read(key_length)  # Read the key&#xA;        &#xA;        if len(key) &lt; key_length:&#xA;            break  # Break the loop when length is too small&#xA;        f.write(key)  # Write data to binary file for testing&#xA;&#xA;        # https://github.com/paretech/klvdata/tree/master/klvdata&#xA;        # Length field&#xA;        len_byte = pipe.read(1)&#xA;&#xA;        if len(len_byte) &lt; 1:&#xA;            break  # Break the loop when length is too small&#xA;        f.write(len_byte)  # Write data to binary file for testing&#xA;&#xA;        byte_length = bytes_to_int(len_byte)&#xA;&#xA;        # https://github.com/paretech/klvdata/tree/master/klvdata                                                &#xA;        if byte_length &lt; 128:&#xA;            # BER Short Form&#xA;            length = byte_length&#xA;            ber_len_bytes = b&#x27;&#x27;&#xA;        else:&#xA;            # BER Long Form&#xA;            ber_len = byte_length - 128&#xA;            ber_len_bytes = pipe.read(ber_len)&#xA;&#xA;            if len(ber_len_bytes) &lt; ber_len:&#xA;                break  # Break the loop when length is too small&#xA;            f.write(ber_len_bytes)  # Write ber_len_bytes to binary file for testing&#xA;&#xA;            length = bytes_to_int(ber_len_bytes)&#xA;&#xA;        # Read the value (length bytes)&#xA;        value = pipe.read(length)&#xA;        if len(value) &lt; length:&#xA;            break  # Break the loop when length is too small&#xA;        f.write(value)  # Write data to binary file for testing&#xA;&#xA;        klv_data = key &#x2B; len_byte &#x2B; ber_len_bytes &#x2B; value  # Concatenate key length and data&#xA;        klv_data_as_bytes_io = BytesIO(klv_data)  # Wrap klv_data with BytesIO (before parsing)&#xA;&#xA;        # Parse the KLV data&#xA;        for packet in klvdata.StreamParser(klv_data_as_bytes_io): &#xA;            metadata = packet.MetadataList()&#xA;            for key, value in metadata.items():&#xA;                print(key, value)&#xA;                &#xA;            print("\n") # New line&#xA;&#xA;# Execute FFmpeg as sub-process&#xA;# Map the video to stderr and map the data to stdout&#xA;process = sp.Popen(shlex.split(&#x27;ffmpeg -hide_banner -loglevel quiet &#x27;                        # Set loglevel to quiet for disabling the prints ot stderr&#xA;                               &#x27;-i "rtp://192.168.0.141:11024" &#x27;                                        # Input video "Day Flight.mpg"&#xA;                               &#x27;-map 0:v -c:v rawvideo -pix_fmt bgr24 -f:v rawvideo pipe:2 &#x27; # rawvideo format is mapped to stderr pipe (raw video codec with bgr24 pixel format)&#xA;                               &#x27;-map 0:d -c copy -copy_unknown -f:d data pipe:1 &#x27;            # Copy the data without ddecoding.&#xA;                               &#x27;-report&#x27;),                                                   # Create a log file (because we can&#x27;t the statuses that are usually printed to stderr).&#xA;                                stdout=sp.PIPE, stderr=sp.PIPE)&#xA;&#xA;&#xA;# Start video reader thread (pass stderr pipe as argument).&#xA;video_thread = threading.Thread(target=video_reader, args=(process.stderr,))&#xA;video_thread.start()&#xA;&#xA;# Start data reader thread (pass stdout pipe as argument).&#xA;data_thread = threading.Thread(target=data_reader, args=(process.stdout,))&#xA;data_thread.start()&#xA;&#xA;&#xA;# Wait for threads (and process) to finish.&#xA;video_thread.join()&#xA;data_thread.join()&#xA;process.wait()&#xA;&#xA;

    &#xA;

    With the above script, I was facing two issues :

    &#xA;

      &#xA;
    1. The second thread resulted in an attribute error
    2. &#xA;

    &#xA;

    Exception in thread Thread-2:&#xA;Traceback (most recent call last):&#xA;  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner&#xA;    self.run()&#xA;  File "/usr/lib/python3.8/threading.py", line 870, in run&#xA;    self._target(*self._args, **self._kwargs)&#xA;  File "video_data_extraction.py", line 97, in data_reader&#xA;    print(packet.MetadataList())&#xA;AttributeError: &#x27;UnknownElement&#x27; object has no attribute &#x27;MetadataList&#x27;&#xA;&#xA;

    &#xA;

      &#xA;
    1. With this though I continuously able to see following output on the terminal regarding reading the images
    2. &#xA;

    &#xA;

    read image&#xA;image showed on window&#xA;read image&#xA;image showed on window&#xA;read image&#xA;image showed on window&#xA;read image&#xA;image showed on window&#xA;read image&#xA;image showed on window&#xA;read image&#xA;image showed on window&#xA;

    &#xA;

    The imshow windows wasnt updating properly ! It seemed stuck after a few frames.

    &#xA;

    Further diving into the lane with the help of following command, I concluded that the video stream that I am reading has H264 encoding

    &#xA;

    ffprobe -i rtp://192.168.0.141:11024 -show_streams -show_formats&#xA;

    &#xA;

    Output of the above command :

    &#xA;

    ffprobe version 4.2.7-0ubuntu0.1 Copyright (c) 2007-2022 the FFmpeg developers&#xA;  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)&#xA;  configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared&#xA;  libavutil      56. 31.100 / 56. 31.100&#xA;  libavcodec     58. 54.100 / 58. 54.100&#xA;  libavformat    58. 29.100 / 58. 29.100&#xA;  libavdevice    58.  8.100 / 58.  8.100&#xA;  libavfilter     7. 57.100 /  7. 57.100&#xA;  libavresample   4.  0.  0 /  4.  0.  0&#xA;  libswscale      5.  5.100 /  5.  5.100&#xA;  libswresample   3.  5.100 /  3.  5.100&#xA;  libpostproc    55.  5.100 / 55.  5.100&#xA;[rtp @ 0xaaaac81ecce0] PES packet size mismatch&#xA;    Last message repeated 62 times&#xA;[NULL @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[rtp @ 0xaaaac81ecce0] PES packet size mismatch&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[rtp @ 0xaaaac81ecce0] PES packet size mismatch&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] non-existing PPS 0 referenced&#xA;    Last message repeated 1 times&#xA;[h264 @ 0xaaaac81f09b0] decode_slice_header error&#xA;[h264 @ 0xaaaac81f09b0] no frame!&#xA;[rtp @ 0xaaaac81ecce0] PES packet size mismatch&#xA;    Last message repeated 187 times&#xA;Input #0, rtp, from &#x27;rtp://192.168.0.141:11024&#x27;:&#xA;  Duration: N/A, start: 1317.040656, bitrate: N/A&#xA;  Program 1 &#xA;    Stream #0:1: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720, 25 fps, 25 tbr, 90k tbn&#xA;    Stream #0:0: Data: klv (KLVA / 0x41564C4B)&#xA;Unsupported codec with id 100356 for input stream 0&#xA;[STREAM]&#xA;index=0&#xA;codec_name=klv&#xA;codec_long_name=SMPTE 336M Key-Length-Value (KLV) metadata&#xA;profile=unknown&#xA;codec_type=data&#xA;codec_tag_string=KLVA&#xA;codec_tag=0x41564c4b&#xA;id=N/A&#xA;r_frame_rate=0/0&#xA;avg_frame_rate=0/0&#xA;time_base=1/90000&#xA;start_pts=118533659&#xA;start_time=1317.040656&#xA;duration_ts=N/A&#xA;duration=N/A&#xA;bit_rate=N/A&#xA;max_bit_rate=N/A&#xA;bits_per_raw_sample=N/A&#xA;nb_frames=N/A&#xA;nb_read_frames=N/A&#xA;nb_read_packets=N/A&#xA;DISPOSITION:default=0&#xA;DISPOSITION:dub=0&#xA;DISPOSITION:original=0&#xA;DISPOSITION:comment=0&#xA;DISPOSITION:lyrics=0&#xA;DISPOSITION:karaoke=0&#xA;DISPOSITION:forced=0&#xA;DISPOSITION:hearing_impaired=0&#xA;DISPOSITION:visual_impaired=0&#xA;DISPOSITION:clean_effects=0&#xA;DISPOSITION:attached_pic=0&#xA;DISPOSITION:timed_thumbnails=0&#xA;[/STREAM]&#xA;[STREAM]&#xA;index=1&#xA;codec_name=h264&#xA;codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10&#xA;profile=Constrained Baseline&#xA;codec_type=video&#xA;codec_time_base=1/50&#xA;codec_tag_string=[27][0][0][0]&#xA;codec_tag=0x001b&#xA;width=1280&#xA;height=720&#xA;coded_width=1280&#xA;coded_height=720&#xA;has_b_frames=0&#xA;sample_aspect_ratio=N/A&#xA;display_aspect_ratio=N/A&#xA;pix_fmt=yuv420p&#xA;level=31&#xA;color_range=unknown&#xA;color_space=unknown&#xA;color_transfer=unknown&#xA;color_primaries=unknown&#xA;chroma_location=left&#xA;field_order=progressive&#xA;timecode=N/A&#xA;refs=1&#xA;is_avc=false&#xA;nal_length_size=0&#xA;id=N/A&#xA;r_frame_rate=25/1&#xA;avg_frame_rate=25/1&#xA;time_base=1/90000&#xA;start_pts=118533659&#xA;start_time=1317.040656&#xA;duration_ts=N/A&#xA;duration=N/A&#xA;bit_rate=N/A&#xA;max_bit_rate=N/A&#xA;bits_per_raw_sample=8&#xA;nb_frames=N/A&#xA;nb_read_frames=N/A&#xA;nb_read_packets=N/A&#xA;DISPOSITION:default=0&#xA;DISPOSITION:dub=0&#xA;DISPOSITION:original=0&#xA;DISPOSITION:comment=0&#xA;DISPOSITION:lyrics=0&#xA;DISPOSITION:karaoke=0&#xA;DISPOSITION:forced=0&#xA;DISPOSITION:hearing_impaired=0&#xA;DISPOSITION:visual_impaired=0&#xA;DISPOSITION:clean_effects=0&#xA;DISPOSITION:attached_pic=0&#xA;DISPOSITION:timed_thumbnails=0&#xA;[/STREAM]&#xA;[FORMAT]&#xA;filename=rtp://192.168.0.141:11024&#xA;nb_streams=2&#xA;nb_programs=1&#xA;format_name=rtp&#xA;format_long_name=RTP input&#xA;start_time=1317.040656&#xA;duration=N/A&#xA;size=N/A&#xA;bit_rate=N/A&#xA;probe_score=100&#xA;[/FORMAT]&#xA;

    &#xA;

    Further, in the log output, I see a lot of statements in regard to missed packets and PES packet mismatch

    &#xA;

    [rtp @ 0xaaaaf31896c0] max delay reached. need to consume packet&#xA;[rtp @ 0xaaaaf31896c0] RTP: missed 98 packets&#xA;[rtp @ 0xaaaaf31896c0] Continuity check failed for pid 40 expected 14 got 10&#xA;[rtp @ 0xaaaaf31896c0] PES packet size mismatch&#xA;rtp://192.168.0.141:11024: corrupt input packet in stream 0&#xA;frame=  124 fps=2.6 q=-0.0 size=  334800kB time=00:00:05.32 bitrate=515406.0kbits/s dup=97 drop=0 speed=0.111x &#xA;

    &#xA;

    What arguments do I provide to ffmpeg and in what order because my stream 0 is metadata and stream 1 is video so as to display image frame by frame with opencv ?&#xA;I would be grateful for any help that you could provide.

    &#xA;

    Further, I also have a query regarding how does ffmpeg know to that it has to first convert the rtp packets into mpeg2 TS packets before segregating video stream and data stream ?

    &#xA;

  • Conversion Funnel Optimisation : 10 Ways to Convert More

    24 janvier 2024, par Erin

    Converting leads into happy customers is the ultimate goal of any sales and marketing team. But there are many steps in between those two events, or in other words, funnel stages. 

    Your sales funnel includes all the steps you take to make your audience aware of your product or services and convince them to purchase. Conversion funnel optimisation strategies can help you move users through the stages of your sales funnel. 

    This article will show you how to optimise your conversion funnel and boost sales — no matter how your funnel looks. We’ll go over practical tips you can implement and how you can analyse and measure results.

    Let’s get started.

    What is conversion funnel optimisation ? 

    Conversion funnel optimisation is the strategic and ongoing process of refining and improving the different stages of a sales or marketing funnel to increase the rate at which users complete desired actions.

    A sales funnel represents the stages a potential customer goes through before purchasing. 

    The typical stages of a sales funnel include :

    • Awareness : At the top of the funnel, potential customers become aware of your product or service. 
    • Consideration : In this stage, prospects evaluate the product or service against alternatives. They may compare features, prices and customer reviews to make an informed decision.
    • Conversion : The prospect completes the transaction and becomes an actual customer by purchasing.
    • Loyalty : You can turn one-time buyers into repeat customers and brand advocates. 

    It’s called a “funnel” because, similar to the shape of a funnel, the number of potential customers decreases as they progress through the various stages of the sales process — as you can see illustrated below.

    Marketing funnel stages

    Sales funnels can vary across industries and business models, but the general concept remains the same. The goal is to guide potential customers through each funnel stage, addressing their needs and concerns at each step, ultimately leading to a successful conversion. 

    You can create and monitor a custom funnel for your site’s user journey with a web analytics solution like Matomo.

    Try Matomo for Free

    Get the web insights you need, without compromising data accuracy.

    No credit card required

    The importance of conversion funnel optimisation 

    At the heart of conversion funnel optimisation is the quest for higher conversion rates

    Refining the customer journey can increase the chances of turning visitors into customers who return repeatedly.

    Specifically, here’s how conversion funnel optimisation can benefit your business :

    • Increased conversions : Marketers can increase the likelihood of turning website visitors into customers by making the user journey more user-friendly and persuasive.
    • Higher revenue : Improved conversion rates aren’t just numbers on a chart ; they translate to tangible revenue. 
    • Increased ROI (return on investment) : By optimising the conversion funnel, you can get more value from your marketing and sales efforts. 
    • Improved customer satisfaction : When customers find it easy and enjoyable to interact with a website or service, it positively influences their satisfaction and likelihood of returning.
    • Data-driven decision-making : Businesses can make informed decisions on budgets and resources based on user behaviour and performance metrics by analysing and optimising conversion funnels.

    ​​Ultimately, conversion funnel optimisation efforts align the entire funnel with overarching business goals.

    10 ways to optimise your conversion funnel 

    Here are 10 ways to optimise your conversion funnel.

    1. Identify and segment your target audience

    The key to a successful conversion funnel begins with a deep understanding of your target audience. 

    Identifying and segmenting your audience lets you speak directly to their pain points, desires and motivations.

    One effective way to know your audience better is by creating detailed buyer personas. These are fictional representations of your ideal customers based on thorough market research and real data. Dive into demographics and behavioural patterns to craft personas that resonate with your audience.

    Audience segmentation

    Note that consumer preferences are not static. They evolve, influenced by trends, technological advancements and shifts in societal values. Staying attuned to these changes is crucial as part of optimising your conversion funnel.

    Thus, you must regularly update your buyer personas and adjust your marketing strategies accordingly.

    2. Create content for every stage of the funnel

    Each funnel stage represents a different mindset and needs for your potential customers. Tailoring your content ensures you deliver the right message at the right time to the right audience. 

    Here’s how to tailor your content to fit prospective customers at every conversion funnel stage.

    Awareness-stage content

    Prospects here are seeking information. Your content should be educational and focused on addressing their pain points. Create blog posts, infographics and videos introducing them to your industry, product or service.

    This video we created at Matomo is a prime example of awareness-stage content, grabbing attention and educating viewers about Matomo.

    Consideration-stage content

    Prospects are evaluating their options. Provide content highlighting your product’s unique selling points, such as case studies, product demonstrations and customer testimonials.

    Here’s how we use a versus landing page at Matomo to persuade prospects at this funnel stage.

    Versus page example from Matomo comparing Google Analytics alternative

    Conversion-stage content

    This is the final push. Ensure a smooth transition to conversion with content like promotional offers, limited-time discounts and clear calls to action (CTA).

    Loyalty-stage content

    In this stage, you might express gratitude for the purchase through personalised thank-you emails. Follow up with additional resources, tips or exclusive offers to reinforce a positive post-purchase experience. This also positions your brand as a helpful resource beyond the initial sale.

    Reward customer loyalty with exclusive offers, discounts or membership in a loyalty program.

    3. Capture leads

    Lead magnets are incentives offered to potential customers in exchange for their contact information, typically their email addresses. 

    Examples of lead magnets include :

    • Ebooks and whitepapers : In-depth resources that delve into specific topics of interest to your target audience.
    • Webinars and workshops : Live or recorded sessions that offer valuable insights, training or demonstrations.
    • Free trials and demos : Opportunities for potential customers to experience your product or service firsthand.
    • Checklists and templates : Practical tools that help your audience solve specific challenges.
    • Exclusive offers and discounts : Special promotions are available to those who subscribe or provide their contact information.

    For instance, here’s how HubSpot uses templates as lead magnets.

    HubSpot templates

    Similarly, you can incorporate your lead magnets into relevant articles or social media posts, email campaigns and other marketing channels.

    4. Optimise your landing pages

    Understanding how visitors interact with your landing pages is a game-changer. So, the first step in optimising your landing pages is to analyse them.

    Enter Matomo’s heatmaps — the secret weapon in landing page optimisation. They visually represent how users interact with your pages, revealing where they linger, what catches their attention and where they may encounter friction. 

    Matomo Heatmaps Feature

    Here are a few landing page elements you should pay attention to :

    • Strategic visual elements : Integrate high-quality images, videos and graphics that support your message and guide visitors through the content.
    • Compelling copy : Develop concise and persuasive copy that emphasises the benefits of your offering, addressing user pain points.
    • Effective CTA : Ensure your CTA is prominently displayed, using compelling language and colours that stand out.
    • Mobile responsiveness : Optimise your landing pages for various devices, especially considering the prevalence of mobile users.
    • Minimal form fields : Reduce friction by keeping form fields to a minimum, requesting only essential information.
    • ​​Leverage social proof : Integrate testimonials, reviews and trust badges to build trust and credibility.
    • A/B testing : Experiment with variations in design, copy and CTAs through A/B testing, allowing data to guide your decisions.

    Try Matomo for Free

    Get the web insights you need, without compromising data accuracy.

    No credit card required

    5. ​​Use compelling Calls to Action (CTAs)

    Crafting compelling CTAs is an art that involves a careful balance of persuasion, clarity and relevance.

    Here are a few tips you can implement to write CTAs that support your goals :

    • Use language that compels action. Instead of generic phrases like “Click Here,” opt for more persuasive alternatives such as “Unlock Exclusive Access” or “Start Your Free Trial.”
    • Make sure your CTAs are clear and straightforward. Visitors should instantly understand what action you want them to take. 
    • Tailor CTAs to the specific content on the page. Whether it’s a blog post, landing page or email, the CTA should seamlessly connect with the surrounding context.
    • Position your CTAs strategically. They should be prominently displayed and easily noticeable, guiding visitors without intruding.
    • Create a sense of urgency. Encourage immediate action by incorporating language that instils a sense of urgency. Phrases like “Limited Time Offer” or “Act Now” can prompt quicker responses.

    6. Have an active social presence

    Social media platforms are bustling hubs of activity where your target audience spends a significant portion of their online time. Cultivating a social media presence allows you to meet your audience where they are, fostering a direct line of communication.

    Moreover, the integration of shopping features directly into social media platforms transforms them into seamless shopping experiences. Nearly half of Instagram users shop weekly through the platform. 

    Also, the US social commerce sales continue to grow each year and are expected to reach $79.64 billion by 2025.

    Graph showing the UD social commerce sales 2019-2025

    7. Build a brand community

    Four in five customers consider communities important to how engaged they are with a brand.

    A strong community fosters a sense of belonging and loyalty among members. When customers feel connected to your brand and each other, they are more likely to remain loyal over the long term. 

    Also, satisfied community members often share their positive experiences with others, expanding your brand’s reach without additional marketing efforts.

    For example, Nike’s community for runners is a digital space where individuals share their running journeys, accomplishments and challenges. 

    Nike Run Club page

    By strategically building and nurturing a community, you not only enhance retention and spur referrals but also create a space where your brand becomes an integral part of your customers’ lives. 

    8. Conduct A/B tests

    A/B testing systematically compares two versions of a webpage, email or other content to determine which performs better.

    Examples of elements to A/B test :

    • CTAs : The language, colour, size and placement of CTAs can significantly impact user engagement. A/B testing allows you to discover which variations prompt the desired actions.
    • Headlines : Crafting compelling headlines is an art. Test different versions to identify which headlines resonate best with your audience, whether they are more drawn to clarity, humour, urgency or curiosity.
    • Images : Test different images to understand your audience’s visual preferences. This could include product images, lifestyle shots or graphics.
    Matomo A/B Test feature

    With Matomo’s A/B testing feature, you can test various elements to see which is successful in converting visitors or moving them to the next stage of the conversion funnel.

    9. Leverage social proof

    In an era where consumers are inundated with choices, the opinions, reviews and endorsements of others serve as beacons, guiding potential customers through the decision-making process. 

    Simply put — when people see that others have had positive experiences with your brand, it instils trust and confidence.

    Importance of social proof

    You can proactively gather social proof and display it prominently across your marketing channels. Here are some examples of social proof you can leverage :

    • Customer reviews : Positive reviews and testimonials from satisfied customers serve as authentic endorsements of your products or services. 
    • Case studies : In-depth case studies that showcase successful collaborations or solutions provided to clients offer a detailed narrative of your brand’s capabilities. These are particularly effective in B2B scenarios or for complex products and services.
    • User-generated content : Encourage customers to share their experiences. This could include photos, videos or posts on social media platforms, providing a dynamic and genuine portrayal of your brand.
    • Influencer endorsements : Collaborating with influencers in your industry or niche can amplify your social proof. When influencers vouch for your products or services, their followers are more likely to take notice.

    10. Measure and analyse performance

    This is a continuous loop of refinement, where you should use analysis and data-driven insights to guide your conversion funnel optimisation efforts.

    Here’s a systematic approach you can take :

    1. Identify the path users take on your site using a feature like Users Flow.
    2. Map the customer journey using a Funnels feature like the one in Matomo. 
    3. Identify the metrics that align with your conversion goals at each stage of the funnel, such as website traffic, conversion rates, click-through rates and customer acquisition costs.
    4. Assess conversion rates at different stages of the funnel. Identify areas with significant drop-offs and investigate factors that might contribute to the decline.
    5. Use heatmaps and session recordings to see first-hand how users interact with your site.
    6. Create an experiment to test and improve a specific area within your funnel using insights from the heatmaps and session recordings.
    7. A/B test, analyse the results to understand which variations performed better. Use this data to refine elements within your funnel.

    See how Concrete CMS 3x their leads with conversion optimisation.

    Conclusion 

    The customer journey is not linear. However, it involves a few specific stages your audience will go through — from first learning about your product or services to considering whether to try it. The goal is to turn them into happy and loyal customers.

    In this article, we went over strategies and practical tips you can use to guide customers through the conversion funnel. From segmenting your audience to capturing leads, optimising landing pages and running A/B tests, there are steps you can take to ensure your audience will move to the next stage.

    And of course, you have to continuously measure and analyse your performance. That’s how you know whether you’re heading in the right direction and, if not, where to correct your course. 

    For that, you need a robust web analytics solution with conversion optimisation features. Try Matomo free for 21 days and start optimising your conversion funnel—no credit card required.