Recherche avancée

Sur d’autres sites (7677)

• Algorithm when recording SegmentTimeline

  24 mars 2021, par jgkim0518

  I packaged a media stream by mpeg-dash transcoded video at twice the speed and audio at normal speed.
Here is the command :

  


  ffmpeg -re -stream_loop -1 -i timing_logic.mp4 -c:v hevc_nvenc -filter:v "setpts=2*PTS" -c:a libfdk_aac -map 0:v -map 0:a -f mpegts udp://xxx.xxx.xxx.xxx:xxxx?pkt_size=1316


  


  The source information used here is as follows :

  


  Input #0, mpegts, from '/home/test/timing_logic/timing_logic.mp4':
Duration: 00:00:30.22, start: 1.400000, bitrate: 16171 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: hevc (Main 10) (HEVC / 0x43564548), yuv420p10le(tv, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:1[0x101](eng): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 128 kb/s Stream mapping:
Stream #0:0 -> #0:0 (hevc (native) -> hevc (hevc_nvenc))
Stream #0:1 -> #0:1 (mp2 (native) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
frame= 0 fps=0.0 q=0.0 size= 0kB time=-577014:32:22.77 bitrate= -0.0kbits/s speed=N/A
frame= 0 fps=0.0 q=0.0 size= 0kB time=-577014:32:22.77 bitrate= -0.0kbits/s speed=N/A
Output #0, mpegts, to 'udp://xxx.xxx.xxx.xxx:xxxx?pkt_size=1316':
Metadata:
encoder : Lavf58.45.100
Stream #0:0: Video: hevc (hevc_nvenc) (Main 10), p010le, 3840x2160 [SAR 1:1 DAR 16:9], q=-1--1, 2000 kb/s, 50 fps, 90k tbn, 50 tbc
Metadata:
encoder : Lavc58.91.100 hevc_nvenc
Side data:
cpb: bitrate max/min/avg: 0/0/2000000 buffer size: 4000000 vbv_delay: N/A
Stream #0:1(eng): Audio: aac (libfdk_aac), 48000 Hz, stereo, s16, 139 kb/s
Metadata:
encoder : Lavc58.91.100 libfdk_aac


  


  My shaka packager command is :

  


  packager \ 'in=udp://xxx.xxx.xxx.xxx:xxxx,stream=video,init_segment=/home/test/timing_logic/package/video/0/video.mp4,segment_template=/home/test/timing_logic/package/video/0/$Time$.m4s' \
'in=udp://xxx.xxx.xxx.xxx:xxxx,stream=audio,init_segment=/home/test/timing_logic/package/audio/0/audio.mp4,segment_template=/home/test/timing_logic/package/audio/0/$Time$.m4a' \
--segment_duration 5 --fragment_duration 5 --minimum_update_period 5 --min_buffer_time 5 \
--preserved_segments_outside_live_window 24 --time_shift_buffer_depth 40 \
--allow_codec_switching --allow_approximate_segment_timeline --log_file_generation_deletion \
--mpd_output $OUTPUT/$output_mpd.mpd &


  


  Looking at the mpd generated as a result of packaging, the duration of the audio is twice that of the video, and the number of segments is half.
The following is the content of mpd :

  


  &lt;?xml version="1.0" encoding="UTF-8"?>&#xA;&#xA;<mpd xmlns="urn:mpeg:dash:schema:mpd:2011" profiles="urn:mpeg:dash:profile:isoff-live:2011" minbuffertime="PT5S" type="dynamic" publishtime="2021-03-23T10:01:57Z" availabilitystarttime="2021-03-23T09:59:55Z" minimumupdateperiod="PT5S" timeshiftbufferdepth="PT40S">&#xA;<period start="PT0S">&#xA;<adaptationset contenttype="audio" segmentalignment="true">&#xA;<representation bandwidth="140020" codecs="mp4a.40.2" mimetype="audio/mp4" audiosamplingrate="48000">&#xA;<audiochannelconfiguration schemeiduri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"></audiochannelconfiguration>&#xA;<segmenttemplate timescale="90000" initialization="/home/test/timing_logic/package/audio/0/audio.mp4" media="/home/test/timing_logic/package/audio/0/$Time$.m4a" startnumber="16">&#xA;<segmenttimeline>&#xA;<s t="6751436" d="449280"></s>&#xA;<s t="7200716" d="451200"></s>&#xA;<s t="7651916" d="449280"></s>&#xA;<s t="8101196" d="374400"></s>&#xA;<s t="8551196" d="449280"></s>&#xA;<s t="9000476" d="451200"></s>&#xA;<s t="9451674" d="449280"></s>&#xA;<s t="9900956" d="449280"></s>&#xA;<s t="10350236" d="451200"></s>&#xA;<s t="10801434" d="374400"></s>&#xA;</segmenttimeline>&#xA;</segmenttemplate>&#xA;</representation>&#xA;</adaptationset>&#xA;<adaptationset contenttype="video" width="3840" height="2160" framerate="90000/3600" segmentalignment="true" par="16:9">&#xA;<representation bandwidth="1520392" codecs="hvc1.2.4.L153" mimetype="video/mp4" sar="1:1">&#xA;<segmenttemplate timescale="90000" initialization="/home/test/timing_logic/package/video/0/video.mp4" media="/home/test/timing_logic/package/video/0/$Time$.m4s" startnumber="19">&#xA;<segmenttimeline>&#xA;<s t="16954440" d="900000" r="4"></s>&#xA;</segmenttimeline>&#xA;</segmenttemplate>&#xA;</representation>&#xA;</adaptationset>&#xA;</period>&#xA;</mpd>&#xA;

  


  When looking at the MPD, it seems that the shaka packager checks the PTS or DTS of the TS when creating a segment and recording the contents of the SegmentTimeline.
But I couldn't understand even by looking at the MPEG standard documentation and the DASH-IF documentation.

  


  My question is whether the packager refers to PTS or DTS when creating a segment.
How are SegmentTimeline's S@t and S@d recorded ?
What algorithm is the SegmentTimeline recorded with ? Please help me. Thank you.

  


• h264 encoding of bgr images into .mp4 file with libav

  10 février 2021, par xyfix

  I'm trying to encode(h264) a series of .png into a mp4 file. A cv::Mat holds the png data (BGR) and that is converted to YUV420P which is then encoded and written to a .mp4 file. I have added two block statements in the code to store images on the disk (before and after encoding). The first image before it gets encoded is correct but the second one after encoding is not. The avcodec_send_frame returns 0 so up to that point everything works.
Edit : I get an mp4 file of 1 MB but I can't open it with vlc
ecodec.h

  


  class ECodec
{
public:

    MovieCodec();

    ~MovieCodec();

    void MatToFrame( cv::Mat& image );

    void encode( AVFrame *frame, AVPacket *pkt );

private:
 
    FILE* m_file;

    AVCodec* m_encoder = NULL;

    AVCodecContext* m_codecContextOut = NULL;

    AVPacket* m_packet = NULL;

};


  


  ecodec.cpp

  


  ECodec::ECodec() :
//    m_encoder( avcodec_find_encoder_by_name( videoCodec.c_str()))
    m_encoder( avcodec_find_encoder( AV_CODEC_ID_H264 ))
{
    m_file = fopen( "c:\\tmp\\outputVideo.mp4", "wb");
}



void ECodec::MatToFrame( cv::Mat& image )
{
    int ret( 0 );
    int frameRate( 24 );
    AVFrame *frame = NULL;

    m_encoder( avcodec_find_encoder( AV_CODEC_ID_H264 ))
    m_codecContextOut = avcodec_alloc_context3( m_encoder );

    m_codecContextOut->width = 800;
    m_codecContextOut->height = 640;
    m_codecContextOut->bit_rate = 400000;//m_codecContextOut->width * m_codecContextOut->height * 3;
    m_codecContextOut->time_base = (AVRational){1, 24};
    m_codecContextOut->framerate = (AVRational){24, 1};
    m_codecContextOut->codec_tag = AV_CODEC_ID_H264;
    m_codecContextOut->pix_fmt = AV_PIX_FMT_YUV420P;
    m_codecContextOut->codec_type = AVMEDIA_TYPE_VIDEO;
    m_codecContextOut->gop_size = 1;
    m_codecContextOut->max_b_frames = 1;

    av_log_set_level(AV_LOG_VERBOSE);

    ret = av_opt_set(m_codecContextOut->priv_data, "preset", "slow", 0);

    ret = avcodec_open2(m_codecContextOut, m_encoder, NULL);

    frame = av_frame_alloc();

    frame->format = AV_PIX_FMT_YUV420P;
    frame->width = image.cols();
    frame->height = image.rows();


    ret = av_image_alloc(frame->data, frame->linesize, frame->width,  frame->height, AV_PIX_FMT_YUV420P, 1);

    if (ret < 0)
    {
        return;
    }

    struct SwsContext *sws_ctx;
    sws_ctx = sws_getContext((int)image.cols(), (int)image.rows(), AV_PIX_FMT_RGB24,
                             (int)image.cols(), (int)image.rows(), AV_PIX_FMT_YUV420P,
                             0, NULL, NULL, NULL);

    const uint8_t* rgbData[1] = { (uint8_t* )image.getData() };
    int rgbLineSize[1] = { 3 * image.cols() };

    sws_scale(sws_ctx, rgbData, rgbLineSize, 0, image.rows(), frame->data, frame->linesize);

    frame->pict_type = AV_PICTURE_TYPE_I;

cv::Mat yuv420p(frame->height + frame->height/2, frame->width, CV_8UC1,frame->data[0]);
cv::Mat cvmIm;
cv::cvtColor(yuv420p,cvmIm,CV_YUV420p2BGR);
cv::imwrite("c:\\tmp\\rawimage.png", cvmIm);
//OK

    m_packet = av_packet_alloc();
    ret = av_new_packet( m_packet, m_codecContextOut->width * m_codecContextOut->height * 3 );

    /* encode the image */
    encode( frame, m_packet );


    avcodec_free_context(&m_codecContextOut);
    av_frame_free(&frame);
    av_packet_free( &m_packet );
}




void ECodec::encode( AVFrame *frame, AVPacket *pkt )
{
    int ret;
    
    /* send the frame to the encoder */
    ret = avcodec_send_frame( m_codecContextOut, frame);

    if (ret < 0)
    {
        fprintf(stderr, "Error sending a frame for encoding\n");
        exit(1);
    }

    do
    {
        ret = avcodec_receive_packet(m_codecContextOut, pkt);
        if (ret == 0)
        {

cv::Mat yuv420p(frame->height + frame->height/2, frame->width, CV_8UC1,pkt->data);
cv::Mat cvmIm;
cv::cvtColor(yuv420p,cvmIm,CV_YUV420p2BGR);
cv::imwrite("c:\\tmp\\rawencodedimage.png", cvmIm);
//NOT OK
            fwrite(pkt->data, 1, pkt->size, m_file );
            av_packet_unref(pkt);

            break;
        }
        else if ((ret < 0) && (ret != AVERROR(EAGAIN)))
        {
            return;
        }
        else if (ret == AVERROR(EAGAIN))
        {
             ret = avcodec_send_frame(m_codecContextOut, NULL);
             if (0 > ret)
             {
                 return;
             }
        }
    } while (ret == 0);
}


  


• ffmpeg - preserve time base and pts of all frames during transcode

  18 mars 2021, par jdizzle

  Context :

  


  I have an application that produces mp4s with HEVC encoding. I want to convert them to AVC for use in browser-based displaying. A very crucial part of how I want to use this is to preserve exact PTS times, as this is how we correlate the frames to other data streams not included in the video.

  


  Question :

  


  How do I make ffmpeg preserve this information across the transcode ? All the obvious flags seem to have no effect and ffmpeg just does whatever it wants.

  


  $ ffprobe -show_streams original.mp4 2>/dev/null | grep time_base
codec_time_base=16666667/500000000
time_base=1/1000


  


  Here is my convert command :

  


  $ ffmpeg -i original.mp4 -copyts -copytb 0 test.mp4


  


  And its result :

  


  $ ffprobe -show_streams test.mp4 2>/dev/null | grep time_base
codec_time_base=1/60
time_base=1/15360


  


  I would expect the time_bases to match. The PTS of the frames also don't match when doing ffprobe -show_frames

  


  EDIT :
@Gyan suggested using -video_track_timescale, but that didn't get the exact behavior I was looking for :

  


  $ sdiff <(ffprobe -show_frames test.mp4  | grep pkt_pts_time) <(ffprobe -show_frames original.mp4 | grep pkt_pts_time)
pkt_pts_time=0.000000                           pkt_pts_time=0.000000
pkt_pts_time=0.033000                           pkt_pts_time=0.033000
pkt_pts_time=0.067000                         | pkt_pts_time=0.066000
pkt_pts_time=0.100000                           pkt_pts_time=0.100000
pkt_pts_time=0.133000                           pkt_pts_time=0.133000
pkt_pts_time=0.167000                         | pkt_pts_time=0.166000
pkt_pts_time=0.200000                           pkt_pts_time=0.200000
pkt_pts_time=0.233000                           pkt_pts_time=0.233000
pkt_pts_time=0.267000                         | pkt_pts_time=0.266000
pkt_pts_time=0.300000                           pkt_pts_time=0.300000
pkt_pts_time=0.333000                           pkt_pts_time=0.333000
pkt_pts_time=0.367000                         | pkt_pts_time=0.366000
pkt_pts_time=0.400000                           pkt_pts_time=0.400000
pkt_pts_time=0.433000                           pkt_pts_time=0.433000
pkt_pts_time=0.467000                           pkt_pts_time=0.467000
pkt_pts_time=0.500000                           pkt_pts_time=0.500000
pkt_pts_time=0.533000                         | pkt_pts_time=0.532000
pkt_pts_time=0.567000                         | pkt_pts_time=0.565000
pkt_pts_time=0.600000                         | pkt_pts_time=0.598000
pkt_pts_time=0.633000                         | pkt_pts_time=0.631000
pkt_pts_time=0.667000                         | pkt_pts_time=0.665000
pkt_pts_time=0.700000                         | pkt_pts_time=0.698000
...