Sur d’autres sites (8237)

• Sync video and data with ffmpeg

  31 août 2022, par plj

  I join a video and data where I copy the video from the vid1.ts along with the data (klv) from the vid2.ts :

  


  ffmpeg -y -i vid1.ts -i vid2.ts -c copy -map 0:0 -map 1:2 -shortest full_exclude.ts

  


  When I probe the resulting video I see that the packets don't line up :

  


  ffprobe -show_packets full_exclude.ts

  


  The 1st data packet is :

  


  [PACKET]
codec_type=data
stream_index=1
pts=126000
pts_time=1.400000
dts=126000
dts_time=1.400000
duration=N/A
duration_time=N/A
size=324
pos=564
flags=K_
[SIDE_DATA]
side_data_type=MPEGTS Stream ID
id=252
[/SIDE_DATA]
[/PACKET]


  


  The 1st video packet is :

  


  [PACKET]
codec_type=video
stream_index=0
pts=341392
pts_time=3.793244
dts=341392
dts_time=3.793244
duration=N/A
duration_time=N/A
size=9270
pos=28952
flags=K_
[SIDE_DATA]
side_data_type=MPEGTS Stream ID
id=224
[/SIDE_DATA]
[/PACKET]
[PACKET]


  


  How do I get the data and video in sync with each other ?

  


• ffmpeg stream mp4 file to Instagram live

  31 mars 2022, par raitech

  I created a live stream session on instafeed.me then used ffmpeg to send an MP4 file to the stream. But I get IO error.

  


  The command is

  


  ffmpeg -rtbufsize 256M -re -i "1.mp4" -acodec libmp3lame -ar 44100 -b:a 128k -pix_fmt yuv420p -profile:v baseline -s 504x896 -bufsize 6000k -vb 400k -maxrate 1500k -deinterlace -vcodec libx264 -preset veryfast -g 30 -r 30 -f flv "rtmps://live-upload.instagram.com:443/rtmp/XXXXXXX?s_sw=0&s_vt=ig&a=XXXXXXX"


  


  The output is

  


  ffmpeg version git-2020-05-04-5767a2e Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9.3.1 (GCC) 20200328
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --disable-w32threads --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 43.100 / 56. 43.100
  libavcodec     58. 82.100 / 58. 82.100
  libavformat    58. 42.102 / 58. 42.102
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 80.100 /  7. 80.100
  libswscale      5.  6.101 /  5.  6.101
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.42.102
  Duration: 01:05:43.70, start: 0.000000, bitrate: 1729 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 1594 kb/s, 24 fps, 24 tbr, 90k tbn, 48 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
[libx264 @ 000001cf5ba53300] using SAR=256/81e=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A
[libx264 @ 000001cf5ba53300] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 000001cf5ba53300] profile Constrained Baseline, level 3.1, 4:2:0, 8-bit
[libx264 @ 000001cf5ba53300] 264 - core 159 - H.264/MPEG-4 AVC codec - Copyleft 2003-2019 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=18 lookahead_threads=6 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=30 keyint_min=3 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=abr mbtree=1 bitrate=400 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=1500 vbv_bufsize=6000 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00
Output #0, flv, to 'rtmps://live-upload.instagram.com:443/rtmp/XXXXX?s_sw=0&s_vt=ig&a=XXXX':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.42.102
    Stream #0:0(und): Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv420p, 504x896 [SAR 256:81 DAR 16:9], q=-1--1, 400 kb/s, 30 fps, 1k tbn, 30 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      encoder         : Lavc58.82.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 1500000/0/400000 buffer size: 6000000 vbv_delay: N/A
    Stream #0:1(und): Audio: mp3 (libmp3lame) ([2][0][0][0] / 0x0002), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      encoder         : Lavc58.82.100 libmp3lame
[tls @ 000001cf5bb3c200] Error in the push function.0:02.90 bitrate= 179.5kbits/s speed=0.957x
av_interleaved_write_frame(): I/O error
    Last message repeated 1 times
[flv @ 000001cf5bd0e980] Failed to update header with correct duration.
[flv @ 000001cf5bd0e980] Failed to update header with correct filesize.
Error writing trailer of rtmps://live-upload.instagram.com:443/rtmp/XXXXXX?s_sw=0&s_vt=ig&a=XXXXXXX: I/O error
frame=   66 fps= 20 q=37.0 Lsize=      89kB time=00:00:03.26 bitrate= 224.1kbits/s speed=0.965x
video:66kB audio:51kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[tls @ 000001cf5bb3c200] The specified session has been invalidated for some reason.
    Last message repeated 1 times
[libx264 @ 000001cf5ba53300] frame I:3     Avg QP:31.09  size:  9012
[libx264 @ 000001cf5ba53300] frame P:63    Avg QP:33.47  size:  1733
[libx264 @ 000001cf5ba53300] mb I  I16..4: 78.1%  0.0% 21.9%
[libx264 @ 000001cf5ba53300] mb P  I16..4: 11.0%  0.0%  0.6%  P16..4: 14.7%  3.6%  0.5%  0.0%  0.0%    skip:69.6%
[libx264 @ 000001cf5ba53300] final ratefactor: 30.81
[libx264 @ 000001cf5ba53300] coded y,uvDC,uvAC intra: 16.6% 12.9% 0.9% inter: 1.8% 0.9% 0.0%
[libx264 @ 000001cf5ba53300] i16 v,h,dc,p: 56% 12% 18% 14%
[libx264 @ 000001cf5ba53300] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 43%  7% 26%  4%  3%  9%  1%  6%  1%
[libx264 @ 000001cf5ba53300] i8c dc,h,v,p: 76%  4% 19%  1%
[libx264 @ 000001cf5ba53300] kb/s:389.26
Conversion failed!


  


• Android h264 decode non-existing PPS 0 referenced

  8 juin 2023, par nmxprime

  In Android JNI, using ffmpeg with libx264 use below codes to encode and decode raw rgb data !. I should use swscale to convert rgb565 to yuv420p as required by H.264. But not clear about this conversion.Please help, where i am wrong, with regard the log i get !

  



  Code for Encoding

  



  codecinit()- called once(JNI wrapper function)

  



  int Java_com_my_package_codecinit (JNIEnv *env, jobject thiz) {
avcodec_register_all();
codec = avcodec_find_encoder(AV_CODEC_ID_H264);//AV_CODEC_ID_MPEG1VIDEO);
if(codec->id == AV_CODEC_ID_H264)
    __android_log_write(ANDROID_LOG_ERROR, "set","h264_encoder");

if (!codec) {
    fprintf(stderr, "codec not found\n");
    __android_log_write(ANDROID_LOG_ERROR, "codec", "not found");

}
    __android_log_write(ANDROID_LOG_ERROR, "codec", "alloc-contest3");
c= avcodec_alloc_context3(codec);
if(c == NULL)
    __android_log_write(ANDROID_LOG_ERROR, "avcodec","context-null");

picture= av_frame_alloc();

if(picture == NULL)
    __android_log_write(ANDROID_LOG_ERROR, "picture","context-null");

c->bit_rate = 400000;
c->height = 800;
c->time_base= (AVRational){1,25};
c->gop_size = 10; 
c->max_b_frames=1;
c->pix_fmt = AV_PIX_FMT_YUV420P;
outbuf_size = 768000;
c->width = 480;

size = (c->width * c->height);

if (avcodec_open2(c, codec,NULL) < 0) {

__android_log_write(ANDROID_LOG_ERROR, "codec", "could not open");


}

ret = av_image_alloc(picture->data, picture->linesize, c->width, c->height,
                     c->pix_fmt, 32);
if (ret < 0) {
        __android_log_write(ANDROID_LOG_ERROR, "image","alloc-failed");
    fprintf(stderr, "could not alloc raw picture buffer\n");

}

picture->format = c->pix_fmt;
picture->width  = c->width;
picture->height = c->height;
return 0;

}


  



  encodeframe()-called in a while loop

  



  int Java_com_my_package_encodeframe (JNIEnv *env, jobject thiz,jbyteArray buffer) {
jbyte *temp= (*env)->GetByteArrayElements(env, buffer, 0);
Output = (char *)temp;
const uint8_t * const inData[1] = { Output }; 
const int inLinesize[1] = { 2*c->width };

//swscale should implement here

    av_init_packet(&pkt);
    pkt.data = NULL;    // packet data will be allocated by the encoder
    pkt.size = 0;

    fflush(stdout);
picture->data[0] = Output;
ret = avcodec_encode_video2(c, &pkt, picture,&got_output);

    fprintf(stderr,"ret = %d, got-out = %d \n",ret,got_output);
     if (ret < 0) {
                __android_log_write(ANDROID_LOG_ERROR, "error","encoding");
        if(got_output > 0)
        __android_log_write(ANDROID_LOG_ERROR, "got_output","is non-zero");

    }

    if (got_output) {
        fprintf(stderr,"encoding frame %3d (size=%5d): (ret=%d)\n", 1, pkt.size,ret);
        fprintf(stderr,"before caling decode");
        decode_inline(&pkt); //function that decodes right after the encode
        fprintf(stderr,"after caling decode");


        av_free_packet(&pkt);
    }


fprintf(stderr,"y val: %d \n",y);


(*env)->ReleaseByteArrayElements(env, buffer, Output, 0);
return ((ret));
}


  



  decode_inline() function

  



  decode_inline(AVPacket *avpkt){
AVCodec *codec;
AVCodecContext *c = NULL;
int frame, got_picture, len = -1,temp=0;

AVFrame *rawFrame, *rgbFrame;
uint8_t inbuf[INBUF_SIZE + FF_INPUT_BUFFER_PADDING_SIZE];
char buf[1024];
char rawBuf[768000],rgbBuf[768000];

struct SwsContext *sws_ctx;

memset(inbuf + INBUF_SIZE, 0, FF_INPUT_BUFFER_PADDING_SIZE);
avcodec_register_all();

c= avcodec_alloc_context3(codec);
if(c == NULL)
    __android_log_write(ANDROID_LOG_ERROR, "avcodec","context-null");

codec = avcodec_find_decoder(AV_CODEC_ID_H264);
if (!codec) {
    fprintf(stderr, "codec not found\n");
    fprintf(stderr, "codec = %p \n", codec);
    }
c->pix_fmt = AV_PIX_FMT_YUV420P;
c->width = 480;
c->height = 800;

rawFrame = av_frame_alloc();
rgbFrame = av_frame_alloc();

if (avcodec_open2(c, codec, NULL) < 0) {
    fprintf(stderr, "could not open codec\n");
    exit(1);
    }
sws_ctx = sws_getContext(c->width, c->height,/*PIX_FMT_RGB565BE*/
            PIX_FMT_YUV420P, c->width, c->height, AV_PIX_FMT_RGB565/*PIX_FMT_YUV420P*/,
            SWS_BILINEAR, NULL, NULL, NULL);


frame = 0;

unsigned short *decodedpixels = &rawBuf;
rawFrame->data[0] = &rawBuf;
rgbFrame->data[0] = &rgbBuf;

fprintf(stderr,"size of avpkt %d \n",avpkt->size);
temp = avpkt->size;
while (temp > 0) {
        len = avcodec_decode_video2(c, rawFrame, &got_picture, avpkt);

        if (len < 0) {
            fprintf(stderr, "Error while decoding frame %d\n", frame);
            exit(1);
            }
        temp -= len;
        avpkt->data += len;

        if (got_picture) {
            printf("saving frame %3d\n", frame);
            fflush(stdout);
        //TODO  
        //memcpy(decodedpixels,rawFrame->data[0],rawFrame->linesize[0]); 
        //  decodedpixels +=rawFrame->linesize[0];

            frame++;
            }

        }

avcodec_close(c);
av_free(c);
//free(rawBuf);
//free(rgbBuf);
av_frame_free(&rawFrame);
av_frame_free(&rgbFrame);


  



  }

  



  The log i get

  



  For the decode_inline() function :

  



  ---

  



  01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] non-existing PPS 0 referenced
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] decode_slice_header error
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] non-existing PPS 0 referenced
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] decode_slice_header error
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] non-existing PPS 0 referenced
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] decode_slice_header error
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] non-existing PPS 0 referenced
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] decode_slice_header error
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] non-existing PPS 0 referenced
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] decode_slice_header error
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] non-existing PPS 0 referenced
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] decode_slice_header error
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] non-existing PPS 0 referenced
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] decode_slice_header error
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] non-existing PPS 0 referenced
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] decode_slice_header error
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] non-existing PPS 0 referenced
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] decode_slice_header error
01-02 14:50:50.160: I/stderr(3407): [h264 @ 0x8db540] Invalid mix of idr and non-idr slices
01-02 14:50:50.160: I/stderr(3407): Error while decoding frame 0


  



  Edit : Changing GOP value :

  



  If i change c->gop_size = 3; as expected it emits one I frame every three frames. The non-existing PPS 0 referenced message is not there for in every third execution, but all other have this message