Sur d’autres sites (2124)

• Files created with "ffmpeg hevc_nvenc" do not play on TV. (with video codec SDK 9.1 of nvidia)

  29 janvier 2020, par Dashhh

  Problem

  • Files created with hevc_nvenc do not play on TV. (samsung smart tv, model unknown)
    Related to my ffmpeg build is below.

  FFmpeg build conf

  $ ffmpeg -buildconf
  
      --enable-cuda
  
      --enable-cuvid
  
      --enable-nvenc
  
      --enable-nonfree
  
      --enable-libnpp
  
      --extra-cflags=-I/path/cuda/include
  
      --extra-ldflags=-L/path/cuda/lib64
  
      --prefix=/prefix/ffmpeg_build
  
      --pkg-config-flags=--static
  
      --extra-libs='-lpthread -lm'
  
      --extra-cflags=-I/prefix/ffmpeg_build/include
  
      --extra-ldflags=-L/prefix/ffmpeg_build/lib
  
      --enable-gpl
  
      --enable-nonfree
  
      --enable-version3
  
      --disable-stripping
  
      --enable-avisynth
  
      --enable-libass
  
      --enable-libfontconfig
  
      --enable-libfreetype
  
      --enable-libfribidi
  
      --enable-libgme
  
      --enable-libgsm
  
      --enable-librubberband
  
      --enable-libshine
  
      --enable-libsnappy
  
      --enable-libssh
  
      --enable-libtwolame
  
      --enable-libwavpack
  
      --enable-libzvbi
  
      --enable-openal
  
      --enable-sdl2
  
      --enable-libdrm
  
      --enable-frei0r
  
      --enable-ladspa
  
      --enable-libpulse
  
      --enable-libsoxr
  
      --enable-libspeex
  
      --enable-avfilter
  
      --enable-postproc
  
      --enable-pthreads
  
      --enable-libfdk-aac
  
      --enable-libmp3lame
  
      --enable-libopus
  
      --enable-libtheora
  
      --enable-libvorbis
  
      --enable-libvpx
  
      --enable-libx264
  
      --enable-libx265
  
      --disable-ffplay
  
      --enable-libopenjpeg
  
      --enable-libwebp
  
      --enable-libxvid
  
      --enable-libvidstab
  
      --enable-libopenh264
  
      --enable-zlib
  
      --enable-openssl

  ---

  ffmpeg Command

  • Command about FFmpeg encoding

  ffmpeg -ss 1800 -vsync 0 -hwaccel cuvid -hwaccel_device 0 \
  
  -c:v h264_cuvid -i /data/input.mp4 -t 10 \
  
  -filter_complex "\
  
  [0:v]hwdownload,format=nv12,format=yuv420p,\
  
  scale=iw*2:ih*2" -gpu 0 -c:v hevc_nvenc -pix_fmt yuv444p16le -preset slow -rc cbr_hq -b:v 5000k -maxrate 7000k -bufsize 1000k -acodec aac -ac 2 -dts_delta_threshold 1000 -ab 128k -flags global_header ./makevideo_nvenc_hevc.mp4

  Full log about This Command - check this full log

  The reason for adding "-color_ " in the command is as follows.

  • HDR video after creating bt2020 + smpte2084 video using nvidia hardware accelerator. (I’m studying to make HDR videos. I’m not sure if this is right.)

  How can I make a video using ffmpeg hevc_nvenc and have it play on TV ?

  ---

  Things i’ve done

  Here’s what I’ve researched about why it doesn’t work.
   The header information is not properly included in the resulting video file. So I used a program called nvhsp to add SEI and VUI information inside the video. See below for the commands and logs used.

  nvhsp is open source for writing VUI and SEI bitstrings in raw video. nvhsp link

  # make rawvideo for nvhsp
  
  $  ffmpeg -vsync 0 -hwaccel cuvid -hwaccel_device 0 -c:v h264_cuvid \
  
  -i /data/input.mp4 -t 10 \
  
  -filter_complex "[0:v]hwdownload,format=nv12,\
  
  format=yuv420p,scale=iw*2:ih*2" \
  
  -gpu 0 -c:v hevc_nvenc -f rawvideo output_for_nvhsp.265
  
  
  
  # use nvhsp
  
  $ python nvhsp.py ./output_for_nvhsp.265 -colorprim bt2020 \
  
  -transfer smpte-st-2084 -colormatrix bt2020nc \
  
  -maxcll "1000,300" -videoformat ntsc -full_range tv \
  
  -masterdisplay "G (13250,34500) B (7500,3000 ) R (34000,16000) WP (15635,16450) L (10000000,1)" \
  
  ./after_nvhsp_proc_output.265
  
  
  
  Parsing the infile:
  
  
  
  ==========================
  
  
  
  Prepending SEI data
  
  Starting new SEI NALu ...
  
  SEI message with MaxCLL = 1000 and MaxFall = 300 created in SEI NAL
  
  SEI message Mastering Display Data G (13250,34500) B (7500,3000) R (34000,16000) WP (15635,16450) L (10000000,1) created in SEI NAL
  
  Looking for SPS ......... [232, 22703552]
  
  SPS_Nals_addresses [232, 22703552]
  
  SPS NAL Size 488
  
  Starting reading SPS NAL contents
  
  Reading of SPS NAL finished. Read 448 of SPS NALu data.
  
  
  
  Making modified SPS NALu ...
  
  Made modified SPS NALu-OK
  
  New SEI prepended
  
  Writing new stream ...
  
  Progress: 100%
  
  =====================
  
  Done!
  
  
  
  File nvhsp_after_output.mp4 created.
  
  
  
  # after process
  
  $ ffmpeg -y -f rawvideo -r 25 -s 3840x2160 -pix_fmt yuv444p16le -color_primaries bt2020 -color_trc smpte2084  -colorspace bt2020nc -color_range tv -i ./1/after_nvhsp_proc_output.265 -vcodec copy  ./1/result.mp4 -hide_banner
  
  
  
  Truncating packet of size 49766400 to 3260044
  
  [rawvideo @ 0x40a6400] Estimating duration from bitrate, this may be inaccurate
  
  Input #0, rawvideo, from './1/nvhsp_after_output.265':
  
    Duration: N/A, start: 0.000000, bitrate: 9953280 kb/s
  
      Stream #0:0: Video: rawvideo (Y3[0][16] / 0x10003359), yuv444p16le(tv, bt2020nc/bt2020/smpte2084), 3840x2160, 9953280 kb/s, 25 tbr, 25 tbn, 25 tbc
  
  [mp4 @ 0x40b0440] Could not find tag for codec rawvideo in stream #0, codec not currently supported in container
  
  Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (copy)
  
      Last message repeated 1 times

  Goal

  • I want to generate matadata normally when encoding a video through hevc_nvenc.

  • I want to create a video through hevc_nvenc and play HDR Video on smart tv with 10bit color depth support.

  ---

  Additional

  • Is it normal for ffmpeg hevc_nvenc not to generate metadata in the resulting video file ? or is it a bug ?

  • Please refer to the image below. (*’알 수 없음’ meaning ’unknown’)

    • if you need more detail file info, check this Gist Link (by ffprobe)
      

  • However, if you encode a file in libx265, the attribute information is entered correctly as shown below.

    • if you need more detail file info, check this Gist Link
      

  However, when using hevc_nvenc, all information is missing.

  • i used option -show_streams -show_programs -show_format -show_data -of json -show_frames -show_log 56 at ffprobe

• libswresample : Why does swr_init() change |in_ch_layout| order so it no longer matches my decoded AVFrames, causing resampling to fail ?

  20 novembre 2023, par CheekyChips

  I am trying to write some code that resamples an audio file to 16kHz and 1 channel and then encodes it to PCM, but I am having an issue with channel layouts.

  


  In a nutshell :

  


  My AVCodecContext and the frames I get from the stream via avcodec_receive_frame() have a channel layout order of AV_CHANNEL_ORDER_UNSPEC. But when I call swr_init() it changes the in_ch_layout order to AV_CHANNEL_ORDER_NATIVE. Then when I call swr_convert_frame() with my AVFrames, because the channel layout orders don't match, the resampling fails because it thinks the input changed.

  


  More details :

  


  I create an AVCodecContext from my audio stream's codec, and it has a channel layout of AV_CHANNEL_ORDER_UNSPEC with 2 channels, and any frames I decode from the stream via avcodec_receive_frame() also have a channel layout order of AV_CHANNEL_ORDER_UNSPEC.

  


  I set SwrContext's |in_ch_layout| to the sample channel layout from the codec context :

  


      AVChannelLayout in_ch_layout = in_codec_context->ch_layout,
    ...
    int ret = swr_alloc_set_opts2(&swr_ctx, ...
                      &in_ch_layout,
                      ...);


  


  But SwrContext->init() changes its internal in_ch_layout from AV_CHANNEL_ORDER_UNSPEC to AV_CHANNEL_ORDER_NATIVE meaning it fails the next time I call swr_convert_frame() because the input frame has a different channel layout to the SwrContext. When swr_init() is called (in my case indirectly by swr_convert_frame(), but also if I alternatively call it directly) the SwrContext->used_ch_layout and SwrContext->in_ch_layout are updated to have channel layout order of AV_CHANNEL_ORDER_NATIVE :

  


      // swresample.c
    av_cold int swr_init(struct SwrContext *s){
        ...
        if (!av_channel_layout_check(&s->used_ch_layout))       <-- This hits if I don't set anything for used_ch_layout
            av_channel_layout_default(&s->used_ch_layout, s->in.ch_count);      <-- default is AV_CHANNEL_ORDER_NATIVE
        ...
        if (s->used_ch_layout.order == AV_CHANNEL_ORDER_UNSPEC) <-- This hits if I do set used_ch_layout
            av_channel_layout_default(&s->used_ch_layout, s->used_ch_layout.nb_channels);   <-- default is AV_CHANNEL_ORDER_NATIVE


  


  Then when I next call swr_convert_frame(), because the frame has the same layout as the audio stream's codec (AV_CHANNEL_ORDER_UNSPEC), and this is different to SwrContext->in_ch_layout (AV_CHANNEL_ORDER_NATIVE), it early exits with ret |= AVERROR_INPUT_CHANGED.

  


  // swresample_frame.c
    int swr_convert_frame(SwrContext *s,
                      AVFrame *out, const AVFrame *in)
    {
        ...
        if ((ret = config_changed(s, out, in)))
            return ret;
        ...


  


      static int config_changed(SwrContext *s,
                            const AVFrame *out, const AVFrame *in)
    {
        ...
        if ((err = av_channel_layout_copy(&ch_layout, &in->ch_layout)) < 0)
            ...
        if (av_channel_layout_compare(&s->in_ch_layout, &ch_layout) || ...) {   <-- This hits the next time I call swr_convert_frame()
            ret |= AVERROR_INPUT_CHANGED;
        }


  


      // channel_layout.c
    int av_channel_layout_compare(const AVChannelLayout *chl, const AVChannelLayout *chl1)
    {
        ...
        // if only one is unspecified -> not equal
        if ((chl->order  == AV_CHANNEL_ORDER_UNSPEC) !=
            (chl1->order == AV_CHANNEL_ORDER_UNSPEC))
            return 1;


  


  If I hardcode the channel layout order of each input AVFrame to AV_CHANNEL_ORDER_NATIVE before resampling, then the resampling and subsequent encoding works, but this feels like a really bad idea and of course wouldn't work as soon as I resample an audio file with a different channel layout.

  


      avcodec_receive_frame(in_codec_context, input_frame);

    AVChannelLayout input_frame_ch_layout;
    av_channel_layout_default(&input_frame_ch_layout, 2 /* = nb_channels*/);
    input_frame->ch_layout = input_frame_ch_layout;
    // Bad idea - but "fixes" my issue!


  


  My questions

  


  What do I need to do to the resampler OR/AND the decoded audio frame to make sure they have the same channel layout order and the resampling works ?

  


  How can I make the channel order of the AVFrames that I get from avcodec_receive_frame() match the input channel order of SwrContext so the resampling works ? My understanding is that the decoded frames should be 'correct' already and I shouldn't need to change any of their values, only values of the output (resampled) frames that I create.

  


  Is there something I need to set on the AVFrame before I resample it ?

  


  Why does the SwrContext choose to change the channel order to AV_CHANNEL_ORDER_NATIVE ?

  


  Note :
A workaround could be to use swr_convert() with the raw data buffer instead of swr_convert_frame(), since it looks like it bypasses this check (since there are no frames involved). I haven't tried this but this shouldn't be necessary and I would like to use swr_convert_frame() as I am working with input and output frames.

  


  Unfortunately I can't find example code using swr_convert_frame() (not even the ffmpeg code seems to ever call it).

  


  My full c++ source code
(error handling omitted for readability) :

  


  std::string fileToUse = "/home/projects/audioFileProject/Audio files/14 Black Cadillacs.wma";
const std::string outputFilename = "out.wav";
const std::string PCMS16BE_encoder_name = "pcm_f32le";

int main()
{
    // Open audio file
    AVFormatContext* in_format_context = avformat_alloc_context();
    avformat_open_input(&in_format_context, fileToUse.c_str(), NULL, NULL);
    avformat_find_stream_info(in_format_context, NULL);
    
    // Get audio stream from file and corresponding decoder
    AVStream* in_stream = in_format_context->streams[0];
    AVCodecParameters* codec_params = in_stream->codecpar;
    const AVCodec* in_codec = avcodec_find_decoder(codec_params->codec_id);
    AVCodecContext *in_codec_context = avcodec_alloc_context3(in_codec);
    avcodec_parameters_to_context(in_codec_context, codec_params);
    avcodec_open2(in_codec_context, in_codec, NULL);

    // Prepare output stream and output encoder (PCM)
    AVFormatContext* out_format_context = nullptr;
    avformat_alloc_output_context2(&out_format_context, NULL, NULL, outputFilename.c_str());
    AVStream* out_stream = avformat_new_stream(out_format_context, NULL);
    const AVCodec* output_codec = avcodec_find_encoder_by_name(PCMS16BE_encoder_name.c_str());
    AVCodecContext* output_codec_context = avcodec_alloc_context3(output_codec);

    // -------------------------------
    
    AVChannelLayout output_ch_layout;
    av_channel_layout_default(&output_ch_layout, 1);    // AV_CHANNEL_LAYOUT_MONO
    output_codec_context->ch_layout = output_ch_layout;
    
    auto out_sample_rate = 16000;
    output_codec_context->sample_rate = out_sample_rate;
    output_codec_context->sample_fmt = output_codec->sample_fmts[0];
    //output_codec_context->bit_rate = output_codec_context->bit_rate;  // TODO Do we need to set the bit rate?
    output_codec_context->time_base = (AVRational){1, out_sample_rate};
    out_stream->time_base = output_codec_context->time_base;

    auto in_sample_rate = in_codec_context->sample_rate;
    AVChannelLayout in_ch_layout = in_codec_context->ch_layout,
                    out_ch_layout = output_ch_layout;   // AV_CHANNEL_LAYOUT_MONO;
    enum AVSampleFormat in_sample_fmt = in_codec_context->sample_fmt,
                        out_sample_fmt = in_codec_context->sample_fmt;

    SwrContext *swr_ctx = nullptr;
    int ret = swr_alloc_set_opts2(&swr_ctx,
                      &out_ch_layout,
                      out_sample_fmt,
                      out_sample_rate,
                      &in_ch_layout,
                      in_sample_fmt,
                      in_sample_rate,
                      0,                    // log_offset
                      NULL);                // log_ctx

    // Probably not necessary - documentation says "This option is
only used for special remapping."
    av_opt_set_chlayout(swr_ctx,    "used_chlayout",     &in_ch_layout, 0);

    // Open output file for writing
    avcodec_open2(output_codec_context, output_codec, NULL);
    avcodec_parameters_from_context(out_stream->codecpar, output_codec_context);
    
    if (out_format_context->oformat->flags & AVFMT_GLOBALHEADER)
        out_format_context->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

    avio_open(&out_format_context->pb, outputFilename.c_str(), AVIO_FLAG_WRITE);
    AVDictionary* muxer_opts = nullptr;
    avformat_write_header(out_format_context, &muxer_opts);

    AVFrame* input_frame = av_frame_alloc();
    AVPacket* in_packet = av_packet_alloc();

    // Loop through decoded input frames. Resample and get resulting samples in a new output frame.
    // I think PCM supports variable number of samples in frames so probably can immediately write out
    while (av_read_frame(in_format_context, in_packet) >= 0) {
        avcodec_send_packet(in_codec_context, in_packet);
        avcodec_receive_frame(in_codec_context, input_frame);

        // I don't want to do this, but it 'fixes' the error where channel layout of input frames
        // doesn't match what the resampler expects - hardcoded the number 2 to fit my sample audio file.
        AVChannelLayout input_frame_ch_layout;
        av_channel_layout_default(&input_frame_ch_layout, 2 /* = nb_channels*/);
        input_frame->ch_layout = input_frame_ch_layout;

        AVFrame* output_frame = av_frame_alloc();
        output_frame->sample_rate = out_sample_rate;
        output_frame->format = out_sample_fmt;
        output_frame->ch_layout = out_ch_layout;
        output_frame->nb_samples = output_codec_context->frame_size;
        
        // TODO Probably need to do maths to calculate new pts properly
        output_frame->pts = input_frame->pts;

        if (swr_convert_frame(swr_ctx, output_frame, input_frame))
            {   logging("Swr Convert failed");  return -1;   }          
            /// ^ Fails here, the second time (since the first time init() is called internally)

        AVPacket *output_packet = av_packet_alloc();
        int response = avcodec_send_frame(output_codec_context, output_frame);

        while (response >= 0) {
            response = avcodec_receive_packet(output_codec_context, output_packet);

            if (response == AVERROR(EAGAIN) || response == AVERROR_EOF) {
                break;
            }

            output_packet->stream_index = 0;
            av_packet_rescale_ts(output_packet, in_stream->time_base, out_stream->time_base);
            av_interleaved_write_frame(out_format_context, output_packet);
        }
        av_packet_unref(output_packet);
        av_packet_free(&output_packet);
        av_frame_unref(input_frame);    // Free references held by the frame before reading new data into it.
        av_frame_unref(output_frame);
    }
    // TODO write last output packet flushing the buffer

    avformat_close_input(&in_format_context);
    return 0;
}


  


• avfilter : do not use AVFrame accessor

  22 avril 2017, par Muhammad Faiz

  avfilter : do not use AVFrame accessor
  Reviewed-by : wm4 <nfxjfg@googlemail.com>
  
  Signed-off-by : Muhammad Faiz <mfcc64@gmail.com>
  

  • [DH] libavfilter/af_afade.c
  • [DH] libavfilter/af_amerge.c
  • [DH] libavfilter/af_apad.c
  • [DH] libavfilter/af_aphaser.c
  • [DH] libavfilter/af_aresample.c
  • [DH] libavfilter/af_ashowinfo.c
  • [DH] libavfilter/af_astats.c
  • [DH] libavfilter/af_biquads.c
  • [DH] libavfilter/af_channelmap.c
  • [DH] libavfilter/af_channelsplit.c
  • [DH] libavfilter/af_dynaudnorm.c
  • [DH] libavfilter/af_join.c
  • [DH] libavfilter/af_pan.c
  • [DH] libavfilter/af_volume.c
  • [DH] libavfilter/af_volumedetect.c
  • [DH] libavfilter/asrc_flite.c
  • [DH] libavfilter/avf_aphasemeter.c
  • [DH] libavfilter/avf_showcqt.c
  • [DH] libavfilter/avf_showspectrum.c
  • [DH] libavfilter/avfilter.c
  • [DH] libavfilter/buffersrc.c
  • [DH] libavfilter/f_drawgraph.c
  • [DH] libavfilter/f_loop.c
  • [DH] libavfilter/f_metadata.c
  • [DH] libavfilter/f_select.c
  • [DH] libavfilter/fifo.c
  • [DH] libavfilter/framepool.c
  • [DH] libavfilter/setpts.c
  • [DH] libavfilter/src_movie.c
  • [DH] libavfilter/vf_bbox.c
  • [DH] libavfilter/vf_blackdetect.c
  • [DH] libavfilter/vf_blackframe.c
  • [DH] libavfilter/vf_colormatrix.c
  • [DH] libavfilter/vf_crop.c
  • [DH] libavfilter/vf_cropdetect.c
  • [DH] libavfilter/vf_deflicker.c
  • [DH] libavfilter/vf_deinterlace_vaapi.c
  • [DH] libavfilter/vf_drawtext.c
  • [DH] libavfilter/vf_eq.c
  • [DH] libavfilter/vf_idet.c
  • [DH] libavfilter/vf_ocr.c
  • [DH] libavfilter/vf_overlay.c
  • [DH] libavfilter/vf_psnr.c
  • [DH] libavfilter/vf_readeia608.c
  • [DH] libavfilter/vf_readvitc.c
  • [DH] libavfilter/vf_scale.c
  • [DH] libavfilter/vf_showinfo.c
  • [DH] libavfilter/vf_ssim.c
  • [DH] libavfilter/vf_swaprect.c
  • [DH] libavfilter/vf_vectorscope.c
  • [DH] libavfilter/vf_waveform.c
  • [DH] libavfilter/vsrc_testsrc.c