Recherche avancée

Sur d’autres sites (9249)

• ffmpeg record screen and save video file to disk as .mp4 or .mpg

  5 janvier 2015, par musimbate

  I want to record the screen of my pc (using gdigrab on my windows machine) and store the saved video file on my disk as an mp4 or mpg file .I have found an example piece of code that grabs the screen and shows it in an SDL window here :http://xwk.iteye.com/blog/2125720 (The code is on the bottom of the page and has an english version) and the ffmpeg muxing example https://ffmpeg.org/doxygen/trunk/muxing_8c-source.html seems to be able to help encode audio and video into a desired output video file.

  I have tried to combine these two by having a format context for grabbing the screen (AVFormatContext *pFormatCtx ; in my code ) and a separate format context to write the desired video file (AVFormatContext *outFormatContextEncoded ;).Within the loop to read packets from the input stream( screen grab stream) I directly encode write packets to the output file as shown in my code.I have kept the SDL code so I can see what I am recording.Below is my code with my modified write_video_frame() function .

  The code builds OK but the output video can’t be played by vlc. When I run the command

  ffmpeg -i filename.mpg

  I get this output

  [mpeg @ 003fed20] probed stream 0 failed
  
  [mpeg @ 003fed20] Stream #0: not enough frames to estimate rate; consider increasing probesize
  
  [mpeg @ 003fed20] Could not find codec parameters for stream 0 (Video: none): unknown codec
  
  Consider increasing the value for the 'analyzeduration' and 'probesize' options
  
  karamage.mpg: could not find codec parameters
  
  Input #0, mpeg, from 'karamage.mpg':
  
    Duration: 19:30:09.25, start: 37545.438756, bitrate: 2 kb/s
  
      Stream #0:0[0x1e0]: Video: none, 90k tbr, 90k tbn
  
  At least one output file must be specified

  Am I doing something wrong here ? I am new to ffmpeg and any guidance on this is highly appreciated.Thank you for your time.

  int main(int argc, char* argv[])
  
  {
  
  
  
      AVFormatContext *pFormatCtx;
  
  
  
      int             i, videoindex;
  
      AVCodecContext  *pCodecCtx;
  
      AVCodec         *pCodec;
  
  
  
      av_register_all();
  
      avformat_network_init();
  
  
  
      //Localy defined structure.
  
      OutputStream outVideoStream = { 0 };
  
  
  
      const char *filename;
  
      AVOutputFormat *outFormatEncoded;
  
      AVFormatContext *outFormatContextEncoded;
  
      AVCodec *videoCodec;
  
  
  
      filename="karamage.mpg";
  
  
  
      int ret1;
  
  
  
      int have_video = 0, have_audio = 0;
  
      int encode_video = 0, encode_audio = 0;
  
  
  
  
  
      AVDictionary *opt = NULL;
  
  
  
  
  
  
  
      //ASSIGN STH TO THE FORMAT CONTEXT.
  
      pFormatCtx = avformat_alloc_context();
  
  
  
      //
  
      //Use this when opening a local file.
  
      //char filepath[]="src01_480x272_22.h265";
  
      //avformat_open_input(&pFormatCtx,filepath,NULL,NULL)
  
  
  
      //Register Device
  
      avdevice_register_all();
  
  
  
      //Use gdigrab
  
      AVDictionary* options = NULL;
  
      //Set some options
  
      //grabbing frame rate
  
      //av_dict_set(&options,"framerate","5",0);
  
      //The distance from the left edge of the screen or desktop
  
      //av_dict_set(&options,"offset_x","20",0);
  
      //The distance from the top edge of the screen or desktop
  
      //av_dict_set(&options,"offset_y","40",0);
  
      //Video frame size. The default is to capture the full screen
  
      //av_dict_set(&options,"video_size","640x480",0);
  
      AVInputFormat *ifmt=av_find_input_format("gdigrab");
  
      if(avformat_open_input(&pFormatCtx,"desktop",ifmt,&options)!=0){
  
          printf("Couldn't open input stream.\n");
  
          return -1;
  
      }
  
  
  
      if(avformat_find_stream_info(pFormatCtx,NULL)<0)
  
      {
  
          printf("Couldn't find stream information.\n");
  
          return -1;
  
      }
  
      videoindex=-1;
  
      for(i=0; inb_streams; i++) 
  
          if(pFormatCtx->streams[i]->codec->codec_type==AVMEDIA_TYPE_VIDEO)
  
          {
  
              videoindex=i;
  
              break;
  
          }
  
      if(videoindex==-1)
  
      {
  
          printf("Didn't find a video stream.\n");
  
          return -1;
  
      }
  
      pCodecCtx=pFormatCtx->streams[videoindex]->codec;
  
      pCodec=avcodec_find_decoder(pCodecCtx->codec_id);
  
      if(pCodec==NULL)
  
      {
  
          printf("Codec not found.\n");
  
          return -1;
  
      }
  
      if(avcodec_open2(pCodecCtx, pCodec,NULL)<0)
  
      {
  
          printf("Could not open codec.\n");
  
          return -1;
  
      }
  
  
  
  
  
      AVFrame *pFrame,*pFrameYUV;
  
      pFrame=avcodec_alloc_frame();
  
      pFrameYUV=avcodec_alloc_frame();
  
  
  
      //PIX_FMT_YUV420P WHAT DOES THIS SAY ABOUT THE FORMAT??
  
      uint8_t *out_buffer=(uint8_t *)av_malloc(avpicture_get_size(PIX_FMT_YUV420P, pCodecCtx->width, pCodecCtx->height));
  
  
  
  
  
      avpicture_fill((AVPicture *)pFrameYUV, out_buffer, PIX_FMT_YUV420P, pCodecCtx->width, pCodecCtx->height);
  
  
  
      //<<<<<<<<<<<-------PREP WORK TO WRITE ENCODED VIDEO FILES-----
  
  
  
      avformat_alloc_output_context2(&outFormatContextEncoded, NULL, NULL, filename);
  
      if (!outFormatContextEncoded) {
  
          printf("Could not deduce output format from file extension: using MPEG.\n");
  
          avformat_alloc_output_context2(&outFormatContextEncoded, NULL, "mpeg", filename);
  
      }
  
      if (!outFormatContextEncoded)
  
          return 1;
  
  
  
      outFormatEncoded=outFormatContextEncoded->oformat;
  
  
  
  
  
       //THIS CREATES THE STREAMS(AUDIO AND VIDEO) ADDED TO OUR OUTPUT STREAM
  
  
  
      if (outFormatEncoded->video_codec != AV_CODEC_ID_NONE) {
  
  
  
          //YOUR VIDEO AND AUDIO PROPS ARE SET HERE.
  
          add_stream(&outVideoStream, outFormatContextEncoded, &videoCodec, outFormatEncoded->video_codec);
  
          have_video = 1;
  
          encode_video = 1;
  
      }
  
  
  
  
  
       // Now that all the parameters are set, we can open the audio and
  
       // video codecs and allocate the necessary encode buffers. 
  
      if (have_video)
  
          open_video(outFormatContextEncoded, videoCodec, &outVideoStream, opt);
  
  
  
       av_dump_format(outFormatContextEncoded, 0, filename, 1);
  
  
  
  
  
        /* open the output file, if needed */
  
      if (!(outFormatEncoded->flags & AVFMT_NOFILE)) {
  
          ret1 = avio_open(&outFormatContextEncoded->pb, filename, AVIO_FLAG_WRITE);
  
          if (ret1 < 0) {
  
              //fprintf(stderr, "Could not open '%s': %s\n", filename,
  
              //        av_err2str(ret));
  
              fprintf(stderr, "Could not open your dumb file.\n");
  
              return 1;
  
          }
  
      }
  
  
  
  
  
      /* Write the stream header, if any. */
  
      ret1 = avformat_write_header(outFormatContextEncoded, &opt);
  
      if (ret1 < 0) {
  
          //fprintf(stderr, "Error occurred when opening output file: %s\n",
  
           //       av_err2str(ret));
  
          fprintf(stderr, "Error occurred when opening output file\n");
  
          return 1;
  
      }
  
  
  
  
  
      //<<<<<<<<<<<-------PREP WORK TO WRITE ENCODED VIDEO FILES-----
  
  
  
      //SDL----------------------------
  
      if(SDL_Init(SDL_INIT_VIDEO | SDL_INIT_AUDIO | SDL_INIT_TIMER)) {  
  
          printf( "Could not initialize SDL - %s\n", SDL_GetError()); 
  
          return -1;
  
      } 
  
      int screen_w=640,screen_h=360;
  
      const SDL_VideoInfo *vi = SDL_GetVideoInfo();
  
      //Half of the Desktop's width and height.
  
      screen_w = vi->current_w/2;
  
      screen_h = vi->current_h/2;
  
      SDL_Surface *screen; 
  
      screen = SDL_SetVideoMode(screen_w, screen_h, 0,0);
  
  
  
      if(!screen) {  
  
          printf("SDL: could not set video mode - exiting:%s\n",SDL_GetError());  
  
          return -1;
  
      }
  
      SDL_Overlay *bmp; 
  
      bmp = SDL_CreateYUVOverlay(pCodecCtx->width, pCodecCtx->height,SDL_YV12_OVERLAY, screen); 
  
      SDL_Rect rect;
  
      //SDL End------------------------
  
      int ret, got_picture;
  
  
  
      AVPacket *packet=(AVPacket *)av_malloc(sizeof(AVPacket));
  
  
  
      //TRY TO INIT THE PACKET HERE
  
       av_init_packet(packet);
  
  
  
  
  
      //Output Information-----------------------------
  
      printf("File Information---------------------\n");
  
      av_dump_format(pFormatCtx,0,NULL,0);
  
      printf("-------------------------------------------------\n");
  
  
  
      struct SwsContext *img_convert_ctx;
  
      img_convert_ctx = sws_getContext(pCodecCtx->width, pCodecCtx->height, pCodecCtx->pix_fmt, pCodecCtx->width, pCodecCtx->height, PIX_FMT_YUV420P, SWS_BICUBIC, NULL, NULL, NULL); 
  
      //------------------------------
  
      //
  
      while(av_read_frame(pFormatCtx, packet)>=0)
  
      {
  
  
  
          if(packet->stream_index==videoindex)
  
          {
  
              //HERE WE DECODE THE PACKET INTO THE FRAME
  
              ret = avcodec_decode_video2(pCodecCtx, pFrame, &got_picture, packet);
  
              if(ret < 0)
  
              {
  
                  printf("Decode Error.\n");
  
                  return -1;
  
              }
  
              if(got_picture)
  
              {
  
  
  
                  //THIS IS WHERE WE DO STH WITH THE FRAME WE JUST GOT FROM THE STREAM
  
                  //FREE AREA--START
  
                  //IN HERE YOU CAN WORK WITH THE FRAME OF THE PACKET.
  
                  write_video_frame(outFormatContextEncoded, &outVideoStream,packet);
  
  
  
  
  
                  //FREE AREA--END
  
                  sws_scale(img_convert_ctx, (const uint8_t* const*)pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameYUV->data, pFrameYUV->linesize);
  
  
  
  
  
                  SDL_LockYUVOverlay(bmp);
  
                  bmp->pixels[0]=pFrameYUV->data[0];
  
                  bmp->pixels[2]=pFrameYUV->data[1];
  
                  bmp->pixels[1]=pFrameYUV->data[2];     
  
                  bmp->pitches[0]=pFrameYUV->linesize[0];
  
                  bmp->pitches[2]=pFrameYUV->linesize[1];   
  
                  bmp->pitches[1]=pFrameYUV->linesize[2];
  
                  SDL_UnlockYUVOverlay(bmp); 
  
                  rect.x = 0;    
  
                  rect.y = 0;    
  
                  rect.w = screen_w;    
  
                  rect.h = screen_h;  
  
                  SDL_DisplayYUVOverlay(bmp, &rect); 
  
                  //Delay 40ms----WHY THIS DELAY????
  
                  SDL_Delay(40);
  
              }
  
          }
  
          av_free_packet(packet);
  
      }//THE LOOP TO PULL PACKETS FROM THE FORMAT CONTEXT ENDS HERE.
  
  
  
      //AFTER THE WHILE LOOP WE DO SOME CLEANING
  
  
  
      //av_read_pause(context);
  
  
  
  
  
       av_write_trailer(outFormatContextEncoded);
  
       close_stream(outFormatContextEncoded, &outVideoStream);
  
       if (!(outFormatContextEncoded->flags & AVFMT_NOFILE))
  
          /* Close the output file. */
  
          avio_close(outFormatContextEncoded->pb);
  
  
  
      /* free the stream */
  
      avformat_free_context(outFormatContextEncoded);
  
  
  
  
  
  
  
      //STOP DOING YOUR CLEANING
  
      sws_freeContext(img_convert_ctx);
  
  
  
  
  
  
  
      SDL_Quit();
  
  
  
      av_free(out_buffer);
  
      av_free(pFrameYUV);
  
      avcodec_close(pCodecCtx);
  
      avformat_close_input(&pFormatCtx);
  
  
  
      return 0;
  
  }
  
  
  
  
  
  
  
  /*
  
   * encode one video frame and send it to the muxer
  
   * return 1 when encoding is finished, 0 otherwise
  
   */
  
  static int write_video_frame(AVFormatContext *oc, OutputStream *ost,AVPacket * pkt11)
  
  {
  
      int ret;
  
      AVCodecContext *c;
  
      AVFrame *frame;
  
      int got_packet = 0;
  
  
  
      c = ost->st->codec;
  
  
  
      //DO NOT NEED THIS FRAME.
  
      //frame = get_video_frame(ost);
  
  
  
      if (oc->oformat->flags & AVFMT_RAWPICTURE) {
  
  
  
          //IGNORE THIS FOR A MOMENT
  
          /* a hack to avoid data copy with some raw video muxers */
  
          AVPacket pkt;
  
          av_init_packet(&pkt);
  
  
  
          if (!frame)
  
              return 1;
  
  
  
          pkt.flags        |= AV_PKT_FLAG_KEY;
  
          pkt.stream_index  = ost->st->index;
  
          pkt.data          = (uint8_t *)frame;
  
          pkt.size          = sizeof(AVPicture);
  
  
  
          pkt.pts = pkt.dts = frame->pts;
  
          av_packet_rescale_ts(&pkt, c->time_base, ost->st->time_base);
  
  
  
          ret = av_interleaved_write_frame(oc, &pkt);
  
      } else {
  
  
  
              ret = write_frame(oc, &c->time_base, ost->st, pkt11);
  
  
  
      }
  
  
  
      if (ret < 0) {
  
          fprintf(stderr, "Error while writing video frame: %s\n");
  
          exit(1);
  
      }
  
  
  
  
  
      return 1;
  
  }

• Resampling audio using FFmpeg API

  13 octobre 2020, par bbdd

  I have a task to decode audio data, re-encode it to another format, and save this encoded data to a buffer. The encoded data that I need to save to the buffer is in AVPacket::data. I save them after this procedure :

  


  

  1. I receive a packet from the input stream

  


  2. I send the packet to the decoder

  


  3. I get the decrypted frame

  


  4. I send it to the encoder

  


  5. I get the encoded packet

  


  6. Save to the buffer

  


  


  All procedures work. But here's the problem. I need to create a "resampling" between points 3 and 4. Before sending data to the encoder, it must pass resampling if required. For example, I get audio data in PCM_ALAW format, with 1 audio channel, and 8000 sample rate. When exiting, I want to get PCM_S32LE, with 2 channels and a sampling rate of 44100. Converting audio format PCM_ALAW to PCM_S32LE works. But I do not know how to implement resampling.

  


  I have an incomplete implementation of the oversampling functions, but I do not know how to put it all together. I was advised this and this example. But I couldn't solve it.

  


  I provide the full code.

  


  audiodecoder.h

  


  class AudioDecoder
{
public:

    AudioDecoder(const AudioDecoderSettings& settings);
    AudioDecoder& operator=(const AudioDecoder& other) = delete;
    AudioDecoder& operator=(AudioDecoder&& other)      = delete;
    AudioDecoder(const AudioDecoder& other)            = delete;
    AudioDecoder(AudioDecoder&& other)                 = delete;
    virtual ~AudioDecoder(void);

    virtual qint32 init(void) noexcept;
    //virtual QByteArray getData(const quint32 &size) noexcept;
    virtual QByteArray get() noexcept;
    virtual qint32 term(void) noexcept;

protected:

    virtual qint32 openInputStream (void) noexcept;
    virtual qint32 openEncoderForStream(void) noexcept;
    virtual qint32 decodeAudioFrame(AVFrame *frame);
    virtual qint32 encodeAudioFrame(AVFrame *frame);
    virtual qint32 initResampler(void);
    virtual qint32 initConvertedSamples(uint8_t ***converted_input_samples, int frame_size);

    class Deleter
    {
    public:
        static void cleanup(AVFormatContext* p);
        static void cleanup(AVCodecContext* p);
        static void cleanup(AudioDecoderSettings* p);
    };

protected:

    bool   m_edf;
    bool   m_initialized{ false };
    qint32 m_streamIndex{ 0 };
    QByteArray                                     m_buffer;
    QScopedPointer p_frmCtx{nullptr};
    QScopedPointer p_iCdcCtx{nullptr};
    QScopedPointer p_oCdcCtx{nullptr};
    QScopedPointer p_settings{nullptr};
    SwrContext *swrCtx;
};


  


  audiodecoder.cpp

  


  static void initPacket(AVPacket *packet)
{
    av_init_packet(packet);
    // Set the packet data and size so that
    // it is recognized as being empty.
    packet->data = nullptr;
    packet->size = 0;
}

static QString error2string(const qint32& code)
{
    if (code < 0) {
        char errorBuffer[255]{ '0' };
        av_strerror(code, errorBuffer, sizeof(errorBuffer));
        return QString(errorBuffer);
    }
    return QString();
}

static void printErrorMessage(const QString &message, const qint32 &code = 0)
{
    qDebug() << "AudioDecoder: " << message << error2string(code);
}

static qint32 initInputFrame(AVFrame **frame)
{
    if (!(*frame = av_frame_alloc())) {
        printErrorMessage(QString("Could not allocate input frame"));
        return AVERROR(ENOMEM);
    }
    return 0;
}

void AudioDecoder::Deleter::cleanup(AVFormatContext* p)
{
    if (p) {
        avformat_close_input(&p);
    }
}

void AudioDecoder::Deleter::cleanup(AVCodecContext* p)
{
    if (p) {
        avcodec_free_context(&p);
    }
}

void AudioDecoder::Deleter::cleanup(AudioDecoderSettings* p)
{
    if (p) {
        delete p;
    }
}

AudioDecoder::AudioDecoder(const AudioDecoderSettings& settings)
    : m_edf(false),
      m_initialized(false)
    , m_streamIndex(0)
    , p_frmCtx(nullptr)
    , p_iCdcCtx(nullptr)
    , p_oCdcCtx(nullptr)
    , p_settings(new AudioDecoderSettings(settings))
{
    av_register_all();
    avcodec_register_all();
}

qint32 AudioDecoder::openInputStream(void) noexcept
{
    qint32           error  = -1;
    AVCodecContext  *avctx  = nullptr;
    AVFormatContext *frmCtx = nullptr;

    // Open the input file to read from it.
    if ((error = avformat_open_input(&frmCtx,
            p_settings->inputFile().toStdString().c_str(), nullptr, nullptr)) < 0) {
        frmCtx = nullptr;
        printErrorMessage(QString("Could not open input file '%1' (error '%2')")
                          .arg(p_settings->inputFile()
                          .arg(error2string(error))));
        return error;
    }

    // Get information on the input file (number of streams etc.).
    if ((error = avformat_find_stream_info(frmCtx, nullptr)) < 0) {
        printErrorMessage(QString("Could not open find stream info (error '%1')")
                          .arg(error2string(error)));
        avformat_close_input(&frmCtx);
        return error;
    }

    // Find audio stream index
    auto getAudioStreamIndex = [](AVFormatContext *frmCtx) -> qint32
    {
        if (frmCtx->nb_streams != 1) {
            for (quint32 i = 0; i < frmCtx->nb_streams; ++i) {
                if (frmCtx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) {
                    return i;
                }
            }
        }
        return 0;
    };

    if (frmCtx->streams[m_streamIndex =
            getAudioStreamIndex(frmCtx)]->codecpar->codec_type != AVMEDIA_TYPE_AUDIO) {
        avformat_close_input(&frmCtx);
        printErrorMessage(QString("The audio stream was not found"));
        return -1;
    }

    AVCodec *codec = nullptr;
    // Find a decoder for the audio stream.
    if (!(codec = avcodec_find_decoder(
              frmCtx->streams[m_streamIndex]->codecpar->codec_id))) {
        printErrorMessage(QString("Could not find input codec"));
        avformat_close_input(&frmCtx);
        return -1;
    }

    // Allocate a new decoding context.
    avctx = avcodec_alloc_context3(codec);
    if (!avctx) {
        printErrorMessage(QString("Could not allocate a decoding context"));
        avformat_close_input(&frmCtx);
        return AVERROR(ENOMEM);
    }

    // Initialize the stream parameters with demuxer information.
    error = avcodec_parameters_to_context(
                avctx, frmCtx->streams[m_streamIndex]->codecpar);
    if (error < 0) {
        avformat_close_input(&frmCtx);
        avcodec_free_context(&avctx);
        return error;
    }

    // Open the decoder for the audio stream to use it later.
    if ((error = avcodec_open2(avctx, codec, nullptr)) < 0) {
        printErrorMessage(QString("Could not open input codec: "), error);
        avcodec_free_context(&avctx);
        avformat_close_input(&frmCtx);
        return error;
    }

    // Save the decoder context for easier access later.
    p_iCdcCtx.reset(avctx);
    p_frmCtx.reset(frmCtx);

    // Print detailed information about the input format
    av_dump_format(p_frmCtx.data(), 0,
                   p_settings->inputFile().toStdString().c_str(), 0);
    return 0;
}

AudioDecoder::~AudioDecoder(void)
{
    term();
}

qint32 AudioDecoder::term(void) noexcept
{
    if (!m_initialized) {
        return -1;
    }
    if (p_frmCtx   != nullptr) {
        p_frmCtx.reset();
    }
    if (p_iCdcCtx  != nullptr) {
        p_iCdcCtx.reset();
    }
    if (p_oCdcCtx  != nullptr) {
        p_oCdcCtx.reset();
    }
    if (p_settings != nullptr) {
        p_settings.reset();
    }
    m_initialized = false;
    return (p_frmCtx && p_iCdcCtx && p_oCdcCtx && p_settings) ? -1 : 0;
}

qint32 AudioDecoder::init(void) noexcept
{
    if (m_initialized) {
        return 0;
    }
    if (p_settings->inputFile().isEmpty()) {
        return -1;
    }
    if (p_settings->audioCodec().isEmpty()) {
        return -1;
    }
    if (openInputStream() < 0) {
        return -1;
    }
    if (openEncoderForStream() < 0) {
        return -1;
    }
    if (initResampler() < 0) {
        return -1;
    }

    m_initialized = true;
    return 0;
}

qint32 AudioDecoder::openEncoderForStream(void) noexcept
{
    AVCodecContext *avctx = nullptr;
    AVCodec        *codec = nullptr;
    qint32          error = 0;

    // Set the basic encoder parameters.
    const quint32 sampleRate   = p_settings->sampleRate()   > 0
            ? p_settings->sampleRate()   : p_iCdcCtx->sample_rate;
    const quint16 channelCount = p_settings->channelCount() > 0
            ? p_settings->channelCount() : p_iCdcCtx->channels;
    const quint32 constBitRate = p_settings->constBitRate() > 0
            ? p_settings->constBitRate() : p_iCdcCtx->bit_rate;
    const QString encodeName   = p_settings->audioCodec() == "copy"
            ? QString(p_iCdcCtx->codec->name) : p_settings->audioCodec();

    if (!(codec = avcodec_find_encoder_by_name(
              encodeName.toStdString().c_str()))) {
        printErrorMessage(QString(
            "Could not find an %1 encoder").arg(p_settings->audioCodec()));
        return -1;
    }

    avctx = avcodec_alloc_context3(codec);
    if (!avctx) {
        printErrorMessage(QString("Could not allocate an encoding context"));
        avcodec_free_context(&avctx);
        return -1;
    }

    if (!codec->sample_fmts) {
        avcodec_free_context(&avctx);
        return -1;
    }

    avctx->channels              = channelCount;
    avctx->channel_layout        = av_get_default_channel_layout(channelCount);
    avctx->sample_rate           = sampleRate;
    avctx->bit_rate              = constBitRate;
    avctx->sample_fmt            = codec->sample_fmts[0];
    // Set the sample rate for the container.
    avctx->time_base.den         = sampleRate;
    avctx->time_base.num         = 1;
    // Allow the use of the experimental encoder.
    avctx->strict_std_compliance = FF_COMPLIANCE_EXPERIMENTAL;

    // Open the encoder for the audio stream to use it later.
    if ((error = avcodec_open2(avctx, codec, nullptr)) < 0) {
        printErrorMessage(QString("Could not open output codec (error '%1')")
                          .arg(error2string(error)));
        avcodec_free_context(&avctx);
        return -1;
    }

    p_oCdcCtx.reset(avctx);
    return 0;
}

qint32 AudioDecoder::decodeAudioFrame(AVFrame *frame)
{
    // Packet used for temporary storage.
    AVPacket input_packet;
    qint32 error = 0;
    initPacket(&input_packet);

    // Read one audio frame from the input file into a temporary packet.
    if ((error = av_read_frame(p_frmCtx.data(), &input_packet)) < 0) {
        // If we are at the end of the file, flush the decoder below.
        if (error == AVERROR_EOF) {
            m_edf = true;
            return 0;
        }
        else {
            printErrorMessage(QString("Could not read frame (error '%1')")
                              .arg(error2string(error)));
            return error;
        }
    }

    if (input_packet.stream_index != m_streamIndex) {
        av_packet_unref(&input_packet);
        return -1;
    }

    // Send the audio frame stored in the temporary packet to the decoder.
    // The input audio stream decoder is used to do this.
    if ((error = avcodec_send_packet(p_iCdcCtx.data(), &input_packet)) < 0) {
        printErrorMessage(QString("Could not send packet for decoding (error '%1')")
                          .arg(error2string(error)));
        return error;
    }

    // Receive one frame from the decoder.
    error = avcodec_receive_frame(p_iCdcCtx.data(), frame);
    // If the decoder asks for more data to be able to decode a frame,
    // return indicating that no data is present.

    if (error == AVERROR(EAGAIN)) {
        error = 0;
    // If the end of the input file is reached, stop decoding.
    } else if (error == AVERROR_EOF) {
        m_edf = true;
        error = 0;
    } else if (error < 0) {
        printErrorMessage(QString("Could not decode frame (error '%1')")
                          .arg(error2string(error)));
    } else {
        error = 0;
    }
    av_packet_unref(&input_packet);
    return error;
}

qint32 AudioDecoder::encodeAudioFrame(AVFrame *frame)
{
    /* Packet used for temporary storage. */
    AVPacket output_packet;
    int error;
    initPacket(&output_packet);
    // Send the audio frame stored in the temporary packet to the encoder.
    // The output audio stream encoder is used to do this.
    error = avcodec_send_frame(p_oCdcCtx.data(), frame);
    // The encoder signals that it has nothing more to encode.
    if (error == AVERROR_EOF) {
        error = 0;
    } else if (error < 0) {
        printErrorMessage(QString("Could not send packet for encoding (error '%1')")
                          .arg(error2string(error)));
    }
    else {

        // Receive one encoded frame from the encoder.
        error = avcodec_receive_packet(p_oCdcCtx.data(), &output_packet);
        // If the encoder asks for more data to be able to provide an
        // encoded frame, return indicating that no data is present.
        if (error == AVERROR(EAGAIN)) {
            error = 0;
        /* If the last frame has been encoded, stop encoding. */
        } else if (error == AVERROR_EOF) {
            error = 0;
        } else if (error < 0) {
            printErrorMessage(QString("Could not encode frame (error '%1')")
                              .arg(error2string(error)));
        } else {

            // Copy packet
            // output_packet.pts      = av_rescale_q_rnd(output_packet.pts,  p_iCdcCtx->time_base, p_oCdcCtx->time_base, (enum AVRounding) (AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX) );
            // output_packet.dts      = av_rescale_q_rnd(output_packet.dts,  p_iCdcCtx->time_base, p_oCdcCtx->time_base, (enum AVRounding) (AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX) );
            // output_packet.duration = av_rescale_q(output_packet.duration, p_iCdcCtx->time_base, p_oCdcCtx->time_base);
            // output_packet.pos      = -1;

            // Save decoded - encoded audio data
            for (int i = 0; i < output_packet.size; ++i) {
                m_buffer.push_back(output_packet.data[i]);
            }
        }
    }
    av_packet_unref(&output_packet);
    return error;
}

QByteArray AudioDecoder::get() noexcept
{
    AVFrame *frame = nullptr;
    if (initInputFrame(&frame) < 0) {
        return m_buffer;
    }

    while (!m_edf) {
        if (decodeAudioFrame(frame) < 0) {
            av_frame_free(&frame);
            return m_buffer;
        }

        // ????
        uint8_t **converted_input_samples = nullptr;
        if (initConvertedSamples(&converted_input_samples, frame->nb_samples) < 0) {
            if (converted_input_samples) {
                av_freep(&converted_input_samples[0]);
                free(converted_input_samples);
            }
            av_frame_free(&frame);
            return {};
        }



        if (encodeAudioFrame(frame) < 0) {
            av_frame_free(&frame);
            return m_buffer;
        }
        av_frame_unref(frame);
    }
    av_frame_free(&frame);
    return m_buffer;
}

qint32 AudioDecoder::initResampler(void)
{
    qint32 error = 0;
    // Create a resampler context for the conversion.
    // Set the conversion parameters. Default channel layouts based on the number of channels
    // are assumed for simplicity (they are sometimes not detected properly by the demuxer and/or decoder).
    swrCtx = swr_alloc_set_opts(
                nullptr,
                av_get_default_channel_layout(p_oCdcCtx->channels), p_oCdcCtx->sample_fmt, p_oCdcCtx->sample_rate,
                av_get_default_channel_layout(p_iCdcCtx->channels), p_iCdcCtx->sample_fmt, p_iCdcCtx->sample_rate,  0,
                nullptr);
    if (!swrCtx) {
        printErrorMessage(QString("Could not allocate resample context"));
        return AVERROR(ENOMEM);
    }

    // Perform a sanity check so that the number of converted samples is
    // not greater than the number of samples to be converted.
    // If the sample rates differ, this case has to be handled differently
    av_assert0(p_oCdcCtx->sample_rate == p_iCdcCtx->sample_rate);

    // Open the resampler with the specified parameters.
    if ((error = swr_init(swrCtx)) < 0) {
        printErrorMessage(QString("Could not open resample context"));
        swr_free(&swrCtx);
        return error;
    }
    return 0;
}

qint32 AudioDecoder::initConvertedSamples(uint8_t ***converted_input_samples, int frame_size)
{
    qint32 error = 0;
    // Allocate as many pointers as there are audio channels.
    // Each pointer will later point to the audio samples of the corresponding
    // channels (although it may be NULL for interleaved formats).
    if (!(*converted_input_samples =
            (uint8_t **) calloc(p_oCdcCtx->channels, sizeof(**converted_input_samples)))) {
        printErrorMessage("Could not allocate converted input sample pointers");
        return AVERROR(ENOMEM);
    }
    
    // Allocate memory for the samples of all channels in one consecutive
    // block for convenience
    if ((error = av_samples_alloc(
             *converted_input_samples,
             nullptr,
             p_oCdcCtx->channels,
             frame_size,
             p_oCdcCtx->sample_fmt,
             0)) < 0) {

        printErrorMessage(QString("Could not allocate converted input samples (error '%1')")
                          .arg(error2string(error)));
        av_freep(&(*converted_input_samples)[0]);
        free(*converted_input_samples);
        return error;
    }
    return 0;
}


  


  This is where I have to implement data resampling before sending it to the encoder :

  


  QByteArray AudioDecoder::get() noexcept
{
    AVFrame *frame = nullptr;
    if (initInputFrame(&frame) < 0) {
        return m_buffer;
    }

    while (!m_edf) {
        if (decodeAudioFrame(frame) < 0) {
            av_frame_free(&frame);
            return m_buffer;
        }

        // ???
        // ???
        // ???
        // This is where I have to implement data 
        // resampling before sending it to the encoder
        // ???
        // ???
        // ???

        if (encodeAudioFrame(frame) < 0) {
            av_frame_free(&frame);
            return m_buffer;
        }
        av_frame_unref(frame);
    }
    av_frame_free(&frame);
    return m_buffer;
}


  


• Formatting ffmpeg arguments correctly in Swift

  22 octobre 2019, par NCrusher

  I deleted my previous post about this because I’ve gone through a lot more trial and error on it and I needed to be sure I was giving current and relevant information.

  I’m trying to create a very simple audio conversion app for MacOS using ffmpeg. Because it’s geared at audiobooks, the audio options are pretty basic.

  func conversionSelection() {
  
      if inputFileUrl != nil {
  
          let conversionChoice = conversionOptionsPopup.indexOfSelectedItem
  
          switch conversionChoice {
  
              case 1 :
  
                  outputExtension = ".mp3"
  
                  ffmpegFilters = ["-c:a libmp3lame", "-ac 1", "-ar 22050", "-q:a 9"]
  
              case 2 :
  
                  outputExtension = ".mp3"
  
                  ffmpegFilters = ["-c:a libmp3lame", "-ac 2", "-ar 44100", "-q:a 5"]
  
              case 3 :
  
                  outputExtension = ".mp3"
  
                  ffmpegFilters = ["-c:a libmp3lame", "-ac 1", "-ar 22050", "-b:a 32k"]
  
              case 4:
  
                  outputExtension = ".flac"
  
                  ffmpegFilters = ["-c:a flac"]
  
              default :
  
                  outputExtension = ".m4b"
  
                  ffmpegFilters = ["-c copy"]
  
          }
  
      }
  
  }

  I don’t want to inundate this post with code, but I’m not sure what all is necessary here for people to help me troubleshoot this problem. This is the code that allows me to get my input and output paths set up :

  func updateOutputText(outputString: String, outputField: NSTextField) {
  
      if inputFileUrl != nil && outputDirectoryUrl == nil {
  
          // derive output path and filename from input and tack on a new extension if a different conversion format is chosen
  
          let outputFileUrl = inputFileUrl!.deletingPathExtension()
  
          let outputPath = outputFileUrl.path
  
          outputFilePath = outputPath + "\(outputExtension)"
  
      } else if inputFileUrl != nil && outputDirectoryUrl != nil {
  
          // derive output directory from outputBrowseButton action, derive filename from input file, and derive output format from conversionSelection
  
          let outputFile = inputFileUrl!.deletingPathExtension()
  
          let outputFilename = outputFile.lastPathComponent
  
          let outputDirectory = outputDirectoryUrl!.path
  
          outputFilePath = outputDirectory + "/" + outputFilename + "\(outputExtension)"
  
      }
  
      outputTextDisplay.stringValue = outputFilePath
  
  }
  
  
  
  // update input and output text fields
  
  func updateInputText(inputString: String, inputField: NSTextField) {
  
      conversionSelection()
  
      inputTextDisplay.stringValue = inputFilePath
  
  }

  Everything on the Swift side appears to be working. The input and output Browse buttons work fine. The input and output file paths are written to text fields exactly as they should be. When I select a different conversion option, it updates the file extension for my output file.

  Here are my methods for actually launching ffmpeg :

  func ffmpegConvert(inputPath: String, filters: String, outputPath: String) {
  
      guard let launchPath = Bundle.main.path(forResource: "ffmpeg", ofType: "") else { return }
  
      do {
  
          let convertTask: Process = Process()
  
          convertTask.launchPath = launchPath
  
          convertTask.arguments = [
  
              "-i", inputPath,
  
              filters,
  
              outputPath
  
          ]
  
          convertTask.standardInput = FileHandle.nullDevice
  
          convertTask.launch()
  
          convertTask.waitUntilExit()
  
      }
  
  }
  
  @IBAction func startConversionClicked(_ sender: Any) {
  
      ffmpegConvert(inputPath: inputFilePath, filters: ffmpegFilters.joined(), outputPath: "outputFilePath")
  
  }

  The errors are coming from ffmpeg. But I’m fairly certain the PROBLEM is that I haven’t figured out how to write them so that Swift will pass them on to ffmpeg properly.

  If I use the argument arrays exactly as they are formatted above, these are my results :

  When I choose the default option (.m4b, "-c copy") I get this error :

  Unrecognized option ’c copy’. Error splitting the argument list :
  Option not found

  If I choose any of the other options (mp3 or flac), I get both a warning that reads

  Trailing options were found on the commandline.

  Then it will actually read in the metadata of my input file (so I know my input path works at least) and then it will tell me :

  At least one output file must be specified

  So while it’s reading my input file correctly, perhaps it’s not reading my output file path ?

  Moving on.

  I then put double hyphens in the argument strings, thinking that perhaps the reason ffmpeg was reading "-c copy" as "c copy" is because the hyphen is a mathematical operator. My results are as follows :

  (Still doesn’t read metadata)
  Unrecognized option ’-c copy’.
  Error splitting the argument list : Option not found

  (choosing .mp3 output options)
  (doesn’t read metadata this time)
  Unrecognized option ’-c:a libmp3lame—ac 1—ar 22050—q:a 9’.
  Error splitting the argument list : Option not found

  (choosing .flac output option)
  (doesn’t read metadata this time)
  Unrecognized option ’-c:a flac’.
  Error splitting the argument list : Option not found

  So. No help there. Time for a different approach.

  This time, I added a whitespace in front of my arguments. Results :

  (reads metadata)
  [NULL @ 0x106800000] Unable to find a suitable output format for ’ -c copy’
  -c copy : Invalid argument

  (reads metadata)
  [NULL @ 0x10b004400] Unable to find a suitable output format for ’ -c:a libmp3lame -ac 1 -ar 22050 -q:a 9’
  -c:a libmp3lame -ac 1 -ar 22050 -q:a 9 : Invalid argument

  (reads metadata)
  NULL @ 0x10580c400] Unable to find a suitable output format for ’ -c:a flac’
  -c:a flac : Invalid argument

  So, again, I have fairly decent confirmation that it’s reading my input file correctly, but possibly not my output file. Which might make sense in that the process of stringing together my output filepath is a lot more complex than the process for my input file, but I have visual confirmation that my output file path is accurate displayed in my outputTextDisplay box, which displays the outputFilePath perfectly.

  I really have no idea what isn’t working here. I’m sorry for the long post, but I’m not able to narrow the pertinent information down any further.