Recherche avancée

Sur d’autres sites (11727)

• Playing MP3 with FFMPEG Library

  13 février 2017, par Keith Chambers

  I am attempting to make a C program that will play an MP3 file (Using FFMPEG & SDL).

  Here’s the general gist of what I’m try to do, based on my understanding thus far.

  1. Open the file + load ’container’ level data into a AVFormatContext
  2. Find the stream inside the container that you want to work with. In my case I believe there is only a single audio steam so format_context->streams[0] references this.
  3. Fetch a decoder(Stored in AVcodec), by the Codec ID found in the stream and populate AVCodecContext
  4. Set up your desired SDL_AudioSpec struct. Notably here you pass it a callback function for filling up an audio buffer provided by SDL itself and your AVCodecContext which is passed to the callback.
  5. Open an SDL audio device using your wanted spec and call SDL_PauseAudioDevice() which creates a new thread to continually call the callback function (start the audio playback).
  6. The main thread continues and continually reads encoded packets/frames and puts them on a global queue until the stream data has been completely processed.
  7. The callback function dequeues the encoded packets, decodes them and writes them to the buffer passed by SDL audio as a parameter.

  Questions :

  1. What exactly is the difference between a frame and a packet in the context of audio ?

     My understanding is that frames are encapsulated packets that are easier to feed as raw data since they are more uniform in size. A frame may require multiple packets to fill it but multiple frames cannot be made from a single packet(I could have mixed that up, as I don’t remember where I read that). Are packets always decoded into frames or are the terms somewhat interchangeable ?

  2. Where is the encoded stream of packets stored ? In my code I’m using av_read_frame(AVFormatContext*, AVPacket) to somehow fetch a packet from somewhere so I can put the packet on my queue.

  3. Lots of things about the callback function..

     `void audio_callback(void *userdata, Uint8 *stream, int len)`

     1. Just to confirm, stream and len are decided by SDL audio ? The programmer has no way of passing values for these arguments ?
     2. Len is the size, in bytes of decoded audio data that SDL Audio is requesting ?
     3. How do I know how many frames is enough to write len data to the buffer ? To my knowledge a frame is an encapsulation of a FIXED amount of decoded audio data, so how do I know how much data each frame contains ?
     4. avcodec_send_packet(AVCodecContext *, AVPacket *) and avcodec_receive_frame(AVCodecContext *, AVFrame *) are the new ways to decode data, how exactly do they work ?

  So it seems like you pass in a valid packet to avcodec_send_packet(), it decodes it and then to get the corresponding frame you call the avcodec_receive_frame().

  Issues that I can see with this is that size-wise a packet mightn’t correspond to a frame, a number of packets might make up a single frame.

  OK here’s the stripped down version of my code, I can link to the full thing if required.

  I omitted all error checking, clean up and the implementation of the PacketQueue structure since it’s pretty intuitive. I haven’t tested this specifically but my original program is segfaulting when it gets to avcodec_send_packet()

  // Includes etc ommited
  
  
  
  // My own type, just assume it works and was allocated in main
  
  PacketQueue *audioq;
  
  
  
  void audio_callback(void *userdata, Uint8 *stream, int requestedLen)
  
  {
  
      AVCodecContext *codec_context = (AVCodecContext *)userdata;
  
      AVPacket * nextPacket;
  
      AVFrame outputFrame;
  
  
  
      // Take packet off of audioq Queue and store in nextPacket
  
      deQueue(audioq, nextPacket);
  
      avcodec_send_packet(codec_context, nextPacket);
  
      avcodec_receive_frame(codec_context, &outputFrame);
  
  
  
      memcpy(stream, (uint8_t *)outputFrame.data, outputFrame.sample_rate);
  
  }
  
  
  
  int main(int argc, char *argv[])
  
  {
  
  
  
      AVCodec * audio_codec;
  
      AVCodecContext *codec_context;
  
      AVFormatContext *format_context = NULL;
  
      SDL_AudioSpec want;
  
      SDL_AudioSpec have;
  
      SDL_AudioDeviceID audio_device_id;
  
      const char* audio_device_name = NULL;
  
  
  
      SDL_Init(SDL_INIT_AUDIO | SDL_INIT_TIMER);
  
      av_register_all();
  
  
  
      format_context = avformat_alloc_context();
  
      avformat_open_input(&format_context, *(argv + 1), NULL, NULL);
  
      avformat_find_stream_info(format_context, NULL);
  
  
  
      audio_codec = avcodec_find_decoder(format_context->streams[0]->codecpar->codec_id);
  
      codec_context = avcodec_alloc_context3(audio_codec);
  
  
  
      avcodec_open2(codec_context, audio_codec, NULL);
  
  
  
      want.freq = codec_context->sample_rate;
  
      want.format = AUDIO_S16SYS;
  
      want.channels = 2;
  
      want.samples = SDL_AUDIO_BUFFER_SIZE;
  
      want.callback = audio_callback;
  
      want.userdata = codec_context;
  
      want.silence = 0;
  
  
  
      audio_device_id = SDL_OpenAudioDevice(NULL, 0, &want, &have, SDL_AUDIO_ALLOW_FORMAT_CHANGE);
  
      audio_device_name = SDL_GetAudioDeviceName(0, 0);
  
  
  
      SDL_PauseAudioDevice(audio_device_id, 0);
  
  
  
      while(quit == 0)
  
      {
  
          AVPacket* packet = malloc(sizeof(AVPacket));
  
          av_read_frame(format_context, packet);
  
          // Put packet onto audioq Queue
  
          enQueue(audioq, packet);
  
  
  
          if(packet)
  
              free(packet);
  
      }
  
      // Cleanup
  
  
  
      return 0;
  
  }

  I’m sure my callback function is highly simplistic and somewhat naive but after I replaced the old avcodec_decode_audio4() much of the code I had seem irrelevant(Don’t worry the program wasn’t working at that point either..)

• FFmpeg : H.265/HEVC - does not work on Sony Smart-TV if you change encoder

  25 juillet 2019, par oigen90

  We have such Sony Bravia Smart-TV devices :

  • Sony KD-49XD7005 (2016) - Android
  • Sony KD-49XE7096 (2017) - Linux

  Those devices does have HEVC support, as well as 4K support.

  Also, we have a one minute Apple ProRes 4K video file. I need to transcode it into HEVC with some simple params, create DASH manifest and stream it into HTML5-player.
  The problem is that if I use the regular ’libx265’ codec passing it after "-c:v" - the video does not play on Sony Bravia (just freezes on ’loadstart’ player’s event). But if I use ’hevc_nvenc’ - it works ! What’s the reason of such behavior ? Both of codecs should work identically, hevc_nvenc just should be faster and was made to work on nvidia graphic cards (that’s all I know about those codecs’ difference so far). But in fact we have working stream encoded into hevc_nvenc and invalid stream encoded into libx265.

  See my FFmpeg command below.

  Which thoughts do you have ? If you want me to provide some more info - feel free to ask.

  ffmpeg -i input.mov -c:v hevc_nvenc -pix_fmt yuv420p -b:v 12000k -maxrate 14000k -bufsize 6000k -c:a copy -f mp4 output.mp4

• ffmpeg errors when playing a playlist - Non monotonous DTS

  20 novembre 2023, par Evilmachine

  i have a strange bug in ffmpeg. I am using ffmpeg to stream Twitch VODs and CLIPs to a 24/7 twitch channel. If i stream the files one by one its working. If i am streaming a playlist containing vods and clips i get this error when the shorter clips are playing :

  


  [flv @ 0x555677050c60] Non-monotonous DTS in output stream 0:0; previous: 56867, current: 31846; changing to 56867. This may result in incorrect timestamps in the output file.

  


  and the video stops. Once the playlist has a longer vod again it is working.

  


  here is my code that i use to stream :

  


  while [ 1 -eq 1 ]
do
ffmpeg  -fflags +igndts -re -f concat -safe 0 -i playlist.txt -codec copy -flvflags no_duration_filesize -f flv rtmp://XXXXXXXXX
done


  


  If i look into ffprobe the files seems both to be normal :

  


  VODs

  


  ffprobe version 5.1.3-1+rpt4 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr --extra-version=1+rpt4 --toolchain=hardened --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --disable-mmal --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sand --enable-sdl2 --disable-sndio --enable-libjxl --enable-neon --enable-v4l2-request --enable-libudev --enable-epoxy --libdir=/usr/lib/aarch64-linux-gnu --arch=arm64 --enable-pocketsphinx --enable-librsvg --enable-libdc1394 --enable-libdrm --enable-vout-drm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Speedrun - Goof Troop - Coop - @olli_wan und luk30994 -- Disney's Goof Troop.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2023-10-17T11:46:25.000000Z
  Duration: 00:28:23.02, start: 0.000000, bitrate: 1004 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 873 kb/s, 30 fps, 30 tbr, 15360 tbn (default)
    Metadata:
      creation_time   : 2023-10-17T11:46:25.000000Z
      handler_name    : ISO Media file produced by Google Inc. Created on: 10/17/2023.
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2023-10-17T11:46:25.000000Z
      handler_name    : ISO Media file produced by Google Inc. Created on: 10/17/2023.
      vendor_id       : [0][0][0][0]


  


  CLIPS

  


  ffprobe version 5.1.3-1+rpt4 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr --extra-version=1+rpt4 --toolchain=hardened --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --disable-mmal --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sand --enable-sdl2 --disable-sndio --enable-libjxl --enable-neon --enable-v4l2-request --enable-libudev --enable-epoxy --libdir=/usr/lib/aarch64-linux-gnu --arch=arm64 --enable-pocketsphinx --enable-librsvg --enable-libdc1394 --enable-libdrm --enable-vout-drm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Clip -- Luk - Terra - Lebensweisheiten mit Lukas Thema Smart.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
  Duration: 00:00:05.00, start: 0.000000, bitrate: 5520 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, unknown/bt709/unknown, progressive), 1280x720, 5374 kb/s, 60.20 fps, 60 tbr, 15360 tbn (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]


  


  Has someone a hint why this error happens and how i can avoid them ?
Is gstreamer better for this approach ?

  


  Thanks a lot in Advance.

  


  I tried changing the files.