Recherche avancée

Sur d’autres sites (7607)

• converting a "gif" to video using swift

  3 décembre 2019, par James Woodrow

  I’ve looked around and found a few things here and there, mainly that I should be using AVAssetWriter to do this but I have 0 experience with this and video editing/creation so it doesn’t help me much since I can’t seem to find anything that does something I can modify easily (or not at my level of knowledge at least) so that it works as I intend it to.

  I have an app which takes n photos every cft (capture frame time which I get from a backend server) seconds (it’s a double for obvious reasons) I then display these frames using a UIImageView and the frames change every dft (display frame time which I also get from a backend server and can be different from cft). Up until this point nothing complicated.

  now what is currently the workflow is that these frames are sent back to a server with any relevant information I want and then the server would use imagemagick to create a real gif file and ffmpeg to create a 15 seconds video using said gif.

  the issue is this makes it so that my heroku server bills aren’t as low as I would like because of the limited memory on the dynos and the time it takes to generate these videos is of about 5-10 seconds I believe (not sure but it’s longer than I’d like)

  So the idea I had was to make the app create the video since he already has all the information he needs for this, and then simply upload it with the rest of the frames and relevant data. Using bandwidth nowadays is much cheaper than buying extra processing power on a server.

  • he has n frames to loop over
  • he has a float value representing how long each frame should last dft
  • he has a gpu or at least a much better cpu than the dynos heroku have to offer

  I’ve also looked around to see if anyone made an extensive tutorial on how to use ffmpeg in swift but I still didn’t find anything at my level and I didn’t even find a tutorial per se, only some GitHub projects which were partially completed and/or without the original tutorial linked to understand the thought process.

  I would appreciate any tips/code sample/tutorials on the subject.

  I’m adding the ffmpeg command line equivalent to what I would love to be able to do (if I could use ffmpeg directly with iOS this could be nice too)

  ffmpeg -framerate 100/13 -loop 1 -i frame%02d.png -c:v libx264 -r 100/13 -pix_fmt yuv420p -t 0:15 instagram.mp4

  where basically I did 100 / (dft * 100) for the input frame rate and just output at the same fps for 15 seconds. by the way if there are any ways to optimise this command to make it run faster without losing quality I might be able to keep the current way of functioning with heroku although I would still prefer some iOS solution.

• ffmpeg ; opus encoded sound in webm does not work with ffplay or YouTube, only VLC [on hold]

  2 août 2017, par Mockarutan

  I’m having trouble getting Opus encoded sound in the webm container to work. I’m using libopus in ffmpeg.

  The file does work in VLC. But not in ffplay or on YouTube. If I take the raw wav data in a wav file and then convert it to Opus/webm with the ffmpeg.exe that comes pre-compiled. It works in VLC, ffplay and YouTube.

  So ffmpeg can obviously do it correctly, I must be doing something wrong in my code.

  The file my code produces : https://drive.google.com/file/d/0B16rIXjPXJCqcU5HVllIYW1iODg/view?usp=sharing

  Edit, More details that I forgot in my frustration : The file can be opened by ffplay and uploaded to youtube (when I interlace it with VP9 video). But the sound is just "ticks", example : https://www.youtube.com/watch?v=j_ShBbuizeo&feature=youtu.be

  I have read though all example codes that I know of from ffmpeg, but all of them is in the old API, not the send/receive api, so a big part of the code does not apply anymore. This code works with all other Codes I’ve tested, including H.264+AAC in mp4, VP8+Opus in ogg and raw PCM F32LE in wav. I would have gone with VP8+Opus in ogg if the license was as straight forward as the webm license

  I’ve looked though the source for the ffmpeg.exe command line tool and coped everything applicable in to my code base.

  (Edit 3, reduced the code as much as I can)
  Here is my code : https://pastebin.com/HTuc0g8K

  Setup :

  int initialize(int sample_rate, int per_frame_audio_samples, int audio_bitrate, const char *filename)
  
      {
  
          int ret;
  
  
  
          avcodec_register_all();
  
          av_register_all();
  
  
  
          ret = avformat_alloc_output_context2(&outctx, NULL, "webm", filename);
  
  
  
          if (ret < 0)
  
              return ret;
  
  
  
          aud_codec = avcodec_find_encoder(aud_codec_id);
  
          avcodec_register(aud_codec);
  
  
  
          if (!aud_codec)
  
              return -1;
  
  
  
          // Setup Audio Stream
  
  
  
          aud_codec_context = avcodec_alloc_context3(aud_codec);
  
          if (!aud_codec_context)
  
              return -1;
  
  
  
          /* select other audio parameters supported by the encoder */
  
          aud_codec_context->bit_rate = audio_bitrate;
  
          aud_codec_context->sample_rate = sample_rate;
  
          aud_codec_context->sample_fmt = sample_fmt;
  
          aud_codec_context->channel_layout = AV_CH_LAYOUT_STEREO;
  
          aud_codec_context->channels = av_get_channel_layout_nb_channels(aud_codec_context->channel_layout);
  
  
  
          aud_codec_context->codec = aud_codec;
  
          aud_codec_context->codec_id = aud_codec_id;
  
  
  
  
  
          AVRational time_base;
  
          time_base.num = per_frame_audio_samples;
  
          time_base.den = aud_codec_context->sample_rate;
  
          aud_codec_context->time_base = time_base;
  
  
  
          ret = avcodec_open2(aud_codec_context, aud_codec, NULL);
  
  
  
          if (ret < 0)
  
              return ret;
  
  
  
          outctx->audio_codec = aud_codec;
  
          outctx->audio_codec_id = aud_codec_id;
  
  
  
          audio_st = avformat_new_stream(outctx, aud_codec);
  
  
  
          avcodec_parameters_from_context(audio_st->codecpar, aud_codec_context);
  
  
  
          conv_time_base.num = aud_codec_context->frame_size;
  
          conv_time_base.den = aud_codec_context->sample_rate;
  
  
  
          // Setup audio frame
  
          aud_frame = av_frame_alloc();
  
          aud_frame->nb_samples = aud_codec_context->frame_size;
  
          aud_frame->format = aud_codec_context->sample_fmt;
  
          aud_frame->channel_layout = aud_codec_context->channel_layout;
  
          aud_frame->sample_rate = aud_codec_context->sample_rate;
  
  
  
          int buffer_size;
  
          if (aud_codec_context->frame_size == 0)
  
          {
  
              buffer_size = per_frame_audio_samples * 2 * 4;
  
              aud_frame->nb_samples = per_frame_audio_samples;
  
          }
  
          else
  
          {
  
              buffer_size = av_samples_get_buffer_size(NULL, aud_codec_context->channels, aud_codec_context->frame_size,
  
                  aud_codec_context->sample_fmt, 0);
  
          }
  
  
  
          if (av_sample_fmt_is_planar(sample_fmt))
  
              ret = av_frame_get_buffer(aud_frame, buffer_size / 2);
  
          else
  
              ret = av_frame_get_buffer(aud_frame, buffer_size);
  
  
  
          if (!aud_frame || ret < 0)
  
              return ret;
  
  
  
          // Setup audio resampler
  
  
  
          audio_swr_ctx = swr_alloc();
  
          if (!audio_swr_ctx)
  
              return -1;
  
  
  
          /* set options */
  
          av_opt_set_int(audio_swr_ctx, "in_channel_layout", aud_codec_context->channel_layout, 0);
  
          av_opt_set_int(audio_swr_ctx, "in_sample_rate", sample_rate, 0);
  
          av_opt_set_int(audio_swr_ctx, "in_frame_size", per_frame_audio_samples, 0);
  
          av_opt_set_sample_fmt(audio_swr_ctx, "in_sample_fmt", AV_SAMPLE_FMT_FLT, 0);
  
  
  
          av_opt_set_int(audio_swr_ctx, "out_channel_layout", aud_codec_context->channel_layout, 0);
  
          av_opt_set_int(audio_swr_ctx, "out_sample_rate", aud_codec_context->sample_rate, 0);
  
          av_opt_set_int(audio_swr_ctx, "out_frame_size", aud_codec_context->frame_size, 0);
  
          av_opt_set_sample_fmt(audio_swr_ctx, "out_sample_fmt", aud_codec_context->sample_fmt, 0);
  
  
  
          /* initialize the resampling context */
  
          if ((ret = swr_init(audio_swr_ctx)) < 0)
  
          {
  
              return ret;
  
          }
  
  
  
          dst_rate = aud_codec_context->sample_rate;
  
          src_rate = sample_rate;
  
  
  
          src_nb_samples = per_frame_audio_samples;
  
          dst_nb_samples = aud_codec_context->frame_size;
  
  
  
          max_dst_nb_samples = av_rescale_rnd(src_nb_samples, dst_rate, src_rate, AV_ROUND_UP);
  
  
  
          dst_nb_channels = av_get_channel_layout_nb_channels(aud_codec_context->channel_layout);
  
  
  
          ret = av_samples_alloc_array_and_samples(&dst_data, &dst_linesize, dst_nb_channels, dst_nb_samples, sample_fmt, 0);
  
  
  
          aud_frame_counter = 0;
  
  
  
          if (ret < 0)
  
              return ret;
  
  
  
          av_dump_format(outctx, 0, filename, 1);
  
  
  
          if (!(outctx->oformat->flags & AVFMT_NOFILE))
  
          {
  
              ret = avio_open(&outctx->pb, filename, AVIO_FLAG_WRITE);
  
              if (ret < 0)
  
              {
  
                  return ret;
  
              }
  
          }
  
  
  
          ret = avformat_write_header(outctx, NULL);
  
          if (ret < 0)
  
              return ret;
  
  
  
          return 0;
  
      }

  Encoding and ending :

  int process_encode_loop(AVFormatContext *local_outctx, AVCodecContext *codec_context, AVStream *stream, AVRational time_base, bool flush)
  
      {
  
          int ret;
  
  
  
          AVPacket pkt;
  
          av_init_packet(&pkt);
  
          pkt.data = NULL;
  
          pkt.size = 0;
  
  
  
          while (true)
  
          {
  
              ret = avcodec_receive_packet(codec_context, &pkt);
  
              if (!ret)
  
              {
  
                  pkt.stream_index = stream->index;
  
                  av_packet_rescale_ts(&pkt, time_base, stream->time_base);
  
                  av_interleaved_write_frame(local_outctx, &pkt);
  
  
  
                  av_packet_unref(&pkt);
  
              }
  
  
  
              if (ret == AVERROR(EAGAIN))
  
                  break;
  
              else if (ret == AVERROR_EOF)
  
                  break;
  
              else if (ret < 0)
  
                  return ret;
  
              else if (flush == false)
  
                  break;
  
          }
  
  
  
          return 0;
  
      }
  
  
  
      int write_audio_frame(float_t *aud_sample)
  
      {
  
          int ret;
  
          if (dst_nb_samples > max_dst_nb_samples)
  
          {
  
              av_free(&aud_frame->data[0]);
  
              ret = av_samples_alloc(aud_frame->data, &dst_linesize, dst_nb_channels, dst_nb_samples, sample_fmt, 1);
  
              if (ret < 0)
  
                  return ret;
  
  
  
              max_dst_nb_samples = dst_nb_samples;
  
          }
  
  
  
          ret = swr_convert(audio_swr_ctx, dst_data, dst_nb_samples, (const uint8_t **)&aud_sample, src_nb_samples);
  
          if (ret < 0)
  
          {
  
              return ret;
  
          }
  
  
  
          aud_frame->data[0] = (uint8_t*)dst_data[0];
  
          aud_frame->extended_data[0] = (uint8_t*)dst_data[0];
  
  
  
          aud_frame->pts = aud_frame_counter++;
  
  
  
          ret = avcodec_send_frame(aud_codec_context, aud_frame);
  
  
  
          ret = process_encode_loop(outctx, aud_codec_context, audio_st, conv_time_base, false);
  
  
  
          if (ret < 0)
  
              return ret;
  
  
  
          return 0;
  
      }
  
  
  
      int finish_audio_encoding()
  
      {
  
          int ret = avcodec_send_frame(aud_codec_context, NULL);
  
          if (ret < 0)
  
              return ret;
  
  
  
          ret = process_encode_loop(outctx, aud_codec_context, audio_st, conv_time_base, true);
  
          if (ret < 0)
  
              return ret;
  
  
  
          av_write_trailer(outctx);
  
  
  
          return ret;
  
      }

  Main :

  void fill_samples(float_t *dst, int nb_samples, int nb_channels, int sample_rate, float_t *t)
  
      {
  
          int i, j;
  
          float_t tincr = 1.0 / sample_rate;
  
          const float_t c = 2 * M_PI * 440.0;
  
          /* generate sin tone with 440Hz frequency and duplicated channels */
  
          for (i = 0; i < nb_samples; i++) {
  
              *dst = sin(c * *t);
  
              for (j = 1; j < nb_channels; j++)
  
                  dst[j] = dst[0];
  
              dst += nb_channels;
  
              *t += tincr;
  
          }
  
      }
  
  
  
      int main()
  
      {
  
          int frame_rate = 30;
  
          int sec = 12;
  
          int bit_rate = 192000;
  
          float t = 0;
  
  
  
          int src_samples_linesize;
  
          int src_nb_samples = 1024;
  
          int src_channels = 2;
  
          int sample_rate = 48000;
  
  
  
          uint8_t **src_data = NULL;
  
  
  
          int ret;
  
  
  
          initialize(sample_rate, src_nb_samples, bit_rate, "sound_test.webm");
  
  
  
          ret = av_samples_alloc_array_and_samples(&src_data, &src_samples_linesize, src_channels,
  
              src_nb_samples, AV_SAMPLE_FMT_FLT, 0);
  
  
  
          for (size_t i = 0; i < frame_rate * sec; i++)
  
          {
  
              fill_samples((float *)src_data[0], src_nb_samples, src_channels, sample_rate, &t);
  
              write_audio_frame((float *)src_data[0]);
  
          }
  
          finish_audio_encoding();
  
  
  
          cleanup();
  
  
  
          return 0;
  
      }

  Edit 2, This code reproduces the issue exactly and is fully self contained, if you have the ffmpeg 3.3.x libraries. It’s tried with 3.3.1 and 3.3.2 is the same results.

  So what could I be missing ? I do not think something is wrong with the sample rates or any other specifications, else it would not work in VLC or an ogg file. I do think the audio stream itself if correct, just some part of the header or how the file is formatted (look further down for some EBML inspection) that is not correct.

  As explained earlier, the licence with VP9+Opus in webm is why I have these specifics. And the exact problem is that I want the audio stream produced to work well when I upload it to YouTube.

  Any suggestion is appreciated, thanks in Advance !

  Some other things I’ve tried :

  I’ve looked at the header with the "MediaInfo" app built in to MVKTool :
  https://i.gyazo.com/3b29b41629a28bd526bf7637ce3f2601.png
  It all looks fine to me.

  I’ve also inspected the raw EBML file with EBML-Viewer (https://code.google.com/archive/p/ebml-viewer/) and in there I can se some difference between the files ;

  My file : https://i.gyazo.com/6fa8c540a2698a8a4d3421d363aede0a.png
  File produced with ffmpeg.exe : https://i.gyazo.com/04d60e64ff3c3040ea83e98cdf507530.png

  In my file it’s "Cluster" -> "BlockGroup" -> "Block", " ?"
  In the other it’s just "Cluster" -> "SimpleBlock"
  And in the webm specs, it says both are supported (https://www.webmproject.org/docs/container/)

  But I do not know much about these specific things, just looking for anything.

• FFmpeg throwing error during video render process using Remotion

  31 mai 2022, par Vince

  I'm using a tool called Remotion, which allows you to create videos using javascript (node & react). I have the server-side rendering process working, unless I try to include an mp3. Per their docs, I have my audio imported like this :

  


  import goodtimes from &#x27;../../music/GoodTimes.mp3&#x27;;&#xA;&#xA;...&#xA;&#xA;<audio src="{goodtimes}"></audio>&#xA;

  


  I'm currently trying to troubleshoot this using the CLI, because the error messaging is better. The command runs, and seems to generate the full video – 2400 out of 2400 frames. But then it fails with this output :

  


  (2/3) [====================] Rendered frames (6x) 176540ms&#xA;    &#x2B; [====================] Downloading http://localhost:3000/1ba90857580e6291.mp3&#xA;(3/3) [====================] Encoding video 2400/2400&#xA;An error occurred:&#xA;Error: Command failed with exit code 1: ffprobe -v error -show_entries stream=channels:format=duration -of default=nw=1 /var/folders/6n/_nfb38t53dgdj092_pcnqqmw0000gn/T/remotion-assets-dir2jc2oppruh/7744374580215663.mp3&#xA;[mp3 @ 0x7fa78152ec40] Failed to read frame size: Could not seek to 5244.&#xA;/var/folders/6n/_nfb38t53dgdj092_pcnqqmw0000gn/T/remotion-assets-dir2jc2oppruh/7744374580215663.mp3: Invalid argument&#xA;    at makeError (/Users/voverson/clm-video/server/node_modules/execa/lib/error.js:60:11)&#xA;    at handlePromise (/Users/voverson/clm-video/server/node_modules/execa/index.js:118:26)&#xA;    at runMicrotasks (<anonymous>)&#xA;    at processTicksAndRejections (internal/process/task_queues.js:93:5)&#xA;    at async getAudioChannelsAndDuration (/Users/voverson/clm-video/server/node_modules/@remotion/renderer/dist/assets/get-audio-channels.js:17:18)&#xA;    at async preprocessAudioTrackUnlimited (/Users/voverson/clm-video/server/node_modules/@remotion/renderer/dist/preprocess-audio-track.js:14:36)&#xA;</anonymous>

  


  I'm hoping some ffmpeg experts might recognize this error and have some idea what might be causing it.