Sur d’autres sites (6383)

• Creating a usable H.264 video file

  4 mai 2019, par Ethan McTague

  I am trying to use libavcodec to generate an mp4 video file from individual frames. Each input frame is a qt QImage, and the output file is written to using the Qt QFile class.

  I’ve done this through a VideoTarget class which opens the given ’target’ file when initialized, records frames when addFrame(image) is called, and then saves/closes the file when its destructor is called.

  The class has the following fields :

  AVCodec* m_codec = nullptr;
  
  AVCodecContext *m_context = nullptr;
  
  AVPacket* m_packet = nullptr;
  
  AVFrame* m_frame = nullptr;
  
  
  
  QFile m_target;

  And looks like this :

  VideoTarget::VideoTarget(QString target, QObject *parent) : QObject(parent), m_target(target)
  
  {
  
      // Find video codec
  
      m_codec = avcodec_find_encoder_by_name("libx264rgb");
  
      if (!m_codec) throw std::runtime_error("Unable to find codec.");
  
  
  
      // Make codec context
  
      m_context = avcodec_alloc_context3(m_codec);
  
      if (!m_context) throw std::runtime_error("Unable to allocate codec context.");
  
  
  
      // Make codec packet
  
      m_packet = av_packet_alloc();
  
      if (!m_packet) throw std::runtime_error("Unable to allocate packet.");
  
  
  
      // Configure context
  
      m_context->bit_rate = 400000;
  
      m_context->width = 1280;
  
      m_context->height = 720;
  
      m_context->time_base = (AVRational){1, 60};
  
      m_context->framerate = (AVRational){60, 1};
  
      m_context->gop_size = 10;
  
      m_context->max_b_frames = 1;
  
      m_context->pix_fmt = AV_PIX_FMT_RGB24;
  
  
  
      if (m_codec->id == AV_CODEC_ID_H264)
  
          av_opt_set(m_context->priv_data, "preset", "slow", 0);
  
  
  
      // Open Codec
  
      int ret = avcodec_open2(m_context, m_codec, nullptr);
  
      if (ret < 0) {
  
          throw std::runtime_error("Unable to open codec.");
  
      }
  
  
  
      // Open file
  
      if (!m_target.open(QIODevice::WriteOnly))
  
          throw std::runtime_error("Unable to open target file.");
  
  
  
      // Allocate frame
  
      m_frame = av_frame_alloc();
  
      if (!m_frame) throw std::runtime_error("Unable to allocate frame.");
  
  
  
      m_frame->format = m_context->pix_fmt;
  
      m_frame->width = m_context->width;
  
      m_frame->height = m_context->height;
  
      m_frame->pts = 0;
  
  
  
      ret = av_frame_get_buffer(m_frame, 24);
  
      if (ret < 0) throw std::runtime_error("Unable to allocate frame buffer.");
  
  }
  
  
  
  void VideoTarget::addFrame(QImage &image)
  
  {
  
      // Ensure frame data is writable
  
      int ret = av_frame_make_writable(m_frame);
  
      if (ret < 0) throw std::runtime_error("Unable to make frame writable.");
  
  
  
      // Prepare image
  
      for (int y = 0; y < m_context->height; y++) {
  
          for (int x = 0; x < m_context->width; x++) {
  
              auto pixel = image.pixelColor(x, y);
  
              int pos = (y * 1024 + x) * 3;
  
              m_frame->data[0][pos] = pixel.red();
  
              m_frame->data[0][pos + 1] = pixel.green();
  
              m_frame->data[0][pos + 2] = pixel.blue();
  
          }
  
      }
  
  
  
      m_frame->pts++;
  
  
  
      // Send the frame
  
      ret = avcodec_send_frame(m_context, m_frame);
  
      if (ret < 0) throw std::runtime_error("Unable to send AV frame.");
  
  
  
      while (ret >= 0) {
  
          ret = avcodec_receive_packet(m_context, m_packet);
  
          if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
  
              return;
  
          else if (ret < 0) throw std::runtime_error("Error during encoding.");
  
  
  
          m_target.write((const char*)m_packet->data, m_packet->size);
  
          av_packet_unref(m_packet);
  
      }
  
  }
  
  
  
  VideoTarget::~VideoTarget()
  
  {
  
      int ret = avcodec_send_frame(m_context, nullptr);
  
      if (ret < 0) throw std::runtime_error("Unable to send AV null frame.");
  
  
  
      while (ret >= 0) {
  
          ret = avcodec_receive_packet(m_context, m_packet);
  
          if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
  
              return;
  
          else if (ret < 0) throw std::runtime_error("Error during encoding.");
  
  
  
          m_target.write((const char*)m_packet->data, m_packet->size);
  
          av_packet_unref(m_packet);
  
      }
  
  
  
      // Magic number at the end of the file
  
      uint8_t endcode[] = { 0, 0, 1, 0xb7 };
  
      m_target.write((const char*)endcode, sizeof(endcode));
  
      m_target.close();
  
  
  
      // Free codec stuff
  
      avcodec_free_context(&m_context);
  
      av_frame_free(&m_frame);
  
      av_packet_free(&m_packet);
  
  }

  When used, the class seems to work, and data is written to the file, except I am unable to play back the resulting file in any application.

  My main suspect is these lines :

      // Prepare image
  
      for (int y = 0; y < m_context->height; y++) {
  
          for (int x = 0; x < m_context->width; x++) {
  
              auto pixel = image.pixelColor(x, y);
  
              int pos = (y * 1024 + x) * 3;
  
              m_frame->data[0][pos] = pixel.red();
  
              m_frame->data[0][pos + 1] = pixel.green();
  
              m_frame->data[0][pos + 2] = pixel.blue();
  
          }
  
      }

  The libavcodec documentation was extremely vague regarding the layout of image data, so I effectively had to guess and be happy with the first thing that didn’t crash, so chances are I’m writing this incorrectly. There’s also the issue of size mismatch between my pixel color data calls (giving int values) and the 24-bits-per-pixel RGB format I have selected.

  How do I tweak this code to output actual, functioning video files ?

• How to make customizable subtitles with FFmpeg ?

  6 août 2022, par Alex Rypun

  I need to generate videos with text (aka subtitles) and styles provided by users.
The styles are a few fonts, text border, text background, text shadow, colors, position on the video, etc.

  


  As I understand there are 2 filters I can use : drawtext and subtitles.

  


  subtitles is easier to use but it's not fully customizable. For example, I can't add shadow and background for the same text.

  


  drawtext is more customizable but quite problematic.

  


  I've implemented almost everything I need with drawtext but have one problem : multiline text with a background.

  


  boxborderw parameter adds specified pixels count from all 4 sides from the extreme text point. I need to make backgrounds touch each other (pixel perfect). That means I have to position text lines with the same space between extreme text points (not between baselines). With crazy workarounds I solved it.
Everything almost works but sometimes some strange border appears between lines :

  


  

  


  Spending ages investigating I figured out that it depends on the actual line height and the position on the video.
Each letter has its own distances between baseline and the highest point and between baseline and the lowest point.
The whole line height is the difference between the highest point of the highest symbol and the lowest point of the lowest symbol :

  


  

  


  Now the most interesting.

  


  For lines that have even pixels height.

  


  If the line is positioned on the odd pixel (y-axis) and has an odd boxborderw (or positioned on the even pixel (y-axis) and has an even boxborderw ), then it's rendered as expected (without any additional borders).
In other cases, the thin dark line is noticeable on the contact line (it's rendered either on top or at bottom of each text block :

  


  

  

  


  For lines that have odd pixels height.

  


  In all cases the thin dark line is noticeable. Depending on y coordinate (odd or even) and boxborderw value (odd or even) that magic line appears on top or bottom :

  


  

  


  I can make text lines overlap a bit. This solves the problem but adds another problem.
I use fade-in/out by smoothly changing alfa (transparency). When the text becomes semi-transparent the overlapped area has a different color :

  


  

  


  Here is the command I use :

  


  ffmpeg "-y" "-f" "lavfi" "-i" "color=#ffffff" "-filter_complex" \
"[0:v]loop=-1:1:0,trim=duration=2.00,format=yuv420p,scale=960:540,setdar=16/9[video];\
[video]drawtext=text='qqq':\
fontfile=ComicSansMS.ttf:\
fontcolor=#ffffff:\
fontsize=58:\
bordercolor=#000000:\
borderw=2:\
box=1:\
boxcolor=#ff0000:\
boxborderw=15:\
x=(w-text_w)/2:\
y=105:\
alpha='if(lt(t,0),0,if(lt(t,0.5),(t-0)/0.5,if(lt(t,1.49),1,if(lt(t,1.99),(0.5-(t-1.49))/0.5,0))))',\
drawtext=text='qqq':\
fontfile=ComicSansMS.ttf:\
fontcolor=#ffffff:\
fontsize=58:\
bordercolor=#000000:\
borderw=2:\
box=1:\
boxcolor=#ff0000:\
boxborderw=15:\
x=(w-text_w)/2:\
y=182:\
alpha='if(lt(t,0),0,if(lt(t,0.5),(t-0)/0.5,if(lt(t,1.49),1,if(lt(t,1.99),(0.5-(t-1.49))/0.5,0))))'[video]" \
"-vsync" "2" "-map" "[video]" "-r" "25" "output_multiline.mp4"


  


  I tried to draw rectangles with drawbox following the same logic (changing the position and height). But there were no problems.

  


  Does anybody know what is the nature of that dark line ?
And how it can depend on the even/odd of the height and position ?
What should I investigate to figure out such a behavior ?

  


  UPD :

  


  Just accidentally figured out that changing the pixel format (from format=yuv420p to format=uyvy422 or many others) solved the problem. At least on my test commands.
Now will learn about what is pixel format)

  


• Museum of Multimedia Software, Part 4

  20 août 2010, par Multimedia Mike — Software Museum

  This is the last part of the series, at least until some more multimedia software shows up at my favorite thrift shop or the other boneyards I scavenge.

  Miscellaneous Multimedia Programs
  This set includes the titles Matinee FMV Screensaver, MetaCreations Painter Classic, and Multimedia JumpStart. The second one is likely a creation program. I have no idea what the third one is, while the first title gives me chills just thinking about the implications.

  
  
  

  Miscellaneous Creativity Software
  Magic Theatre and Microsoft Home : Creative Writer. I think I loaded up the former once to find a very basic animation program. The latter isn’t necessarily multimedia-related but certainly classifies as creative software. It also reminds me of the ad I once spied in Entertainment Weekly magazine during the mid-1990s for a Microsoft music history CD-ROM. MS branched out into all kinds of niches.

  
  
  

  More Multimedia Creativity Software
  VideoCraft and U-Create Games & Animation. I wager these would be fun to play around with if I had the time.

  
  
  

  Showcase CD-ROMs
  "What Can You Make ? Showcase 7" from Macromedia and Microsoft Multimedia Pack 10.

  
  
  

  Basic Multimedia Software Discs
  As a multimedia nerd, these Apple QuickTime and Microsoft Video for Windows discs make me sentimental.

  
  
  

  Real Software Collection
  Grit your teeth and gaze upon CD-ROM distributions of Real’s software. There is a RealAudio disc back from when Real still called themselves Progressive Networks. "Everything you need to hear the web roar !"

  
  
  

  Clips
  And a few multimedia clip CD-ROMs, along with a disc that promises to test and tune your MPC setup.

  
  
  

  Wrap-Up
  I would be remiss if I neglected to mention a few more pieces of multimedia creation software in my collection. First, there’s the Barbie Storymaker. I actually gave that one a go, as you can tragically see from that link. Further, the Taco Bell fast food restaurant chain ran, as one of their many kids meal promotions, a series of 4 simple Comics Constructor CD-ROMs. I played briefly with it here and again during an exploration of XML data formats and the parsing thereof (which the software uses).