Sur d’autres sites (10796)

• My C++ software uses ffmpeg in streaming mode, but I want to decode the data as quickly as possible, how do I do that ?

  14 septembre 2022, par Alexis Wilke

  When I run ffmpeg in my command line to convert an m4a file to mp3, it says ×45 to ×55 for the speed at which is decodes the input data. In this case, the 45 minute audio file gets converter in 2 minutes.

  


  When I run the same process through my C++ code, I stream the data. This is because my code accepts data coming from the network so often it will a little faster to do streaming (unfortunately, not with m4a data since the header is placed at the end of the file...)

  


  However, there seems to be something in ffmpeg that makes it think that if I want to stream the data it needs to be done in realtime. In other words, the frames come through at a speed of ×1 instead of the possible average of ×50.

  


  Is there a flag/setup that I need to turn ON or OFF so the streaming process goes ×50 or so ?

  


  ---

  


  I allocate the context like so :

  


  size_t const avio_buffer_size(4096);&#xA;unsigned char * avio_buffer(reinterpret_cast<unsigned char="char">(av_malloc(avio_buffer_size)));&#xA;AVIOContext * context(avio_alloc_context(&#xA;                          avio_buffer&#xA;                        , avio_buffer_size&#xA;                        , 0             // write flag&#xA;                        , this          // opaque&#xA;                        , decoder_read_static&#xA;                        , nullptr       // write func.&#xA;                        , decoder_seek_static));&#xA;</unsigned>

  


  To do the streaming, I use custom I/O in the context :

  


  AVFormatContext * format_context(avformat_alloc_context());
format_context->pb = context;
format_context->flags |= AVFMT_FLAG_CUSTOM_IO;
avformat_open_input(
          &format_context
        , "input"           // filename (unused)
        , nullptr           // input format
        , nullptr);         // options


  


  Next I get the audio stream index :

  


  avformat_find_stream_info(format_context, nullptr);
AVCodec * decoder_codec(nullptr);
int const index(av_find_best_stream(
              format_context
            , AVMEDIA_TYPE_AUDIO
            , -1            // wanted stream number
            , -1            // related stream
            , &decoder_codec
            , 0));          // flags


  


  That has the side effect of telling us which decoder to use :

  


  AVCodecContext * decoder_context = avcodec_alloc_context3(decoder_codec);
avcodec_parameters_to_context(
          decoder_context
        , format_context->streams[index]->codecpar);
avcodec_open2(
              decoder_context
            , decoder_codec
            , nullptr);        // options


  


  And finally, I loop through the frames :

  


  AVFrame *frame(av_frame_alloc());
AVPacket av_packet;
av_init_packet(&av_packet);
for(;;)
{
    av_read_frame(format_context, &av_packet);
    if(end-detected)
    {
        break;
    }
    if(av_packet.stream_index != index) continue;
    avcodec_send_packet(decoder_context, &av_packet);
    for(;;)
    {
        avcodec_receive_frame(decoder_context, frame);
        if(end-detected)
        {
            break;
        }
        ...copy data from frame...
        av_frame_unref(frame);
    }
    av_packet_unref(&av_packet);
}


  


  Note : I don't show all the error handling, use of RAII, etc. in an attempt to show the code in its simplest form.

  


• How to make customizable subtitles with FFmpeg ?

  6 août 2022, par Alex Rypun

  I need to generate videos with text (aka subtitles) and styles provided by users.
The styles are a few fonts, text border, text background, text shadow, colors, position on the video, etc.

  


  As I understand there are 2 filters I can use : drawtext and subtitles.

  


  subtitles is easier to use but it's not fully customizable. For example, I can't add shadow and background for the same text.

  


  drawtext is more customizable but quite problematic.

  


  I've implemented almost everything I need with drawtext but have one problem : multiline text with a background.

  


  boxborderw parameter adds specified pixels count from all 4 sides from the extreme text point. I need to make backgrounds touch each other (pixel perfect). That means I have to position text lines with the same space between extreme text points (not between baselines). With crazy workarounds I solved it.
Everything almost works but sometimes some strange border appears between lines :

  


  

  


  Spending ages investigating I figured out that it depends on the actual line height and the position on the video.
Each letter has its own distances between baseline and the highest point and between baseline and the lowest point.
The whole line height is the difference between the highest point of the highest symbol and the lowest point of the lowest symbol :

  


  

  


  Now the most interesting.

  


  For lines that have even pixels height.

  


  If the line is positioned on the odd pixel (y-axis) and has an odd boxborderw (or positioned on the even pixel (y-axis) and has an even boxborderw ), then it's rendered as expected (without any additional borders).
In other cases, the thin dark line is noticeable on the contact line (it's rendered either on top or at bottom of each text block :

  


  

  

  


  For lines that have odd pixels height.

  


  In all cases the thin dark line is noticeable. Depending on y coordinate (odd or even) and boxborderw value (odd or even) that magic line appears on top or bottom :

  


  

  


  I can make text lines overlap a bit. This solves the problem but adds another problem.
I use fade-in/out by smoothly changing alfa (transparency). When the text becomes semi-transparent the overlapped area has a different color :

  


  

  


  Here is the command I use :

  


  ffmpeg "-y" "-f" "lavfi" "-i" "color=#ffffff" "-filter_complex" \
"[0:v]loop=-1:1:0,trim=duration=2.00,format=yuv420p,scale=960:540,setdar=16/9[video];\
[video]drawtext=text='qqq':\
fontfile=ComicSansMS.ttf:\
fontcolor=#ffffff:\
fontsize=58:\
bordercolor=#000000:\
borderw=2:\
box=1:\
boxcolor=#ff0000:\
boxborderw=15:\
x=(w-text_w)/2:\
y=105:\
alpha='if(lt(t,0),0,if(lt(t,0.5),(t-0)/0.5,if(lt(t,1.49),1,if(lt(t,1.99),(0.5-(t-1.49))/0.5,0))))',\
drawtext=text='qqq':\
fontfile=ComicSansMS.ttf:\
fontcolor=#ffffff:\
fontsize=58:\
bordercolor=#000000:\
borderw=2:\
box=1:\
boxcolor=#ff0000:\
boxborderw=15:\
x=(w-text_w)/2:\
y=182:\
alpha='if(lt(t,0),0,if(lt(t,0.5),(t-0)/0.5,if(lt(t,1.49),1,if(lt(t,1.99),(0.5-(t-1.49))/0.5,0))))'[video]" \
"-vsync" "2" "-map" "[video]" "-r" "25" "output_multiline.mp4"


  


  I tried to draw rectangles with drawbox following the same logic (changing the position and height). But there were no problems.

  


  Does anybody know what is the nature of that dark line ?
And how it can depend on the even/odd of the height and position ?
What should I investigate to figure out such a behavior ?

  


  UPD :

  


  Just accidentally figured out that changing the pixel format (from format=yuv420p to format=uyvy422 or many others) solved the problem. At least on my test commands.
Now will learn about what is pixel format)

  


• How to Drop all matched frames that are the same with a specified frame

  2 octobre 2022, par zgno

  After researching into this for a couple of days, I found that mpdecimate only drop frames that is similar to the PREVIOUS frame. My problem is that just one same frame is duplicated throughout the whole video, and it is duplicated between frames(in other words, not sequentially,so mpdecimate always treats them as not duplicated) . It's almost like that frame is inserted randomly for a 1000+ times throughout the video. And I use ffmpeg -err_detect ignore_err -i only to find that no errors are detected. This strange situation happens in some hls streaming video. I will provide a small segment of the whole video for anyone who are interested to test.
A 6 seconds video down below, it is a badminton game.
A segment from the m3u8 file list