Sur d’autres sites (8757)

• How to make customizable subtitles with FFmpeg ?

  6 août 2022, par Alex Rypun

  I need to generate videos with text (aka subtitles) and styles provided by users.
The styles are a few fonts, text border, text background, text shadow, colors, position on the video, etc.

  


  As I understand there are 2 filters I can use : drawtext and subtitles.

  


  subtitles is easier to use but it's not fully customizable. For example, I can't add shadow and background for the same text.

  


  drawtext is more customizable but quite problematic.

  


  I've implemented almost everything I need with drawtext but have one problem : multiline text with a background.

  


  boxborderw parameter adds specified pixels count from all 4 sides from the extreme text point. I need to make backgrounds touch each other (pixel perfect). That means I have to position text lines with the same space between extreme text points (not between baselines). With crazy workarounds I solved it.
Everything almost works but sometimes some strange border appears between lines :

  


  

  


  Spending ages investigating I figured out that it depends on the actual line height and the position on the video.
Each letter has its own distances between baseline and the highest point and between baseline and the lowest point.
The whole line height is the difference between the highest point of the highest symbol and the lowest point of the lowest symbol :

  


  

  


  Now the most interesting.

  


  For lines that have even pixels height.

  


  If the line is positioned on the odd pixel (y-axis) and has an odd boxborderw (or positioned on the even pixel (y-axis) and has an even boxborderw ), then it's rendered as expected (without any additional borders).
In other cases, the thin dark line is noticeable on the contact line (it's rendered either on top or at bottom of each text block :

  


  

  

  


  For lines that have odd pixels height.

  


  In all cases the thin dark line is noticeable. Depending on y coordinate (odd or even) and boxborderw value (odd or even) that magic line appears on top or bottom :

  


  

  


  I can make text lines overlap a bit. This solves the problem but adds another problem.
I use fade-in/out by smoothly changing alfa (transparency). When the text becomes semi-transparent the overlapped area has a different color :

  


  

  


  Here is the command I use :

  


  ffmpeg "-y" "-f" "lavfi" "-i" "color=#ffffff" "-filter_complex" \
"[0:v]loop=-1:1:0,trim=duration=2.00,format=yuv420p,scale=960:540,setdar=16/9[video];\
[video]drawtext=text='qqq':\
fontfile=ComicSansMS.ttf:\
fontcolor=#ffffff:\
fontsize=58:\
bordercolor=#000000:\
borderw=2:\
box=1:\
boxcolor=#ff0000:\
boxborderw=15:\
x=(w-text_w)/2:\
y=105:\
alpha='if(lt(t,0),0,if(lt(t,0.5),(t-0)/0.5,if(lt(t,1.49),1,if(lt(t,1.99),(0.5-(t-1.49))/0.5,0))))',\
drawtext=text='qqq':\
fontfile=ComicSansMS.ttf:\
fontcolor=#ffffff:\
fontsize=58:\
bordercolor=#000000:\
borderw=2:\
box=1:\
boxcolor=#ff0000:\
boxborderw=15:\
x=(w-text_w)/2:\
y=182:\
alpha='if(lt(t,0),0,if(lt(t,0.5),(t-0)/0.5,if(lt(t,1.49),1,if(lt(t,1.99),(0.5-(t-1.49))/0.5,0))))'[video]" \
"-vsync" "2" "-map" "[video]" "-r" "25" "output_multiline.mp4"


  


  I tried to draw rectangles with drawbox following the same logic (changing the position and height). But there were no problems.

  


  Does anybody know what is the nature of that dark line ?
And how it can depend on the even/odd of the height and position ?
What should I investigate to figure out such a behavior ?

  


  UPD :

  


  Just accidentally figured out that changing the pixel format (from format=yuv420p to format=uyvy422 or many others) solved the problem. At least on my test commands.
Now will learn about what is pixel format)

  


• Minimal Understanding of VP8′s Forward Transform

  16 novembre 2010, par Multimedia Mike — VP8

  Regarding my toy VP8 encoder, Pengvado mentioned in the comments of my last post, “x264 looks perfect using only i16x16 DC mode. You must be doing something wrong in computing residual or fdct or quantization.” This makes a lot of sense. The encoder generates a series of elements which describe how to reconstruct the original image. Intra block reconstruction takes into consideration the following elements :

  
  
  

  I have already verified that both my encoder and FFmpeg’s VP8 decoder agree precisely on how to reconstruct blocks based on the predictors, coefficients, and quantizers. Thus, if the decoded image still looks crazy, the elements the encoder is generating to describe the image must be wrong.

  So I started studying the forward DCT, which I had cribbed wholesale from the original libvpx 0.9.0 source code. It should be noted that the formal VP8 spec only defines the inverse transform process, not the forward process. I was using a version designated as the “short” version, vs. the “fast” version. Then I looked at the 0.9.5 FDCT. Then I got the idea of comparing the results of each.

  input:   92 91 89 86 91 90 88 86 89 89 89 88 89 87 88 93

  • libvpx 0.9.0 “short” :

    forward : -314 5 1 5 4 5 -2 0 0 1 -1 -1 1 11 -3 -4
    inverse : 92 91 89 86 89 86 91 90 91 90 88 86 88 86 89 89
    

  • libvpx 0.9.0 “fast” :

    forward : -314 4 0 5 4 4 -2 0 0 1 0 -1 1 11 -2 -5
    inverse : 91 91 89 86 88 86 91 90 91 90 88 86 88 86 89 89
    

  • libvpx 0.9.5 “short” :

    forward : -312 7 1 0 1 12 -5 2 2 -3 3 -1 1 0 -2 1
    inverse : 92 91 89 86 91 90 88 86 89 89 89 88 89 87 88 93
    

  I was surprised when I noticed that input[] != idct(fdct(input[])) in some of the above cases. Then I remembered that the aforementioned property isn’t what is meant by a “bit-exact” transform– only that all implementations of the inverse transform are supposed to produce bit-exact output for a given vector of input coefficients.

  Anyway, I tried applying each of these forward transforms. I got slightly differing results, with the latest one I tried (the fdct from libvpx 0.9.5) producing the best results (to my eye). At least the trees look better in the Big Buck Bunny logo image :

  
  
  The dense trees of the Big Buck Bunny logo using one of the libvpx 0.9.0 forward transforms
  
  
  The same segment of the image using the libvpx 0.9.5 forward transform
  

  Then again, it could be that the different numbers generated by the newer forward transform triggered different prediction modes to be chosen. Overall, adapting the newer FDCT did not dramatically improve the encoding quality.

  Working on the intra 4×4 mode encoding is generating some rather more accurate blocks than my intra 16×16 encoder. Pengvado indicated that x264 generates perfectly legible results when forcing the encoder to only use intra 16×16 mode. To be honest, I’m having trouble understanding how that can possibly occur thanks to the Walsh-Hadamard transform (WHT). I think that’s where a lot of the error is creeping in with my intra 16×16 encoder. Then again, FFmpeg implements an inverse WHT function that bears ‘vp8′ in its name. This implies that it’s custom to the algorithm and not exactly shared with H.264.

• ffmpeg does not produce result with C# process

  20 août 2020, par Ali Agha

  So, I'm facing a really strange issue. When I run "ffmpeg" through C# process, it doesn't work for multiple commands (concatenated with &). However, it works for simple/single command and even produces output. I checked the logs through "Standard Error" and it appears that it fails in "concat" command.

  


  This line specifically indicates the error in the ffmpeg created log :
concat:CoverImage_Video_Landscape_fd8dbbb5-63c0-4ff0-b0c6-e6c3c8f590aa.ts|Downloaded_Video_fd8dbbb5-63c0-4ff0-b0c6-e6c3c8f590aa.ts: No such file or directory

  


  Now, the code snippet is as follows :

  


  string arguments = $"-loop 1 -i \"{physicalPath}bin\\CoverImage_Video_Landscape_{guid}.png\" -f lavfi -i anullsrc -t 3 -c:v libx264 -c:a aac -b:a 160k -t 3 -pix_fmt yuv420p -vf scale={videoWidth}:{videoHeight} -y \"{physicalPath}bin\\CoverImage_Video_Landscape_{guid}.ts\"" 
+ $" & -i \"{physicalPath}bin\\{objectNameWithoutExtension}.mp4\" -c:v libx264 -c:a aac -b:a 160k -f mpegts \"{physicalPath}bin\\{objectNameWithoutExtension}.ts\"" 
+ $" & -i \"concat:CoverImage_Video_Landscape_{guid}.ts|{objectNameWithoutExtension}.ts\" -c copy \"{fileAssetRootPath}\\Video\\Output\\{outputVideoName}.mp4\"";

            process = new Process();
            process.StartInfo.FileName = ffmpegPath;
            process.StartInfo.Arguments = arguments;
            process.StartInfo.CreateNoWindow = true;
            process.StartInfo.UseShellExecute = false;
            process.StartInfo.RedirectStandardOutput = true;
            process.StartInfo.RedirectStandardError = true;

            process.EnableRaisingEvents = true;
            process.Exited += new EventHandler(process_Exited);
            process.Start();
            string stResult = process.StandardOutput.ReadToEnd();
            string erResult = process.StandardError.ReadToEnd();
            process.WaitForExit();


  


  "Physical Path" -> Application directory (bin) folder
"File Asset Path" -> The directory in which I'm storing uploaded resources

  


  Now, the strange part. During debugging, if I just copy the "arguments" variable value and execute it in CMD directly in "bin" folder, I get perfect results. Everything works. Here's the actual query generated :

  


  ffmpeg -loop 1 -i "E:\Projects\Freelance\UP - Inspire3 (Karl)\AWS S3 Bucket Editing Routine\Code\AWSAPI\AWSAPI\bin\CoverImage_Video_Landscape_fd8dbbb5-63c0-4ff0-b0c6-e6c3c8f590aa.png" -f lavfi -i anullsrc -t 3 -c:v libx264 -c:a aac -b:a 160k -t 3 -pix_fmt yuv420p -vf scale=1920:964 -y "E:\Projects\Freelance\UP - Inspire3 (Karl)\AWS S3 Bucket Editing Routine\Code\AWSAPI\AWSAPI\bin\CoverImage_Video_Landscape_fd8dbbb5-63c0-4ff0-b0c6-e6c3c8f590aa.ts" & ffmpeg -i "E:\Projects\Freelance\UP - Inspire3 (Karl)\AWS S3 Bucket Editing Routine\Code\AWSAPI\AWSAPI\bin\Downloaded_Video_fd8dbbb5-63c0-4ff0-b0c6-e6c3c8f590aa.mp4" -c:v libx264 -c:a aac -b:a 160k -f mpegts "E:\Projects\Freelance\UP - Inspire3 (Karl)\AWS S3 Bucket Editing Routine\Code\AWSAPI\AWSAPI\bin\Downloaded_Video_fd8dbbb5-63c0-4ff0-b0c6-e6c3c8f590aa.ts" & ffmpeg -i "concat:CoverImage_Video_Landscape_fd8dbbb5-63c0-4ff0-b0c6-e6c3c8f590aa.ts|Downloaded_Video_fd8dbbb5-63c0-4ff0-b0c6-e6c3c8f590aa.ts" -c copy "E:\Projects\Freelance\UP - Inspire3 (Karl)\AWS S3 Bucket Editing Routine\Code\AWSAPI\AWSAPI\FileAssets\Video\Output\Downloaded_Video_fd8dbbb5-63c0-4ff0-b0c6-e6c3c8f590aa.mp4"


  


  I have added "ffmpeg" keyword before every actual query otherwise I can't run it in CMD. Rest of the query is exactly the same

  


  This query doesn't produce anything when running through C# process and throws the "concat" error I just mentioned above.

  


  FYI, all files needed for conversion are present in the "bin" directory (temp files) and output file is supposed to be placed in another directory. I did that, because concat command apparently has issues with paths, so I just placed ffmpeg.exe along with all files needed directly in "bin" folder.

  


  Also, the code explanation is that it's a 3-step process. It's supposed to convert a png image to ts, then another video to ts, and then combine the 2 and produce output.

  


  Any help would be highly appreciated, thanks !!