Advanced search

On other websites (10574)

• FFMPEG is really slow at extracting subtitles

  3 March 2021, by Gustav P Svensson

  I'm trying to extract the subtitles from a 1080P video, around 40min long. I'm currently using this command (using fluent-ffmpeg in node, but the translated command is this):

  


  ffmpeg -threads 3  -map 0: -c copy 


  


  This command takes about 20-30 min to complete. I've searched quite a lot on how to speed up this process, if I look at my task manager I can see that ffmpeg is using 0.1% of the CPU which makes me think that it's possible to speed up this process.

  


  I'm not sure if the -threads option is actually doing anything when extracting subtitles but I don't think it should make it slower atleast?

  


  So my question is, is it possible to speed up this process? I appriciate any help.

  


  Full FFMPEG log:
fmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  libavutil      55. 78.100 / 55. 78.100
  libavcodec     57.107.100 / 57.107.100
  libavformat    57. 83.100 / 57. 83.100
  libavdevice    57. 10.100 / 57. 10.100
  libavfilter     6.107.100 /  6.107.100
  libavresample   3.  7.  0 /  3.  7.  0
  libswscale      4.  8.100 /  4.  8.100
  libswresample   2.  9.100 /  2.  9.100
  libpostproc    54.  7.100 / 54.  7.100
Input #0, matroska,webm, from '/home/test.mkv:
  Metadata:
    encoder         : libebml v1.3.6 + libmatroska v1.4.9
    creation_time   : 2019-03-14T16:46:55.000000Z
  Duration: 00:41:20.29, start: 0.000000, bitrate: 6430 kb/s
    Stream #0:0: Video: h264 (Main), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Metadata:
      BPS-eng         : 5788926
      DURATION-eng    : 00:41:20.020000000
      NUMBER_OF_FRAMES-eng: 59461
      NUMBER_OF_BYTES-eng: 1794581562
      _STATISTICS_WRITING_APP-eng: mkvmerge v31.0.0 ('Dolores In A Shoestand') 64-bit
      _STATISTICS_WRITING_DATE_UTC-eng: 2019-03-14 16:46:55
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:1(eng): Audio: eac3, 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
    Metadata:
      title           : English
      BPS-eng         : 640000
      DURATION-eng    : 00:41:20.288000000
      NUMBER_OF_FRAMES-eng: 77509
      NUMBER_OF_BYTES-eng: 198423040
      _STATISTICS_WRITING_APP-eng: mkvmerge v31.0.0 ('Dolores In A Shoestand') 64-bit
      _STATISTICS_WRITING_DATE_UTC-eng: 2019-03-14 16:46:55
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:2(eng): Subtitle: subrip
    Metadata:
      BPS-eng         : 80
      DURATION-eng    : 00:40:22.295000000
      NUMBER_OF_FRAMES-eng: 645
      NUMBER_OF_BYTES-eng: 24473
      _STATISTICS_WRITING_APP-eng: mkvmerge v31.0.0 ('Dolores In A Shoestand') 64-bit
      _STATISTICS_WRITING_DATE_UTC-eng: 2019-03-14 16:46:55
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
    Stream #0:3(eng): Subtitle: subrip
    Metadata:
      title           : SDH
      BPS-eng         : 86
      DURATION-eng    : 00:40:31.012000000
      NUMBER_OF_FRAMES-eng: 709
      NUMBER_OF_BYTES-eng: 26142
      _STATISTICS_WRITING_APP-eng: mkvmerge v31.0.0 ('Dolores In A Shoestand') 64-bit
      _STATISTICS_WRITING_DATE_UTC-eng: 2019-03-14 16:46:55
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Output #0, srt, to 'test6.srt':
  Metadata:
    encoder         : Lavf57.83.100
    Stream #0:0(eng): Subtitle: subrip
    Metadata:
      BPS-eng         : 80
      DURATION-eng    : 00:40:22.295000000
      NUMBER_OF_FRAMES-eng: 645
      NUMBER_OF_BYTES-eng: 24473
      _STATISTICS_WRITING_APP-eng: mkvmerge v31.0.0 ('Dolores In A Shoestand') 64-bit
      _STATISTICS_WRITING_DATE_UTC-eng: 2019-03-14 16:46:55
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream mapping:
  Stream #0:2 -> #0:0 (copy)
Press [q] to stop, [?] for help
size=      46kB time=00:40:36.68 bitrate=   0.2kbits/s speed=11.6x    
video:0kB audio:0kB subtitle:24kB other streams:0kB global headers:0kB muxing overhead: 94.438766%


  


• Create Panorama from Non-Sequential Video Frames

  6 May 2021, by M.Innat

  There is a similar question (not that detailed and no exact solution).

  


  ---

  


  I want to create a single panorama image from video frames. And for that, I need to get minimum non-sequential video frames at first. A demo video file is uploaded here.

  


  What I Need

  


  A mechanism that can produce not-only non-sequential video frames but also in such a way that can be used to create a panorama image. A sample is given below. As we can see to create a panorama image, all the input samples must contain minimum overlap regions to each other otherwise it can not be done.

  


  

  


  So, if I have the following video frame's order

  


  A, A, A, B, B, B, B, C, C, A, A, C, C, C, B, B, B ...


  


  To create a panorama image, I need to get something as follows - reduced sequential frames (or adjacent frames) but with minimum overlapping.

  


       [overlap]  [overlap]  [overlap] [overlap]  [overlap]
 A,    A,B,       B,C,       C,A,       A,C,      C,B,  ...


  


  What I've Tried and Stuck

  


  A demo video clip is given above. To get non-sequential video frames, I primarily rely on ffmpeg software.

  


  Trial 1 Ref.

  


  ffmpeg -i check.mp4 -vf mpdecimate,setpts=N/FRAME_RATE/TB -map 0:v out.mp4


  


  After that, on the out.mp4, I applied slice the video frames using opencv

  


  import cv2, os 
from pathlib import Path

vframe_dir = Path("vid_frames/")
vframe_dir.mkdir(parents=True, exist_ok=True)

vidcap = cv2.VideoCapture('out.mp4')
success,image = vidcap.read()
count = 0

while success:
    cv2.imwrite(f"{vframe_dir}/frame%d.jpg" % count, image)     
    success,image = vidcap.read()
    count += 1


  


  Next, I rotated these saved images horizontally (as my video is a vertical view).

  


  vframe_dir = Path("out/")
vframe_dir.mkdir(parents=True, exist_ok=True)

vframe_dir_rot = Path("vframe_dir_rot/")
vframe_dir_rot.mkdir(parents=True, exist_ok=True)

for i, each_img in tqdm(enumerate(os.listdir(vframe_dir))):
    image = cv2.imread(f"{vframe_dir}/{each_img}")[:, :, ::-1] # Read (with BGRtoRGB)
    
    image = cv2.rotate(image,cv2.cv2.ROTATE_180)
    image = cv2.rotate(image,cv2.ROTATE_90_CLOCKWISE)

    cv2.imwrite(f"{vframe_dir_rot}/{each_img}", image[:, :, ::-1]) # Save (with RGBtoBGR)


  


  The output is ok for this method (with ffmpeg) but inappropriate for creating the panorama image. Because it didn't give some overlapping frames sequentially in the results. Thus panorama can't be generated.

  
 


  Trail 2 - Ref

  


  ffmpeg -i check.mp4 -vf decimate=cycle=2,setpts=N/FRAME_RATE/TB -map 0:v out.mp4


  


  didn't work at all.

  


  Trail 3

  


  ffmpeg -i check.mp4 -ss 0 -qscale 0 -f image2 -r 1 out/images%5d.png


  


  No luck either. However, I've found this last ffmpeg command was close by far but wasn't enough. Comparatively to others, this gave me a small amount of non-duplicate frames (good) but the bad thing is still do not need frames, and I kinda manually pick some desired frames, and then the opecv stitching algorithm works. So, after picking some frames and rotating (as mentioned before):

  


  stitcher = cv2.Stitcher.create()
status, pano = stitcher.stitch(images) # images: manually picked video frames -_- 


  
 


  Update

  


  After some trials, I am kinda adopting the non-programming solution. But would love to see an efficient programmatic approach.

  


  On the given demo video, I used Adobe products (premiere pro and photoshop) to do this task, video instruction. But the issue was, I kind of took all video frames at first (without dropping to any frames and that will computationally cost further) via premier and use photoshop to stitching them (according to the youtube video instruction). It was too heavy for these editor tools and didn't look better way but the output was better than anything until now. Though I took few (400+ frames) video frames only out of 1200+.

  


  

  


  ---

  


  Here are some big challenges. The original video clips have some conditions though, and it's too serious. Unlike the given demo video clips:

  


  

  • It's not straight forward, i.e. camera shaking

  


  • Lighting condition, i.e. causes different visual look at the same spot

  


  • Cameral flickering or banding

  


  


  This scenario is not included in the given demo video. And this brings additional and heavy challenges to create panorama images from such videos. Even with the non-programming way (using adobe tools) I couldn't make it any good.

  


  ---

  


  However, for now, all I'm interest to get a panorama image from the given demo video which is without the above condition. But I would love to know any comment or suggestion on that.

  


• Cannot extract audio tracks from video using ffmpeg

  29 July 2021, by Mentos

  I tried to extract and transcode audio tracks from video, but I got an error: Filter split:output1 has an unconnected output. How to fix this problem?

  


  ffmpeg command:

  


  ffmpeg -i video.mp4 -filter_complex [0]split=2[s0][s1] -map [s0] -c:a aac -dn -map -0:v? -map 0:1 -map_chapters -1 -map_metadata -1 -sn video-0.aac -map [s1] -ac 2 -c:a aac -dn -map -0:v? -map 0:2 -map_chapters -1 -map_metadata -1 -sn video-1.aac -y


  


  fprobe full output (strip information about chapters):

  


  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'video.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    title           : SomeTitle
    encoder         : Lavf58.45.100
  Duration: 00:21:35.58, start: 0.000000, bitrate: 10591 kb/s
  Chapters:
    Chapter #0:0: start 0.000000, end 70.000000
      Metadata:
        title           : Chapter 1
...
...
    Chapter #0:20: start 1278.000000, end 1295.584000
      Metadata:
        title           : Last Chapter
  Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 9754 kb/s, 23.98 fps, 23.98 tbr, 16k tbn, 47.95 tbc (default)
    Metadata:
      handler_name    : VideoHandler
      vendor_id       : [0][0][0][0]
  Stream #0:1(rus): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, stereo, fltp, 192 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
    Side data:
      audio service type: main
  Stream #0:2(eng): Audio: eac3 (ec-3 / 0x332D6365), 48000 Hz, 5.1(side), fltp, 640 kb/s
    Metadata:
      handler_name    : SoundHandler
      vendor_id       : [0][0][0][0]
    Side data:
      audio service type: main
  Stream #0:3(eng): Data: bin_data (text / 0x74786574), 0 kb/s
    Metadata:
      handler_name    : SubtitleHandler