Recherche avancée

Sur d’autres sites (6351)

• ffmpeg "Manifest too large" when downloading youtube video

  22 avril 2024, par ynn

  TL ;DR

  


  I'm trying to programmatically download a part of a YouTube video. The widely known procedure doesn't work for some videos and I'd like to overcome this situation.

  


  ---

  


  Context

  


  I'm trying to programmatically download a part of a YouTube video. As described in How to download portion of video with youtube-dl command ?, you can achieve this by the following commands.

  


  #Converts a human-readable URL to longer URLs for internal use.
~ $ youtube-dl --get-url 'https://www.youtube.com/watch?v=POrfo478HRI'
https://r1---sn-3pm76n76.googlevideo.com/videoplayback?expire=1597593009&ei=UQE5X7bXEYqO4QLs17KACQ&ip=10.100.238.99&id=o-AD0HfadoeimIbuf9t8Anru5X9V7cER08YyOz4iKZOCHL&itag=299&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278%2C298%2C299%2C302%2C303&source=youtube&requiressl=yes&mh=NV&mm=31%2C26&mn=sn-3pm76n76%2Csn-oguesnze&ms=au%2Conr&mv=m&mvi=1&pl=17&initcwndbps=908750&vprv=1&mime=video%2Fmp4&gir=yes&clen=9205054989&dur=17931.000&lmt=1597347724488304&mt=1597571322&fvip=1&keepalive=yes&fexp=23883098&c=WEB&txp=7316222&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIgYDqTm0CnLfG0kxsYNrAdtAUYB7alnowiaBArU8R5bBwCIQDeCRDJHFO_PWcnbeFaJvip80deuboN4Pi1x3eRhJBxlQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAKTkyTIGVkbWWVx4lqFz77rOdG1FzkM5XuTnYOqtHT4oAiEA9u1R47jPz-mjj4DRXpqqqpUbfaFMW36KbG_pR2jovFM%3D&ratebypass=yes
https://r1---sn-3pm76n76.googlevideo.com/videoplayback?expire=1597593009&ei=UQE5X7bXEYqO4QLs17KACQ&ip=10.100.238.99&id=o-AD0HfadoeimIbuf9t8Anru5X9V7cER08YyOz4iKZOCHL&itag=251&source=youtube&requiressl=yes&mh=NV&mm=31%2C26&mn=sn-3pm76n76%2Csn-oguesnze&ms=au%2Conr&mv=m&mvi=1&pl=17&initcwndbps=908750&vprv=1&mime=audio%2Fwebm&gir=yes&clen=268945126&dur=17931.001&lmt=1597348111385641&mt=1597571322&fvip=1&keepalive=yes&fexp=23883098&c=WEB&txp=7311222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRgIhAK8XGu3vHucHwqERZ7TQniuvKEd-NTbMkwZbu8EXJ5E3AiEAiS7OYFOcsIrD2xp-AJNzucj1H9ZKMlmkCl_sU7__dZo%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAKTkyTIGVkbWWVx4lqFz77rOdG1FzkM5XuTnYOqtHT4oAiEA9u1R47jPz-mjj4DRXpqqqpUbfaFMW36KbG_pR2jovFM%3D&ratebypass=yes

#Stores the result to a variable.
~ $ url_video_part='https://r1---sn-3pm76n76.googlevideo.com/videoplayback?expire=1597593009&ei=UQE5X7bXEYqO4QLs17KACQ&ip=10.100.238.99&id=o-AD0HfadoeimIbuf9t8Anru5X9V7cER08YyOz4iKZOCHL&itag=299&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C278%2C298%2C299%2C302%2C303&source=youtube&requiressl=yes&mh=NV&mm=31%2C26&mn=sn-3pm76n76%2Csn-oguesnze&ms=au%2Conr&mv=m&mvi=1&pl=17&initcwndbps=908750&vprv=1&mime=video%2Fmp4&gir=yes&clen=9205054989&dur=17931.000&lmt=1597347724488304&mt=1597571322&fvip=1&keepalive=yes&fexp=23883098&c=WEB&txp=7316222&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRQIgYDqTm0CnLfG0kxsYNrAdtAUYB7alnowiaBArU8R5bBwCIQDeCRDJHFO_PWcnbeFaJvip80deuboN4Pi1x3eRhJBxlQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAKTkyTIGVkbWWVx4lqFz77rOdG1FzkM5XuTnYOqtHT4oAiEA9u1R47jPz-mjj4DRXpqqqpUbfaFMW36KbG_pR2jovFM%3D&ratebypass=yes'

#Stores the result to a variable.
~ $ url_audio_part='https://r1---sn-3pm76n76.googlevideo.com/videoplayback?expire=1597593009&ei=UQE5X7bXEYqO4QLs17KACQ&ip=10.100.238.99&id=o-AD0HfadoeimIbuf9t8Anru5X9V7cER08YyOz4iKZOCHL&itag=251&source=youtube&requiressl=yes&mh=NV&mm=31%2C26&mn=sn-3pm76n76%2Csn-oguesnze&ms=au%2Conr&mv=m&mvi=1&pl=17&initcwndbps=908750&vprv=1&mime=audio%2Fwebm&gir=yes&clen=268945126&dur=17931.001&lmt=1597348111385641&mt=1597571322&fvip=1&keepalive=yes&fexp=23883098&c=WEB&txp=7311222&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRgIhAK8XGu3vHucHwqERZ7TQniuvKEd-NTbMkwZbu8EXJ5E3AiEAiS7OYFOcsIrD2xp-AJNzucj1H9ZKMlmkCl_sU7__dZo%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRgIhAKTkyTIGVkbWWVx4lqFz77rOdG1FzkM5XuTnYOqtHT4oAiEA9u1R47jPz-mjj4DRXpqqqpUbfaFMW36KbG_pR2jovFM%3D&ratebypass=yes'

#Downloads the video for one minute from 12:45.
~ $ ffmpeg    -ss 12:45 -i "${url_video_part}"    -ss 12:45 -i "${url_audio_part}"    -t 1:00 output.mkv
frame= 3600 fps= 49 q=-1.0 Lsize=   27047kB time=00:00:59.99 bitrate=3692.9kbits/s speed=0.813x


  


  This does work perfectly.

  


  ---

  


  Problem

  


  For some videos^†1, the internal URLs are of the form https://manifest.googlevideo.com/... instead of https://RANDOM-STRING.googlevideo.com/... as in the example above.

  


  ~ $ youtube-dl --get-url 'https://www.youtube.com/watch?v=KRMfnvLODZQ'
https://manifest.googlevideo.com/api/manifest/dash/expire/1597593866/ei/qgQ5X5maN8-ilQSu15W4CQ/ip/10.100.238.99/id/KRMfnvLODZQ.1/source/yt_live_broadcast/requiressl/yes/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/force_finished/1/vprv/1/keepalive/yes/fexp/23883098/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cforce_finished%2Cvprv%2Citag%2Cplaylist_type/sig/AOq0QJ8wRQIhANYKQvKTN8BRmhp7--4tQp9aqRp_qIGjh472BTYLyJ62AiAautFF86xGDHOe7mYlYaZ1W86EM4k2e6UiJS9Q91dMog%3D%3D
https://manifest.googlevideo.com/api/manifest/dash/expire/1597593866/ei/qgQ5X5maN8-ilQSu15W4CQ/ip/10.100.238.99/id/KRMfnvLODZQ.1/source/yt_live_broadcast/requiressl/yes/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/force_finished/1/vprv/1/keepalive/yes/fexp/23883098/itag/0/playlist_type/DVR/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cforce_finished%2Cvprv%2Citag%2Cplaylist_type/sig/AOq0QJ8wRQIhANYKQvKTN8BRmhp7--4tQp9aqRp_qIGjh472BTYLyJ62AiAautFF86xGDHOe7mYlYaZ1W86EM4k2e6UiJS9Q91dMog%3D%3D


  


  And downloading such videos always fails.

  


  ~ $ ffmpeg    -ss 12:45 -i "${url_video_part}"    -ss 12:45 -i "${url_audio_part}"    -t 1:00 output.mkv
[dash @ 0x55fa6a1a4400] Manifest too large: 3776722
https://manifest.googlevideo.com/...: Invalid data found when processing input


  


  Please note downloading full of such videos using youtube-dl alone succeeds.

  


  _{†1 : As far as we tested, recently (e.g. yesterday) uploaded videos have such URLs and when we retry the same youtube-dl --get-url command after a day or so, we get "innocent" URLs like in the first example.}

  


  ---

  


  Questions

  


  How can I download parts of such videos

  


  


  • by specifying additional options for ffmpeg,

    


  


  • by somehow forcefully retrieving internal URLs of the form https://RANDOM-STRING.googlevideo.com/...,

    


  


  • or by something which I have no idea at all ?

    


  


  


  My current workaround is just to wait for a day or two until the internal URLs change, which is much slower than downloading the full video and then cropping it.

  


  ---

  


  Environments

  


  ~ $ lsb_release -d
Description:    Arch Linux

~ $ ffmpeg -version
ffmpeg version n4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 10.1.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avisynth --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-nvdec --enable-nvenc --enable-omx --enable-shared --enable-version3
libavutil      56. 51.100 / 56. 51.100
libavcodec     58. 91.100 / 58. 91.100
libavformat    58. 45.100 / 58. 45.100
libavdevice    58. 10.100 / 58. 10.100
libavfilter     7. 85.100 /  7. 85.100
libswscale      5.  7.100 /  5.  7.100
libswresample   3.  7.100 /  3.  7.100
libpostproc    55.  7.100 / 55.  7.100

~ $ youtube-dl --version
2020.07.28


  


• How to accelerate minterpolate in ffmpeg ? [closed]

  24 août 2020, par Ken

  I'm using minterpolate of ffmpeg(version git-2020-08-21) with this command on Windows 10 :

  


  ffmpeg -i $file -filter:v "minterpolate=mi_mode=mci:me_mode=bidir:mc_mode=aobmc:vsbmc=1:fps=60" -c:a copy "$file_60fps.mp4"


  


  But the CPU load is only about 15% to 20% while working.

  


  My CPU is AMD 4800H with 8 cores and 16 threads. And I have a NVIDIA GTX1650 with CUDA.

  


  But I cannot find any document about GPU acceleration or multithread methods of minterpolate.

  


  Have tried -threads 16 but nothing changed.

  


  So how to accelerate the process ?

  


• ffmpeg extract frame and get timestamp based on creation time for each extracted frame in exif or txt

  4 août 2021, par Scaling

  I try to extract frames from a underwater survey video using ffmpeg. the extracted frames should be merged with a gps position based on a timestamp later in the process. the code i use to extract the frames is :

  



  ffmpeg -i GOPR0173.MP4 -qscale:v 2 -r 1 frames/%4d.jpg.

  



  I need a solution to get the real timestamp of the frame based on the survey datetime, so that the timestamp of the frame equals the real time when the video was created. Ideally the timestamp is saved in the images exif or if this is not possible in a seperate textfile and the timestamp is used as name of the files like 2020_01_01_HH_MM_SS.jpg.

  



  My idea is to use the creation_time of the video add the timestamp of the frame to this and store the result in the exif of the extracted frame or if this is not possible in a txt file.

  



  Im new to ffmpeg and really lost and would appreciate any help to solve this task !
If anyone could provide me the necessary code, that would be awesome !

  



  C:\Unterwasserfotos>ffmpeg -i GOPR0173.MP4 -vstats -qscale:v 2 -r 0.1 frames/%4d.jpg
ffmpeg version git-2020-01-21-fc6fde2 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9.2.1 (GCC) 20200122
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls
 --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enabl
e-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-
libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwol
ame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enab
le-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable
-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable
-libaom --enable-libmfx --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enabl
e-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 38.100 / 56. 38.100
  libavcodec     58. 66.100 / 58. 66.100
  libavformat    58. 35.104 / 58. 35.104
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 71.100 /  7. 71.100
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'GOPR0173.MP4':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    creation_time   : 2020-01-21T12:33:10.000000Z
    firmware        : HD4.02.05.00.00
  Duration: 00:02:21.85, start: 0.000000, bitrate: 30152 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, bt709), 1920x1080 [SAR
 1:1 DAR 16:9], 30000 kb/s, 50 fps, 50 tbr, 90k tbn, 100 tbc (default)
    Metadata:
      creation_time   : 2020-01-21T12:33:10.000000Z
      handler_name    :         GoPro AVC
      encoder         : GoPro AVC encoder
      timecode        : 12:33:11:16
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 127 kb/s (defa
ult)
    Metadata:
      creation_time   : 2020-01-21T12:33:10.000000Z
      handler_name    :         GoPro AAC
      timecode        : 12:33:11:16
    Stream #0:2(eng): Data: none (tmcd / 0x64636D74) (default)
    Metadata:
      creation_time   : 2020-01-21T12:33:10.000000Z
      handler_name    :         GoPro TCD
      timecode        : 12:33:11:16
    Stream #0:3(eng): Data: none (fdsc / 0x63736466), 12 kb/s (default)
    Metadata:
      creation_time   : 2020-01-21T12:33:10.000000Z
      handler_name    :         GoPro SOS
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native))
Press [q] to stop, [?] for help
Output #0, image2, to 'frames/%4d.jpg':
  Metadata:
    major_brand     : mp41
    minor_version   : 538120216
    compatible_brands: mp41
    firmware        : HD4.02.05.00.00
    encoder         : Lavf58.35.104
    Stream #0:0(eng): Video: mjpeg, yuvj420p(pc), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s,
 0.10 fps, 0.10 tbn, 0.10 tbc (default)
    Metadata:
      creation_time   : 2020-01-21T12:33:10.000000Z
      handler_name    :         GoPro AVC
      timecode        : 12:33:11:16
      encoder         : Lavc58.66.100 mjpeg
    Side data:
      cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: N/A
frame=   16 fps=0.3 q=2.0 Lsize=N/A time=00:02:40.00 bitrate=N/A dup=0 drop=7076 speed=3.26x
video:1938kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown´´´