Recherche avancée

Sur d’autres sites (16773)

• why it is taking a lot of time to convert image to video in ffmpeg ?

  20 avril 2021, par al pacino

  Below is a snippet of code i am using to convert images into videos but It is taking around 3 minutes to convert image to 5 second video.
If I run the command on Image present on a directory then it converts images to videos very swiftly. but i need to use try except to catch ConnectionError..
can anyone tell what I am doing wrong ?

  


  video = []
video_list = []
for video in videos:
    try:
        r = requests.get(video, stream=True)
        print(r.status_code)
        if not r.status_code == 200:
            message = "broken url process could not be completed"
            return message
        if r.status_code == 200:
            if video.endswith(".jpg") or video.endswith(".png") or video.endswith(".jpeg"):
                filename = str(uuid.uuid4())
                url2 = download_path + filename +".webm"
                #cmd = "ffmpeg -loop 1 -i '{}' -c:v libx264 -t 5 -pix_fmt yuv420p -vf scale={}:{} ".format(video, width, height)+ url2
                cmd = "ffmpeg -loop 1 -i '{}' -t 5 -vf scale={}:{} ".format(video, width, height) + url2
                os.system(cmd)
                path_remover(video)
                video_list.append(url2)
            else:
                video_list.append(video)
    except ConnectionError:
        for i in video_list:
            path_remover(i)
        message = "broken url process could not be completed"
        return message


  


  full log after running the command with online url :

  


  ffmpeg -loop 1 -i 'https://cdn.pixabay.com/photo/2020/10/23/12/03/arch-5678549__340.jpg' -t 5 -vf scale=1280:780 output.mp4 


ffmpeg version n4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --prefix= --prefix=/usr --disable-debug --disable-doc --disable-static --enable-cuda --enable-cuda-sdk --enable-cuvid --enable-libdrm --enable-ffplay --enable-gnutls --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-libnpp --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopus --enable-libpulse --enable-sdl2 --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libv4l2 --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-nonfree --enable-nvenc --enable-omx --enable-openal --enable-opencl --enable-runtime-cpudetect --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-xlib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, image2, from 'https://cdn.pixabay.com/photo/2021/04/17/18/26/woman-6186493__340.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 5538 kb/s
    Stream #0:0: Video: mjpeg (Progressive), yuvj420p(pc, bt470bg/unknown/unknown), 511x340 [SAR 1:1 DAR 511:340], 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[swscaler @ 0x5580ac8e7c40] deprecated pixel format used, make sure you did set range correctly
[libx264 @ 0x5580ac186b40] using SAR=1034/1129
[libx264 @ 0x5580ac186b40] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
[libx264 @ 0x5580ac186b40] profile High, level 3.2
[libx264 @ 0x5580ac186b40] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'output.mp4':
  Metadata:
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc), 1280x780 [SAR 19929:21760 DAR 511:340], q=-1--1, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.91.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=  100 fps=0.0 q=28.0 size=       0kB time=00:00:01.64 bitrate=   0.2kbits/frame=  125 fps=0.0 q=-1.0 Lsize=      90kB time=00:00:04.88 bitrate= 151.7kbits/s speed=6.79x    
video:88kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.618829%
[libx264 @ 0x5580ac186b40] frame I:1     Avg QP:16.31  size: 75247
[libx264 @ 0x5580ac186b40] frame P:31    Avg QP:16.13  size:   314
[libx264 @ 0x5580ac186b40] frame B:93    Avg QP:31.33  size:    49
[libx264 @ 0x5580ac186b40] consecutive B-frames:  0.8%  0.0%  0.0% 99.2%
[libx264 @ 0x5580ac186b40] mb I  I16..4: 10.6% 77.6% 11.8%
[libx264 @ 0x5580ac186b40] mb P  I16..4:  0.0%  0.0%  0.0%  P16..4:  2.5%  0.2%  0.2%  0.0%  0.0%    skip:97.2%
[libx264 @ 0x5580ac186b40] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.4%  0.0%  0.0%  direct: 0.0%  skip:99.6%  L0: 8.3% L1:91.7% BI: 0.0%
[libx264 @ 0x5580ac186b40] 8x8 transform intra:77.5% inter:98.7%
[libx264 @ 0x5580ac186b40] coded y,uvDC,uvAC intra: 88.4% 95.3% 79.5% inter: 0.1% 0.5% 0.0%
[libx264 @ 0x5580ac186b40] i16 v,h,dc,p:  2%  3%  0% 95%
[libx264 @ 0x5580ac186b40] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 18%  7%  6% 11% 10% 10%  8%  8%
[libx264 @ 0x5580ac186b40] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 18%  7%  5% 16% 11%  9%  6%  4%
[libx264 @ 0x5580ac186b40] i8c dc,h,v,p: 33% 21% 22% 24%
[libx264 @ 0x5580ac186b40] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x5580ac186b40] ref P L0: 85.7%  0.1% 13.2%  1.0%
[libx264 @ 0x5580ac186b40] ref B L0: 53.6% 46.4%
[libx264 @ 0x5580ac186b40] ref B L1: 92.8%  7.2%
[libx264 @ 0x5580ac186b40] kb/s:143.20
aman@aman:~/Desktop/a$ ffmpeg -loop 1 -i 'https://cdn.pixabay.com/photo/2021/04/17/18/26/woman-6186493__340.jpg' -t 5 -vf scale=1280:780 output.webm
ffmpeg version n4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --prefix= --prefix=/usr --disable-debug --disable-doc --disable-static --enable-cuda --enable-cuda-sdk --enable-cuvid --enable-libdrm --enable-ffplay --enable-gnutls --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-libnpp --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopus --enable-libpulse --enable-sdl2 --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libv4l2 --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxvid --enable-nonfree --enable-nvenc --enable-omx --enable-openal --enable-opencl --enable-runtime-cpudetect --enable-shared --enable-vaapi --enable-vdpau --enable-version3 --enable-xlib
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, image2, from 'https://cdn.pixabay.com/photo/2021/04/17/18/26/woman-6186493__340.jpg':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 5538 kb/s
    Stream #0:0: Video: mjpeg (Progressive), yuvj420p(pc, bt470bg/unknown/unknown), 511x340 [SAR 1:1 DAR 511:340], 25 fps, 25 tbr, 25 tbn, 25 tbc
File 'output.webm' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> vp9 (libvpx-vp9))
Press [q] to stop, [?] for help
[swscaler @ 0x5630553879c0] deprecated pixel format used, make sure you did set range correctly
[libvpx-vp9 @ 0x563054cd2500] v1.7.0
[libvpx-vp9 @ 0x563054cd2500] Neither bitrate nor constrained quality specified, using default CRF of 32
Output #0, webm, to 'output.webm':
  Metadata:
    encoder         : Lavf58.45.100
    Stream #0:0: Video: vp9 (libvpx-vp9), yuv420p, 1280x780 [SAR 19929:21760 DAR 511:340], q=-1--1, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc58.91.100 libvpx-vp9
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=   57 fps=0.0 q=0.0 size=       1kB time=00:00:01.28 bitrate=   3.3kbits/sframe=  103 fps=102 q=0.0 size=       1kB time=00:00:03.12 bitrate=   1.3kbits/sframe=  125 fps= 85 q=0.0 Lsize=      51kB time=00:00:04.96 bitrate=  84.0kbits/s speed=3.37x    
video:50kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.571023%


  


• Trying to create mp4 proxy from Sony RAW file using FFMPEG

  16 avril 2021, par Daniel Akerman

  I'm trying to create a viewable mp4 proxy file (MPEG-4 AVC with AAC LC audio) from a Sony RAW SQ mxf file.

  


  I was able to create an mp4 with a single track of audio but no video by using this :

  


  ffmpeg -i Raw.mxf -c:v libx264 -c:a aac -strict experimental Raw-test.mp4


  


  I have a Mac, using ffmpeg version 4.3.2

  


  This is the file I'm trying to convert:
File spec:
General
Complete name                            : Raw.mxf
Format                                   : MXF
Format version                           : 1.3
Format profile                           : OP-1a
Format settings                          : Closed / Complete
File size                                : 2.33 GiB
Duration                                 : 20 s 200 ms
Overall bit rate                         : 990 Mb/s
Encoded date                             : 2019-06-23 19:48:44.000
Writing application                      : Sony AXS 2.0.0.0.1
Writing library                          : Sony SC Platform 8.2.0.0.1

Video
ID                                       : 2
Format                                   : Sony RAW SQ
Format settings, wrapping mode           : Frame (D-10)
Codec ID                                 : 0E060D0302010100-0E06040102040201
Duration                                 : 20 s 200 ms
Bit rate                                 : 986 Mb/s
Width                                    : 4 096 pixels
Original width                           : 4 128 pixels
Height                                   : 2 160 pixels
Original height                          : 2 192 pixels
Display aspect ratio                     : 1.896
Frame rate                               : 25.000 FPS
Scan type                                : Progressive
Bits/(Pixel*Frame)                       : 4.457
Stream size                              : 2.32 GiB (100%)

Audio #1
ID                                       : 3
Format                                   : PCM
Format settings                          : Little
Format settings, wrapping mode           : Frame (AES)
Codec ID                                 : 0D01030102060300
Duration                                 : 20 s 200 ms
Bit rate mode                            : Constant
Bit rate                                 : 1 152 kb/s
Channel(s)                               : 1 channel
Sampling rate                            : 48.0 kHz
Frame rate                               : 25.000 FPS (1920 SPF)
Bit depth                                : 24 bits
Stream size                              : 2.77 MiB (0%)
Locked                                   : Yes

Audio #2
ID                                       : 4
Format                                   : PCM
Format settings                          : Little
Format settings, wrapping mode           : Frame (AES)
Codec ID                                 : 0D01030102060300
Duration                                 : 20 s 200 ms
Bit rate mode                            : Constant
Bit rate                                 : 1 152 kb/s
Channel(s)                               : 1 channel
Sampling rate                            : 48.0 kHz
Frame rate                               : 25.000 FPS (1920 SPF)
Bit depth                                : 24 bits
Stream size                              : 2.77 MiB (0%)
Locked                                   : Yes

Audio #3
ID                                       : 5
Format                                   : PCM
Format settings                          : Little
Format settings, wrapping mode           : Frame (AES)
Codec ID                                 : 0D01030102060300
Duration                                 : 20 s 200 ms
Bit rate mode                            : Constant
Bit rate                                 : 1 152 kb/s
Channel(s)                               : 1 channel
Sampling rate                            : 48.0 kHz
Frame rate                               : 25.000 FPS (1920 SPF)
Bit depth                                : 24 bits
Stream size                              : 2.77 MiB (0%)
Locked                                   : Yes

Audio #4
ID                                       : 6
Format                                   : PCM
Format settings                          : Little
Format settings, wrapping mode           : Frame (AES)
Codec ID                                 : 0D01030102060300
Duration                                 : 20 s 200 ms
Bit rate mode                            : Constant
Bit rate                                 : 1 152 kb/s
Channel(s)                               : 1 channel
Sampling rate                            : 48.0 kHz
Frame rate                               : 25.000 FPS (1920 SPF)
Bit depth                                : 24 bits
Stream size                              : 2.77 MiB (0%)
Locked                                   : Yes

Other #1
ID                                       : 1-Material
Type                                     : Time code
Format                                   : MXF TC
Frame rate                               : 25.000 FPS
Time code of first frame                 : 20:50:05:08
Time code settings                       : Material Package
Time code, striped                       : Yes

Other #2
ID                                       : 1-Source
Type                                     : Time code
Format                                   : MXF TC
Frame rate                               : 25.000 FPS
Time code of first frame                 : 20:50:05:08
Time code settings                       : Source Package
Time code, striped                       : Yes

Other #3
ID                                       : 7
Format                                   : Acquisition Metadata
Muxing mode                              : Ancillary data / RDD 18
Duration                                 : 20 s 200 ms
Frame rate                               : 25.000 FPS
CaptureGammaEquation_FirstFrame          : SceneLinear
MacroSetting_FirstFrame                  : Off
OpticalExtenderMagnification_FirstFrame  : 100%
LensAttributes_FirstFrame                : Unknown
AutoExposureMode_FirstFrame              : Manual
NeutralDensityFilterWheelSetting_FirstFr : Clear
ImageSensorDimensionEffectiveWidth_First : 24.003 mm
ImageSensorDimensionEffectiveHeight_Firs : 12.658 mm
CaptureFrameRate_FirstFrame              : 25.000 fps
ImageSensorReadoutMode_FirstFrame        : Progressive frame
ShutterSpeed_Angle_FirstFrame            : 180.0°
ShutterSpeed_Time_FirstFrame             : 1/50 s
CameraMasterGainAdjustment_FirstFrame    : 0.00 dB
ISOSensitivity_FirstFrame                : 1250
ElectricalExtenderMagnification_FirstFra : 100%
AutoWhiteBalanceMode_FirstFrame          : Preset
WhiteBalance_FirstFrame                  : 3200 K
CameraMasterBlackLevel_FirstFrame        : 3.0%
CameraAttributes_FirstFrame              : F55 101044
ExposureIndexofPhotoMeter_FirstFrame     : 1250
GammaForCDL_FirstFrame                   : Undefined
ASC_CDL_V12_FirstFrame                   : sR=1.0 sG=1.0 sB=1.0 oR=0.0 oG=0.0 oB=0.0 pR=1.0 pG=1.0 pB=1.0 sat=1.0
EffectiveMarkerCoverage_FirstFrame       : 4096x4096
EffectiveMarkerAspectRatio_FirstFrame    : 4096x2160
CameraProcessDiscriminationCode_FirstFra : 201
RawBlackCodeValue_FirstFrame             : 512
RawGrayCodeValue_FirstFrame              : 1504
RawWhiteCodeValue_FirstFrame             : 5472
MonitoringDescriptions_FirstFrame        : SonyProf2:LC-709typA
MonitoringBaseCurve_FirstFrame           : 0E06040101010408

Other #4
Type                                     : Time code
Format                                   : SMPTE TC
Muxing mode                              : SDTI
Frame rate                               : 25.000 FPS
Time code of first frame                 : 20:50:05:08


  


  I'm not sure which camera it is from, I only have the information extracted by MediaInfo that I included.

  


  The FFMEG log is:
$ ffmpeg -i /Users/Aker84150/Downloads/Raw.mxf -c:v libx264 -c:a aac -strict experimental /Users/Aker84150/Downloads/Raw-test.mp4ffmpeg version 4.3.2 Copyright (c) 2000-2021 the FFmpeg developers  built with Apple clang version 11.0.0 (clang-1100.0.33.17)  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.3.2_4 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox  libavutil      56. 51.100 / 56. 51.100  libavcodec     58. 91.100 / 58. 91.100  libavformat    58. 45.100 / 58. 45.100  libavdevice    58. 10.100 / 58. 10.100  libavfilter     7. 85.100 /  7. 85.100  libavresample   4.  0.  0 /  4.  0.  0  libswscale      5.  7.100 /  5.  7.100  libswresample   3.  7.100 /  3.  7.100  libpostproc    55.  7.100 / 55.  7.100[mxf @ 0x7f808f008800] could not resolve sub descriptor strong ref[mxf @ 0x7f808f008800] source track 2: stream 0, no descriptor found[mxf @ 0x7f808f008800] could not resolve sub descriptor strong ref    Last message repeated 4 times[mxf @ 0x7f808f008800] Could not find codec parameters for stream 0 (Video: none, none): unknown codecConsider increasing the value for the 'analyzeduration' and 'probesize' optionsGuessed Channel Layout for Input Stream #0.1 : monoGuessed Channel Layout for Input Stream #0.2 : monoGuessed Channel Layout for Input Stream #0.3 : monoGuessed Channel Layout for Input Stream #0.4 : monoInput #0, mxf, from '/Users/Aker84150/Downloads/Raw.mxf':  Metadata:    operational_pattern_ul: 060e2b34.04010101.0d010201.01010900    product_uid     : cede1604-8280-11de-8a39-08004678031c    uid             : e8f36160-95ef-11e9-83ab-08004632275a    generation_uid  : e8f3616a-95ef-11e9-b502-08004632275a    company_name    : Sony    product_name    : AXS     product_version : 2.0    application_platform: Sony SC Platform    modification_date: 2019-06-23T19:48:44.000000Z    material_package_umid: 0x060A2B340101010501010D43130000009CDA54D3578605DC080046020132275A    timecode        : 20:50:05:08  Duration: 00:00:20.20, start: 0.000000, bitrate: 990426 kb/s    Stream #0:0: Video: none, none, 25 tbr, 25 tbn, 25 tbc    Stream #0:1: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s    Metadata:      file_package_umid: 0x060A2B340101010501010D43130000009DDA54D3578605DC080046020132275A    Stream #0:2: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s    Metadata:      file_package_umid: 0x060A2B340101010501010D43130000009DDA54D3578605DC080046020132275A    Stream #0:3: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s    Metadata:      file_package_umid: 0x060A2B340101010501010D43130000009DDA54D3578605DC080046020132275A    Stream #0:4: Audio: pcm_s24le, 48000 Hz, mono, s32 (24 bit), 1152 kb/s    Metadata:      file_package_umid: 0x060A2B340101010501010D43130000009DDA54D3578605DC080046020132275A    Stream #0:5: Data: none    Metadata:      file_package_umid: 0x060A2B340101010501010D43130000009DDA54D3578605DC080046020132275A      data_type       : vbi_vanc_smpte_436MStream mapping:  Stream #0:1 -> #0:0 (pcm_s24le (native) -> aac (native))Press [q] to stop, [?] for helpOutput #0, mp4, to '/Users/Aker84150/Downloads/Raw-test.mp4':  Metadata:    operational_pattern_ul: 060e2b34.04010101.0d010201.01010900    product_uid     : cede1604-8280-11de-8a39-08004678031c    uid             : e8f36160-95ef-11e9-83ab-08004632275a    generation_uid  : e8f3616a-95ef-11e9-b502-08004632275a    company_name    : Sony    product_name    : AXS     product_version : 2.0    application_platform: Sony SC Platform    modification_date: 2019-06-23T19:48:44.000000Z    material_package_umid: 0x060A2B340101010501010D43130000009CDA54D3578605DC080046020132275A    timecode        : 20:50:05:08    encoder         : Lavf58.45.100    Stream #0:0: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp (24 bit), 69 kb/s    Metadata:      file_package_umid: 0x060A2B340101010501010D43130000009DDA54D3578605DC080046020132275A      encoder         : Lavc58.91.100 aacsize=     176kB time=00:00:20.20 bitrate=  71.3kbits/s speed=66.6x    video:0kB audio:171kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.619946%[aac @ 0x7f808f89a800] Qavg: 121.639


  


  Thanks

  


• VP8 Codec Optimization Update

  16 juin 2010, par noreply@blogger.com (John Luther) — inside webm

  Since WebM launched in May, the team has been working hard to make the VP8 video codec faster. Our community members have contributed improvements, but there’s more work to be done in some interesting areas related to performance (more on those below).

  
  

  Encoder

  
  The VP8 encoder is ripe for speed optimizations. Scott LaVarnway’s efforts in writing an x86 assembly version of the quantizer will help in this goal significantly as the quantizer is called many times while the encoder makes decisions about how much detail from the image will be transmitted.

  For those of you eager to get involved, one piece of low-hanging fruit is writing a SIMD version of the ARNR temporal filtering code. Also, much of the assembly code only makes use of the SSE2 instruction set, and there surely are newer extensions that could be made use of. There are also redundant code removal and other general cleanup to be done ; (Yaowu Xu has submitted some changes for these).
  
  At a higher level, someone can explore some alternative motion search strategies in the encoder. Eventually the motion search can be decoupled entirely to allow motion fields to be calculated elsewhere (for example, on a graphics processor).
  

  Decoder

  
  Decoder optimizations can bring higher resolutions and smoother playback to less powerful hardware.

  Jeff Muizelaar has submitted some changes which combine the IDCT and summation with the predicted block into a single function, helping us avoid storing the intermediate result, thus reducing memory transfers and avoiding cache pollution. This changes the assembly code in a fundamental way, so we will need to sync the other platforms up or switch them to a generic C implementation and accept the performance regression. Johann Koenig is working on implementing this change for ARM processors, and we’ll merge these changes into the mainline soon.

  In addition, Tim Terriberry is attacking a different method of bounds checking on the "bool decoder." The bool decoder is performance-critical, as it is called several times for each bit in the input stream. The current code handles this check with a simple clamp in the innermost loops and a less-frequent copy into a circular buffer. This can be expensive at higher data rates. Tim’s patch removes the circular buffer, but uses a more complex clamp in the innermost loops. These inner loops have historically been troublesome on embedded platforms.
  
  To contribute in these efforts, I’ve started working on rewriting higher-level parts of the decoder. I believe there is an opportunity to improve performance by paying better attention to data locality and cache layout, and reducing memory bus traffic in general. Another area I plan to explore is improving utilization in the multi-threaded decoder by separating the bitstream decoding from the rest of the image reconstruction, using work units larger than a single macroblock, and not tying functionality to a specific thread. To get involved in these areas, subscribe to the codec-devel mailing list and provide feedback on the code as it’s written.
  

  Embedded Processors

  
  We want to optimize multiple platforms, not just desktops. Fritz Koenig has already started looking at the performance of VP8 on the Intel Atom platform. This platform need some attention as we wrote our current x86 assembly code with an out-of-order processor in mind. Since Atom is an in-order processor (much like the original Pentium), the instruction scheduling of all of the x86 assembly code needs to be reexamined. One option we’re looking at is scheduling the code for the Atom processor and seeing if that impacts the performance on other x86 platforms such as the Via C3 and AMD Geode. This is shaping up to be a lot of work, but doing it would provide us with an opportunity to tighten up our assembly code.
  
  These issues, along with wanting to make better use of the larger register file on x86_64, may reignite every assembly programmer’s (least ?) favorite debate : whether or not to use intrinsics. Yunqing Wang has been experimenting with this a bit, but initial results aren’t promising. If you have experience in dealing with a lot of assembly code across several similar-but-kinda-different platforms, these maintainability issues might be familiar to you. I hope you’ll share your thoughts and experiences on the codec-devel mailing list.
  
  Optimizing codecs is an iterative (some would say never-ending) process, so stay tuned for more posts on the progress we’re making, and by all means, start hacking yourself.
  
  It’s exciting to see that we’re starting to get substantial code contributions from developers outside of Google, and I look forward to more as WebM grows into a strong community effort.
  
  John Koleszar is a software engineer at Google.