Recherche avancée

Sur d’autres sites (8945)

• ffmpeg mp4 x264 encoding -> playback causes "pending" in both crome and IE causing 10+ seconds delay

  29 septembre 2014, par user1978645

  After encoding a video to mp4(x264 with aac) ; I have the following weird behaviour in both crome and IE :

  Im serving the content from https with spdy enables.

  It takes up to 30 seconds before the video is actually played, in the mean time i cannot reload the page it shows up as "pending". (even after i have visual on the video sometimes it takes 10-20 seconds before i can actually reload the page, or navigate to another url on the same domain)

  After looking at the "network" tab in developer tools, i see the following requests for 1 page/video :

  Path        Method Status   Type         Initiator   Size/content  Time/latency
  
  video.mp4   GET    206 OK   video/mp4    other       32.3KB/32.0kb  600ms/339MS 
  
  video.mp4   GET    206 OK   video/mp4    other       123kb/123b     21.85s/21.46s
  
  video.mp4   GET    206 OK   video/mp4    other       7.1MB/7.2MB    1.4min/2ms

  I tried to isolate the problem, When i use an mp4 video from the internet (for example the demo video of jplayer) and load it from my server, it loads rapidly, without delays.

  So it makes me think the problem lies within the encoding. I tried various things.
  FFmpeg :

  • csr 69 (low quality)

  • various options

  HTML :

  • preload="none"

  • javascript loading/playing of the movie

  • type=’mp4/video’

  • no posterimage

  But i cannot resolve the problem. Does anyone have a clue what is causing this ?

  I have a download speed of 300kb/s and the movie is 6MB.

  After the video starts, the video isn’t fully buffered, so i wonder : What is the html5 videoplayer doing all this time ?

  The problem also blocks the connections. When i press "F5" in both chrome and IE the page beeing reloaded comes up in the network tab as "pending" and it can take 10 to 20 seconds before the page actually reloads.

  ffmpeg command : (i used various commands but this is just 1 example which causes the problem)

  /root/bin/ffmpeg    -threads 1 -y  -i /home/flirtzo/public_html//userfiles/files/94e76a18a7838e62ecb23cf0c374b1b798e7b936  -threads 0 -codec:a libfdk_aac -b:a 128k  -vf "scale=-2:320" -preset veryslow -vcodec h264 -acodec aac -strict -2  /home/flirtzo/public_html/userfiles/files/b2/72/695f4eba95169a3f29564bf9571c703b05f1b5974f5156da633eb139c80a1575452e2858dfc61cc82bfca02d2b156aa64d4503695756481dc2a5d1c673a4cdea-94e76a18a7838e62ecb23cf0c374b1b798e7b936.mp4

  Output :

  ffmpeg version git-2014-04-16-c150e2c Copyright (c) 2000-2014 the FFmpeg developers
  
    built on Sep 28 2014 21:08:17 with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-4)
  
    configuration: --prefix=/root/ffmpeg_build --extra-cflags=-I/root/ffmpeg_build/include --extra-        ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --extra-libs=-ldl --enable-gpl --enable-nonfree -    -enable-libfdk_aac --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-    libx264
  
    libavutil      52. 76.100 / 52. 76.100
  
    libavcodec     55. 58.103 / 55. 58.103
  
    libavformat    55. 37.100 / 55. 37.100
  
    libavdevice    55. 13.100 / 55. 13.100
  
    libavfilter     4.  4.100 /  4.  4.100
  
    libswscale      2.  6.100 /  2.  6.100
  
    libswresample   0. 18.100 /  0. 18.100
  
    libpostproc    52.  3.100 / 52.  3.100
  
  Input #0, mpeg, from         '/home/flirtzo/public_html//userfiles/files/94e76a18a7838e62ecb23cf0c374b1b798e7b936':
  
    Duration: 00:00:25.97, start: 0.340078, bitrate: 29004 kb/s
  
      Stream #0:0[0x1e0]: Video: mpeg1video, yuv420p(tv), 352x240 [SAR 200:219 DAR 880:657], 1150     kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 29.97 tbc
  
      Stream #0:1[0x1c0]: Audio: mp2, 44100 Hz, stereo, s16p, 224 kb/s
  
  [libx264 @ 0x2b189c0] using SAR=1199/1314
  
  [libx264 @ 0x2b189c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
  
  [libx264 @ 0x2b189c0] profile High, level 1.3
  
  [libx264 @ 0x2b189c0] 264 - core 142 r2 d6b4e63 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 -     http://www.videolan.org/x264.html - options: cabac=1 ref=2 deblock=1:0:0 analyse=0x3:0x113 me=hex     subme=4 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0     deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 lookahead_threads=1 sliced_threads=0 nr=0     decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0     direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0     rc_lookahead=20 rc=crf mbtree=1 crf=51.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
  
  Output #0, mp4, to     '/home/flirtzo/public_html/userfiles/files/b2/72/695f4eba95169a3f29564bf9571c703b05f1b5974f5156da633e    b139c80a1575452e2858dfc61cc82bfca02d2b156aa64d4503695756481dc2a5d1c673a4cdea-    94e76a18a7838e62ecb23cf0c374b1b798e7b936.mp4':
  
    Metadata:
  
      encoder         : Lavf55.37.100
  
      Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 320x218 [SAR 1199:1314        DAR 880:657], q=-1--1, 30k tbn, 29.97 tbc
  
      Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 44100 Hz, stereo, fltp, 128 kb/s
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mpeg1video -> libx264)
  
    Stream #0:1 -> #0:1 (mp2 -> aac)
  
  Press [q] to stop, [?] for help
  
  frame=16127 fps=786 q=-1.0 Lsize=   10559kB time=00:08:58.12 bitrate= 160.7kbits/s dup=12 drop=0
  
  video:1586kB audio:8410kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead:     5.638589%
  
  [libx264 @ 0x2b189c0] frame I:109   Avg QP:50.58  size:   494
  
  [libx264 @ 0x2b189c0] frame P:9537  Avg QP:51.00  size:   138
  
  [libx264 @ 0x2b189c0] frame B:6481  Avg QP:51.00  size:    40
  
  [libx264 @ 0x2b189c0] consecutive B-frames: 21.8% 72.1%  5.4%  0.7%
  
  [libx264 @ 0x2b189c0] mb I  I16..4: 46.1% 53.9%  0.0%
  
  [libx264 @ 0x2b189c0] mb P  I16..4:  6.0%  6.1%  0.0%  P16..4: 12.7%  1.1%  0.1%  0.0%  0.0%        skip:74.1%
  
  [libx264 @ 0x2b189c0] mb B  I16..4:  0.1%  0.1%  0.0%  B16..8:  6.3%  0.0%  0.0%  direct: 0.7%      skip:92.9%  L0:38.6% L1:61.2% BI: 0.2%
  
  [libx264 @ 0x2b189c0] 8x8 transform intra:50.8% inter:85.6%
  
  [libx264 @ 0x2b189c0] coded y,uvDC,uvAC intra: 5.9% 39.5% 0.1% inter: 0.1% 1.0% 0.0%
  
  [libx264 @ 0x2b189c0] i16 v,h,dc,p: 56% 30%  7%  7%
  
  [libx264 @ 0x2b189c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 12% 11% 62%  3%  3%  3%  3%  2%  2%
  
  [libx264 @ 0x2b189c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 19%  5% 73%  2%  0%  0%  0%  0%  2%
  
  [libx264 @ 0x2b189c0] i8c dc,h,v,p: 97%  1%  2%  0%
  
  [libx264 @ 0x2b189c0] Weighted P-Frames: Y:5.5% UV:2.7%
  
  [libx264 @ 0x2b189c0] ref P L0: 64.8% 35.2%
  
  [libx264 @ 0x2b189c0] ref B L0: 75.2% 24.8%
  
  [libx264 @ 0x2b189c0] ref B L1: 99.3%  0.7%
  
  [libx264 @ 0x2b189c0] kb/s:24.13
  
  ffmpeg version git-2014-04-16-c150e2c Copyright (c) 2000-2014 the FFmpeg developers
  
    built on Sep 28 2014 21:08:17 with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-4)
  
    configuration: --prefix=/root/ffmpeg_build --extra-cflags=-I/root/ffmpeg_build/include --extra-    ldflags=-L/root/ffmpeg_build/lib --bindir=/root/bin --extra-libs=-ldl --enable-gpl --enable-nonfree -    -enable-libfdk_aac --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-    libx264
  
    libavutil      52. 76.100 / 52. 76.100
  
    libavcodec     55. 58.103 / 55. 58.103
  
    libavformat    55. 37.100 / 55. 37.100
  
    libavdevice    55. 13.100 / 55. 13.100
  
    libavfilter     4.  4.100 /  4.  4.100
  
    libswscale      2.  6.100 /  2.  6.100
  
    libswresample   0. 18.100 /  0. 18.100
  
    libpostproc    52.  3.100 / 52.  3.100
  
  Input #0, mpeg, from     '/home/flirtzo/public_html//userfiles/files/94e76a18a7838e62ecb23cf0c374b1b798e7b936':
  
    Duration: 00:00:25.97, start: 0.340078, bitrate: 29004 kb/s
  
      Stream #0:0[0x1e0]: Video: mpeg1video, yuv420p(tv), 352x240 [SAR 200:219 DAR 880:657], 1150         kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 29.97 tbc
  
      Stream #0:1[0x1c0]: Audio: mp2, 44100 Hz, stereo, s16p, 224 kb/s
  
  [libx264 @ 0x300d9c0] using SAR=200/219
  
  [libx264 @ 0x300d9c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
  
  [libx264 @ 0x300d9c0] profile High, level 2.2
  
  [libx264 @ 0x300d9c0] 264 - core 142 r2 d6b4e63 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 -                     http://www.videolan.org/x264.html - options: cabac=1 ref=16 deblock=1:0:0 analyse=0x3:0x133 me=umh         subme=10 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=24 chroma_me=1 trellis=2 8x8dct=1 cqm=0     deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=1 sliced_threads=0 nr=0     decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=8 b_pyramid=2 b_adapt=2 b_bias=0     direct=3 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0     rc_lookahead=60 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
  
  Output #0, mp4, to     '/home/flirtzo/public_html/userfiles/files/b2/72/73d4a3245c0b0e174ab7ce0f872ba3f649f8b93f73a6deeab364    4a994009d73638ce61aecc7dc2e0250c4e74ff2d9a4d479ed35cef26b3f6e1a77e8bf5938518-    94e76a18a7838e62ecb23cf0c374b1b798e7b936.mp4':
  
    Metadata:
  
      encoder         : Lavf55.37.100
  
      Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 352x240 [SAR 200:219     DAR 880:657], q=-1--1, 30k tbn, 29.97 tbc
  
      Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 44100 Hz, stereo, fltp, 128 kb/s
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mpeg1video -> libx264)
  
    Stream #0:1 -> #0:1 (mp2 -> aac)
  
     Press [q] to stop, [?] for help
  
  frame=16127 fps= 88 q=-1.0 Lsize=   29190kB time=00:08:58.12 bitrate= 444.4kbits/s dup=12 drop=0
  
  video:20221kB audio:8410kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead:     1.954086%
  
  [libx264 @ 0x300d9c0] frame I:73    Avg QP:24.31  size:  8024
  
  [libx264 @ 0x300d9c0] frame P:4399  Avg QP:26.97  size:  2600
  
  [libx264 @ 0x300d9c0] frame B:11655 Avg QP:32.51  size:   745
  
  [libx264 @ 0x300d9c0] consecutive B-frames:  3.8%  5.0% 27.2% 18.4%  8.4% 33.9%  1.7%  0.7%  0.9%
  
  [libx264 @ 0x300d9c0] mb I  I16..4: 10.8% 68.5% 20.7%
  
  [libx264 @ 0x300d9c0] mb P  I16..4:  2.9%  7.5%  0.8%  P16..4: 45.1% 18.4% 12.4%  0.5%  0.1%        skip:12.3%
  
  [libx264 @ 0x300d9c0] mb B  I16..4:  0.6%  1.3%  0.1%  B16..8: 40.1%  8.7%  1.7%  direct: 2.1%      skip:45.5%  L0:47.2% L1:41.7% BI:11.1%
  
  [libx264 @ 0x300d9c0] 8x8 transform intra:66.7% inter:76.7%
  
  [libx264 @ 0x300d9c0] direct mvs  spatial:99.9% temporal:0.1%
  
  [libx264 @ 0x300d9c0] coded y,uvDC,uvAC intra: 60.3% 75.8% 16.4% inter: 17.3% 16.8% 0.6%
  
  [libx264 @ 0x300d9c0] i16 v,h,dc,p: 16% 27% 11% 47%
  
  [libx264 @ 0x300d9c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 13% 18%  6%  9%  9% 11%  9% 11%
  
  [libx264 @ 0x300d9c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 15%  7%  7% 14% 14% 13%  8% 10%
  
  [libx264 @ 0x300d9c0] i8c dc,h,v,p: 26% 32% 20% 22%
  
  [libx264 @ 0x300d9c0] Weighted P-Frames: Y:10.1% UV:6.9%
  
  [libx264 @ 0x300d9c0] ref P L0: 40.6% 12.1% 10.0%  5.2%  5.2%  4.7%  4.6%  3.0%  2.5%  2.1%  1.9%      1.8%  1.7%  1.7%  1.6%  1.3%
  
  [libx264 @ 0x300d9c0] ref B L0: 71.4%  5.6%  3.2%  3.0%  3.1%  2.8%  2.5%  1.8%  1.3%  1.0%  1.1%      1.2%  1.0%  0.7%  0.5%
  
  [libx264 @ 0x300d9c0] ref B L1: 96.3%  3.7%
  
  [libx264 @ 0x300d9c0] kb/s:307.82

• FFmpeg saving rtmp live stream cuts off after 3 minutes

  20 janvier 2013, par user1636922

  I was playing with ffmpeg and was able to save a live stream to a file. The command to do so is :

  ffmpeg -re -i "rtmp://<ip addr="addr">/livestream live=1" -f h264 test.flv
  
  </ip>

  However, I have tested this twice, and both times ffmpeg stops after grabbing 3:28 worth of live video.

  The entire output is here :

  bash-4.2$ ffmpeg -re -i "rtmp://<ip addr="addr">/livestream live=1" -vcodec libx264 -f h264 test.flv
  
  WARNING: gnome-keyring:: couldn&#39;t connect to: /home/me/.cache/keyring-bpajcJ/pkcs11: No such file or directory
  
  ffmpeg version 0.10.4 Copyright (c) 2000-2012 the FFmpeg developers
  
    built on Jul 20 2012 22:01:52 with gcc 4.7.0 20120507 (Red Hat 4.7.0-5)
  
    configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib --mandir=/usr/share/man --arch=i686 --extra-cflags=&#39;-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables&#39; --enable-bzlib --disable-crystalhd --enable-gnutls --enable-libass --enable-libcdio --enable-libcelt --enable-libdc1394 --disable-indev=jack --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopenjpeg --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab --enable-avfilter --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib --cpu=i686 --enable-runtime-cpudetect
  
    libavutil      51. 35.100 / 51. 35.100
  
    libavcodec     53. 61.100 / 53. 61.100
  
    libavformat    53. 32.100 / 53. 32.100
  
    libavdevice    53.  4.100 / 53.  4.100
  
    libavfilter     2. 61.100 /  2. 61.100
  
    libswscale      2.  1.100 /  2.  1.100
  
    libswresample   0.  6.100 /  0.  6.100
  
    libpostproc    52.  0.100 / 52.  0.100
  
  WARNING: gnome-keyring:: couldn&#39;t connect to: /home/me/.cache/keyring-bpajcJ/pkcs11: No such file or directory
  
  Metadata:
  
    videocodecid          avc1
  
    width                 320.00
  
    height                240.00
  
    frameWidth            320.00
  
    frameHeight           240.00
  
    displayWidth          320.00
  
    displayHeight         240.00
  
    framerate             29.97
  
  trackinfo:
  
    timescale             90000.00
  
    language              eng
  
  sampledescription:
  
    sampletype            H264
  
    type                  video
  
    profile-level-id      42e00c
  
    sprop-parameter-sets  Z0LgDNoFB+wEQAAC7sAAr8gh,aM4zyA==
  
    description           {H264CodecConfigInfo: profile: "Baseline", level: 1.2, frameSize: 320x240, displaySize: 320x240, PAR: 1:1, frameRate: 29.97}
  
  rtpsessioninfo:
  
    name                  H264 Stream 1
  
    origin                - 1486490083 118668671 IN IP4 10.93.183.3
  
    timing                0 0
  
    protocolversion       0
  
  attributes:
  
    range                 npt=now-
  
  [flv @ 0x9578ee0] Estimating duration from bitrate, this may be inaccurate
  
  Input #0, flv, from &#39;rtmp://<ip addr="addr">/livestream live=1&#39;:
  
    Duration: N/A, start: 0.000000, bitrate: N/A
  
      Stream #0:0: Video: h264 (Constrained Baseline), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 14.99 tbr, 1k tbn, 59.94 tbc
  
  [buffer @ 0x99ee900] w:320 h:240 pixfmt:yuv420p tb:1/1000000 sar:1/1 sws_param:
  
  [libx264 @ 0x9584540] using SAR=1/1
  
  [libx264 @ 0x9584540] using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.2
  
  [libx264 @ 0x9584540] profile High, level 1.2
  
  Output #0, h264, to &#39;test.flv&#39;:
  
    Metadata:
  
      encoder         : Lavf53.32.100
  
      Stream #0:0: Video: h264, yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 90k tbn, 14.99 tbc
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (h264 -> libx264)
  
  Press [q] to stop, [?] for help
  
  RTMP_ReadPacket, failed to read RTMP packet body. len: 16582bitrate= 212.1kbits/s    
  
  frame= 3111 fps= 15 q=-2.0 Lsize=    5385kB time=00:03:27.47 bitrate= 212.6kbits/s    
  
  video:5385kB audio:0kB global headers:0kB muxing overhead 0.000000%
  
  [libx264 @ 0x9584540] frame I:13    Avg QP:18.70  size: 31866
  
  [libx264 @ 0x9584540] frame P:1908  Avg QP:22.29  size:  2392
  
  [libx264 @ 0x9584540] frame B:1190  Avg QP:29.24  size:   451
  
  [libx264 @ 0x9584540] consecutive B-frames: 39.5% 24.0% 13.0% 23.4%
  
  [libx264 @ 0x9584540] mb I  I16..4:  0.2%  1.2% 98.6%
  
  [libx264 @ 0x9584540] mb P  I16..4:  0.0%  0.0%  0.7%  P16..4: 22.0%  3.5%  2.6%  0.0%  0.0%    skip:71.2%
  
  [libx264 @ 0x9584540] mb B  I16..4:  0.0%  0.0%  0.2%  B16..8: 15.6%  2.8%  1.1%  direct: 1.3%  skip:78.9%  L0:47.0% L1:42.2% BI:10.8%
  
  [libx264 @ 0x9584540] 8x8 transform intra:2.2% inter:8.5%
  
  [libx264 @ 0x9584540] coded y,uvDC,uvAC intra: 98.0% 94.9% 73.9% inter: 14.5% 16.0% 11.3%
  
  [libx264 @ 0x9584540] i16 v,h,dc,p:  0% 50% 17% 33%
  
  [libx264 @ 0x9584540] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu:  7% 21% 11%  3% 13%  6% 21%  4% 14%
  
  [libx264 @ 0x9584540] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 11% 21% 14%  5% 11%  7% 14%  5% 12%
  
  [libx264 @ 0x9584540] i8c dc,h,v,p: 65% 15% 10% 10%
  
  [libx264 @ 0x9584540] Weighted P-Frames: Y:0.5% UV:0.1%
  
  [libx264 @ 0x9584540] ref P L0: 80.8% 10.4%  7.6%  1.2%  0.0%
  
  [libx264 @ 0x9584540] ref B L0: 94.9%  4.8%  0.2%
  
  [libx264 @ 0x9584540] ref B L1: 96.4%  3.6%
  
  [libx264 @ 0x9584540] kb/s:212.48
  
  </ip></ip>

  Although I don't see any errors. It just looks like ffmpeg thought the stream had ended ? But that's not possible since it's a live stream.

• The use cases for a element in HTML

  1er janvier 2014, par silvia

  The W3C HTML WG and the WHATWG are currently discussing the introduction of a <main> element into HTML.

  The <main> element has been proposed by Steve Faulkner and is specified in a draft extension spec which is about to be accepted as a FPWD (first public working draft) by the W3C HTML WG. This implies that the W3C HTML WG will be looking for implementations and for feedback by implementers on this spec.

  I am supportive of the introduction of a <main> element into HTML. However, I believe that the current spec and use case list don’t make a good enough case for its introduction. Here are my thoughts.

  Main use case : accessibility

  In my opinion, the main use case for the introduction of <main> is accessibility.

  Like any other users, when blind users want to perceive a Web page/application, they need to have a quick means of grasping the content of a page. Since they cannot visually scan the layout and thus determine where the main content is, they use accessibility technology (AT) to find what is known as “landmarks”.

  “Landmarks” tell the user what semantic content is on a page : a header (such as a banner), a search box, a navigation menu, some asides (also called complementary content), a footer, …. and the most important part : the main content of the page. It is this main content that a blind user most often wants to skip to directly.

  In the days of HTML4, a hidden “skip to content” link at the beginning of the Web page was used as a means to help blind users access the main content.

  In the days of ARIA, the aria @role=main enables authors to avoid a hidden link and instead mark the element where the main content begins to allow direct access to the main content. This attribute is supported by AT – in particular screen readers – by making it part of the landmarks that AT can directly skip to.

  Both the hidden link and the ARIA @role=main approaches are, however, band aids : they are being used by those of us that make “finished” Web pages accessible by adding specific extra markup.

  A world where ARIA is not necessary and where accessibility developers would be out of a job because the normal markup that everyone writes already creates accessible Web sites/applications would be much preferable over the current world of band-aids.

  Therefore, to me, the primary use case for a <main> element is to achieve exactly this better world and not require specialized markup to tell a user (or a tool) where the main content on a page starts.

  An immediate effect would be that pages that have a <main> element will expose a “main” landmark to blind and vision-impaired users that will enable them to directly access that main content on the page without having to wade through other text on the page. Without a <main> element, this functionality can currently only be provided using heuristics to skip other semantic and structural elements and is for this reason not typically implemented in AT.

  Other use cases

  The <main> element is a semantic element not unlike other new semantic elements such as <header>, <footer>, <aside>, <article>, <nav>, or <section>. Thus, it can also serve other uses where the main content on a Web page/Web application needs to be identified.

  Data mining

  For data mining of Web content, the identification of the main content is one of the key challenges. Many scholarly articles have been published on this topic. This stackoverflow article references and suggests a multitude of approaches, but the accepted answer says “there’s no way to do this that’s guaranteed to work”. This is because Web pages are inherently complex and many <div>, <p>, <iframe> and other elements are used to provide markup for styling, notifications, ads, analytics and other use cases that are necessary to make a Web page complete, but don’t contribute to what a user consumes as semantically rich content. A <main> element will allow authors to pro-actively direct data mining tools to the main content.

  Search engines

  One particularly important “data mining” tool are search engines. They, too, have a hard time to identify which sections of a Web page are more important than others and employ many heuristics to do so, see e.g. this ACM article. Yet, they still disappoint with poor results pointing to findings of keywords in little relevant sections of a page rather than ranking Web pages higher where the keywords turn up in the main content area. A <main> element would be able to help search engines give text in main content areas a higher weight and prefer them over other areas of the Web page. It would be able to rank different Web pages depending on where on the page the search words are found. The <main> element will be an additional hint that search engines will digest.

  Visual focus

  On small devices, the display of Web pages designed for Desktop often causes confusion as to where the main content can be found and read, in particular when the text ends up being too small to be readable. It would be nice if browsers on small devices had a functionality (maybe a default setting) where Web pages would start being displayed as zoomed in on the main content. This could alleviate some of the headaches of responsive Web design, where the recommendation is to show high priority content as the first content. Right now this problem is addressed through stylesheets that re-layout the page differently depending on device, but again this is a band-aid solution. Explicit semantic markup of the main content can solve this problem more elegantly.

  Styling

  Finally, naturally, <main> would also be used to style the main content differently from others. You can e.g. replace a semantically meaningless <div id=”main”> with a semantically meaningful <main> where their position is identical. My analysis below shows, that this is not always the case, since oftentimes <div id=”main”> is used to group everything together that is not the header – in particular where there are multiple columns. Thus, the ease of styling a <main> element is only a positive side effect and not actually a real use case. It does make it easier, however, to adapt the style of the main content e.g. with media queries.

  Proposed alternative solutions

  It has been proposed that existing markup serves to satisfy the use cases that <main> has been proposed for. Let’s analyse these on some of the most popular Web sites. First let’s list the propsed algorithms.

  Proposed solution No 1 : Scooby-Doo

  On Sat, Nov 17, 2012 at 11:01 AM, Ian Hickson <ian@hixie.ch> wrote :
  | The main content is whatever content isn’t
  | marked up as not being main content (anything not marked up with <header>,
  | <aside>, <nav>, etc).
  

  This implies that the first element that is not a <header>, <aside>, <nav>, or <footer> will be the element that we want to give to a blind user as the location where they should start reading. The algorithm is implemented in https://gist.github.com/4032962.

  Proposed solution No 2 : First article element

  On Sat, Nov 17, 2012 at 8:01 AM, Ian Hickson  wrote :
  | On Thu, 15 Nov 2012, Ian Yang wrote :
  | >
  | > That’s a good idea. We really need an element to wrap all the <p>s,
  | > <ul>s, <ol>s, <figure>s, <table>s ... etc of a blog post.
  |
  | That’s called <article>.
  

  This approach identifies the first <article> element on the page as containing the main content. Here’s the algorithm for this approach.

  Proposed solution No 3 : An example heuristic approach

  The readability plugin has been developed to make Web pages readable by essentially removing all the non-main content from a page. An early source of readability is available. This demonstrates what a heuristic approach can perform.

  Analysing alternative solutions

  Comparison

  I’ve picked 4 typical Websites (top on Alexa) to analyse how these three different approaches fare. Ideally, I’d like to simply apply the above three scripts and compare pictures. However, since the semantic HTML5 elements <header>, <aside>, <nav>, and <footer> are not actually used by any of these Web sites, I don’t actually have this choice.

  So, instead, I decided to make some assumptions of where these semantic elements would be used and what the outcome of applying the first two algorithms would be. I can then compare it to the third, which is a product so we can take screenshots.

  Google.com

  http://google.com – search for “Scooby Doo”.

  The search results page would likely be built with :

  • a <nav> menu for the Google bar
  • a <header> for the search bar
  • another <header> for the login section
  • another <nav> menu for the search types
  • a <div> to contain the rest of the page
  • a <div> for the app bar with the search number
  • a few <aside>s for the left and right column
  • a set of <article>s for the search results

  “Scooby Doo” would find the first element after the headers as the “main content”. This is the element before the app bar in this case. Interestingly, there is a <div @id=main> already in the current Google results page, which “Scooby Doo” would likely also pick. However, there are a nav bar and two asides in this div, which clearly should not be part of the “main content”. Google actually placed a @role=main on a different element, namely the one that encapsulates all the search results.

  “First Article” would find the first search result as the “main content”. While not quite the same as what Google intended – namely all search results – it is close enough to be useful.

  The “readability” result is interesting, since it is not able to identify the main text on the page. It is actually aware of this problem and brings a warning before displaying this page :

  

  Facebook.com

  https://facebook.com

  A user page would likely be built with :

  • a <header> bar for the search and login bar
  • a <div> to contain the rest of the page
  • an <aside> for the left column
  • a <div> to contain the center and right column
  • an <aside> for the right column
  • a <header> to contain the center column “megaphone”
  • a <div> for the status posting
  • a set of <article>s for the home stream

  “Scooby Doo” would find the first element after the headers as the “main content”. This is the element that contains all three columns. It’s actually a <div @id=content> already in the current Facebook user page, which “Scooby Doo” would likely also pick. However, Facebook selected a different element to place the @role=main : the center column.

  “First Article” would find the first news item in the home stream. This is clearly not what Facebook intended, since they placed the @role=main on the center column, above the first blog post’s title. “First Article” would miss that title and the status posting.

  The “readability” result again disappoints but warns that it failed :

  

  YouTube.com

  http://youtube.com

  A video page would likely be built with :

  • a <header> bar for the search and login bar
  • a <nav> for the menu
  • a <div> to contain the rest of the page
  • a <header> for the video title and channel links
  • a <div> to contain the video with controls
  • a <div> to contain the center and right column
  • an <aside> for the right column with an <article> per related video
  • an <aside> for the information below the video
  • a <article> per comment below the video

  “Scooby Doo” would find the first element after the headers as the “main content”. This is the element that contains the rest of the page. It’s actually a <div @id=content> already in the current YouTube video page, which “Scooby Doo” would likely also pick. However, YouTube’s related videos and comments are unlikely to be what the user would regard as “main content” – it’s the video they are after, which generously has a <div id=watch-player>.

  “First Article” would find the first related video or comment in the home stream. This is clearly not what YouTube intends.

  The “readability” result is not quite as unusable, but still very bare :

  

  Wikipedia.com

  http://wikipedia.com (“Overscan” page)

  A Wikipedia page would likely be built with :

  • a <header> bar for the search, login and menu items
  • a <div> to contain the rest of the page
  • an &ls ; article> with title and lots of text
  • <article> an <aside> with the table of contents
  • several <aside>s for the left column

  Good news : “Scooby Doo” would find the first element after the headers as the “main content”. This is the element that contains the rest of the page. It’s actually a <div id=”content” role=”main”> element on Wikipedia, which “Scooby Doo” would likely also pick.

  “First Article” would find the title and text of the main element on the page, but it would also include an <aside>.

  The “readability” result is also in agreement.

  

  Results

  In the following table we have summarised the results for the experiments :

  Site	Scooby-Doo	First article	Readability
  Google.com	FAIL	SUCCESS	FAIL
  Facebook.com	FAIL	FAIL	FAIL
  YouTube.com	FAIL	FAIL	FAIL
  Wikipedia.com	SUCCESS	SUCCESS	SUCCESS

  Clearly, Wikipedia is the prime example of a site where even the simple approaches find it easy to determine the main content on the page. WordPress blogs are similarly successful. Almost any other site, including news sites, social networks and search engine sites are petty hopeless with the proposed approaches, because there are too many elements that are used for layout or other purposes (notifications, hidden areas) such that the pre-determined list of semantic elements that are available simply don’t suffice to mark up a Web page/application completely.

  Conclusion

  It seems that in general it is impossible to determine which element(s) on a Web page should be the “main” piece of content that accessibility tools jump to when requested, that a search engine should put their focus on, or that should be highlighted to a general user to read. It would be very useful if the author of the Web page would provide a hint through a <main> element where that main content is to be found.

  I think that the <main> element becomes particularly useful when combined with a default keyboard shortcut in browsers as proposed by Steve : we may actually find that non-accessibility users will also start making use of this shortcut, e.g. to get to videos on YouTube pages directly without having to tab over search boxes and other interactive elements, etc. Worthwhile markup indeed.