Sur d’autres sites (7090)

• Evolution #4493 (Nouveau) : Ajouter des robots à l’écran de sécurité

  17 mai 2020

  Bonjour,

  J’ai quelques sites qui ont des nombres de visites bizarres (trop grand, d’un facteur 10 à 30 certains jours).

  J’ai fait un patch : https://git.spip.net/RealET/spip/commit/ce2ce832b697ab0754a78e0fc8b298ae1e9c992b
  Mais je ne sais pas comment en faire un PR.

  Pour info, voici un extrait d’awstats pour le serveur concerné :
  Pour info :
  MJ12bot 2 241 164+29470 32.60 Go 17 Mai 2020 - 11:50
  AhrefsBot 1 388 315+9242 28.07 Go 17 Mai 2020 - 11:50
  SemrushBot 1 251 075+74309 23.91 Go 17 Mai 2020 - 11:50
  bingbot 619 488+28872 22.38 Go 17 Mai 2020 - 11:50
  DotBot 394 445+7038 22.52 Go 17 Mai 2020 - 11:50
  Googlebot 297 801+35895 11.64 Go 17 Mai 2020 - 11:50
  crawl 243 784+325 4.66 Go 17 Mai 2020 - 11:39
  MegaIndex.ru 172 567+51 5.41 Go 17 Mai 2020 - 04:50
  BLEXBot 161 178+4405 4.24 Go 17 Mai 2020 - 06:42
  Applebot 136 660+2443 3.20 Go 17 Mai 2020 - 11:50
  robot 129 533+3017 7.80 Go 17 Mai 2020 - 11:50
  empty user agent string 117 782+778 6.55 Go 17 Mai 2020 - 11:49
  Firefox version 10 and lower - various robots 106 252+366 1.87 Go 17 Mai 2020 - 11:50
  qwantify 83 743+3013 3.61 Go 17 Mai 2020 - 11:50
  Unknown robot identified by bot* 53 246+1846 2.25 Go 17 Mai 2020 - 11:37
  spip 46 253 554.75 Mo 17 Mai 2020 - 11:50
  link 44 510+567 761.38 Mo 17 Mai 2020 - 11:50
  Googlebot-Image 36 704 1.24 Go 17 Mai 2020 - 11:48
  facebookexternalhit 35 176+1 1.31 Go 17 Mai 2020 - 11:49
  MSIE 6 - ( Rogue Robot ) 33 090 1.10 Go 17 Mai 2020 - 11:47
  YandexBot 21 272+9245 822.57 Mo 17 Mai 2020 - 11:50
  Unknown robot (identified by hit on robots.txt) 0+27213 98.21 Mo 17 Mai 2020 - 11:48
  LinkpadBot 17 954 289.02 Mo 17 Mai 2020 - 06:46
  infobot 15 038+1887 198.28 Mo 17 Mai 2020 - 04:19
  MSIE 5 - ( Rogue Robot ) 16 110+1 980.04 Mo 17 Mai 2020 - 11:18
  Sogou web spider 14 514+453 239.14 Mo 17 Mai 2020 - 11:49
  ltx71 10 593+3663 378.89 Mo 17 Mai 2020 - 11:50
  BingPreview 11 920 1.87 Go 17 Mai 2020 - 11:40
  nbot 8 476+1023 230.66 Mo 17 Mai 2020 - 11:42
  Apache-HttpClient 7 897+43 99.64 Mo 17 Mai 2020 - 11:01
  Curl 7 752 151.93 Mo 17 Mai 2020 - 11:49
  CFNetwork 7 313 779.65 Mo 17 Mai 2020 - 11:25
  core 7 295+2 174.93 Mo 17 Mai 2020 - 11:35
  oBot 7 064+230 338.14 Mo 17 Mai 2020 - 08:06
  SeznamBot 3 677+3204 148.02 Mo 16 Mai 2020 - 22:36
  twitterbot 4 264+2518 100.64 Mo 17 Mai 2020 - 11:44
  panscient 6 298+397 97.66 Mo 17 Mai 2020 - 06:37
  A PHP script 6 395 13.36 Mo 17 Mai 2020 - 11:50
  archive.org_bot 4 917+385 157.65 Mo 17 Mai 2020 - 11:50
  vagabondo 5 022+12 197.93 Mo 17 Mai 2020 - 09:58
  Feedfetcher-Google 4 808 137.78 Mo 17 Mai 2020 - 11:41
  zend_http_client 4 535 20.98 Mo 17 Mai 2020 - 11:34
  ia_archiver 4 411+98 302.37 Mo 17 Mai 2020 - 03:58
  feed 4 401 127.40 Mo 17 Mai 2020 - 11:45
  dataprovider.com 4 003+167 169.38 Mo 17 Mai 2020 - 10:31
  universalfeedparser 4 156 82.26 Mo 17 Mai 2020 - 11:36
  Baidu ( catchall ) 3 956 260.02 Mo 17 Mai 2020 - 11:11
  coccoc 1 509+2374 33.71 Mo 17 Mai 2020 - 10:37
  python 3 607+87 54.82 Mo 17 Mai 2020 - 11:44
  Mail.RU Bot 1 969+1492 177.25 Mo 17 Mai 2020 - 11:13

• FFmpeg Error "No such filter : 'libvmaf' Mac

  4 janvier 2021, par ringola27

  I want to use FFmpeg to calculate PSNR for video sequences.
At first I installed libvmaf with the instruction from here and here. I installed ffmpeg like here. I tried for a test :

  



  ffmpeg -i output4.mpg -i output1.mpg -lavfi libvmaf -f null -


  



  But I get an Error (below). I think I could solve it with ./configure --enable-libvmaf --enable-version3 or ./configure --enable-libvmaf but unfortunately I can not run it (I don't know how).

  



  I hope anyone can help me to solve this problem.

  



  ffmpeg version 4.2.3 Copyright (c) 2000-2020 the FFmpeg developers
  built with Apple clang version 11.0.3 (clang-1103.0.32.59)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.3 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags=-fno-stack-check --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mpeg, from 'output4.mpg':
  Duration: 00:00:09.88, start: 0.533367, bitrate: 363 kb/s
    Stream #0:0[0x1e0]: Video: mpeg1video, yuv420p(tv), 352x288 [SAR 178:163 DAR 1958:1467], 104857 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 29.97 tbc
Input #1, mpeg, from 'output1.mpg':
  Duration: 00:00:10.01, start: 0.533367, bitrate: 345 kb/s
    Stream #1:0[0x1e0]: Video: mpeg1video, yuv420p(tv), 352x288 [SAR 178:163 DAR 1958:1467], 104857 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 29.97 tbc
[AVFilterGraph @ 0x7ffedbe23180] No such filter: 'libvmaf'
Error initializing complex filters.
Invalid argument


  


• The SSIM values calculated using FFMPEG are not what I expected

  18 mai 2020, par fd-alexer

  I'm trying to encode an m2ts (MPEG-2 Transport Stream) file to mp4 (H.264) and get the ssim value.
  
I did get some SSIM values, but the results were contrary to my expectations.

  



  Are there wrong command options for ffmpeg ?

  



  Encoding and SSIM calculation commands

  



  # encode&#xA;$ ffmpeg -hide_banner -fflags &#x2B;discardcorrupt -i input.m2ts \&#xA;  -c:v libx264 -crf <crf> -preset:v medium \&#xA;  -c:a copy -bsf:a aac_adtstoasc \&#xA;  output_ff_crf-<crf>.mp4&#xA;&#xA;# calculate ssim&#xA;$ ffmpeg -hide_banner -i <a> -i <b> \&#xA;  -lavfi  "[0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" \&#xA;  -f null -&#xA;</b></a></crf></crf>

  



  The results of the SSIM

  



  (a) A=input.m2ts,          B=input.m2ts,           ssim=0.973266
(b) A=input.m2ts,          B=output_ff_crf-0.mp4,  ssim=0.813347
(c) A=input.m2ts,          B=output_ff_crf-30.mp4, ssim=0.819897
(d) A=output_ff_crf-0.mp4, B=output_ff_crf-0.mp4,  ssim=1.000000
(e) A=output_ff_crf-0.mp4, B=output_ff_crf-30.mp4, ssim=0.972911


  



  

  • (d)(e) : These are what I expected.

  


  • (a) : The files are the same, but ssim≠1.

  


  • (b)(c) : SSIMs with CRF=0 and CRF=30 have almost the same value, although the image quality is different.

  


  



  In the case of HandBrakeCLI

  



  To determine if there was a problem with input.m2ts, I ran HandBrakeCLI with almost the same parameters as ffmpeg.

  



  # encode&#xA;HandBrakeCLI --verbose --format av_mp4 --encoder x264 --quality <crf> --x264-preset medium \&#xA;  --aencoder copy \&#xA;  --input input.m2ts --output output_hb_crf-<crf>.mp4&#xA;&#xA;# calculate ssim (same as ffmpeg)&#xA;$ ffmpeg -hide_banner -i <a> -i <b> \&#xA;  -lavfi  "[0:v]settb=AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" \&#xA;  -f null -&#xA;</b></a></crf></crf>

  



  (b') A=input.m2ts,          B=output_hb_crf-0.mp4,  ssim=0.999999
(c') A=input.m2ts,          B=output_hb_crf-30.mp4, ssim=0.972886
(d') A=output_hb_crf-0.mp4, B=output_hb_crf-0.mp4,  ssim=1.000000
(e') A=output_hb_crf-0.mp4, B=output_hb_crf-30.mp4, ssim=0.972886


  



  It's all as I expected. (although A is not ssim=1.0) 
Therefore, I don't see a problem with input.m2ts.

  



  Informations about video files and tools

  



  Results of the ffprobe

  



  input.m2ts

  



  [mpeg2video @ 0x5655577c1680] Invalid frame dimensions 0x0.
    Last message repeated 1 times
[mpegts @ 0x5655577bd080] start time for stream 2 is not set in estimate_timings_from_pts
[mpegts @ 0x5655577bd080] PES packet size mismatch
Input #0, mpegts, from 'input.m2ts':
  Duration: 00:30:02.68, start: 39593.392600, bitrate: 19019 kb/s
  Program 211
    Stream #0:0[0x140]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, top first), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1[0x141]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 255 kb/s
    Stream #0:2[0x138]: Data: bin_data ([6][0][0][0] / 0x0006)
Unsupported codec with id 100359 for input stream 2


  



  output_ff_crf-0.mp4

  



  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'output_ff_crf-0.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.83.100
  Duration: 00:30:02.67, start: 0.000000, bitrate: 109301 kb/s
    Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 109040 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 252 kb/s (default)
    Metadata:
      handler_name    : SoundHandler


  



  output_hb_crf-0.mp4

  



  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'output_hb_crf-0.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    creation_time   : 2020-05-17T06:22:06.000000Z
    encoder         : HandBrake 1.1.0 2018042400
  Duration: 00:30:02.22, start: 0.000000, bitrate: 109661 kb/s
    Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 109405 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 180k tbc (default)
    Metadata:
      creation_time   : 2020-05-17T06:22:06.000000Z
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 252 kb/s (default)
    Metadata:
      creation_time   : 2020-05-17T06:22:06.000000Z
      handler_name    : Stereo


  



  Tools

  



  version

  



  

  • ffmpeg : 3.4.6-0ubuntu0.18.04.1

  


  • HandBrakeCLI : 1.1.0

  


  



  ldd

  



  $ ldd /usr/bin/ffmpeg
...
libx264.so.152 => /usr/lib/x86_64-linux-gnu/libx264.so.152 (0x00007efbf1f33000)
...

$ ldd /usr/bin/HandBrakeCLI
...
libx264.so.152 => /usr/lib/x86_64-linux-gnu/libx264.so.152 (0x00007efbfb38f000)
...


  



  ffmpeg and HandBrakeCLI are using the same libx264.