Sur d’autres sites (9019)

• What is Google Analytics data sampling and what’s so bad about it ?

  16 août 2019, par Joselyn Khor — Analytics Tips, Development
  What is Google Analytics data sampling, and what’s so bad about it ?

  Google (2019) explains what data sampling is :

  “In data analysis, sampling is the practice of analysing a subset of all data in order to uncover the meaningful information in the larger data set.”[1]

  This is basically saying instead of analysing all of the data, there’s a threshold on how much data is analysed and any data after that will be an assumption based on patterns.

  Google’s (2019) data sampling thresholds :

  Ad-hoc queries of your data are subject to the following general thresholds for sampling :
  [Google] Analytics Standard : 500k sessions at the property level for the date range you are using
  [Google] Analytics 360 : 100M sessions at the view level for the date range you are using (para. 3) [2]

  This threshold is limiting because your data in GA may become more inaccurate as the traffic to your website increases.

  Say you’re looking through all your traffic data from the last year and find you have 5 million page views. Only 500K of that 5 million is accurate ! The data for the remaining 4.5 million (90%) is an assumption based on the 500K sample size.

  This is a key weapon Google uses to sell to large businesses. In order to increase that threshold for more accurate reporting, upgrading to premium Google Analytics 360 for approximately US$150,000 per year seems to be the only choice.

  What’s so bad about data sampling ?

  It’s unfair to say sampled data is to be disregarded completely. There is a calculation ensuring it is representative and can allow you to get good enough insights. However, we don’t encourage it as we don’t just want “good enough” data. We want the actual facts.

  In a recent survey sent to Matomo customers, we found a large proportion of users switched from GA to Matomo due to the data sampling issue.

  The two reasons why data sampling isn’t preferable : 

  1. If the selected sample size is too small, you won’t get a good representative of all the data. 
  2. The bigger your website grows, the more inaccurate your reports will become.

  An example of why we don’t fully trust sampled data is, say you have an ecommerce store and see your GA revenue reports aren’t matching the actual sales data, due to data sampling. In GA you may be seeing revenue for the month as $1 million, instead of actual sales of $800K.

  The sampling here has caused an inaccuracy that could have negative financial implications. What you get in the GA report is an estimated dollar figure rather than the actual sales. Making decisions based on inaccurate data can be costly in this case. 

  Another disadvantage to sampled data is that you might be missing out on opportunities you would’ve noticed if you were given a view of the whole. E.g. not being able to see real patterns occurring due to the data already being predicted. 

  By not getting a chance to see things as they are and only being able to jump to the conclusions and assumptions made by GA is risky. The bigger your business grows, the less you can risk making business decisions based on assumptions that could be inaccurate. 

  If you feel you could be missing out on opportunities because your GA data is sampled data, get 100% accurately reported data. 

  The benefits of 100% accurate data

  Matomo doesn’t use data sampling on any of our products or plans. You get to see all of your data and not a sampled data set.

  Data quality is necessary for high impact decision-making. It’s hard to make strategic changes if you don’t have confidence that your data is reliable and accurate.

  Learn about how Matomo is a serious contender to Google Analytics 360. 

  Now you can import your Google Analytics data directly into your Matomo

  If you’re wanting to make the switch to Matomo but worried about losing all your historic Google Analytics data, you can now import this directly into your Matomo with the Google Analytics Importer tool.

  
  Take the challenge !

  Compare your Google Analytics data (sampled data) against your Matomo data, or if you don’t have Matomo data yet, sign up to our 30-day free trial and start tracking !

  References :

  [1 & 2] About data sampling. (2019). In Analytics Help About data sampling. Retrieved August 14, 2019, from https://support.google.com/analytics/answer/2637192

• Bad video quality and only 3 fps

  25 juillet 2019, par ApplowPi

  There are a few things wrong with the quality of the output file when I run this command.

  arecord -d 0 -r 48000 -c 2 -f S16_LE -t wav -D sysdefault:CARD=tc358743 | avconv -t 0 -i pipe:0 -f v4l2 -i /dev/video0 -c:v libx264 -r 30 -acodec aac -ar 48000 -vcodec flv -strict experimental -y -f flv test.flv

  The audio seems okay at the moment but, if I’m reading it correctly (at the very bottom of the output below), the fps is about 3 and the quality is at 31, which I understand is the worse. This is the output during the process.

  I’m looking to get 30 fps (ideally 60 if possible).

  I’m pretty new to this, any help would be great. Thanks in advance.

  ffmpeg version 3.2.10-1~deb9u1+rpt2 Copyright (c) 2000-2018 the FFmpeg developers
  
    built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516
  
    configuration: --prefix=/usr --extra-version='1~deb9u1+rpt2' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx-rpi --enable-mmal --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --arch=armhf --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  
    libavutil      55. 34.101 / 55. 34.101
  
    libavcodec     57. 64.101 / 57. 64.101
  
    libavformat    57. 56.101 / 57. 56.101
  
    libavdevice    57.  1.100 / 57.  1.100
  
    libavfilter     6. 65.100 /  6. 65.100
  
    libavresample   3.  1.  0 /  3.  1.  0
  
    libswscale      4.  2.100 /  4.  2.100
  
    libswresample   2.  3.100 /  2.  3.100
  
    libpostproc    54.  1.100 / 54.  1.100
  
  Guessed Channel Layout for Input Stream #0.0 : stereo
  
  Input #0, wav, from 'pipe:0':
  
    Duration: N/A, bitrate: 1536 kb/s
  
      Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, stereo, s16, 1536 kb/s
  
  [video4linux2,v4l2 @ 0x172e110] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
  
  [video4linux2,v4l2 @ 0x172e110] Time per frame unknown
  
  Input #1, video4linux2,v4l2, from '/dev/video0':
  
    Duration: N/A, start: 927.631204, bitrate: N/A
  
      Stream #1:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1280x720, 120 tbr, 1000k tbn, 1000k tbc
  
  Output #0, flv, to 'test.flv':
  
    Metadata:
  
      encoder         : Lavf57.56.101
  
      Stream #0:0: Video: flv1 (flv) ([2][0][0][0] / 0x0002), yuv420p, 1280x720, q=2-31, 200 kb/s, 120 fps, 1k tbn, 120 tbc
  
      Metadata:
  
        encoder         : Lavc57.64.101 flv
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1
  
      Stream #0:1: Audio: aac (LC) ([10][0][0][0] / 0x000A), 48000 Hz, stereo, fltp, 128 kb/s
  
      Metadata:
  
        encoder         : Lavc57.64.101 aac
  
  Stream mapping:
  
    Stream #1:0 -> #0:0 (rawvideo (native) -> flv1 (flv))
  
    Stream #0:0 -> #0:1 (pcm_s16le (native) -> aac (native))
  
  [wav @ 0x1728730] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
  
  [video4linux2,v4l2 @ 0x172e110] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
  
  overrun!!! (at least 9513.101 ms long)44kB time=00:00:00.97 bitrate=2890.9kbits/s speed=0.0987x
  
  overrun!!! (at least 25.077 ms long) 363kB time=00:00:01.84 bitrate=1614.1kbits/s speed=0.162x
  
  overrun!!! (at least 254.192 ms long)421kB time=00:00:02.97 bitrate=1159.1kbits/s speed=0.226x
  
  overrun!!! (at least 93.349 ms long) 441kB time=00:00:03.86 bitrate= 935.3kbits/s speed=0.263x
  
  overrun!!! (at least 198.942 ms long)461kB time=00:00:04.97 bitrate= 758.6kbits/s speed=0.303x
  
  overrun!!! (at least 155.909 ms long)526kB time=00:00:06.14 bitrate= 701.0kbits/s speed=0.337x
  
  frame=   54 fps=2.8 q=31.0 Lsize=     534kB time=00:00:06.67 bitrate= 655.6kbits/s speed=0.352x

  Running different commands to see what happens. This command :

  avconv -t 0 -i /dev/video0 -c:v libx264 -preset ultrafast -y newtest.h264

  Slowly builds up to 40 fps but the result is still poor. Feels like it’s 5 fps not 40. The CPU percentage when running this is 65-70%. Output is :

  ffmpeg version 3.2.10-1~deb9u1+rpt2 Copyright (c) 2000-2018 the FFmpeg developers
  
    built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516
  
    configuration: --prefix=/usr --extra-version='1~deb9u1+rpt2' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx-rpi --enable-mmal --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --arch=armhf --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  
    libavutil      55. 34.101 / 55. 34.101
  
    libavcodec     57. 64.101 / 57. 64.101
  
    libavformat    57. 56.101 / 57. 56.101
  
    libavdevice    57.  1.100 / 57.  1.100
  
    libavfilter     6. 65.100 /  6. 65.100
  
    libavresample   3.  1.  0 /  3.  1.  0
  
    libswscale      4.  2.100 /  4.  2.100
  
    libswresample   2.  3.100 /  2.  3.100
  
    libpostproc    54.  1.100 / 54.  1.100
  
  [video4linux2,v4l2 @ 0x16d4600] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
  
  [video4linux2,v4l2 @ 0x16d4600] Time per frame unknown
  
  Input #0, video4linux2,v4l2, from '/dev/video0':
  
    Duration: N/A, start: 5015.769341, bitrate: N/A
  
      Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1280x720, 59.94 tbr, 1000k tbn, 1000k tbc
  
  No pixel format specified, yuv422p for H.264 encoding chosen.
  
  Use -pix_fmt yuv420p for compatibility with outdated media players.
  
  [libx264 @ 0x16d77e0] using cpu capabilities: ARMv6 NEON
  
  [libx264 @ 0x16d77e0] profile High 4:2:2, level 3.2, 4:2:2 8-bit
  
  Output #0, h264, to 'newtest.h264':
  
    Metadata:
  
      encoder         : Lavf57.56.101
  
      Stream #0:0: Video: h264 (libx264), yuv422p, 1280x720, q=-1--1, 59.94 fps, 59.94 tbn, 59.94 tbc
  
      Metadata:
  
        encoder         : Lavc57.64.101 libx264
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
  
  Press [q] to stop, [?] for help
  
  frame= 1813 fps= 37 q=-1.0 Lsize=    3218kB time=00:00:30.24 bitrate= 871.5kbits/s dup=1700 drop=0 speed=0.617x
  
  video:3218kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000000%
  
  [libx264 @ 0x16d77e0] frame I:8     Avg QP:14.00  size:170114
  
  [libx264 @ 0x16d77e0] frame P:1805  Avg QP:15.46  size:  1071
  
  [libx264 @ 0x16d77e0] mb I  I16..4: 100.0%  0.0%  0.0%
  
  [libx264 @ 0x16d77e0] mb P  I16..4:  0.4%  0.0%  0.0%  P16..4:  2.0%  0.0%  0.0%  0.0%  0.0%    skip:97.6%
  
  [libx264 @ 0x16d77e0] coded y,uvDC,uvAC intra: 43.4% 28.3% 19.2% inter: 1.1% 0.8% 0.2%
  
  [libx264 @ 0x16d77e0] i16 v,h,dc,p: 61% 26%  8%  5%
  
  [libx264 @ 0x16d77e0] i8c dc,h,v,p: 68% 12% 17%  4%
  
  [libx264 @ 0x16d77e0] kb/s:871.45

  Running this command :

  avconv -t 0 -i /dev/video0 -r 60 -c:v libx264 -preset ultrafast -y newtest.flv 

  (just specifying a different output file format, results in a CPU usage of about 30% and produces this output :

  avconv -t 0 -i /dev/video0 -r 60 -c:v libx264 -preset ultrafast -y newtest.flv
  
  ffmpeg version 3.2.10-1~deb9u1+rpt2 Copyright (c) 2000-2018 the FFmpeg developers
  
    built with gcc 6.3.0 (Raspbian 6.3.0-18+rpi1+deb9u1) 20170516
  
    configuration: --prefix=/usr --extra-version='1~deb9u1+rpt2' --toolchain=hardened --libdir=/usr/lib/arm-linux-gnueabihf --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx-rpi --enable-mmal --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --arch=armhf --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared
  
    libavutil      55. 34.101 / 55. 34.101
  
    libavcodec     57. 64.101 / 57. 64.101
  
    libavformat    57. 56.101 / 57. 56.101
  
    libavdevice    57.  1.100 / 57.  1.100
  
    libavfilter     6. 65.100 /  6. 65.100
  
    libavresample   3.  1.  0 /  3.  1.  0
  
    libswscale      4.  2.100 /  4.  2.100
  
    libswresample   2.  3.100 /  2.  3.100
  
    libpostproc    54.  1.100 / 54.  1.100
  
  [video4linux2,v4l2 @ 0x527620] ioctl(VIDIOC_G_PARM): Inappropriate ioctl for device
  
  [video4linux2,v4l2 @ 0x527620] Time per frame unknown
  
  Input #0, video4linux2,v4l2, from '/dev/video0':
  
    Duration: N/A, start: 6071.912862, bitrate: N/A
  
      Stream #0:0: Video: rawvideo (UYVY / 0x59565955), uyvy422, 1280x720, 120 tbr, 1000k tbn, 1000k tbc
  
  No pixel format specified, yuv422p for H.264 encoding chosen.
  
  Use -pix_fmt yuv420p for compatibility with outdated media players.
  
  [libx264 @ 0x52a870] using cpu capabilities: ARMv6 NEON
  
  [libx264 @ 0x52a870] profile High 4:2:2, level 3.2, 4:2:2 8-bit
  
  [libx264 @ 0x52a870] 264 - core 148 r2748 97eaef2 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
  
  Output #0, flv, to 'newtest.flv':
  
    Metadata:
  
      encoder         : Lavf57.56.101
  
      Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuv422p, 1280x720, q=-1--1, 60 fps, 1k tbn, 60 tbc
  
      Metadata:
  
        encoder         : Lavc57.64.101 libx264
  
      Side data:
  
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
  
  Press [q] to stop, [?] for help
  
  frame=  158 fps=4.1 q=-1.0 Lsize=     878kB time=00:00:30.70 bitrate= 234.2kbits/s speed=0.803x
  
  video:874kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.385400%
  
  [libx264 @ 0x52a870] frame I:1     Avg QP:20.00  size: 68854
  
  [libx264 @ 0x52a870] frame P:157   Avg QP: 8.93  size:  5261
  
  [libx264 @ 0x52a870] mb I  I16..4: 100.0%  0.0%  0.0%
  
  [libx264 @ 0x52a870] mb P  I16..4:  1.4%  0.0%  0.0%  P16..4:  7.7%  0.0%  0.0%  0.0%  0.0%    skip:90.9%
  
  [libx264 @ 0x52a870] coded y,uvDC,uvAC intra: 28.1% 13.8% 13.4% inter: 3.4% 2.7% 2.3%
  
  [libx264 @ 0x52a870] i16 v,h,dc,p: 76% 22%  1%  1%
  
  [libx264 @ 0x52a870] i8c dc,h,v,p: 88%  4%  7%  0%
  
  [libx264 @ 0x52a870] kb/s:231.42

• lavc/h265_profile_level : Add new profiles

  6 mai 2019, par Mark Thompson

  lavc/h265_profile_level : Add new profiles
  Monochrome 10 and Main 10 Still Picture profiles are added by the 2018
  
  version of the standard.
  

  • [DH] libavcodec/h265_profile_level.c