Recherche avancée

Sur d’autres sites (8001)

• Stopping Referrer Spam

  13 mai 2015, par Piwik Core Team — Community

  In this blog post we explain what is Referrer spam, this new kind of spam that has recently appeared on the Internet. We also provide solutions to stop it and preserve the quality of your analytics data.

  What is Referrer Spam ?

  Referrer spam (also known as log spam or referrer bombing) is a kind of spamming aimed at web analytics tools. A spammer bot makes repeated web site requests using a fake referrer URL to the site the spammer wishes to advertise.

  Here is an example of referrer spam in action :

  

  Half of those referrers are spams, here are some well know spammers that you may have seen in your logs : buttons-for-you-website.com, best-seo-offer.com, semalt.com…

  The benefit for spammers is that their website will appear in analytics tools like Piwik or Google Analytics :

  • public analytics reports (or logs) will be indexed by search engines : links to the spammer’s website will improve its ranking
  • curious webmasters are likely to visit their referrers, thus bringing traffic to the spammer’s website

  How to deal with Referrer Spam ?

  Referrer spam is still new and analytics tools are all handling it differently.

  Referrer Spam in Piwik

  At Piwik we started working on mitigating Referrer spam more than a year ago. If you use Piwik and keep it up to date, you do not need to do anything.

  Referrer spammers are automatically excluded from your reports to keep your data clean and useful.

  New spammers are continuously detected and added to Piwik’s blacklist on each update. If you find a new spammer in your analytics data, you can even report it so that it is added to the Piwik’s open referrer blacklist and blocked for everyone.

  Referrer Spam in Google Analytics

  Google Analytics doesn’t offer any spam protection by default. It can however be configured manually using a custom Filter.

  To create a filter in Google Analytics go to the Admin section and click on All Filters. Create a new custom filter that excludes based on the Campaign Source field. In the Filter pattern enter the spammers domains you want to exclude (this is a regular expression) :

  

  If new spammers arise you will need to update this list. You can also use Piwik’s referrer blacklist to exclude all the spammers currently detected.

  Other Analytics Tools

  Many web analytics tools do not yet handle Referrer spam and when using these tools, you will often find a lot of spam data in your Referrer Websites analytics reports.

  If you use an analytics tool that does not exclude Referrer spam, we recommend to contact the vendor and ask them to implement a mechanism to remove these referrer spammers. As of today many analytics vendors still have not mitigated this issue.

  Public List of Referrer Spammers

  At Piwik with the help of our large community we have decided to tackle this growing spam issue. We have created a list of up to date referrer spammers that anyone can edit.

  The list is available in a simple text file on Github : github.com/piwik/referrer-spam-blacklist.

  The list is released under the Public Domain and anyone can use it within their applications to exclude referrer spammers.

  Many people have already contributed new spammers to the list. We invite you to use the list in your apps and websites and help us keep the list up to date !

  Let’s unite and fight the spammers together.

  Happy Analytics !

• Real-time streaming of rawdata images to Android with FFMPEG

  29 juillet 2015, par Philies

  CONDITIONS

  I have an C++ server (Linux) and want to transmit rawdata images (RGB, 32bit) to an Android device in real-time.
  The server generates the rawdata images (with or without bitmap header) continuously every x miliseconds. Now, I want to put the rawdata images into a stream and transmit them without much delay to the Android client.

  I’ve chosen FFMPEG for this kind of job.

  The input for FFMPEG should be the rawdata images, which are generated just in time. The output should be an rtsp stream (h264 or is another format better ?).
  On client side I will play the stream with the Android MediaPlayer. That works for a RTSP url like rtsp ://184.72.239.149/vod/mp4:BigBuckBunny_115k.mov

  ---

  PREVIOUS APPROCH :

  I’ve installed FFMPEG on my server.

  Depending on this question : https://ffmpeg.org/pipermail/ffmpeg-user/2013-April/014617.html ... here is my FFMEG command :

  ffmpeg -an -f rawvideo -vcodec rawvideo -pix_fmt rgb32 -r 10 -i -vcodec libx264 -r 30 -tune zerolatency -preset ultrafast -bsf:v h264_mp4toannexb -f mpegts udp://192.168.1.20:1234

  Explanation of the parameters :

  • -an = ignore audio
  • -report = log file in current directory

  INPUT :

  • -f = rawvideo (video format)
  • -vcodec = rawvideo (video codec)
  • -pix_fmt = rgb32
  • -r = 10 (frame rate)

  OUTPUT :

  • -vcodec = libx264
  • -r = 30 (frame rate)
  • -bsf:v = h264_mp4toannexb
  • -f = mpegts udp ://192.168.1.20:1234

  ---

  C++ Server Code Snippet :

  /* -------  Get image rawdata from source ------- */
  
  ...
  
  /* -------  Create image header which fits to the image rawdata ------- */
  
  ...
  
  
  
  /* ------- Store the picture local (not necessary?) ------- */
  
  FILE *f;
  
  f = fopen("/home/philies/test.bmp","wb");
  
  
  
  //Write Bitmap Headers
  
  fwrite(&imageFileHeader,1,sizeof(imageFileHeader),f);
  
  fwrite(imageInfoHeader,1, imageInfoHeader,f);
  
  
  
  //Write Bitmap Rawdata
  
  fwrite(lastImage.GetBitmapRawData(),1,imageSize,f);
  
  
  
  
  
  —> ffmpeg ???

  ---

  PROBLEMS/QUESTIONS :

  1. How can I specify the rawvideo of the ffmpeg command respectively how can I specify my bitmap rawdata (with or without header ?) as the input of the FFMPEG ?

  2. Is the FFMPEG command correct ?

  If I fire the FFMPEG command in the terminal I get this error :

  vcodec : no such file or directory

  EDIT :
  In the first step, I will open the udp stream with the VLC player on my Android device. After that I will set up a FFserver to create a RTSP stream

• How to make Matomo GDPR compliant in 12 steps

  3 avril 2018, par InnoCraft

  Important note : this blog post has been written by digital analysts, not lawyers. The purpose of this article is to briefly show you where Matomo is entering into play within the GDPR process. This work comes from our interpretation of the UK privacy commission : ICO. It cannot be considered as professional legal advice. So as GDPR, this information is subject to change. We strongly advise you to have a look at the different privacy authorities in order to have up to date information.

  The General Data Protection Regulation (EU) 2016/679, also referred to RGPD in French, Datenschutz-Grundverordnung, DS-GVO in German, is a regulation on data protection and privacy for all individuals within the European Union. It concerns organizations worldwide dealing with EU citizens and will come into force on the 25th May 2018.

  The GDPR applies to ‘personal data’ meaning any information relating to an identifiable person who can be directly or indirectly identified in particular by reference to an identifier. It includes cookies, IP addresses, User ID, location, and any other data you may have collected.

  We will list below the 12 steps recommended by the UK privacy commissioner in order to be GDPR compliant and what you need to do for each step.

  The 12 steps of GDPR compliance according to ICO and how it fit with Matomo

  As mentioned in one of our previous blog post about GDPR, if you are not collecting any personal data with Matomo, then you are not concerned about what is written below.

  If you are processing personal data in any way, here are the 12 steps to follow along with some recommendations on how to be GDPR compliant with Matomo :

  1 – Awareness

  Make sure that people within your organization know that you are using Matomo in order to analyze traffic on the website/app. If needed, send them the link to the “What is Matomo ?” page.

  2 – Information you hold

  List all the personal data you are processing with Matomo within your record of processing activities. We are personally using the template provided by ICO which is composed of a set of 30 questions you need to answer regarding your use of Matomo. We have published an article which walks you through the list of questions specifically in the use case of Matomo Analytics. Please be aware that personal data may be also tracked in non-obvious ways for example as part of page URLs or page titles.

  3 – Communicating privacy information

  a – Add a privacy notice

  Add a privacy notice wherever you are using Matomo in order to collect personal data. Please refer to the ICO documentation in order to learn how to write a privacy notice. You can learn more in our article about creating your privacy notice for Matomo Analytics. Make sure that a privacy policy link is always available on your website or app.

  b – Add Matomo to your privacy policy page

  Add Matomo to the list of technologies you are using on your privacy policy page and add all the necessary information to it as requested in the following checklist. To learn more check out our article about Privacy policy.

  4 – Individuals’ rights

  Make sure that your Matomo installation respects all the individuals’ rights. To make it short, you will need to know the features in Matomo that you need to use to respect user rights (right of access, right of rectification, right of erasure…). These features are available starting in Matomo 3.5.0 released on May 8th : GDPR tools for Matomo (User guide).

  5 – Subject access requests

  Make sure that you are able to answer an access request from a data subject for Matomo. For example, when a person would like to access her or his personal data that you have collected about her or him, then you will need to be you able to provide her or him with this information. We recommend you design a process for this like “Who is dealing with it ?” and check that it is working. If you can answer to the nightmare letter, then you are ready. The needed features for this in Matomo will be available soon.

  6 – Lawful basis for processing personal data

  There are different lawful basis you can use under GDPR. It can be either “Legitimate interest” or “Explicit consent”. Do not forget to mention it within your privacy policy page. Read more in our article about lawful basis.

  7 – Consent

  Users should be able to remove their consent at any time. By chance, Matomo is providing a feature in order to do just that : add the opt-out feature to your privacy policy page.
  We are also offering a tool that allows you optionally to require consent before any data is tracked. This will be useful if a person should be only tracked after she or he has given explicit consent to be tracked.

  8 – Children

  If your website or app is targeted for children and you are using Matomo, extra measures will need to be taken. For example you will need to write your privacy policy even more clear and moreover getting parents consent if the child is below 13. As it is a very specific case, we strongly recommend you to follow this link for further information.

  9 – Data breaches

  As you may be collecting personal data with Matomo, you should also check your “data breach procedure” to define if a leak may have consequences on the privacy of the data subject. Please consult ICO’s website for further information.

  10 – Data Protection by Design and Data Protection Impact Assessments

  Ask yourself if you really need to process personal data within Matomo. If the data you are processing within Matomo is sensitive, we strongly recommend you to make a Data Protection Impact Assessment. A software is available from the The open source PIA software helps to carry out data protection impact assessment, by French Privacy Commissioner : CNIL.

  11 – Data Protection Officers

  If you are reading this article and you are the Data Protection Officer (DPO), you will not be concerned by this step. If that’s not the case, your duty is to provide to the DPO (if your business has a DPO) our blog post in order for her or him to ask you questions regarding your use of Matomo. Note that your DPO can also be interested in the different data that Matomo can process : “What data does Matomo track ?” (FAQ).

  12 – International

  Matomo data is hosted wherever you want. So according to the location of the data, you will need to show specific safeguard except for EU. For example regarding the USA, you will have to check if your web hosting platform is registered to the Privacy Shield : privacyshield.gov/list
  Note : our Matomo cloud infrastructure is based in France.

  That’s the end of this blog post. As GDPR is a huge topic, we will release many more blog posts in the upcoming weeks. If there are any Matomo GDPR topic related posts you would like us to write, please feel free to contact us.

  The post How to make Matomo GDPR compliant in 12 steps appeared first on Analytics Platform - Matomo.