Sur d’autres sites (5794)

• Stopping Referrer Spam

  13 mai 2015, par Piwik Core Team — Community

  In this blog post we explain what is Referrer spam, this new kind of spam that has recently appeared on the Internet. We also provide solutions to stop it and preserve the quality of your analytics data.

  What is Referrer Spam ?

  Referrer spam (also known as log spam or referrer bombing) is a kind of spamming aimed at web analytics tools. A spammer bot makes repeated web site requests using a fake referrer URL to the site the spammer wishes to advertise.

  Here is an example of referrer spam in action :

  

  Half of those referrers are spams, here are some well know spammers that you may have seen in your logs : buttons-for-you-website.com, best-seo-offer.com, semalt.com…

  The benefit for spammers is that their website will appear in analytics tools like Piwik or Google Analytics :

  • public analytics reports (or logs) will be indexed by search engines : links to the spammer’s website will improve its ranking
  • curious webmasters are likely to visit their referrers, thus bringing traffic to the spammer’s website

  How to deal with Referrer Spam ?

  Referrer spam is still new and analytics tools are all handling it differently.

  Referrer Spam in Piwik

  At Piwik we started working on mitigating Referrer spam more than a year ago. If you use Piwik and keep it up to date, you do not need to do anything.

  Referrer spammers are automatically excluded from your reports to keep your data clean and useful.

  New spammers are continuously detected and added to Piwik’s blacklist on each update. If you find a new spammer in your analytics data, you can even report it so that it is added to the Piwik’s open referrer blacklist and blocked for everyone.

  Referrer Spam in Google Analytics

  Google Analytics doesn’t offer any spam protection by default. It can however be configured manually using a custom Filter.

  To create a filter in Google Analytics go to the Admin section and click on All Filters. Create a new custom filter that excludes based on the Campaign Source field. In the Filter pattern enter the spammers domains you want to exclude (this is a regular expression) :

  

  If new spammers arise you will need to update this list. You can also use Piwik’s referrer blacklist to exclude all the spammers currently detected.

  Other Analytics Tools

  Many web analytics tools do not yet handle Referrer spam and when using these tools, you will often find a lot of spam data in your Referrer Websites analytics reports.

  If you use an analytics tool that does not exclude Referrer spam, we recommend to contact the vendor and ask them to implement a mechanism to remove these referrer spammers. As of today many analytics vendors still have not mitigated this issue.

  Public List of Referrer Spammers

  At Piwik with the help of our large community we have decided to tackle this growing spam issue. We have created a list of up to date referrer spammers that anyone can edit.

  The list is available in a simple text file on Github : github.com/piwik/referrer-spam-blacklist.

  The list is released under the Public Domain and anyone can use it within their applications to exclude referrer spammers.

  Many people have already contributed new spammers to the list. We invite you to use the list in your apps and websites and help us keep the list up to date !

  Let’s unite and fight the spammers together.

  Happy Analytics !

• C# : how to pass through complex command line arguments in Process.Arguments with forbidden characters ?

  5 juin 2015, par Louis de la Verité

  In C#, I have a complex set of arguments to pass through to Process.Arguments. I have circumvented the problem by using a one-line script of compiled Powershell. The exact "train" of arguments is :

  ffmpeg -framerate $fr -f image2 -i $temp\JPEG%01d -c:v libx264 -threads $threads -crf $encqual $video_out

  Problematic arguments are the -i (with the JPEG%01 pattern) & the -c argument. I have tried double escaping, but it’s not working any way I try. I don’t like the /<arg>: <val></val></arg> notation because it conflicts with the -c:v. As mentioned, I got it working by pass it off to PowerShell, but that’s a hack. The directory also gets parsed because it has spaces. Looking for some real help here. Working with MS is truly exasperating.

• How to analyse 404 pages

  1er juillet 2019, par Matomo Core Team — Development, Plugins

  How to analyse “not found” pages (404) in digital analytics

  Have you ever sent out a newsletter and one link wasn’t active yet ? Would you like to know how many users get affected when this happens ? Would you like to know if your visitors are encountering 404 pages ? 

  In this article we’re describing an easy way to analyse “not found” pages on your website with Matomo to increase your visitors’ user experience, user acquisition, and SEO (search engine optimization).

  How to know the number of 404s on my website ?

  There are different ways to get this information. Depending on how your website is built, you may or may not collect this data.

  The easiest way to answer this question is to fire a 404 page on your website, you do this by accessing a wrong url :

  

  As you can see here, in our case, the page title starts with “Page non trouvée” which stands for “Page not found” when translated in English (as the website we are considering here is in French) :

  

  In this example 19 page views have been fired and it generated a bounce rate of 67%. As a result ⅔ of the visits ended here.

  In some cases, the information related to a “not found” page can be found either within the title or within the URL, as some websites redirect you to a specific web page when a page can’t be found.

  If you can’t identify “not found” pages via a page title or a page URL, we strongly advise you to use this specific tracking code method on your 404 page : “How to track error pages in Matomo ?”. 

  You can easily set it with Matomo Tag Manager with a custom HTML tag :

  

  where the trigger is the following :

  

  You will however, have to define this trigger as an exclusion for all the other tags which may conflict with it (here below is the new trigger defined for the generic Matomo tags we are inserting on all pages) :

  

  Once this specific tracking is set, you will be able to track the source of the 404 and will gather all the “not found” pages in a specific group within your Page Title report :

  

  Here, for example, you can identify that the homepage of this website had a link pointing to a 404, in our case it was https://www.webassoc.org/pro-du-web.

  Note that this is just one technique. You could also create a custom dimension report and decide to send the 404 there also.

  How to get notified when a 404 page is visited ?

  Trust us, you’re not going to check everyday whether a 404 page has been visited. In order to avoid checking it manually, you can define custom alerts.

  There are three possible scenarios when “not found” pages can be fired :

  • internal 404 : one link within your website is pointing to a wrong url on the same website.
  • external 404 : someone from an external website made a link to yours and the link is not correct.
  • direct access 404 : someone access directly to a not found page on your website.

  You can define all those three within Matomo, but in your case, you will only have to focus on the first two only. In fact, you can’t really fix the third scenario. That’s the reason why we’re not focusing on it. It would result in irrelevant alerts.

  Custom alert for internal 404

  An internal 404 is defined from a 404 where the source is an internal web page. As a result, it will look like the following in your report :

  

  In this example, we’re using this specific custom implementation, the title of the page will contain “From = https://www.webassoc.org/”. So set our custom alert accordingly :

  

  Now every time a 404 page will be fired from an internal page, you’ll be notified by email.

  Note that you can also decide to not receive any email and track the evolution of alerts with the History of triggered alerts feature.

  Custom alert for external 404

  External 404 is almost the same setup. The only thing you need to keep in mind is that we want to exclude the 404 where the source is not indicated. As a result, your configuration will look like the following :

  

  Here your regular expression pattern is the following one :

  404/URL = .*From = (?!https://www.webassoc.org)[^\s]+

  as you’ll want to have any referrer coming from a website which is not Matomo and not a direct 404.

   

  You can now be notified every time that a 404 is fired from any link.

  Note that this configuration may slightly differ from website to website. So always double check your tracking code and the way the values are sent to your reports. Also try to trigger those alerts first before validating them.

  How to follow the evolution of your 404 over time ?

  It may be interesting to know how good or how bad you are performing in terms of 404.

  In order to check this information, you can click on the evolution icon near the 404 title :

  

  But you may be interested in accessing this information more regularly without having to create this report each time.

  So, one way to analyse the evolution of your 404 is to create a segment such as :

  

  and to click after that on evolution icon :

  

  As you can see below the number of “not found” pages is quite low in general, but we can also notice that a period received an increase in terms of 404 not found pages on May 27. It may be interesting to investigate it :

  

  You can start from the overview of referrers :

  

  As you can notice here the main source of 404 is coming from direct entries which is the most difficult channel to analyse as we don’t really know where the visitors are coming from.

  How to perform your analysis even faster ?

  As you can see analysing reports in Matomo in order to detect 404 pages is a time-consuming activity. In order to make it faster, you can already create a report about it within the Email reports feature with the following settings :

  • Segment : 404
  • Email schedule : never.
  • Visits summary and Page titles as selected report.

  You will then end up with a saved report listing all the URLs concerned :

  

  You can also have a look at the “Custom reports” premium feature.

  It will provide you with more flexibility. You will then be able to focus on the most important thing : the cause of 404.

  Good luck and happy analytics !