Sur d’autres sites (10650)

• Introducing the Data Warehouse Connector feature

  30 janvier, par Matomo Core Team

  Matomo is built on a simple truth : your data belongs to you, and you should have complete control over it. That’s why we’re excited to launch our new Data Warehouse Connector feature for Matomo Cloud, giving you even more ways to work with your analytics data. 

  Until now, getting raw data from Matomo Cloud required APIs and custom scripts, or waiting for engineering help.  

  Our new Data Warehouse Connector feature removes those barriers. You can now access your raw, unaggregated data and schedule regular exports straight to your data warehouse. 

  The feature works with all major data warehouses including (but not limited to) : 

  • Google BigQuery 
  • Amazon Redshift 
  • Snowflake 
  • Azure Synapse Analytics 
  • Apache Hive 
  • Teradata 

  You can schedule exports, combine your Matomo data with other data sources in your data warehouse, and easily query data with SQL-like queries. 

  Direct raw data access for greater data portability 

  Waiting for engineering support can delay your work. Managing API connections and writing scripts can be time-consuming. This keeps you from focusing on what you do best—analysing data. 

  

  With the Data Warehouse Connector feature, you get direct access to your raw Matomo data without the technical setup. So, you can spend more time analysing data and finding insights that matter. 

  Bringing your data together 

  Answering business questions often requires data from multiple sources. A single customer interaction might span your CRM, web analytics, sales systems, and more. Piecing this data together manually is time-consuming—what starts as a seemingly simple question from stakeholders can turn into hours of work collecting and comparing data across different tools. 

  This feature lets you combine your Matomo data with data from other business systems in your data warehouse. Instead of switching between tools or manually comparing spreadsheets, you can analyse all your data in one place to better understand how customers interact with your business. 

  Easy, custom analysis with SQL-like queries 

  Standard, pre-built reports often don’t address the specific, detailed questions that analysts need to answer.  

  When you use the Data Warehouse Connector feature, you can use SQL-like queries in your data warehouse to do detailed, customised analysis. This flexibility allows you to explore your data in depth and uncover specific insights that aren’t possible with pre-built reports. 

  

  Here is an example of how you might use SQL-like query to compare the behaviours of paying vs. non-paying users : 

  				
                                          <xmp>SELECT  
  
  
  
  custom_dimension_value AS user_type, -- Assuming 'user_type' is stored in a custom dimension 
  
  
  
  COUNT(*) AS total_visits,  
  
  
  
  AVG(visit_total_time) AS avg_duration, 
  
  
  
  SUM(conversion.revenue) AS total_spent  
  
  
  
  FROM  
  
  
  
  `your_project.your_dataset.matomo_log_visit` AS visit 
  
  
  
  LEFT JOIN  
  
  
  
  `your_project.your_dataset.matomo_log_conversion` AS conversion  
  
  
  
  ON  
  
  
  
  visit.idvisit = conversion.idvisit  
  
  
  
  GROUP BY  
  
  
  
  custom_dimension_value; </xmp>
  
                                  
  			

  This query helps you compare metrics such as the number of visits, average session duration, and total amount spent between paying and non-paying users. It provides a full view of behavioural differences between these groups. 

  Advanced data manipulation and visualisation 

  When you need to create detailed reports or dive deep into data analysis, working within the constraints of a fixed user interface (UI) can limit your ability to draw insights. 

  Exporting your Matomo data to a data warehouse like BigQuery provides greater flexibility for in-depth manipulation and advanced visualisations, enabling you to uncover deeper insights and tailor your reports more effectively. 

  Getting started 

  To set up data warehouse exports in your Matomo : 

  1. Go to System Admin (cog icon in the top right corner) 
  2. Select ‘Export’ from the left-hand menu 
  3. Choose ‘Data Warehouse Connector’ 

  You’ll find detailed instructions in our data warehouse exports guide.  

  Please note, enabling this feature will cost an additional 10% of your current subscription. You can view the exact cost by following the steps above. 

  New to Matomo ? Start your 21-day free trial now (no credit card required), or request a demo. 

  Try Matomo for Free

  21 day free trial. No credit card required.

  
  

  Your email address
  

  Your website address
  

  Your Analytics subdomain will be .matomo.cloud (change)

  Choose your analytics subdomain
  

  .matomo.cloud

  I accept the terms and conditions and data processing agreement (DPA)

  
  Your information will be used to create an account on our cloud service. For more information please consult our privacy policy.
  
  
  

  Start improving your websites now

  

• Hung out to dry

  31 mai 2013, par Mans — Law and liberty

  Outrage was the general reaction when Google recently announced their dropping of XMPP server-to-server federation from Hangouts, as the search giant’s revamped instant messaging platform is henceforth to be known. This outrage is, however, largely unjustified ; Google’s decision is merely a rational response to issues of a more fundamental nature. To see why, we need to step back and look at the broader instant messaging landscape.

  A brief history of IM

  The term instant messaging (IM) gained popularity in the mid-1990s along with the rise of chat clients such as ICQ, AOL Instant Messenger, and later MSN Messenger. These all had one thing in common : they were closed systems. Although global in the sense of allowing access from anywhere on the Internet, communication was possible only within each network, and only using the officially sanctioned client software. Contrast this with email, where users are free to choose any service provider as well as client software, inter-server communication over open protocols delivering messages to their proper destinations.

  The email picture has, however, not always been so rosy. During the 1970s and 80s a multitude of incompatible email systems (e.g. UUCP and X.400) were in more or less widespread use on various networks. As these networks gave way to the ARPANET/Internet, so did their mail systems to the SMTP email we all use today. A similar consolidation has yet to occur in the area of instant messaging.

  Over the years, a few efforts towards a cross-domain instant messaging have been undertaken. One early example is the Zephyr system created as part of Project Athena at MIT in the late 1980s. While it never saw significant uptake, it is still in use at a few universities. A more successful story is that of XMPP. Conceived under the name Jabber in the late 1990s, XMPP is an open standard specified in a set of IETF RFCs. In addition to being open, a distinguishing feature of XMPP compared to other contemporary IM systems is its decentralised nature, server-to-server connections allowing communication between users with accounts on different systems. Just like email.

  The social network

  A more recent emergence on the Internet is the social network. Although not the first of its kind, Facebook was the first to achieve its level of penetration, both geographically and across social groups. A range of messaging options, including email-style as well as instant messaging (chat), are available, all within the same web interface. What it does not allow is communication outside the Facebook network. Other social networks operate in the same spirit.

  The popularity of social networks, to the extent that they for many constitute the primary means of communication, has in a sense brought back fragmented networks of the 1980s. Even though they share infrastructure, up to and including the browser application, the social networks create walled-off regions of the Internet between which little or no exchange is possible.

  The house that Google built

  In 2005, Google launched Talk, an XMPP-based instant messaging service allowing users to connect using either Google’s official client application or any third-party XMPP client. Soon after, server-to-server federation was activated, enabling anyone with a Google account to exchange instant messages with users of any other federated XMPP service. An in-browser chat interface was also added to Gmail.

  It was arguably only with the 2011 introduction of Google+ that Google, despite its previous endeavours with Orkut and Buzz, had a viable contender in the social networking space. Since its inception, Google+ has gone through a number of changes where features have been added or reworked. Instant messaging within Google+ was until recently available only in mobile clients. On the desktop, the sole messaging option was Hangouts which, although featuring text chat, cannot be considered instant messaging in the usual sense.

  With a sprawling collection of messaging systems (Talk, Google+ Messenger, Hangouts), some action to consolidate them was a logical step. What we got was a unification under the Hangouts name. A redesigned Google+ now sports in-browser instant messaging similar the the Talk interface already present in Gmail. At the same time, the standalone desktop Talk client is discontinued, as is the Messenger feature in mobile Google+. All together, the changes make for a much less confusing user experience.

  The sky is falling down

  Along with the changes to the messaging platform, one announcement stoked anger on the Internet : Google’s intent to discontinue XMPP federation (as of this writing, it is still operational). Google, the (self-described) champions of openness on the Internet were seen to be closing their doors to the outside world. The effects of the change are, however, not quite so earth-shattering. Of the other major messaging networks to offer XMPP at all (Facebook, Skype, and the defunct Microsoft Messenger), none support federation ; a Google user has never been able to chat with a Facebook user.

  XMPP federation appears to be in use mainly by non-profit organisations or individuals running their own servers. The number of users on these systems is hard to assess, though it seems fair to assume it is dwarfed by the hundreds of millions using Google or Facebook. As such, the overall impact of cutting off communication with the federated servers is relatively minor, albeit annoying for those affected.

  A fragmented world

  Rather than chastising Google for making a low-impact, presumably founded, business decision, we should be asking ourselves why instant messaging is still so fragmented in the first place, whereas email is not. The answer can be found by examining the nature of entities providing these services.

  Ever since the commercialisation of the Internet started in the 1990s, email has been largely seen as being part of the Internet. Access to email was a major selling point for Internet service providers ; indeed, many still use the email facilities of their ISP. Instant messaging, by contrast, has never come as part of the basic offering, rather being a third-party service running on top of the Internet.

  Users wishing to engage in instant messaging have always had to seek out and sign up with a provider of such a service. As the IM networks were isolated, most would choose whichever service their friends were already using, and a small number of networks, each with a sustainable number of users, came to dominate. In the early days, dedicated IM services such as ICQ were popular. Today, social networks have taken their place with Facebook currently in the dominant position. With the new Hangouts, Google offers its users the service they want in the way they have come to expect.

  Follow the money

  We now have all the pieces necessary to see why inter-domain instant messaging has never taken off, and the answer is simple : the major players have no commercial incentive to open access to their IM networks. In fact, they have good reason to keep the networks closed. Ensuring that a person leaving the network loses contact with his or her friends, increases user retention by raising the cost of switching to another service. Monetising users is also better facilitated if they are forced to remain on, say, Facebook’s web pages while using its services rather than accessing them indirectly, perhaps even through a competing (Google, say) frontend. The users do not generally care much, since all their friends are already on the same network as themselves.

  While Google Talk was a standalone service, only loosely coupled to other Google products, these aspects were of lesser importance. After all, Google still had access to all the messages passing through the system and could analyse them for advert targeting purposes. Now that messaging is an integrated part of Google+, and thus serves as a direct competitor to the likes of Facebook, the situation has changed. All the reasons for Facebook not to open its network now apply equally to Google as well.

• Exceeded GA’s 10M hits data limit, now what ?

  1er décembre 2021, par Joselyn Khor
  Exceeded Google Analytics’ 10M hits data limit, now what ?

  “Your data volume (1XXM hits) exceeds the limit of 10M hits per month as outlined in our Terms of Service. If you continue to exceed the limit, we will stop processing new data on XXX 21, 2019. Learn more about possible solutions.”

  Yikes. Alarm bells were ringing when a Google Analytics free user came to us faced with this notice. Let’s call him ‘Mark’. Mark had reached the limits on the data he could collect through Google Analytics and was shocked by the limited options available to fix the problem, without blowing the budget. The thoughts racing through his head were :

  • “What happens to all my data ?”
  • “What if Google starts charging USD150K now ?”

  Then he came across Matomo and decided to get in touch with our support team …

  “Can you fix this issue ?” he asked us.

  “Absolutely !” we said.

  We’ll get back to helping Mark in a minute. For now let’s go over why this was such a dilemma for him.

  In order to resolve this data limits issue, one of the solutions was for him to upgrade to Google Analytics 360, which meant shelling out USD150,000 per year for their 1 billion hits per month option. Going from free to USD150,000 was too much of a stretch for a growing company.

  “Your data volume (1XXM hits) exceeds the limit of 10M hits per month …”, what did this message mean ?

  With the free version, Mark could collect up to 10 million “hits” per month, per account. Going over meant Google Analytics could stop collecting any more data for free as outlined in their Terms.

  Google Analytics’ Terms of Service (2018, sec. 2) states, “Subject to Section 15, the Service is provided without charge to You for up to 10 million Hits per month per account.”[1]

  What is a "hit" in Google Analytics ?

  Data being sent to Google Analytics. It can be a transaction, event, social interaction or pageview - these all produce what Google calls a “hit”.

  

  And their Analytics Help Data Limits (n.d.) support page makes clear that : “If a property sends more hits per month to Analytics than allowed by the Analytics Terms of Service, there is no assurance that the excess hits will be processed. If the property’s hit volume exceeds this limit, a warning may be displayed in the user interface and you may be prevented from accessing reports.”[2]

  

  Possible solutions

  So the possible solutions given by Google Analytics’ Data Limits support page were (also shown in image below) :

  • To pay USD150K to upgrade to Google Analytics 360
  • To send fewer hits by setting up sampling
  • Or choose the slightly less relevant option to upgrade mobile app tracking to Google Analytics for Firebase.

  Without the means to pay, the free version was fast becoming inaccessible for Mark as he was facing a future where he risked no longer having access to up-to-date data used in his business’ reporting.

  Mark was facing a problem that potentially didn’t have a cost-effective solution.

  

  So what can you really do about it ?

  This is where we can help provide some assistance. If you’re reading this article, we’ll assume you can relate to Mark and share with you the advice on options we gave him.

  Options :

  One option posed by Google is for you to send fewer hits by auditing your data collection processes

  If you really don’t have the budget, you’ll need to reassess your data collection priorities and go over your strategies to see what is necessary to track, and what isn’t.

  • Make sure you know what you’re tracking and why. Look at what websites are being tracked by Google and into what properties.
  • Go through what data you’re tracking and decide what is or isn’t of value.
  • Set up data sampling, this however, will lead to inaccurate data.

  From here you can start to course correct. If you’ve found data you’re not using for analysis, get rid of these events/pageviews in your Google Analytics.

  But the limitations here are that eventually, you’re going to run out of irrelevant metrics and everything you’re tracking will be essential. So you’ll hit another brick wall and return to the same situation.

  Option 2 – Ignore and continue using the free version of Google Analytics

  With this option, you’ll have to bear the business risks involved by basing decisions off of analytics reports that may or may not be updated. In this case, you may still get contacted about exceeding the limits. As the free service is provided for only up to 10 million hits, once you’ve gone over them, you’re violating what’s stipulated in the Terms of Service. 

  There’s also the warning that “… you may be prevented from accessing reports” (Data limits, n.d.). So while we may not know for certain what Google Analytics will do, in this case it may be better to be safe rather than sorry by acting quickly to resolve it. 

  Option 3 -The Matomo solution – a privacy-friendly Google Analytics alternative

  Upgrade to a web analytics platform that can handle your demanding data requirements. Save money while continuing to gain valuable insights by moving over to Matomo Analytics (recommended)

  This is where you can save up to USD130,000 a year. As well as that, the transition from Google Analytics to the Matomo Cloud is a seamless experience as setup and maintenance is taken care of by our experts.

  For example, you can get up to 25M hits for USD3,241/month (or USD38,900/year) on the Essentials plan.

  Or even 25M hits for USD4,991/month (or USD59,900/year) on the Business plan – which offers additional web analytics and conversion optimization resources.

  Matomo Cloud is a great option if you’re looking for a secure, cost-effective and powerful analytics solution. You also get what Google Analytics could never offer you : full control and ownership of your own data and privacy. 

  Try Matomo free for 21 days – no credit card required.

  No need to worry about losing your Google Analytics data because …

  Now you can import your historic Google Analytics data directly into your Matomo with the Google Analytics Importer tool. Simply follow the step-by-step guide to get started for free.

  Along with savings you can get :

  • A solution for the data limits issue forever. You choose the right plan to suit your data needs and adapt as you continue growing
  • 100% accurate data (no data sampling)
  • 100% data ownership of all your information without signing away your data to a third party
  • Powerful web analytics and conversion optimization features
  • Matomo Tag Manager
  • Easy setup
  • Support from Matomo’s specialists

  Learn more about Matomo Cloud pricing.

  

  Or go for Matomo On-Premise

  If you have the in-house infrastructure to support self-hosting Matomo on your own servers then there’s also the option of Matomo On-Premise. Here you’ll get full security knowing the data is on your own servers. 

  Setup will also require technical knowledge. There will also be costs associated with acquiring your own servers, and keeping up with regular maintenance and updates. With On-Premise you get maximum flexibility, with no data limits whatsoever. But if you’re coming over from Google Analytics and don’t have the infrastructure and team to host On-Premise, the Matomo Cloud could be right for you.

  Learn more about Matomo On-Premise.

  Where do you go from here ?

  Getting 10 millions hits per month is no small feat, it’s actually pretty fantastic. But if it means having to shell out USD150,000 just to be able to continue with Google Analytics, we feel your problem could be fixed with Matomo Cloud. You could then put the rest of the money you save to better use.

  If you choose Matomo, you now have the option to : 

  • Raise your data limits for a fraction of Google Analytics 360’s price
  • Get a comprehensive range of analytics features for the most impactful insights to ensure your website continues excelling
  • Get data that’s not sampled – meaning 100% accuracy in your reports
  • Migrate your data easily with the help of Matomo’s support team

  We’ll have you covered. 

  By sharing with you the options and advice we gave to Mark, we hope you’ll be able to find a solution that makes your life easier and solves the issue of data restrictions forever.

  The team at Matomo is here to help you every step of the way to ensure a stress-free transition from Google Analytics if that is what works best for you.

  For next steps, check out our live online Matomo demo and start your free 21-day trial.

  References :

  [1] Terms of Service. (2018, July 24). In Google Analytics Terms of Service. Retrieved June 12, 2019, from https://www.google.com/analytics/terms/us.html

  [2] Data limits. (n.d.). In Analytics Help Data limits. Retrieved June 12, 2019, from https://support.google.com/analytics/answer/1070983?hl=en