Les articles publiés sur le site
-
Our latest improvement to QA : Screenshot Testing
2 octobre 2013, par benaka — DevelopmentIntroduction to QA in Piwik
Like any piece of good software, Piwik comes with a comprehensive QA suite that includes unit and integration tests. The unit tests make sure core components of Piwik work properly. The integration tests make sure Piwik’s tracking and report aggregation and APIs work properly.
To complete our QA suite, we’ve recently added a new type of tests: Screenshot tests, that we use to make sure Piwik’s controller and JavaScript code works properly.
This blog post will explain how they work and describe our experiences setting them up; we hope to show you an example of innovative QA practices in an active open source project.
Screenshot Tests
As the name implies, our screenshot tests (1) first capture a screenshot of a URL, then (2) compare the result with an expected image. This lets us test the code in Piwik’s controllers and Piwik’s JavaScript simply by specifying a URL.
Contrast this with conventional UI tests that test for page content changes. Such tests require writing large amounts of test code that, at most, check for changes in HTML. Our tests, on the otherhand, will be able to show regressions in CSS and JavaScript rendering logic with a bare minimum of testing code.
Capturing Screenshots
Screenshots are captured using a 3rd party tool. We tried several tools before settling on PhantomJS. PhantomJS executes a JavaScript file with an environment that allows it to create WebKit powered web views. When capturing a screenshot, we supply PhantomJS with a script that:
- opens a web page view,
- loads a URL,
- waits for all AJAX requests to be completed,
- waits for all images to be loaded
- waits for all JavaScript to be run.
Then it renders the completed page to an PNG file.
- To see how we use PhantomJS see capture.js.
- To see how we wait for AJAX requests to complete and images to load see override.js.
Comparing Screenshots
Once a screenshot is generated we test for UI regressions by comparing it with an expected image. There is no sort of fuzzy matching involved. We just check that the images consist of the same bytes.
If a screenshot test fails we use ImageMagick’s compare command line tool to generate an image diff:
In this example above, there was a change that caused the Search box to be hidden in the datatable. This resulted in the whole Data table report being shifted up a few pixels. The differences are visible in red color which gives rapid feedback to the developers what has changed in the last commit.
Screenshot Tests on Travis
We experienced trouble generating identical screenshots on different machines, so our tests were not initially automated by Travis. Once we surpassed this hurdle, we created a new github repo to store our UI tests and screenshots and then enabled the travis build for it. We also made sure that every time a commit is pushed to the Piwik repo, our travis build will push a commit to the UI test repo to run the UI tests.
We decided to create a new repository so the main repository wouldn’t be burdened with the large screenshot files (which git would not handle very well). We also made sure the travis build would upload all the generated screenshots to a server so debugging failures would be easier.
Problems we experienced
Getting generated screenshots to render identically on separate machines was quite a challenge. It took months to figure out how to get it right. Here’s what we learned:
Fonts will render identically on different machines, but different machines can pick the wrong fonts. When we first tried getting these tests to run on Travis, we noticed small differences in the way fonts were rendered on different machines. We thought this was an insurmountable problem that would occur due to the libraries installed on these machines. It turns out, the machines were just picking the wrong fonts. After installing certain fonts during our Travis build, everything started working.
Different versions of GD can generate slightly different images. GD is used in Piwik to, among other things, generate sparkline images. Different versions of GD will result in slightly different images. They look the same to the naked eye, but some pixels will have slightly different colors. This is, unfortunately, a problem we couldn’t solve. We couldn’t make sure that everyone who runs the tests uses the same version of GD, so instead we disabled sparklines for UI testing.
What we learned about existing screenshot capturing tools
We tried several screenshot capturing tools before finding one that would work adequately. Here’s what we learned about them:
-
CutyCapt This is the first screenshot capturing tool we tried. CutyCapt is a C++ program that uses QtWebKit to load and take a screenshot of a page. It can’t be used to capture multiple screenshots in one run and it can’t be used to wait for all AJAX/Images/JavaScript to complete/load (at least not currently).
-
PhantomJS This is the solution we eventually chose. PhantomJS is a headless scriptable browser that currently uses WebKit as its rendering engine.
For the most part, PhantomJS is the best solution we found. It reliably renders screenshots, allows JavaScript to be injected into pages it loads, and since it essentially just runs JavaScript code that you provide, it can be made to do whatever you want.
-
SlimerJS SlimerJS is a clone of PhantomJS that uses Gecko as the rendering engine. It is meant to function similarly to PhantomJS. Unfortunately, due to some limitations hard-coded in Mozilla’s software, we couldn’t use it.
For one, SlimerJS is not headless. There is, apparently, no way to do that when embedding Mozilla. You can, however, run it through xvfb, however the fact that it has to create a window means some odd things can happen. When using SlimerJS, we would sometimes end up with images where tooltips would display as if the mouse was hovering over an element. This inconsistency meant we couldn’t use it for our tests.
One tool we didn’t try was Selenium Webdriver. Although Selenium is traditionally used to create tests that check for HTML content, it can be used to generate screenshots. (Note: PhantomJS supports using a remote WebDriver.)
Our Future Plans for Screenshot Testing
At the moment we render a couple dozen screenshots. We test how our PHP code, JavaScript code and CSS makes Piwik’s UI look, but we don’t test how it behaves. This is our next step.
We want to create Screenshot Unit Tests for each UI control Piwik uses (for example, the Data Table View or the Site Selector). These tests would use the Widgetize plugin to load a control by itself, then execute JavaScript that simulates events and user behavior, and finally take a screenshot. This way we can test how our code handles clicks and hovers and all sorts of other behavior.
Screenshots Tests will make Piwik more stable and keep us agile and able to release early and often. Thank you for your support & Spreading the word about Piwik!
-
Hello world !
13 septembre 2013, par clearcode — UncategorizedWelcome to Piwik Sites. This is your first post. Edit or delete it, then start blogging!
-
Piwik at Free and Open Source Software Conference
26 août 2013, par thomas — Community
Last weekend Fabian and Thomas from the Piwik team have represented Piwik at the 8th FrOSCon in St. Augustin (Germany, near Cologne).At our booth we handed out Piwik brochures, our new awesome Stickers, and of course sweets. Many Piwik users came by and we were glad to gather some interesting feature ideas and feedback. Thank you! We also helped a few people troubleshoot issues on their servers and we introduced Piwik to many visitors stopping by the booth.
We cannot put into words what the incredible FrOSCon team and all the people who helped have achieved. Thank you for making this conference possible! We really enjoyed FrOSCon and we hope to be there next year again.
-
Piwik 1.12, New Features, API Improvements, Stability — The Last Piwik 1.X Release
30 mai 2013, par Piwik team — DevelopmentWe are very excited to announce the immediate availability of Piwik v1.12!
- Download Link
- How to update Piwik?
- List of all tickets closed: Changelog
Piwik v1.12 is a major new release with four big new features, seven smaller new features, several API improvements and all together 82 tickets fixed. This is also the last major 1.X release, which means after this release we will be working on releasing Piwik 2.0. This also means that you should upgrade to PHP 5.3 or higher if you haven’t already, since Piwik 2.0 will only support PHP 5.3 and above.
Finally, this release contains two breaking changes to the API. If you use the Piwik API click here or scroll down to see if you’re affected.
Table of Contents:
New Big Feature – Beta Release Channel
For those of you who want to help test Piwik 2.0-beta releases as soon as they come up, we’ve made it easier to use our beta releases. Navigate to the Settings > General Settings page and click the The latest beta release radio button. You will then be able to upgrade to beta releases.
This isn’t truly a major feature, but we think it’s just as important because it will allow us to create more beta releases and thus catch more bugs before we make a final release. This means more releases and more stability for you.
New Big Feature – Segment Editor
The Segment Editor is a long-awaited new feature that allows you to view, save and edit your segments.
Piwik has supported segmentation (filtering visits and reports by arbitrary criteria, like browser family) for quite some time now, but it has never been possible to visually create and modify them. Nor could they be saved for later recall.
Thanks to the eighty individuals and company who funded this feature, it is now possible to:
- visually segment your visitors, instead of creating URLs.
- save segments and easily switch between them, instead of remembering URLs.
- get suggestions for segments that might be helpful to view.
- learn more in the Segmentating Analytics reports user documentation..
New Big Feature – Page Speed Reports
You can now see how long it took your webserver to generate and send pages over HTTP through the new Avg. Generation Time metric.
This metric can be viewed on both the Pages and Page Titles reports:
And the average page generation time for all the pages in your website/webapp is displayed on the visitors overview:
You can use this new information to benchmark your webapp and web server.
New Big Feature – Device Detection Reports
Piwik 1.12 also includes a new plugin that provides reports on the device types (tablet, desktop, smartphone, etc.), device brands (Apple, Google, Samsung, etc.) and device models (iPad, Nexus 7, etc.) your visitors use to access your website:
The new plugin also enhances Operating system detections (detecting sub versions of Linux, Windows, and more).
Note: This plugin is not enabled by default, but will be in Piwik 2.0. If you want to view these reports now, you can activate the plugin in the Installed Plugins admin page. Navigate to Visitors > Devices to see the new reports. You may also use the new (beta) ‘Device type’.
The new plugin was developed with the support of Clearcode.cc our technology partner
Other improvements
Majestic SEO Metrics
We’ve added two new SEO metrics to the SEO widget, both of which are calculated by MajesticSEO.com. These metrics will tell you the number of external backlinks (the number of links to your site from other sites) and the number of referrer domains (the number of domains that link to your site).
We thank the team at Majestic for their support and hard work in bringing you these metrics to your Piwik dashboards!
Real-time Visitor Count Dashboard Widget
There is now a simple new widget you can use to see the number of visitors, visits and actions that occurred in the last couple minutes. We call it the Real Time Visitor Counter!
New segment parameter: siteSearchKeyword.
There is now a new segment parameter you can use to segment your visits: siteSearchKeyword. This parameter will let you select visits that had site searches with a specific keyword.
Ignore URL letter case when importing log files.
We’ve added a new option to the log import script, –force-lowercase-path. When used, the importer will change URL paths to lowercase before tracking them. This way http://domain.com/MY/BLOG will be treated the same as http://domain.com/my/blog.
Updated ISP Names
We’ve also modified the Providers report so prettier and more up-to-date names of ISPs are displayed.
Customize the background/text/axis color of graphs.
It is now possible to change the background color, text color and/or axis color of the graph images generated by the ImageGraph plugin. To access this functionality, use the following URL query parameters when generating an image:
- backgroundColor
- textColor
- axisColor
For example:
http://demo.piwik.org/index.php?module=API&method=ImageGraph.get&idSite=7&apiModule=UserSettings&apiAction=getBrowser&token_auth=anonymous&period=day&date=2013-03-21,2013-04-19&language=en&width=779&height=150&fontSize=9&showMetricTitle=0&aliasedGraph=1&legendAppendMetric=0&backgroundColor=efefef&gridColor=dcdcdc&colors=cb2026Send your users to a custom URL after they logout.
If you manage a Piwik installation with many users and you want to send them to a custom page or website after they log out of Piwik, you can now specify the URL to redirect users after they log out.
API Changes and Improvements
BREAKING CHANGE – renamed segment parameters.
The following segment parameters have been renamed:
- continent renamed to: continentCode
- browserName renamed to: browserCode
- operatingSystem renamed to: operatingSystemCode
- lat renamed to: latitude
- long renamed to: longitude
- region renamed to: regionCode
- country renamed to: countryCode
- continent renamed to: continentCode
If you use one of the old segment parameter names, Piwik will throw an exception, so you should notice when you’re using an old name.
BREAKING CHANGE – changes to the input & output of the Live.getLastVisitsDetails method.
The following changes were made to the Live.getLastVisitsDetails API method:
- The method no longer uses the maxIdVisit query parameter. It has been replaced by the filter_offset parameter.
- Site search keywords are now displayed in a <siteSearchKeyword> element. They were formerly in <pageTitle> elements.
- Custom variables with page scope now have ‘Page’ in their element names when displayed. For example, <customVariablePageName1>, <customVariablePageName2>, etc.
Filter results of MultiSites.getAll by website name.
It is now possible to filter the results of MultiSites.getAll by website name. To do this, set the pattern query parameter to the desired regex pattern.
Get suggested values to use for a segment parameter.
The new API method API.getSuggestedValuesForSegment can now be used to get suggested values for a segment parameter. This method will return a list of the most seen values (in the last 60 days) for a certain segment parameter. So for browserCode, this would return the codes for the browsers most visitors used in the last 60 days.
Use extra tracking query parameters with the JS tracker (such as ‘lat’ & ‘long’).
We’ve added a new method to the JavaScript tracker named appendToTrackingUrl. You can use this method to add extra query parameters to a tracking request, like so:
_paq.push(['appendToTrackingUrl', 'lat=X&long=Y']);What we’re working on
As we said above, Piwik v1.12 is the last in the 1.X series of releases. This means we are now officially working on Piwik 2.0.
Piwik 2.0 will be a big release, to be sure, but it’s going to bring you more than just a couple new features and a bag of bug fixes. For Piwik 2.0 we will be revisiting the user needs and the ideals that originally prompted us to create Piwik in order to build our vision of the future of web analytics.
Piwik 2.0 won’t just be a bigger, better web app, but a new platform for observing and analyzing the things that matter to you.
Participate in Piwik
Are you a talented developer or an experienced User Interface designer? Or maybe you like to write documentation or are a marketing guru?
If you have some free time and if you want to contribute to one of the most awesome open source projects around, please get in touch with the Piwik team, or read this page to learn more…
Summary
For the full list of changes in Piwik 1.12 check out the Changelog.
Thank you to the core developers, all the beta testers and users, our official supporters, the translators & everyone who reported bugs or feature requests. Also thank you to softwares we use, and the libraries we use.
If you are a company and would like to help an important project like Piwik grow, please get in touch, it means a lot to us. You can also participate in the project —
–> if you like what you read, please tell your friends and colleagues or write on your website, blog, forums, stackoverflow, etc. <–
Peace. Enjoy!
-
Presentation of Piwik’s collaborative translations platform : oTrance [Interview]
Piwik enables domain administrators, hobbyists, power users, personal website builders and everyone in between to access enormous amounts of data for website analytics. To support all those users, Piwik needs to be available in a number of different languages. From the start, we made internationalization (i18n) part of Piwik’s DNA. There are now dozens active volunteers who help make sure each language is well represented in the latest official release of Piwik. As of now, Piwik is available in 48 languages.Recently a new tool became available that makes the translation of Piwik much easier. The software we are using is an open source platform called oTrance. It has made our translation architecture more robust, and it allows us to expedite the timely delivery of high quality and up-to-date translations to the thousands of people who rely on Piwik every day.
We’ve met with oTrance creator and lead developer Daniel Schlichtholz who answered a few questions for us.
What is oTrance?
oTranCe is the short form of “Online Translation Center”. It was born because I needed a translation platform for my project MySQLDumper.
Many languages have been added by the community and manual maintenance became more and more time consuming. I wanted to change that. So I searched for an existing platform I could use and tested a lot of approaches. To put a long story short: none of the given solutions satisfied my needs.
From the view of a translator maintaining a language should be as easy as possible. In most cases they have to install a program on their local machine or the workflow was too difficult. A translator doesn’t want to struggle with technical things; he just wants to translate the phrases and wants to know the progress.
That’s the main goal we want to reach: to make the translation process as easy as possible.
What sets oTrance apart from the other ways to manage translations?
Ease of use is one advantage of oTranCe compared to other solutions. Another advantage is that project administrators can install oTranCe on their own server – so nobody is dependant of a third party provider.
We love to get feedback from other users. User feedback influences the way oTranCe is developed. We believe that this way oTranCe satisfies the requirements of the real world.
We also have extensive user documentation, in our “Working with oTranCe” wiki. We try to document use cases in an understandable way. We don’t write down marketing buzz words, but try to explain the use from the view of the user/administrator.
Now that oTranCe 1.0 is out, what will you be working on next?
The language files can be exported to version control and oTranCe can commit changes to the target repository. Currently we support export to Subversion, and we are working on a Git export adapter, which will be released soon.
Another issue we are trying to solve is the context problem. When your project uses many different phrases the translator often doesn’t know in which context the current phrase is used. Version 1.1.0 (not released yet, but you can grab the latest developer version from GitHub) introduces the oTranCe-connector. The idea behind it: a small plug in grabs the used phrases/keys on the current page, and on click this list is submitted to oTranCe, where the translator can edit the words. This way the translator knows in which context these phrases are used. I wrote a small plug in for OXID eShop. Since it is really easy to implement, my hope is that other plug ins for other applications will be added by the community.
Matthieu: Congratulations Daniel for having created such an awesome Translation Platform. At Piwik we are really thankful for oTranCe, which has resulted in much better translation process, and happier translators. Keep up the good work!
If you are a Piwik user, and if you want to participate in translating Piwik, please sign up for an account on oTrance and become part of the team making Piwik available in more languages across the world.
