Recherche avancée

Sur d’autres sites (11223)

• Generating test data – Introducing the Piwik Platform

  9 octobre 2014, par Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to create a command). This time you’ll learn how to generate test data.

  Developers are developing on their local Piwik instance which usually does not contain useful data compared to a real Piwik installation in production (only a few test visits and a few tests users and websites). The ‘VisitorGenerator’ plugin lets you generate any number of visits, websites, users, goals and more. The generator makes sure there will be data for each report so you can easily test anything.

  Getting started

  In this series of posts, we assume that you have already installed Piwik. If not, visit the Piwik Developer Zone where you’ll find the Installation guide for developers.

  Installing the VisitorGenerator plugin

  The easiest way to install the plugin is by using the Marketplace in Piwik itself. It is accessible via Settings => Marketplace => Get new functionality. There you’ll find the plugin “VisitorGenerator” which you can install and activate in one click.

  If your Piwik instance is not connected to the internet you can download the plugin from the VisitorGenerator page on the Marketplace. Afterwards you can install the plugin by going to Settings => Marketplace => Uploading a plugin and uploading the previously downloaded ZIP file.

  If you have already installed the plugin make sure it is activated by going to Settings => Plugins.

  Generating websites

  After you have installed the plugin you can add as many websites as you need. This is useful for instance when you want to test something that affects many websites such as the ‘All Websites’ dashboard or the Websites manager. To generate any number of websites use the following command :

  ./console visitorgenerator:generate-website --limit=10

  This will generate 10 websites. If you need more websites simply specify a higher limit. In case you are wondering the names and URLs of the websites are randomly generated by the Faker PHP library.

  Generating goals

  In case you want to test anything related to Goals you should execute the following command :

  ./console visitorgenerator:generate-goals --idsite=1

  This will generate a few goals for the specified site. The generated goals are defined in a way to make sure there will be conversions when generating the visits in the next step.

  Generating visits

  To generate visits there are two possibilities. Either via the Piwik UI by going to Settings => Visitor Generator or by using the command line. The UI is a bit limited in generating visits so we recommend to use the command line. There you can generate visits as follows :

  ./console visitorgenerator:generate-visits --idsite=1

  This will generate many different visits for the current day. Don’t worry if it takes a while, it will insert quite a few visits by default.

  In case you want to generate visits for multiple days in the past as well you can specify the --days option.

  ./console visitorgenerator:generate-visits --idsite=1 --days=5

  Providing your own logs

  Half of the generated visits are randomly generated and half of the visits are based on real logs to make sure there is data for each report. If you want to generate visits based on your own logs for a more realistic testing just place your log files in the plugins/VisitorGenerator/data folder and make sure the file name ends with .log. You can find a few examples in the VisitorGenerator data folder.

  To generate visits based only on real log files then use the --no-fake option.

  ./console visitorgenerator:generate-visits --idsite=1 --no-fake

  All generated visits will come from the logs and no random visits nor random fake data will be used.

  Advanced features

  We are regularly adding new commands, tools and runtime checks to make your life as a developer easier. For instance you can also generate users and annotations. In the future we want to extend the plugin to create visits in the background to make sure there will be constantly new actions in the real time report.

  Are you missing any kind of generator or any other feature to make your life as a developer easier ? Let us know by email, we are listening !

  Would you like to know more about the Piwik platform ? Go to our Piwik Developer Zone where you’ll find guides and references on how to develop plugin and themes.

• Piwik 3 Development Update #4 – Important notes and breaking changes

  2 octobre 2016, par Piwik Core Team — Community, Development

  As mentioned in the previous development updates #1 New UI design, API changes & release date, #2 Git master branch will become Piwik 3 and #3 Marketplace we are actively working on the Piwik 3 update and we will release a first beta of it soon. While the update brings many new features and bugfixes, it also comes with some important changes that may require your attention which we detail in this blog post.

  Database updates

  To keep Piwik upgrades as easy as possible, we have usually avoided database schema changes that affect large tables during the Piwik 2.X release cycle. The Piwik 3 update will however require several database updates in order to fix some limitations and bugs in the Piwik database schema. If you have a fair amount of traffic and/or large Piwik database, we recommend to execute the database updates for Piwik 3 on the command line.

  If you struggle updating Piwik you can also get in touch with the Piwik Support and ask for a quote so we can help you with upgrading.

  New server & browser Requirements

  Piwik 3 updates the server requirements. Piwik 3 now requires :

  • PHP 5.5.9 or newer (we used to require PHP 5.3+),
  • MySQL 5.5 or newer or MariaDB (we used to require MySQL 4.1+).

  If you are still on an old PHP and MySQL version we highly recommend to update directly to PHP 7 and a recent MySQL version (MySQL 5.7+).

  Piwik 3 also drops support for the older browsers IE8 and IE9.

  Default filter_limit now applies to all HTTP API methods (#8458)

  In the past the default filter_limit of 100 rows was applied to HTTP API methods that return a report. This means that by default :

  • when you requested a method like SitesManager.getAllSites, UsersManager.getUsers or Goals.getGoals (which are not reports), we returned all rows.
  • when you fetched a report like Actions.getPageUrls (which is a report), the result was limited to 100 rows.

  This behaviour was inconsistent and it was hard to understand which methods apply the default limit and which did not. We now apply the default limit to all HTTP API methods by default.

  If your API consumer or app needs to fetch all results for an API method, you can append the following parameter to the URL : &filter_limit=-1. Alternatively, if you wish to change the default filter_limit automatically for all APIs, you can override the API_datatable_default_limit config setting in your config/config.ini.php file.

  For more information about the various APIs parameters you can use, see Piwik Analytics HTTP API Reference.

  Proxy client IP detection (#10342)

  This is only important if you are using the proxy_client_headers[] config in the config/config.ini.php file, ie. your Piwik server is behind a proxy.

  When Piwik is set up to fetch the client IP address from the proxy headers, and the header contained more than one IP address, we used to use the last IP address. From Piwik 3 we will be using the first IP address which is in line with the protocol definition. For example if HTTP_X_FORWARDED_FOR contains multiple IP addresses, we now use the first IP instead of the last one.

  Writable Piwik JavaScript Tracker file piwik.js

  From Piwik 3 we recommend to have the file /piwik.js writable in your Piwik directory, so that the PHP and webserver processes can edit this file. It is not required to have the file writable for Piwik to function well, but we are working on some enhanced features that will only work when the file is writable. If you don’t know whether this file is writable, go to “Administration => System Check” as a Super User and Piwik will display a warning if the file is not writable. If needed, you might be able to make the file writable by executing a command like chmod 0755 piwik.js or chown $webuser piwik.js (on Linux).

  Changes for plugin developers

  In the previous development updates we have already introduced some of the changes that affect Piwik plugin developers. We have made several improvements to the Marketplace that allow developers to promote their plugin better and to maintain their plugin page easier. We have also created a migration guide for plugin developers to make it easier to migrate a plugin to Piwik 3.

  For a list of all changes have a look at the developer changelog.

  Until our next Piwik 3 update, be well, and Happy Analytics !

• Quick way to extract frames in low resolution

  29 mars 2013, par Lescott

  I have a video (640x320) which I need to decode in frame array (BufferedImage's in 100x100 size).

  I can solve this problem with IMediaReader and MediaListenerAdapter by the following slow way.

  // create reader and add listener
  
  reader = ToolFactory.makeReader("video.mp4");
  
  reader.setBufferedImageTypeToGenerate(BufferedImage.TYPE_3BYTE_BGR);
  
  reader.addListener(new ImageSnapListener());
  
  
  
  //change scale of pictures in onVideoPicture
  
  public void onVideoPicture(IVideoPictureEvent event) {
  
    ...check the stream number...
  
    results.add(event.getImage().getScaledInstance(100, 100, Image.SCALE_SMOOTH));
  
  }

  But this way is really slow. Even without images scaling it works 30 seconds on 54 MB file, whereas ffmpeg can extract 100x100 frames in 8 seconds.

  ffmpeg -vf select='eq(pic_type\,I)' -i video.mp4 -s 100x100 /tmp/frames/%0d.bmp

  So, is there any faster in-memory way to extract frames in low resolution ?