Recherche avancée

Sur d’autres sites (6475)

• Generating test data – Introducing the Piwik Platform

  9 octobre 2014, par Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to create a command). This time you’ll learn how to generate test data.

  Developers are developing on their local Piwik instance which usually does not contain useful data compared to a real Piwik installation in production (only a few test visits and a few tests users and websites). The ‘VisitorGenerator’ plugin lets you generate any number of visits, websites, users, goals and more. The generator makes sure there will be data for each report so you can easily test anything.

  Getting started

  In this series of posts, we assume that you have already installed Piwik. If not, visit the Piwik Developer Zone where you’ll find the Installation guide for developers.

  Installing the VisitorGenerator plugin

  The easiest way to install the plugin is by using the Marketplace in Piwik itself. It is accessible via Settings => Marketplace => Get new functionality. There you’ll find the plugin “VisitorGenerator” which you can install and activate in one click.

  If your Piwik instance is not connected to the internet you can download the plugin from the VisitorGenerator page on the Marketplace. Afterwards you can install the plugin by going to Settings => Marketplace => Uploading a plugin and uploading the previously downloaded ZIP file.

  If you have already installed the plugin make sure it is activated by going to Settings => Plugins.

  Generating websites

  After you have installed the plugin you can add as many websites as you need. This is useful for instance when you want to test something that affects many websites such as the ‘All Websites’ dashboard or the Websites manager. To generate any number of websites use the following command :

  ./console visitorgenerator:generate-website --limit=10

  This will generate 10 websites. If you need more websites simply specify a higher limit. In case you are wondering the names and URLs of the websites are randomly generated by the Faker PHP library.

  Generating goals

  In case you want to test anything related to Goals you should execute the following command :

  ./console visitorgenerator:generate-goals --idsite=1

  This will generate a few goals for the specified site. The generated goals are defined in a way to make sure there will be conversions when generating the visits in the next step.

  Generating visits

  To generate visits there are two possibilities. Either via the Piwik UI by going to Settings => Visitor Generator or by using the command line. The UI is a bit limited in generating visits so we recommend to use the command line. There you can generate visits as follows :

  ./console visitorgenerator:generate-visits --idsite=1

  This will generate many different visits for the current day. Don’t worry if it takes a while, it will insert quite a few visits by default.

  In case you want to generate visits for multiple days in the past as well you can specify the --days option.

  ./console visitorgenerator:generate-visits --idsite=1 --days=5

  Providing your own logs

  Half of the generated visits are randomly generated and half of the visits are based on real logs to make sure there is data for each report. If you want to generate visits based on your own logs for a more realistic testing just place your log files in the plugins/VisitorGenerator/data folder and make sure the file name ends with .log. You can find a few examples in the VisitorGenerator data folder.

  To generate visits based only on real log files then use the --no-fake option.

  ./console visitorgenerator:generate-visits --idsite=1 --no-fake

  All generated visits will come from the logs and no random visits nor random fake data will be used.

  Advanced features

  We are regularly adding new commands, tools and runtime checks to make your life as a developer easier. For instance you can also generate users and annotations. In the future we want to extend the plugin to create visits in the background to make sure there will be constantly new actions in the real time report.

  Are you missing any kind of generator or any other feature to make your life as a developer easier ? Let us know by email, we are listening !

  Would you like to know more about the Piwik platform ? Go to our Piwik Developer Zone where you’ll find guides and references on how to develop plugin and themes.

• Generating test data – Introducing the Piwik Platform

  9 octobre 2014, par Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to create a command). This time you’ll learn how to generate test data.

  Developers are developing on their local Piwik instance which usually does not contain useful data compared to a real Piwik installation in production (only a few test visits and a few tests users and websites). The ‘VisitorGenerator’ plugin lets you generate any number of visits, websites, users, goals and more. The generator makes sure there will be data for each report so you can easily test anything.

  Getting started

  In this series of posts, we assume that you have already installed Piwik. If not, visit the Piwik Developer Zone where you’ll find the Installation guide for developers.

  Installing the VisitorGenerator plugin

  The easiest way to install the plugin is by using the Marketplace in Piwik itself. It is accessible via Settings => Marketplace => Get new functionality. There you’ll find the plugin “VisitorGenerator” which you can install and activate in one click.

  If your Piwik instance is not connected to the internet you can download the plugin from the VisitorGenerator page on the Marketplace. Afterwards you can install the plugin by going to Settings => Marketplace => Uploading a plugin and uploading the previously downloaded ZIP file.

  If you have already installed the plugin make sure it is activated by going to Settings => Plugins.

  Generating websites

  After you have installed the plugin you can add as many websites as you need. This is useful for instance when you want to test something that affects many websites such as the ‘All Websites’ dashboard or the Websites manager. To generate any number of websites use the following command :

  ./console visitorgenerator:generate-website --limit=10

  This will generate 10 websites. If you need more websites simply specify a higher limit. In case you are wondering the names and URLs of the websites are randomly generated by the Faker PHP library.

  Generating goals

  In case you want to test anything related to Goals you should execute the following command :

  ./console visitorgenerator:generate-goals --idsite=1

  This will generate a few goals for the specified site. The generated goals are defined in a way to make sure there will be conversions when generating the visits in the next step.

  Generating visits

  To generate visits there are two possibilities. Either via the Piwik UI by going to Settings => Visitor Generator or by using the command line. The UI is a bit limited in generating visits so we recommend to use the command line. There you can generate visits as follows :

  ./console visitorgenerator:generate-visits --idsite=1

  This will generate many different visits for the current day. Don’t worry if it takes a while, it will insert quite a few visits by default.

  In case you want to generate visits for multiple days in the past as well you can specify the --days option.

  ./console visitorgenerator:generate-visits --idsite=1 --days=5

  Providing your own logs

  Half of the generated visits are randomly generated and half of the visits are based on real logs to make sure there is data for each report. If you want to generate visits based on your own logs for a more realistic testing just place your log files in the plugins/VisitorGenerator/data folder and make sure the file name ends with .log. You can find a few examples in the VisitorGenerator data folder.

  To generate visits based only on real log files then use the --no-fake option.

  ./console visitorgenerator:generate-visits --idsite=1 --no-fake

  All generated visits will come from the logs and no random visits nor random fake data will be used.

  Advanced features

  We are regularly adding new commands, tools and runtime checks to make your life as a developer easier. For instance you can also generate users and annotations. In the future we want to extend the plugin to create visits in the background to make sure there will be constantly new actions in the real time report.

  Are you missing any kind of generator or any other feature to make your life as a developer easier ? Let us know by email, we are listening !

  Would you like to know more about the Piwik platform ? Go to our Piwik Developer Zone where you’ll find guides and references on how to develop plugin and themes.

• (issue) ffmpeg - usage of map & map_metadata options for correctly setting stream metadata & copying attachments

  1er mai 2022, par deep_rooted

  Using ffmpeg, I need to encode a series of video files with video, audio, subtitle and attachment streams (5 fonts). I would like to have all the streams in my output file while having fresh metadata for the audio and video streams (not copied from input).

  


  However I see that ffmpeg is quite troublesome in metadata management. Because it retains all input file metadata by default, including audio, video bitrate, file size etc. This is undesirable.

  


  After a lot of digging, I came up with the following solution which should work :

  


  ffmpeg -i input.mkv -map_metadata 0:s:t -map 0 -c:v libx264 -c:a libopus output.mkv


  


  The command above should copy all the streams, but will only copy metadata of the attachment files, since without attachment metadata, attachment streams can not be copied. This method skips copying audio, video metadata.

  


  But this does not work. It shows the following error :

  


  


  [matroska @ 000001d5fa6ad5c0] Attachment stream 3 has no filename tag.
Could not write header for output file #0 (incorrect codec parameters ?) : Invalid argument
Error initializing output stream 0:0 —

  


  


  For some strange reason, it says Attachment stream 3 (first font file) has no filename tag. If no map_metadata option is used & only -map 0 is used, there is no error. So I tried setting the font file name with -metadata:s:3 filename="test.ttf". This solved the issue at hand but brought the same issue for next font stream (s#4).

  


  Font Attachments have two metadata : (1) filename, (2) mimetype. It did not raise any issue for 2nd metadata. Honestly, it should not raise any issue for 1st metadata either, since these info should have been copied with -map_metadata 0:s:t Now of course I don't want to manually set all the font name, rather I would like to know a correct approach for this problem.

  


  I have of course tried -map_metadata -1. But same problem. Since no metadata is copied, it raises the same error that it can't copy the attachment streams without filename.