Recherche avancée

Sur d’autres sites (6035)

• How to create a widget – Introducing the Piwik Platform

  4 septembre 2014, par Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to create a scheduled task in Piwik). This time you’ll learn how to create a new widget. For this tutorial you will need to have basic knowledge of PHP.

  What is a widget in Piwik ?

  Widgets can be added to your dashboards or exported via a URL to embed it on any page. Most widgets in Piwik represent a report but a widget can display anything. For instance a RSS feed of your corporate news. If you prefer to have most of your business relevant data in one dashboard why not display the number of offline sales, the latest stock price, or other key metrics together with your analytics data ?

  Getting started

  In this series of posts, we assume that you have already set up your development environment. If not, visit the Piwik Developer Zone where you’ll find the tutorial Setting up Piwik.

  To summarize the things you have to do to get setup :

  • Install Piwik (for instance via git).
  • Activate the developer mode : ./console development:enable --full.
  • Generate a plugin : ./console generate:plugin --name="MyWidgetPlugin". There should now be a folder plugins/MyWidgetPlugin.
  • And activate the created plugin under Settings => Plugins.

  Let’s start creating a widget

  We start by using the Piwik Console to create a widget template :

  ./console generate:widget

  The command will ask you to enter the name of the plugin the widget should belong to. I will simply use the above chosen plugin name “MyWidgetPlugin”. It will ask you for a widget category as well. You can select any existing category, for instance “Visitors”, “Live !” or “Actions”, or you can define a new category, for instance your company name. There should now be a file plugins/MyWidgetPlugin/Widgets.php which contains already some examples to get you started easily :

  class Widgets extends \Piwik\Plugin\Widgets
  {
      /**
       * Here you can define the category the widget belongs to. You can reuse any existing widget category or define your own category.
       * @var string
       */
      protected $category = 'ExampleCompany';
   
      /**
       * Here you can add one or multiple widgets. You can add a widget by calling the method "addWidget()" and pass the name of the widget as well as a method name that should be called to render the widget. The method can be defined either directly here in this widget class or in the controller in case you want to reuse the same action for instance in the menu etc.
       */
      protected function init()
      {
          $this->addWidget('Example Widget Name', $method = 'myExampleWidget');
          $this->addWidget('Example Widget 2',    $method = 'myExampleWidget', $params = array('myparam' => 'myvalue'));
      }
   
      /**
       * This method renders a widget as defined in "init()". It's on you how to generate the content of the widget. As long as you return a string everything is fine. You can use for instance a "Piwik\View" to render a twig template. In such a case don't forget to create a twig template (eg. myViewTemplate.twig) in the "templates" directory of your plugin.
       *
       * @return string
       */
      public function myExampleWidget()
      {
          $view = new View('@MyWidgetPlugin/myViewTemplate');
          return $view->render();
      }
  }
  Télécharger

  As you might have noticed in the generated template we put emphasis on adding comments to explain you directly how to continue and where to get more information. Ideally this saves you some time and you don’t even have to search for more information on our developer pages. The category is defined in the property $category and can be changed at any time. Starting from Piwik 2.6.0 the generator will directly create a translation key if necessary to make it easy to translate the category into any language. Translations will be a topic in one of our future posts until then you can explore this feature on our Internationalization guide.

  A simple example

  We can define one or multiple widgets in the init method by calling addWidget($widgetName, $methodName). To do so we define the name of a widget which will be seen by your users as well as the name of the method that shall render the widget.

  protected $category = 'Example Company';
  
  
  
  public function init()
  
  {
  
      // Registers a widget named 'News' under the category 'Example Company'. 
  
      // The method 'myCorporateNews' will be used to render the widget.
  
      $this->addWidget('News', $method = 'myCorporateNews'); 
  
  }
  
  
  
  public function myCorporateNews()
  
  {
  
      return file_get_contents('http://example.com/news');
  
  }

  This example would display the content of the specified URL within the widget as defined in the method myCorporateNews. It’s on you how to generate the content of the widget. Any string returned by this method will be displayed within the widget. You can use for example a View to render a Twig template. For simplification we are fetching the content from another site. A more complex version would cache this content for faster performance. Caching and views will be covered in one of our future posts as well.

  

  Did you know ? To make your life as a developer as stress-free as possible the platform checks whether the registered method actually exists and whether the method is public. If not, Piwik will display a notification in the UI and advice you with the next step.

  Checking permissions

  Often you do not want to have the content of a widget visible to everyone. You can check for permissions by using one of our many convenient methods which all start with \Piwik\Piwik::checkUser*. Just to introduce some of them :

  // Make sure the current user has super user access
  
  \Piwik\Piwik::checkUserHasSuperUserAccess(); 
  
  
  
  // Make sure the current user is logged in and not anonymous
  
  \Piwik\Piwik::checkUserIsNotAnonymous(); 

  And here is an example how you can use it within your widget :

  public function myCorporateNews()
  
  {
  
      // Make sure there is an idSite URL parameter
  
      $idSite = Common::getRequestVar('idSite', null, 'int');
  
  
  
      // Make sure the user has at least view access for the specified site. This is useful if you want to display data that is related to the specified site.
  
      Piwik::checkUserHasViewAccess($idSite); 
  
  
  
      $siteUrl = \Piwik\Site::getMainUrlFor($idSite);
  
  
  
      return file_get_contents($siteUrl . '/news');
  
  }

  In case any condition is not met an exception will be thrown and an error message will be presented to the user explaining that he does not have enough permissions. You’ll find the documentation for those methods in the Piwik class reference.

  How to test a widget

  After you have created your widgets you are surely wondering how to test it. First, you should write a unit or integration test which we will cover in one of our future blog posts. Just one hint : You can use the command ./console generate:test to create a test. To manually test a widget you can add a widget to a dashboard or export it.

  Publishing your Plugin on the Marketplace

  In case you want to share your widgets with other Piwik users you can do this by pushing your plugin to a public GitHub repository and creating a tag. Easy as that. Read more about how to distribute a plugin.

  Advanced features

  Isn’t it easy to create a widget ? We never even created a file ! Of course, based on our API design principle “The complexity of our API should never exceed the complexity of your use case.” you can accomplish more if you want : You can clarify parameters that will be passed to your widget, you can create a method in the Controller instead of the Widget class to make the same method also reusable for adding it to the menu, you can assign different categories to different widgets, you can remove any widgets that were added by the Piwik core or other plugins and more.

  Would you like to know more about widgets ? Go to our Widgets class reference in the Piwik Developer Zone.

  If you have any feedback regarding our APIs or our guides in the Developer Zone feel free to send it to us.

• Visualizing Call Graphs Using Gephi

  1er septembre 2014, par Multimedia Mike — General

  When I was at university studying computer science, I took a basic chemistry course. During an accompanying lab, the teaching assistant chatted me up and asked about my major. He then said, “Computer science ? Well, that’s just typing stuff, right ?”

  My impulsive retort : “Sure, and chemistry is just about mixing together liquids and coming up with different colored liquids, as seen on the cover of my high school chemistry textbook, right ?”

  
  
  

  In fact, pure computer science has precious little to do with typing (as is joked in CS circles, computer science is about computers in the same way that astronomy is about telescopes). However, people who study computer science often pursue careers as programmers, or to put it in fancier professional language, software engineers.

  So, what’s a software engineer’s job ? Isn’t it just typing ? That’s where I’ve been going with this overly long setup. After thinking about it for long enough, I like to say that a software engineer’s trade is managing complexity.

  A few years ago, I discovered Gephi, an open source tool for graph and data visualization. It looked neat but I didn’t have much use for it at the time. Recently, however, I was trying to get a better handle on a large codebase. I.e., I was trying to manage the project’s complexity. And then I thought of Gephi again.

  Prior Work
  One way to get a grip on a large C codebase is to instrument it for profiling and extract details from the profiler. On Linux systems, this means compiling and linking the code using the -pg flag. After running the executable, there will be a gmon.out file which is post-processed using the gprof command.

  GNU software development tools have a reputation for being rather powerful and flexible, but also extremely raw. This first hit home when I was learning how to use the GNU tool for code coverage — gcov — and the way it outputs very raw data that you need to massage with other tools in order to get really useful intelligence.

  And so it is with gprof output. The output gives you a list of functions sorted by the amount of processing time spent in each. Then it gives you a flattened call tree. This is arranged as “during the profiled executions, function c was called by functions a and b and called functions d, e, and f ; function d was called by function c and called functions g and h”.

  How can this call tree data be represented in a more instructive manner that is easier to navigate ? My first impulse (and I don’t think I’m alone in this) is to convert the gprof call tree into a representation suitable for interpretation by Graphviz. Unfortunately, doing so tends to generate some enormous and unwieldy static images.

  Feeding gprof Data To Gephi
  I learned of Gephi a few years ago and recalled it when I developed an interest in gaining better perspective on a large base of alien C code. To understand what this codebase is doing for a particular use case, instrument it with gprof, gather execution data, and then study the code paths.

  How could I feed the gprof data into Gephi ? Gephi supports numerous graphing formats including an XML-based format named GEXF.

  Thus, the challenge becomes converting gprof output to GEXF.

  Which I did.

  Demonstration
  I have been absent from FFmpeg development for a long time, which is a pity because a lot of interesting development has occurred over the last 2-3 years after a troubling period of stagnation. I know that 2 big video codec developments have been HEVC (next in the line of MPEG codecs) and VP9 (heir to VP8’s throne). FFmpeg implements them both now.

  I decided I wanted to study the code flow of VP9. So I got the latest FFmpeg code from git and built it using the options "--extra-cflags=-pg --extra-ldflags=-pg". Annoyingly, I also needed to specify "--disable-asm" because gcc complains of some register allocation snafus when compiling inline ASM in profiling mode (and this is on x86_64). No matter ; ASM isn’t necessary for understanding overall code flow.

  After compiling, the binary ‘ffmpeg_g’ will have symbols and be instrumented for profiling. I grabbed a sample from this VP9 test vector set and went to work.

  ./ffmpeg_g -i vp90-2-00-quantizer-00.webm -f null /dev/null
  gprof ./ffmpeg_g > vp9decode.txt
  convert-gprof-to-gexf.py vp9decode.txt > /bigdisk/vp9decode.gexf
  

  Gephi loads vp9decode.gexf with no problem. Using Gephi, however, can be a bit challenging if one is not versed in any data exploration jargon. I recommend this Gephi getting starting guide in slide deck form. Here’s what the default graph looks like :

  
  
  

  Not very pretty or helpful. BTW, that beefy arrow running from mid-top to lower-right is the call from decode_coeffs_b -> iwht_iwht_4x4_add_c. There were 18774 from the former to the latter in this execution. Right now, the edge thicknesses correlate to number of calls between the nodes, which I’m not sure is the best representation.

  Following the tutorial slide deck, I at least learned how to enable the node labels (function symbols in this case) and apply a layout algorithm. The tutorial shows the force atlas layout. Here’s what the node neighborhood looks like for probing file type :

  
  
  

  Okay, so that’s not especially surprising– avprobe_input_format3 calls all of the *_probe functions in order to automatically determine input type. Let’s find that decode_coeffs_b function and see what its neighborhood looks like :

  
  
  

  That’s not very useful. Perhaps another algorithm might help. I select the Fruchterman–Reingold algorithm instead and get a slightly more coherent representation of the decoding node neighborhood :

  
  
  

  Further Work
  Obviously, I’m just getting started with this data exploration topic. One thing I would really appreciate in such a tool is the ability to interactively travel the graph since that’s what I’m really hoping to get out of this experiment– watching the code flows.

  Perhaps someone else can find better use cases for visualizing call graph data. Thus, I have published the source code for this tool at Github.

• How to create a scheduled task – Introducing the Piwik Platform

  28 août 2014, par Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to create a custom theme in Piwik). This time you’ll learn how to execute scheduled tasks in the background, for instance sending a daily email. For this tutorial you will need to have basic knowledge of PHP.

  What can you do with scheduled tasks ?

  Scheduled tasks let you execute tasks regularly (hourly, weekly, …). For instance you can :

  • create and send custom reports or summaries
  • sync users and websites with other systems
  • clear any caches
  • import third-party data into Piwik
  • monitor your Piwik instance
  • execute any other task you can think of

  Getting started

  In this series of posts, we assume that you have already set up your development environment. If not, visit the Piwik Developer Zone where you’ll find the tutorial Setting up Piwik.

  To summarize the things you have to do to get setup :

  • Install Piwik (for instance via git).
  • Activate the developer mode : ./console development:enable --full.
  • Generate a plugin : ./console generate:plugin --name="MyTasksPlugin". There should now be a folder plugins/MyTasksPlugin.
  • And activate the created plugin under Settings => Plugins.

  Let’s start creating a scheduled task

  We start by using the Piwik Console to create a tasks template :

  ./console generate:scheduledtask

  The command will ask you to enter the name of the plugin the task should belong to. I will simply use the above generated plugin name “MyTasksPlugin”. There should now be a file plugins/MyTasksPlugin/Tasks.php which contains some examples to get you started easily :

  class Tasks extends \Piwik\Plugin\Tasks
  
  {
  
      public function schedule()
  
      {
  
          $this->hourly('myTask');  // method will be executed once every hour
  
          $this->daily('myTask');   // method will be executed once every day
  
          $this->weekly('myTask');  // method will be executed once every week
  
          $this->monthly('myTask'); // method will be executed once every month
  
  
  
          // pass a parameter to the task
  
          $this->weekly('myTaskWithParam', 'anystring');
  
  
  
          // specify a different priority
  
          $this->monthly('myTask', null, self::LOWEST_PRIORITY);
  
          $this->monthly('myTaskWithParam', 'anystring', self::HIGH_PRIORITY);
  
      }
  
  
  
      public function myTask()
  
      {
  
          // do something
  
      }
  
  
  
      public function myTaskWithParam($param)
  
      {
  
          // do something
  
      }
  
  }

  A simple example

  As you can see in the generated template you can execute tasks hourly, daily, weekly and monthly by registering a method which represents the actual task :

  public function schedule()
  
  {
  
      // register method remindMeToLogIn to be executed once every day
  
      $this->daily('remindMeToLogIn');  
  
  }
  
  
  
  public function remindMeToLogIn()
  
  {
  
      $mail = new \Piwik\Mail();
  
      $mail->addTo('me@example.com');
  
      $mail->setSubject('Check stats');
  
      $mail->setBodyText('Log into your Piwik instance and check your stats!');
  
      $mail->send();
  
  }

  This example sends you an email once a day to remind you to log into your Piwik daily. The Piwik platform makes sure to execute the method remindMeToLogIn exactly once every day.

  How to pass a parameter to a task

  Sometimes you want to pass a parameter to a task method. This is useful if you want to register for instance one task for each user or for each website. You can achieve this by specifying a second parameter when registering the method to execute.

  public function schedule()
  
  {
  
      foreach (\Piwik\Site::getSites() as $site) {
  
          // create one task for each site and pass the URL of each site to the task
  
          $this->hourly('pingSite', $site['main_url']); 
  
      }
  
  }
  
  
  
  public function pingSite($siteMainUrl)
  
  {
  
      file_get_contents($siteMainUrl);
  
  }

  How to test scheduled tasks

  After you have created your task you are surely wondering how to test it. First, you should write a unit or integration test which we will cover in one of our future blog posts. Just one hint : You can use the command ./console generate:test to create a test. To manually execute all scheduled tasks you can execute the API method CoreAdminHome.runScheduledTasks by opening the following URL in your browser :

  http://piwik.example.com/index.php?module=API&method=CoreAdminHome.runScheduledTasks&token_auth=YOUR_API_TOKEN

  Don’t forget to replace the domain and the token_auth URL parameter.

  There is one problem with executing the scheduled tasks : The platform makes sure they will be executed only once an hour, a day, etc. This means you can’t simply reload the URL and test the method again and again as you would have to wait for the next hour or day. The proper solution is to set the constant DEBUG_FORCE_SCHEDULED_TASKS to true within the file Core/TaskScheduler.php. Don’t forget to set it back to false again once you have finished testing it.

  Starting from Piwik 2.6.0 you can alternatively execute the following command :

  ./console core:run-scheduled-tasks --force --token-auth=YOUR_TOKEN_AUTH

  The option “–force” will make sure to execute even tasks that are not due to run at this time. So you won’t have to modify any files.

  Which tasks are registered and when is the next execution time of my task ?

  The TasksTimetable plugin from the Marketplace can answer this question for you. Simply install and activate the plugin with one click by going to Settings => Marketplace => Get new functionality. It’ll add a new admin menu item under Settings named Scheduled Tasks.

  Publishing your Plugin on the Marketplace

  In case you want to share your task(s) with other Piwik users you can do this by pushing your plugin to a public GitHub repository and creating a tag. Easy as that. Read more about how to distribute a plugin.

  Advanced features

  Isn’t it easy to create scheduled tasks ? We never even created a file ! Of course, based on our API design principle “The complexity of our API should never exceed the complexity of your use case.” you can accomplish more if you want. For instance, you can define priorities, you can directly register methods from different objects and classes, you can specify at which time of a day a task should run and more.

  Would you like to know more about tasks ? Go to our Tasks class reference in the Piwik Developer Zone.

  If you have any feedback regarding our APIs or our guides in the Developer Zone feel free to send it to us.