Recherche avancée

Sur d’autres sites (4891)

• How to create a command – Introducing the Piwik Platform

  2 octobre 2014, par Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to publish your plugin or theme on the Piwik Marketplace). This time you’ll learn how to create a new command. For this tutorial you will need to have basic knowledge of PHP.

  What is a command ?

  A command can execute any task on the command line. Piwik provides currently about 50 commands via the Piwik Console. These commands let you start the archiver, change the number of available custom variables, enable the developer mode, clear caches, run tests and more. You could write your own command to sync users or websites with another system for instance.

  Getting started

  In this series of posts, we assume that you have already set up your development environment. If not, visit the Piwik Developer Zone where you’ll find the tutorial Setting up Piwik.

  To summarize the things you have to do to get setup :

  • Install Piwik (for instance via git).
  • Activate the developer mode : ./console development:enable --full.
  • Generate a plugin : ./console generate:plugin --name="MyCommandPlugin". There should now be a folder plugins/MyCommandPlugin.
  • And activate the created plugin under Settings => Plugins.

  Let’s start creating a command

  We start by using the Piwik Console to create a new command. As you can see there is even a command that lets you easily create a new command :

  ./console generate:command

  The command will ask you to enter the name of the plugin the created command should belong to. I will simply use the above chosen plugin name “MyCommandPlugin”. It will ask you for a command name as well. I will use “SyncUsers” in this example. There should now be a file plugins/MyCommandPlugin/Commands/Syncusers.php which contains already an example to get you started easily :

  class Syncusers extends ConsoleCommand
  {
      protected function configure()
      {
          $this->setName('mycommandplugin:syncusers');
          $this->setDescription('MyCommandPlugin');
          $this->addOption('name', null, InputOption::VALUE_REQUIRED, 'Your name:');
      }
   
      /**
       * Execute command like: ./console mycommandplugin:syncusers --name="The Piwik Team"
       */
      protected function execute(InputInterface $input, OutputInterface $output)
      {
          $name    = $input->getOption('name');
   
          $message = sprintf('Syncusers: %s', $name);
   
          $output->writeln($message);
      }
  }
  Télécharger

  Any command that is placed in the “Commands” folder of your plugin will be available on the command line automatically. Therefore, the newly created command can now be executed via ./console mycommandplugin:syncusers --name="The Piwik Team".

  The code template explained

  protected function configure()
  {
      $this->setName('mycommandplugin:checkdatabase');
      $this->setDescription('MyCommandPlugin');
      $this->addOption('name', null, InputOption::VALUE_REQUIRED, 'Your name:');
  }
  Télécharger

  As the name says the method configure lets you configure your command. You can define the name and description of your command as well as all the options and arguments you expect when executing it.

  protected function execute(InputInterface $input, OutputInterface $output)
  {
      $name    = $input->getOption('name');
      $message = sprintf('Syncusers: %s', $name);
      $output->writeln($message);
  }
  Télécharger

  The actual task is defined in the execute method. There you can access any option or argument that was defined on the command line via $input and write anything to the console via $output argument.

  In case anything went wrong during the execution you should throw an exception to make sure the user will get a useful error message. Throwing an exception when an error occurs will make sure the command does exit with a status code different than 0 which can sometimes be important.

  Advanced features

  The Piwik Console is based on the powerful Symfony Console component. For instance you can ask a user for any interactive input, you can use different output color schemes and much more. If you are interested in learning more all those features have a look at the Symfony console website.

  How to test a command

  After you have created a command you are surely wondering how to test it. Ideally, the actual command is quite short as it acts like a controller. It should only receive the input values, execute the task by calling a method of another class and output any useful information. This allows you to easily create a unit or integration test for the classes behind the command. We will cover this topic in one of our future blog posts. Just one hint : You can use another command ./console generate:test to create a test. If you want to know how to test a command have a look at the Testing Commands documentation.

  Publishing your Plugin on the Marketplace

  In case you want to share your commands with other Piwik users you can do this by pushing your plugin to a public GitHub repository and creating a tag. Easy as that. Read more about how to distribute a plugin and best practices when publishing a plugin.

  Isn’t it easy to create a command ? We never even created a file ! If you have any feedback regarding our APIs or our guides in the Developer Zone feel free to send it to us.

• How to create a command – Introducing the Piwik Platform

  2 octobre 2014, par Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to publish your plugin or theme on the Piwik Marketplace). This time you’ll learn how to create a new command. For this tutorial you will need to have basic knowledge of PHP.

  What is a command ?

  A command can execute any task on the command line. Piwik provides currently about 50 commands via the Piwik Console. These commands let you start the archiver, change the number of available custom variables, enable the developer mode, clear caches, run tests and more. You could write your own command to sync users or websites with another system for instance.

  Getting started

  In this series of posts, we assume that you have already set up your development environment. If not, visit the Piwik Developer Zone where you’ll find the tutorial Setting up Piwik.

  To summarize the things you have to do to get setup :

  • Install Piwik (for instance via git).
  • Activate the developer mode : ./console development:enable --full.
  • Generate a plugin : ./console generate:plugin --name="MyCommandPlugin". There should now be a folder plugins/MyCommandPlugin.
  • And activate the created plugin under Settings => Plugins.

  Let’s start creating a command

  We start by using the Piwik Console to create a new command. As you can see there is even a command that lets you easily create a new command :

  ./console generate:command

  The command will ask you to enter the name of the plugin the created command should belong to. I will simply use the above chosen plugin name “MyCommandPlugin”. It will ask you for a command name as well. I will use “SyncUsers” in this example. There should now be a file plugins/MyCommandPlugin/Commands/Syncusers.php which contains already an example to get you started easily :

  class Syncusers extends ConsoleCommand
  {
      protected function configure()
      {
          $this->setName('mycommandplugin:syncusers');
          $this->setDescription('MyCommandPlugin');
          $this->addOption('name', null, InputOption::VALUE_REQUIRED, 'Your name:');
      }
   
      /**
       * Execute command like: ./console mycommandplugin:syncusers --name="The Piwik Team"
       */
      protected function execute(InputInterface $input, OutputInterface $output)
      {
          $name    = $input->getOption('name');
   
          $message = sprintf('Syncusers: %s', $name);
   
          $output->writeln($message);
      }
  }
  Télécharger

  Any command that is placed in the “Commands” folder of your plugin will be available on the command line automatically. Therefore, the newly created command can now be executed via ./console mycommandplugin:syncusers --name="The Piwik Team".

  The code template explained

  protected function configure()
  {
      $this->setName('mycommandplugin:checkdatabase');
      $this->setDescription('MyCommandPlugin');
      $this->addOption('name', null, InputOption::VALUE_REQUIRED, 'Your name:');
  }
  Télécharger

  As the name says the method configure lets you configure your command. You can define the name and description of your command as well as all the options and arguments you expect when executing it.

  protected function execute(InputInterface $input, OutputInterface $output)
  {
      $name    = $input->getOption('name');
      $message = sprintf('Syncusers: %s', $name);
      $output->writeln($message);
  }
  Télécharger

  The actual task is defined in the execute method. There you can access any option or argument that was defined on the command line via $input and write anything to the console via $output argument.

  In case anything went wrong during the execution you should throw an exception to make sure the user will get a useful error message. Throwing an exception when an error occurs will make sure the command does exit with a status code different than 0 which can sometimes be important.

  Advanced features

  The Piwik Console is based on the powerful Symfony Console component. For instance you can ask a user for any interactive input, you can use different output color schemes and much more. If you are interested in learning more all those features have a look at the Symfony console website.

  How to test a command

  After you have created a command you are surely wondering how to test it. Ideally, the actual command is quite short as it acts like a controller. It should only receive the input values, execute the task by calling a method of another class and output any useful information. This allows you to easily create a unit or integration test for the classes behind the command. We will cover this topic in one of our future blog posts. Just one hint : You can use another command ./console generate:test to create a test. If you want to know how to test a command have a look at the Testing Commands documentation.

  Publishing your Plugin on the Marketplace

  In case you want to share your commands with other Piwik users you can do this by pushing your plugin to a public GitHub repository and creating a tag. Easy as that. Read more about how to distribute a plugin and best practices when publishing a plugin.

  Isn’t it easy to create a command ? We never even created a file ! If you have any feedback regarding our APIs or our guides in the Developer Zone feel free to send it to us.

• Working on images asynchronously

  15 décembre 2013, par Mikko Koppanen — Imagick, PHP stuff

  To get my quota on buzzwords for the day we are going to look at using ZeroMQ and Imagick to create a simple asynchronous image processing system. Why asynchronous ? First of all, separating the image handling from a interactive PHP scripts allows us to scale the image processing separately from the web heads. For example we could do the image processing on separate servers, which have SSDs attached and more memory. In this example making the images available to all worker nodes is left to the reader.

  Secondly, separating the image processing from a web script can provide more responsive experience to the user. This doesn’t necessarily mean faster, but let’s say in a multiple image upload scenario this method allows the user to do something else on the site while we process the images in the background. This can be beneficial especially in cases where users upload hundreds of images at a time. To achieve a simple distributed image processing infrastructure we are going to use ZeroMQ for communicating between different components and Imagick to work on the images.

  The first part we are going to create is a simple “Worker” -process skeleton. Naturally for a live environment you would like to have more error handling and possibly use pcntl for process control, but for the sake of brewity the example is barebones :

  php

  < view plain text >

  1. < ?php
  2.  
  3. define (’THUMBNAIL_ADDR’, ’tcp ://127.0.0.1:5000’) ;
  4. define (’COLLECTOR_ADDR’, ’tcp ://127.0.0.1:5001’) ;
  5.  
  6. class Worker {
  7.  
  8.   private $in ;
  9.   private $out ;
  10.  
  11.   public function __construct ($in_addr, $out_addr)
  12.   {
  13.     $context = new ZMQContext () ;
  14.  
  15.     $this->in = new ZMQSocket ($context, ZMQ: :SOCKET_PULL) ;
  16.     $this->in->bind ($in_addr) ;
  17.  
  18.     $this->out = new ZMQSocket ($context, ZMQ: :SOCKET_PUSH) ;
  19.     $this->out->connect ($out_addr) ;
  20.   }
  21.  
  22.   public function work () {
  23.     while ($command = $this->in->recvMulti ()) {
  24.       if (isset ($this->commands [$command [0]])) {
  25.         echo "Received work" . PHP_EOL ;
  26.  
  27.         $callback = $this->commands [$command [0]] ;
  28.  
  29.         array_shift ($command) ;
  30.         $response = call_user_func_array ($callback, $command) ;
  31.  
  32.         if (is_array ($response))
  33.           $this->out->sendMulti ($response) ;
  34.         else
  35.           $this->out->send ($response) ;
  36.       }
  37.       else {
  38.         error_log ("There is no registered worker for $command [0]") ;
  39.       }
  40.     }
  41.   }
  42.  
  43.   public function register ($command, $callback)
  44.   {
  45.     $this->commands [$command] = $callback ;
  46.   }
  47. }
  48.  ?>

  The Worker class allows us to register commands with callbacks associated with them. In our case the Worker class doesn’t actually care or know about the parameters being passed to the actual callback, it just blindly passes them on. We are using two separate sockets in this example, one for incoming work requests and one for passing the results onwards. This allows us to create a simple pipeline by adding more workers in the mix. For example we could first have a watermark worker, which takes the original image and composites a watermark on it, passes the file onwards to thumbnail worker, which then creates different sizes of thumbnails and passes the final results to event collector.

  The next part we are going to create a is a simple worker script that does the actual thumbnailing of the images :

  php

  < view plain text >

  1. < ?php
  2. include __DIR__ . ’/common.php’ ;
  3.  
  4. // Create worker class and bind the inbound address to ’THUMBNAIL_ADDR’ and connect outbound to ’COLLECTOR_ADDR’
  5. $worker = new Worker (THUMBNAIL_ADDR, COLLECTOR_ADDR) ;
  6.  
  7. // Register our thumbnail callback, nothing special here
  8. $worker->register (’thumbnail’, function ($filename, $width, $height) {
  9.                   $info = pathinfo ($filename) ;
  10.  
  11.                   $out = sprintf ("%s/%s_%dx%d.%s",
  12.                           $info [’dirname’],
  13.                           $info [’filename’],
  14.                           $width,
  15.                           $height,
  16.                           $info [’extension’]) ;
  17.  
  18.                   $status = 1 ;
  19.                   $message = ’’ ;
  20.  
  21.                   try {
  22.                     $im = new Imagick ($filename) ;
  23.                     $im->thumbnailImage ($width, $height) ;
  24.                     $im->writeImage ($out) ;
  25.                   }
  26.                   catch (Exception $e) {
  27.                     $status = 0 ;
  28.                     $message = $e->getMessage () ;
  29.                   }
  30.  
  31.                   return array (
  32.                         ’status’  => $status,
  33.                         ’filename’ => $filename,
  34.                         ’thumbnail’ => $out,
  35.                         ’message’ => $message,
  36.                     ) ;
  37.                 }) ;
  38.  
  39. // Run the worker, will block
  40. echo "Running thumbnail worker.." . PHP_EOL ;
  41. $worker->work () ;

  As you can see from the code the thumbnail worker registers a callback for ‘thumbnail’ command. The callback does the thumbnailing based on input and returns the status, original filename and the thumbnail filename. We have connected our Workers “outbound” socket to event collector, which will receive the results from the thumbnail worker and do something with them. What the “something” is depends on you. For example you could push the response into a websocket to show immediate feeedback to the user or store the results into a database.

  Our example event collector will just do a var_dump on every event it receives from the thumbnailer :

  php

  < view plain text >

  1. < ?php
  2. include __DIR__ . ’/common.php’ ;
  3.  
  4. $socket = new ZMQSocket (new ZMQContext (), ZMQ: :SOCKET_PULL) ;
  5. $socket->bind (COLLECTOR_ADDR) ;
  6.  
  7. echo "Waiting for events.." . PHP_EOL ;
  8. while (($message = $socket->recvMulti ())) {
  9.   var_dump ($message) ;
  10. }
  11.  ?>

  The final piece of the puzzle is the client that pumps messages into the pipeline. The client connects to the thumbnail worker, passes on filename and desired dimensions :

  php

  < view plain text >

  1. < ?php
  2. include __DIR__ . ’/common.php’ ;
  3.  
  4. $socket = new ZMQSocket (new ZMQContext (), ZMQ: :SOCKET_PUSH) ;
  5. $socket->connect (THUMBNAIL_ADDR) ;
  6.  
  7. $socket->sendMulti (
  8.       array (
  9.         ’thumbnail’,
  10.         realpath (’./test.jpg’),
  11.         50,
  12.         50,
  13.       )
  14. ) ;
  15. echo "Sent request" . PHP_EOL ;
  16.  ?>

  After this our processing pipeline will look like this :

  

  Now, if we notice that thumbnail workers or the event collectors can’t keep up with the rate of images we are pushing through we can start scaling the pipeline by adding more processes on each layer. ZeroMQ PUSH socket will automatically round-robin between all connected nodes, which makes adding more workers and event collectors simple. After adding more workers our pipeline will look like this :

  

  Using ZeroMQ also allows us to create more flexible architectures by adding forwarding devices in the middle, adding request-reply workers etc. So, the last thing to do is to run our pipeline and see the results :

  Let’s create our test image first :

  $ convert magick:rose test.jpg
  

  From the command-line run the thumbnail script :

  $ php thumbnail.php 
  Running thumbnail worker..
  

  In a separate terminal window run the event collector :

  $ php collector.php 
  Waiting for events..
  

  And finally run the client to send the thumbnail request :

  $ php client.php 
  Sent request
  $
  

  If everything went according to the plan you should now see the following output in the event collector window :

  array(4) 
    [0]=>
    string(1) "1"
    [1]=>
    string(56) "/test.jpg"
    [2]=>
    string(62) "/test_50x50.jpg"
    [3]=>
    string(0) ""
  
  

  Happy hacking !