Advanced search

On other websites (12683)

• How to create a scheduled task – Introducing the Piwik Platform

  28 August 2014, by Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to create a custom theme in Piwik). This time you’ll learn how to execute scheduled tasks in the background, for instance sending a daily email. For this tutorial you will need to have basic knowledge of PHP.

  What can you do with scheduled tasks?

  Scheduled tasks let you execute tasks regularly (hourly, weekly, …). For instance you can:

  • create and send custom reports or summaries
  • sync users and websites with other systems
  • clear any caches
  • import third-party data into Piwik
  • monitor your Piwik instance
  • execute any other task you can think of

  Getting started

  In this series of posts, we assume that you have already set up your development environment. If not, visit the Piwik Developer Zone where you’ll find the tutorial Setting up Piwik.

  To summarize the things you have to do to get setup:

  • Install Piwik (for instance via git).
  • Activate the developer mode: ./console development:enable --full.
  • Generate a plugin: ./console generate:plugin --name="MyTasksPlugin". There should now be a folder plugins/MyTasksPlugin.
  • And activate the created plugin under Settings => Plugins.

  Let’s start creating a scheduled task

  We start by using the Piwik Console to create a tasks template:

  ./console generate:scheduledtask

  The command will ask you to enter the name of the plugin the task should belong to. I will simply use the above generated plugin name “MyTasksPlugin”. There should now be a file plugins/MyTasksPlugin/Tasks.php which contains some examples to get you started easily:

  class Tasks extends \Piwik\Plugin\Tasks
  
  {
  
      public function schedule()
  
      {
  
          $this->hourly('myTask');  // method will be executed once every hour
  
          $this->daily('myTask');   // method will be executed once every day
  
          $this->weekly('myTask');  // method will be executed once every week
  
          $this->monthly('myTask'); // method will be executed once every month
  
  
  
          // pass a parameter to the task
  
          $this->weekly('myTaskWithParam', 'anystring');
  
  
  
          // specify a different priority
  
          $this->monthly('myTask', null, self::LOWEST_PRIORITY);
  
          $this->monthly('myTaskWithParam', 'anystring', self::HIGH_PRIORITY);
  
      }
  
  
  
      public function myTask()
  
      {
  
          // do something
  
      }
  
  
  
      public function myTaskWithParam($param)
  
      {
  
          // do something
  
      }
  
  }

  A simple example

  As you can see in the generated template you can execute tasks hourly, daily, weekly and monthly by registering a method which represents the actual task:

  public function schedule()
  
  {
  
      // register method remindMeToLogIn to be executed once every day
  
      $this->daily('remindMeToLogIn');  
  
  }
  
  
  
  public function remindMeToLogIn()
  
  {
  
      $mail = new \Piwik\Mail();
  
      $mail->addTo('me@example.com');
  
      $mail->setSubject('Check stats');
  
      $mail->setBodyText('Log into your Piwik instance and check your stats!');
  
      $mail->send();
  
  }

  This example sends you an email once a day to remind you to log into your Piwik daily. The Piwik platform makes sure to execute the method remindMeToLogIn exactly once every day.

  How to pass a parameter to a task

  Sometimes you want to pass a parameter to a task method. This is useful if you want to register for instance one task for each user or for each website. You can achieve this by specifying a second parameter when registering the method to execute.

  public function schedule()
  
  {
  
      foreach (\Piwik\Site::getSites() as $site) {
  
          // create one task for each site and pass the URL of each site to the task
  
          $this->hourly('pingSite', $site['main_url']); 
  
      }
  
  }
  
  
  
  public function pingSite($siteMainUrl)
  
  {
  
      file_get_contents($siteMainUrl);
  
  }

  How to test scheduled tasks

  After you have created your task you are surely wondering how to test it. First, you should write a unit or integration test which we will cover in one of our future blog posts. Just one hint: You can use the command ./console generate:test to create a test. To manually execute all scheduled tasks you can execute the API method CoreAdminHome.runScheduledTasks by opening the following URL in your browser:

  http://piwik.example.com/index.php?module=API&method=CoreAdminHome.runScheduledTasks&token_auth=YOUR_API_TOKEN

  Don’t forget to replace the domain and the token_auth URL parameter.

  There is one problem with executing the scheduled tasks: The platform makes sure they will be executed only once an hour, a day, etc. This means you can’t simply reload the URL and test the method again and again as you would have to wait for the next hour or day. The proper solution is to set the constant DEBUG_FORCE_SCHEDULED_TASKS to true within the file Core/TaskScheduler.php. Don’t forget to set it back to false again once you have finished testing it.

  Starting from Piwik 2.6.0 you can alternatively execute the following command:

  ./console core:run-scheduled-tasks --force --token-auth=YOUR_TOKEN_AUTH

  The option “–force” will make sure to execute even tasks that are not due to run at this time. So you won’t have to modify any files.

  Which tasks are registered and when is the next execution time of my task?

  The TasksTimetable plugin from the Marketplace can answer this question for you. Simply install and activate the plugin with one click by going to Settings => Marketplace => Get new functionality. It’ll add a new admin menu item under Settings named Scheduled Tasks.

  Publishing your Plugin on the Marketplace

  In case you want to share your task(s) with other Piwik users you can do this by pushing your plugin to a public GitHub repository and creating a tag. Easy as that. Read more about how to distribute a plugin.

  Advanced features

  Isn’t it easy to create scheduled tasks? We never even created a file! Of course, based on our API design principle “The complexity of our API should never exceed the complexity of your use case.” you can accomplish more if you want. For instance, you can define priorities, you can directly register methods from different objects and classes, you can specify at which time of a day a task should run and more.

  Would you like to know more about tasks? Go to our Tasks class reference in the Piwik Developer Zone.

  If you have any feedback regarding our APIs or our guides in the Developer Zone feel free to send it to us.

• How to create a scheduled task – Introducing the Piwik Platform

  28 August 2014, by Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to create a custom theme in Piwik). This time you’ll learn how to execute scheduled tasks in the background, for instance sending a daily email. For this tutorial you will need to have basic knowledge of PHP.

  What can you do with scheduled tasks?

  Scheduled tasks let you execute tasks regularly (hourly, weekly, …). For instance you can:

  • create and send custom reports or summaries
  • sync users and websites with other systems
  • clear any caches
  • import third-party data into Piwik
  • monitor your Piwik instance
  • execute any other task you can think of

  Getting started

  In this series of posts, we assume that you have already set up your development environment. If not, visit the Piwik Developer Zone where you’ll find the tutorial Setting up Piwik.

  To summarize the things you have to do to get setup:

  • Install Piwik (for instance via git).
  • Activate the developer mode: ./console development:enable --full.
  • Generate a plugin: ./console generate:plugin --name="MyTasksPlugin". There should now be a folder plugins/MyTasksPlugin.
  • And activate the created plugin under Settings => Plugins.

  Let’s start creating a scheduled task

  We start by using the Piwik Console to create a tasks template:

  ./console generate:scheduledtask

  The command will ask you to enter the name of the plugin the task should belong to. I will simply use the above generated plugin name “MyTasksPlugin”. There should now be a file plugins/MyTasksPlugin/Tasks.php which contains some examples to get you started easily:

  class Tasks extends \Piwik\Plugin\Tasks
  
  {
  
      public function schedule()
  
      {
  
          $this->hourly('myTask');  // method will be executed once every hour
  
          $this->daily('myTask');   // method will be executed once every day
  
          $this->weekly('myTask');  // method will be executed once every week
  
          $this->monthly('myTask'); // method will be executed once every month
  
  
  
          // pass a parameter to the task
  
          $this->weekly('myTaskWithParam', 'anystring');
  
  
  
          // specify a different priority
  
          $this->monthly('myTask', null, self::LOWEST_PRIORITY);
  
          $this->monthly('myTaskWithParam', 'anystring', self::HIGH_PRIORITY);
  
      }
  
  
  
      public function myTask()
  
      {
  
          // do something
  
      }
  
  
  
      public function myTaskWithParam($param)
  
      {
  
          // do something
  
      }
  
  }

  A simple example

  As you can see in the generated template you can execute tasks hourly, daily, weekly and monthly by registering a method which represents the actual task:

  public function schedule()
  
  {
  
      // register method remindMeToLogIn to be executed once every day
  
      $this->daily('remindMeToLogIn');  
  
  }
  
  
  
  public function remindMeToLogIn()
  
  {
  
      $mail = new \Piwik\Mail();
  
      $mail->addTo('me@example.com');
  
      $mail->setSubject('Check stats');
  
      $mail->setBodyText('Log into your Piwik instance and check your stats!');
  
      $mail->send();
  
  }

  This example sends you an email once a day to remind you to log into your Piwik daily. The Piwik platform makes sure to execute the method remindMeToLogIn exactly once every day.

  How to pass a parameter to a task

  Sometimes you want to pass a parameter to a task method. This is useful if you want to register for instance one task for each user or for each website. You can achieve this by specifying a second parameter when registering the method to execute.

  public function schedule()
  
  {
  
      foreach (\Piwik\Site::getSites() as $site) {
  
          // create one task for each site and pass the URL of each site to the task
  
          $this->hourly('pingSite', $site['main_url']); 
  
      }
  
  }
  
  
  
  public function pingSite($siteMainUrl)
  
  {
  
      file_get_contents($siteMainUrl);
  
  }

  How to test scheduled tasks

  After you have created your task you are surely wondering how to test it. First, you should write a unit or integration test which we will cover in one of our future blog posts. Just one hint: You can use the command ./console generate:test to create a test. To manually execute all scheduled tasks you can execute the API method CoreAdminHome.runScheduledTasks by opening the following URL in your browser:

  http://piwik.example.com/index.php?module=API&method=CoreAdminHome.runScheduledTasks&token_auth=YOUR_API_TOKEN

  Don’t forget to replace the domain and the token_auth URL parameter.

  There is one problem with executing the scheduled tasks: The platform makes sure they will be executed only once an hour, a day, etc. This means you can’t simply reload the URL and test the method again and again as you would have to wait for the next hour or day. The proper solution is to set the constant DEBUG_FORCE_SCHEDULED_TASKS to true within the file Core/TaskScheduler.php. Don’t forget to set it back to false again once you have finished testing it.

  Starting from Piwik 2.6.0 you can alternatively execute the following command:

  ./console core:run-scheduled-tasks --force --token-auth=YOUR_TOKEN_AUTH

  The option “–force” will make sure to execute even tasks that are not due to run at this time. So you won’t have to modify any files.

  Which tasks are registered and when is the next execution time of my task?

  The TasksTimetable plugin from the Marketplace can answer this question for you. Simply install and activate the plugin with one click by going to Settings => Marketplace => Get new functionality. It’ll add a new admin menu item under Settings named Scheduled Tasks.

  Publishing your Plugin on the Marketplace

  In case you want to share your task(s) with other Piwik users you can do this by pushing your plugin to a public GitHub repository and creating a tag. Easy as that. Read more about how to distribute a plugin.

  Advanced features

  Isn’t it easy to create scheduled tasks? We never even created a file! Of course, based on our API design principle “The complexity of our API should never exceed the complexity of your use case.” you can accomplish more if you want. For instance, you can define priorities, you can directly register methods from different objects and classes, you can specify at which time of a day a task should run and more.

  Would you like to know more about tasks? Go to our Tasks class reference in the Piwik Developer Zone.

  If you have any feedback regarding our APIs or our guides in the Developer Zone feel free to send it to us.

• ffmpeg with Quick Sync Video (qsv) gets stuck

  28 April 2023, by Juliano B. Nequirito

  I've compiled ffmpeg in an Arch Linux distribution with support to Intel Quick Sync Video. But ffmpeg gets stuck when I try to run it:

  


  $ ffmpeg -debug -hwaccel qsv -qsv_device /dev/dri/renderD128 -c:v h264_qsv -i test_video.mp4 -c:v hevc_qsv output.mp4

ffmpeg version N-109562-g0431f9805e Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (GCC)
  configuration: --prefix=/usr --extra-cflags=-I/opt/cuda/include --extra-ldflags=-L/opt/cuda/lib64 --enable-lto --disable-rpath --enable-gpl --enable-version3 --enable-nonfree --enable-shared --disable-static --disable-stripping --enable-gray --enable-alsa --enable-avisynth --enable-bzlib --enable-chromaprint --enable-frei0r --enable-gcrypt --enable-gmp --enable-gnutls --enable-iconv --enable-ladspa --enable-lcms2 --enable-libaom --enable-libaribb24 --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcelt --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libdavs2 --enable-libdc1394 --enable-libfdk-aac --enable-libflite --enable-fontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libiec61883 --enable-libilbc --enable-libjack --enable-libjxl --enable-libklvanc --enable-libkvazaar --enable-liblensfun --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-libopencv --enable-libopenh264 --enable-libopenjpeg --enable-libopenmpt --disable-libopenvino --enable-libopus --enable-libplacebo --enable-libpulse --enable-librabbitmq --enable-librav1e --enable-librist --enable-librsvg --enable-librubberband --enable-librtmp --enable-libshine --enable-libsmbclient --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libsvthevc --enable-libsvtvp9 --disable-libtensorflow --enable-libtesseract --enable-libtheora --disable-libtls --enable-libtwolame --enable-libuavs3d --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxavs2 --enable-libxcb --enable-libxcb-shm --enable-libxcb-xfixes --enable-libxcb-shape --enable-libxvid --enable-libxml2 --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-lzma --enable-decklink --disable-mbedtls --enable-libmysofa --enable-openal --enable-opencl --enable-opengl --disable-openssl --enable-pocketsphinx --enable-sndio --enable-sdl2 --enable-vapoursynth --enable-vulkan --enable-xlib --enable-zlib --enable-amf --enable-cuda-nvcc --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-libdrm --enable-libvpl --enable-libnpp --enable-nvdec --enable-nvenc --enable-omx --enable-rkmpp --enable-v4l2-m2m --enable-vaapi --enable-vdpau
  libavutil      57. 43.100 / 57. 43.100
  libavcodec     59. 56.100 / 59. 56.100
  libavformat    59. 34.102 / 59. 34.102
  libavdevice    59.  8.101 / 59.  8.101
  libavfilter     8. 53.100 /  8. 53.100
  libswscale      6.  8.112 /  6.  8.112
  libswresample   4.  9.100 /  4.  9.100
  libpostproc    56.  7.100 / 56.  7.100
 matched as AVOption 'debug' with argument '-hwaccel'.
Reading option 'qsv' ... matched as output url.
Reading option '-qsv_device' ... matched as option 'qsv_device' (set QSV hardware device (DirectX adapter index, DRM path or X11 display name)) with argument '/dev/dri/renderD128'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'h264_qsv'.
Reading option '-i' ... matched as input url with argument 'test_video.mp4'.
Reading option '-c:v' ... matched as option 'c' (codec name) with argument 'hevc_qsv'.
Reading option 'output.mp4' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option qsv_device (set QSV hardware device (DirectX adapter index, DRM path or X11 display name)) with argument /dev/dri/renderD128.
[AVHWDeviceContext @ 0x5590d517ee40] libva: VA-API version 1.17.0
[AVHWDeviceContext @ 0x5590d517ee40] libva: User requested driver 'iHD'
[AVHWDeviceContext @ 0x5590d517ee40] libva: Trying to open /usr/lib/dri/iHD_drv_video.so
[AVHWDeviceContext @ 0x5590d517ee40] libva: Found init function __vaDriverInit_1_16
[AVHWDeviceContext @ 0x5590d517ee40] libva: va_openDriver() returns 0
[AVHWDeviceContext @ 0x5590d517ee40] Initialised VAAPI connection: version 1.17
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x41524742 -> bgra.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x42475241 -> argb.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x41424752 -> rgba.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x52474241 -> abgr.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x58524742 -> bgr0.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x42475258 -> 0rgb.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x58424752 -> rgb0.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x52474258 -> 0bgr.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x30335241 -> unknown.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x30334241 -> unknown.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x30335258 -> x2rgb10le.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x30334258 -> unknown.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x36314752 -> unknown.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x50424752 -> unknown.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x50524742 -> unknown.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x56555941 -> unknown.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x30303859 -> gray.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x3231564e -> nv12.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x3132564e -> unknown.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x32595559 -> yuyv422.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x59565955 -> uyvy422.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x32315659 -> yuv420p.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x30323449 -> yuv420p.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x50313134 -> yuv411p.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x48323234 -> yuv422p.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x56323234 -> yuv440p.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x50343434 -> yuv444p.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x33434d49 -> unknown.
[AVHWDeviceContext @ 0x5590d517ee40] Format 0x30313050 -> p010le.
[AVHWDeviceContext @ 0x5590d517ee40] VAAPI driver: Intel iHD driver for Intel(R) Gen Graphics - 22.6.4 ().
[AVHWDeviceContext @ 0x5590d517ee40] Driver not found in known nonstandard list, using standard behaviour.
[AVHWDeviceContext @ 0x5590d517e940] Use Intel(R) oneVPL to create MFX session, API version is 2.6, the required implementation version is 1.3


  


  CPU starts being used intensively and then nothing happens.
Does anybody have any clue on it?

  


  I have an Intel i7-8550U (8th gen Kaby Lake) processor with 4 cores (8 threads).

  


  I have also used these options to ffmpeg, getting the same results:

  


  ffmpeg -loglevel debug -init_hw_device qsv=hw -filter_hw_device hw -hwaccel qsv -hwaccel_output_format qsv -i test_video.mp4 -vf 'format=qsv,hwupload=extra_hw_frames=64' -c:v hevc_qsv -f mp4 output.mp4


  


  With VAAPI, transcoding runs flowlessly:

  


  ffmpeg -init_hw_device vaapi=intel:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device intel -filter_hw_device intel -i test_video.mp4 -c:v hevc_vaapi -f mp4 output.mp4