Sur d’autres sites (4415)

• PHP - Read and write the same file hangs

  2 février 2016, par Adracat

  I’m trying to use FFMPEG to make some works with video on the server, and something I need to do is to get the progress of the process.

  I searched a little and I found this solution which tells to write the log into a file and then reading and parsing it.

  The problem

  What is driving me crazy is that I tell FFMPEG - with exec - (process A) to write the log into a file, but when I try to read it - with file_get_contents() - (process B) it does not show the contents until process A is finished (or interrupted the PHP script).

  So, when process A finishes or it says "PHP script timeout", then I can read the file as times as I want, refreshing the page (process B) and showing the contents at the time.

  What I’ve tried

  I’ve tried to use fopen() to create the file with w, w+ and a parameters, using - and without using - fclose(). I’ve tried to use also flock() just in case it gets faster to read to process B if it knows it’s already locked and does not have to wait, but then FFMPEG is not able to write into the file.

  I’ve searched for multithreading too, but I think there must be an easier and simpler way.

  I’ve used also CURL and HTTP context, as this link suggests, but no luck.

  I’ve tried, too, to use PHP-FFMPEG but it’s not supporting the last FFMPEG version, so I cannot use it.

  When I said before "(or interrupted the PHP script)" is because I tried to wait and, when PHP got a timeout, process B worked alright and the file was still updating.

  The code

  Process A (fileA.php)

  exec('ffmpeg -y -i input_file.mp4 output_file.avi 2> C:\Full\Path\To\File\log.txt 1>&2');

  Process B (fileB.php)

  $content = file_get_contents($file);
  
  
  
  if($content){
  
      //get duration of source
  
      preg_match("/Duration: (.*?), start:/", $content, $matches);
  
  
  
      $rawDuration = $matches[1];
  
  
  
      //rawDuration is in 00:00:00.00 format. This converts it to seconds.
  
      $ar = array_reverse(explode(":", $rawDuration));
  
      $duration = floatval($ar[0]);
  
      if (!empty($ar[1])) $duration += intval($ar[1]) * 60;
  
      if (!empty($ar[2])) $duration += intval($ar[2]) * 60 * 60;
  
  
  
      //get the time in the file that is already encoded
  
      preg_match_all("/time=(.*?) bitrate/", $content, $matches);
  
  
  
      $rawTime = array_pop($matches);
  
  
  
      //this is needed if there is more than one match
  
      if (is_array($rawTime)){$rawTime = array_pop($rawTime);}
  
  
  
      //rawTime is in 00:00:00.00 format. This converts it to seconds.
  
      $ar = array_reverse(explode(":", $rawTime));
  
      $time = floatval($ar[0]);
  
      if (!empty($ar[1])) $time += intval($ar[1]) * 60;
  
      if (!empty($ar[2])) $time += intval($ar[2]) * 60 * 60;
  
  
  
      //calculate the progress
  
      $progress = round(($time/$duration) * 100);
  
  
  
      echo "Duration: " . $duration . "<br />";
  
      echo "Current Time: " . $time . "<br />";
  
      echo "Progress: " . $progress . "%";
  
  
  
  }

  The process

  I just open fileA.php on a Chrome tab and, after a few seconds, I open fileB.php on another Chrome tab (and it stays as loading).

  What I need

  I need to be able to load the file and show the information I want to show while the file is being written (by exec and FFMPEG or other PHP scripts), so I can update the progress percentage with some AJAX calls.

  Extra information

  At this point, I’m using PHP 5.4 on a IIS 7.5 with Windows 7 Professional.

  Thank you everyone for your time, help and patience !

  Best regards.

• Increasing CPU usage over time in FFmpeg with Widevine DRM stream

  25 octobre 2024, par nabil fernandez kiker

  Increasing CPU usage over time in FFmpeg with __memset_avx2_unaligned_erms while processing DRM .mpd streams
I'm facing a problem where CPU usage gradually increases when using FFmpeg to process a DRM-protected DASH stream (.mpd with Widevine encryption). Initially, the CPU usage is low, but after around 2 hours, the usage doubles, and it keeps increasing over time.
I used perf to analyze what's happening, and this is what I see after a few hours :

  


  Overhead  Shared Object     Symbol
  25.57%  ffmpeg            [.] __memset_avx2_unaligned_erms
   9.28%  ffmpeg            [.] aes_encrypt
   8.25%  [kernel]          [k] clear_page_rep
   3.17%  [kernel]          [k] asm_exc_page_fault
   1.68%  [kernel]          [k] __handle_mm_fault
   1.38%  ffmpeg            [.] __memmove_avx_unaligned_erms
   1.14%  ffmpeg            [.] _aesni_ctr32_ghash_6x
   1.01%  ffmpeg            [.] malloc_consolidate


  


  Context
I am processing a Widevine DRM-protected DASH stream (.mpd) using FFmpeg to decrypt it and convert it into HLS segments. -cenc_decryption_key
The issue arises after a few hours, and it seems to center around the function __memset_avx2_unaligned_erms, which ends up using about 25% of the CPU after several hours.
The workload is running with FFmpeg built statically with OpenSSL, AVX2 support, and other necessary dependencies.

  


  Questions :
Why is __memset_avx2_unaligned_erms being called more frequently as time progresses, and why does the CPU usage keep rising ?
Could this be related to misaligned memory or an issue with the AVX2 SIMD implementation in FFmpeg or glibc ?
What strategies can I use to prevent this CPU usage increase in long-running DRM-protected stream processing ?
Should I consider disabling SIMD optimizations in FFmpeg for certain parts of the process ? If so, how can I do this without losing performance benefits ?
Any guidance or suggestions would be greatly appreciated, as I'm running out of ideas to resolve this issue.

  


  What I’ve tried :
Perf analysis : I've confirmed the problem seems related to __memset_avx2_unaligned_erms being called increasingly over time.
Memory buffer optimization : I've attempted to reuse memory buffers to avoid excessive memory allocation and initialization, but the issue persists.
Reducing memset calls : I've reduced the size of memory buffers, trying to limit the impact of memset, but CPU usage still increases after extended processing.

  


• We are hiring engineers to build an awesome product and platform used by millions of people

  16 février 2016, par Piwik Core Team — Uncategorized

  Are you ready for a new challenge ? Or maybe you know someone who is looking for a change ? We have some exciting problems to solve and are looking for senior developers to work with us and our community on our open source Piwik Analytics platform. Piwik is used by more than one million websites all over the world. It is deployed on more than 300.000 servers and some users track more than 1 billion actions per month.

  What is it like to work on Piwik ?

  We develop this software using modern PHP, MySQL, Redis, AngularJS and more. We provide several kind of APIs and a plugin architecture to allow developers to extend and change Piwik to their needs. However, we would not be Piwik if we stopped at this point ! We want to turn Piwik into an even more awesome product and platform.
  You can imagine there is a lot to do and many challenges to face !

  While one part is to always make Piwik scale better and to improve UI and UX, we also want to provide simple APIs to make the life of developers as pleasant as possible. We aim to solve things the right way and our thousands of unit, integration, system, JavaScript and screenshot tests help us to innovate and to not be afraid of change. We like clean code and constant improvements.

  The Piwik team lives in New Zealand and Europe (Germany). We do the vast majority of our collaboration online. Our values include being open, transparent and sharing knowledge. For this we use tools like GitHub and Slack to communicate and Quake servers to take our minds off complex challenges. We are a small, flexible team, so when you come aboard, you will play an integral part in engineering and have a big impact on the product loved by so many people. You’ll help to create a welcoming environment for new contributors and set an example with your development practices and communications skills.

  Apply now, or spread the word !

  If you have strong skills in PHP send us an email with your CV and tell us a little about yourself and your experience in engineering complex applications.

  Apply for a job here http://piwik.org/jobs/ and if you’re maybe not the right candidate, contribute to the project by sharing this blog post and by sending it to your friends !