Recherche avancée

Sur d’autres sites (9966)

• Reduce FFmpeg CPU Usage with cpulimit and PHP

  31 mars 2017, par Matt McManis

  I run FFmpeg on my server using PHP exec().

  It currently uses 100% of the cpu thread while encoding.

  I followed this gude that uses a program called cpulimit to reduce it to 30%.

  ---

  PHP

  $args = "nice -19 cpulimit -l 30 -- ffmpeg -y -i intput.avi -vcodec libx264 -acodec aac -b:a 192k -threads 1 output.mp4"
  
  exec(escapeshellcmd($args));

  FFmpeg/PHP works, and it will work with nice/cpulimit through the terminal, but after adding
  nice -19 cpulimit -l 30 -- to the PHP script it no longer works with exec().

  ---

  Output

  FFmpeg output returns blank. I’m not able to see the full output, I tried using :

  $output = shell_exec($args);
  
  echo "<p>$output</p>"

  and

  file_put_contents("/var/www/mysite/logs/$output.log", $line . PHP_EOL, FILE_APPEND);

  But they return 1 empty line.

  ---

  Solution

  My thought is that www-data runs FFmpeg and nice/cpulimit may need root ?

  How can I get PHP exec() to work with FFmpeg args and cpulimit ?

  Or is there an alternative way to limit the usage % ?

• GDPR Compliance and Personal Data : The Ultimate Guide

  22 septembre 2023, par Erin — GDPR

  According to the International Data Corporation (IDC), the world generated 109 zettabytes of data in 2022 alone, and that number is on track to nearly triple to 291 zettabytes in 2027. For scale, that’s one trillion gigs or one followed by 21 zeros in bytes.

  A major portion of that data is generated online, and the conditions for securing that digital data can have major real-world consequences. For example, online identifiers that fall into the wrong hands can be used nefariously for cybercrime, identity theft or unwanted targeting. Users also want control over how their actions are tracked online and transparency into how their information is used.

  Therefore, regional and international regulations are necessary to set the terms for respecting users’ privacy and control over personal information. Perhaps the most widely known of these laws is the European Union’s General Data Protection Regulation (GDPR).

  What is personal data under GDPR ?

  Under the General Data Protection Regulation (GDPR), “personal data” refers to information linked to an identifiable natural person. An “identifiable natural person” is someone directly or indirectly recognisable via individually specific descriptors such as physical, genetic, economic, cultural, employment and social details.

  It’s important to note that under GDPR, the definition of personal data is very broad, and it encompasses both information that is commonly considered personal (e.g., names and addresses) and more technical or specialised data (e.g., IP addresses or device IDs) that can be used to identify individuals indirectly.

  Organisations that handle personal data must adhere to strict rules and principles regarding the processing and protection of this data to ensure individuals’ privacy rights are respected and upheld.

  Personal data can include, but is not limited to, the following :

  1. Basic Identity Information : This includes a person’s name, government-issued ID number, social address, phone number, email address or other similar identifiers.
  2. Biographical Information : Details such as date of birth, place of birth, nationality and gender.
  3. Contact Information : Information that allows communication with the individual, such as phone numbers, email addresses or mailing addresses.
  4. Financial Information : Data related to a person’s finances, including credit card numbers, bank account numbers, income records or financial transactions.
  5. Health and Medical Information : Information about a person’s health, medical history or healthcare treatments.
  6. Location Data : Data that can pinpoint a person’s geographical location, such as GPS coordinates or information derived from mobile devices.
  7. Online Identifiers : Information like IP addresses, cookies or other online tracking mechanisms that can be used to identify or track individuals online.
  8. Biometric Data : Unique physical or behavioural characteristics used for identification, such as fingerprints, facial recognition data or voiceprints.

  Sensitive Data

  Sensitive data is a special category of personal data prohibited from processing unless specific conditions are met, including users giving explicit consent. The data must also be necessary to fulfil one or more of a limited set of allowed purposes, such as reasons related to employment, social protections or legal claims.

  Sensitive information includes details about a person’s racial or ethnic origin, sexual orientation, political opinions, religion, trade union membership, biometric data or genetic data.

  What are the 7 main principles of GDPR ?

  The 7 principles of GDPR guide companies in how to properly handle personal data gathered from their users.

  

  The seven principles of GDPR are :

  1. Lawfulness, fairness and transparency

  Lawfulness means having legal grounds for data processing, such as consent, legitimate interests, contract and legal obligation. If you can achieve your objective without processing personal data, the basis is no longer lawful.

  Fairness means you’re processing data reasonably and in line with users’ best interests, and they wouldn’t be shocked if they find out what you’re using it for.

  Transparency means being open regarding when you’re processing user data, what you’re using it for and who you’re collecting it from.

  To get started with this, use our guide on creating a GDPR-compliant privacy policy.

  2. Purpose limitation

  You should only process user data for the original purposes you communicated to users when requesting their explicit consent. If you aim to undertake a new purpose, it must be compatible with the original stated purpose. Otherwise, you’ll need to ask for consent again.

  3. Data minimisation

  You should only collect as much data as you need to accomplish compliant objectives and nothing more, especially not other personally identifiable information (PII).

  Matomo provides several features for extensive data minimisation, including the ability to anonymize IP addresses.

  Data minimisation is well-liked by users. Around 70% of people have taken active steps towards protecting their identity online, so they’ll likely appreciate any principles that help them in this effort.

  

  4. Accuracy

  The user data you process should be accurate and up-to-date where necessary. You should have reasonable systems to catch inaccurate data and correct or delete it. If there are mistakes that you need to store, then you need to label them clearly as mistakes to keep them from being processed as accurate.

  5. Storage limitation

  This principle requires you to eliminate data you’re no longer using for the original purposes. You must implement time limits, after which you’ll delete or anonymize any user data on record. Matomo allows you to configure your system such that logs are automatically deleted after some time.

  6. Integrity and confidentiality

  This requires that data processors have security measures in place to protect data from threats such as hackers, loss and damage. As an open-source web analytics solution, Matomo enables you to verify its security first-hand.

  7. Accountability

  Accountability means you’re responsible for what you do with the data you collect. It’s your duty to maintain compliance and document everything for audits. Matomo tracks a lot of the data you’d need for this, including activity, task and application logs.

  Who does GDPR apply to ?

  The GDPR applies to any company that processes the personal data of EU citizens and residents (regardless of the location of the company). 

  If this is the first time you’ve heard about this, don’t worry ! Matomo provides tools that allow you to determine exactly what kinds of data you’re collecting and how they must be handled for full compliance. 

  Best practices for processing personal data under GDPR

  Companies subject to the GDPR need to be aware of several key principles and best practices to ensure they process personal data in a lawful and responsible manner.

  Here are some essential practices to implement :

  1. Lawful basis for processing : Organisations must have a lawful basis for processing personal data. Common lawful bases include the necessity of processing for compliance with a legal obligation, the performance of a contract, the protection of vital interests and tasks carried out in the public interest. Your organisation’s legitimate interests for processing must not override the individual’s legal rights. 
  2. Data minimisation : Collect and process only the personal data that is necessary for the specific purpose for which it was collected. Matomo’s anonymisation capabilities help you avoid collecting excessive or irrelevant data.
  3. Transparency : Provide clear and concise information to individuals about how their data will be processed. Privacy statements should be clear and accessible to users to allow them to easily understand how their data is used.
  4. Consent : If you are relying on consent as a lawful basis, make sure you design your privacy statements and consent forms to be usable. This lets you ensure that consent is freely given, specific, informed and unambiguous. Also, individuals must be able to withdraw their consent at any time.
  5. Data subject rights : You must have mechanisms in place to uphold the data subject’s individual rights, such as the rights to access, erase, rectify errors and restrict processing. Establish internal processes for handling such requests.
  6. Data protection impact assessments (DPIAs) : Conduct DPIAs for high-risk processing activities, especially when introducing new technologies or processing sensitive data.
  7. Security measures : You must implement appropriate technical security measures to maintain the safety of personal data. This can include ‌security tools such as encryption, firewalls and limited access controls, as well as organisational practices like regular security assessments. 
  8. Data breach response : Develop and maintain a data breach response plan. Notify relevant authorities and affected individuals of data breaches within the required timeframe.
  9. International data transfers : If transferring personal data outside the EU, ensure that appropriate safeguards are in place and consider GDPR provisions. These provisions allow data transfers from the EU to non-EU countries in three main ways :
     1. When the destination country has been deemed by the European Commission to have adequate data protection, making it similar to transferring data within the EU.
     2. Through the use of safeguards like binding corporate rules, approved contractual clauses or adherence to codes of conduct.
     3. In specific situations when none of the above apply, such as when an individual explicitly consents to the transfer after being informed of the associated risks.
  10. Data protection officers (DPOs) : Appoint a data protection officer if required by GDPR. DPOs are responsible for overseeing data protection compliance within the organisation.
  11. Privacy by design and default : Integrate data protection into the design of systems and processes. Default settings should prioritise user privacy, as is the case with something like Matomo’s first-party cookies.
  12. Documentation : Maintain records of data processing activities, including data protection policies, procedures and agreements. Matomo logs and backs up web server access, activity and more, providing a solid audit trail.
  13. Employee training : Employees who handle personal data must be properly trained to uphold data protection principles and GDPR compliance best practices. 
  14. Third-party contracts : If sharing data with third parties, have data processing agreements in place that outline the responsibilities and obligations of each party regarding data protection.
  15. Regular audits and assessments : Conduct periodic audits and assessments of data processing activities to ensure ongoing compliance. As mentioned previously, Matomo tracks and saves several key statistics and metrics that you’d need for a successful audit.
  16. Accountability : Demonstrate accountability by documenting and regularly reviewing compliance efforts. Be prepared to provide evidence of compliance to data protection authorities.
  17. Data protection impact on data analytics and marketing : Understand how GDPR impacts data analytics and marketing activities, including obtaining valid consent for marketing communications.

  Organisations should be on the lookout for GDPR updates, as the regulations may evolve over time. When in doubt, consult legal and privacy professionals to ensure compliance, as non-compliance could potentially result in significant fines, damage to reputation and legal consequences.

  What constitutes a GDPR breach ?

  Security incidents that compromise the confidentiality, integrity and/or availability of personal data are considered a breach under GDPR. This means a breach is not limited to leaks ; if you accidentally lose or delete personal data, its availability is compromised, which is technically considered a breach.

  What are the penalty fines for GDPR non-compliance ?

  The penalty fines for GDPR non-compliance are up to €20 million or up to 4% of the company’s revenue from the previous fiscal year, whichever is higher. This makes it so that small companies can also get fined, no matter how low-profile the breach is.

  In 2022, for instance, a company found to have mishandled user data was fined €2,000, and the webmaster responsible was personally fined €150.

  Is Matomo GDPR compliant ?

  Matomo is fully GDPR compliant and can ensure you achieve compliance, too. Here’s how :

  • Data anonymization and IP anonymization
  • GDPR Manager that helps you identify gaps in your compliance and address them effectively
  • Users can opt-out of all tracking
  • First-party cookies by default
  • Users can view the data collected
  • Capabilities to delete visitor data when requested
  • You own your data and it is not used for any other purposes (like advertising)
  • Visitor logs and profiles can be disabled
  • Data is stored in the EU (Matomo Cloud) or in any country of your choice (Matomo On-Premise)

  Is there a GDPR in the US ?

  There is no GDPR-equivalent law that covers the US as a whole. That said, US-based companies processing data from persons in the EU still need to adhere to GDPR principles.

  While there isn’t a federal data protection law, several states have enacted their own. One notable example is the California Consumer Privacy Act (CCPA), which Matomo is fully compliant with.

  Ready for GDPR-compliant analytics ?

  The GDPR lays out a set of regulations and penalties that govern the collection and processing of personal data from EU citizens and residents. A breach under GDPR attracts a fine of either up to €20 million or 4% of the company’s revenue, and the penalty applies to companies of all sizes.

  Matomo is fully GDPR compliant and provides several features and advanced privacy settings to ensure you ‌are as well, without sacrificing the resources you need for effective analytics. If you’re ready to get started, sign up for a 21-day free trial of Matomo — no credit card required.

  Disclaimer
  We are not lawyers and don’t claim to be. The information provided here is to help give an introduction to GDPR. We encourage every business and website to take data privacy seriously and discuss these issues with your lawyer if you have any concerns.

• Reduce latency in ffmpeg snapshot

  3 décembre 2015, par Acorian0

  I have a latency problem with FFMPEG.

  I have a streaming server running and I occasionally need to take multiple snapshots over a period of time (in the example below, 5s), and then examine the snapshots taken. Time is critical.

  With this command I take 25 frames per second over 5 seconds, meaning I will have 125 snapshots in my folder.

  ffmpeg.exe -ss 0.05 -re -i udp://239.255.0.xx:xxxx -ss 0 -vf fps=25 -to 5 -y \Test\%5d.jpg

  The -vf fps is forcing the 25 frames per second even if the server can’t provide them.

  The problem is that -ss 0.05 is not doing its job. It is supposed to delay the initial snapshot for 50 milliseconds but FFMPEG takes around 200ms to "start" the service on itself :/. This means the first snapshot is taken after around 200ms after I called the service. (200ms is way too big of a latency for my purpose... I can survive with 100ms)

  How can I force FFMPEG to start taking snapshots earlier ? Or is there a way to get snapshots, let’s say, from the buffer (e.g. start taking snapshots 150ms in the past) ?

  PS : Changing the -ss from 0.05 to 0.00 is not doing anything, I can only see the -ss doing something if the value is bigger than 0.2/0.25. Another thing, using a newer version of FFMPEG is by now impossible

  Sample OUTPUT :

  ffmpeg version 2.4.5 Copyright (c) 2000-2014 the FFmpeg developers
  
    built on Dec 30 2014 14:53:50 with gcc 4.9.2 (GCC)
  
    configuration: --disable-static --enable-shared --enable-gpl --enable-version3 --disable-w32thread
  
  s --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-icon
  
  v --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable
  
  -libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencor
  
  e-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-l
  
  ibschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-li
  
  bvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-l
  
  ibwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --ena
  
  ble-lzma --enable-decklink --enable-zlib
  
    libavutil      54.  7.100 / 54.  7.100
  
    libavcodec     56.  1.100 / 56.  1.100
  
    libavformat    56.  4.101 / 56.  4.101
  
    libavdevice    56.  0.100 / 56.  0.100
  
    libavfilter     5.  1.100 /  5.  1.100
  
    libswscale      3.  0.100 /  3.  0.100
  
    libswresample   1.  1.100 /  1.  1.100
  
    libpostproc    53.  0.100 / 53.  0.100
  
  [mpeg2video @ 01d77120] Invalid frame dimensions 0x0.
  
  Input #0, mpegts, from 'udp://239.255.0.14:5014':
  
    Duration: N/A, start: 159.051978, bitrate: 105241 kb/s
  
    Program 1
  
      Metadata:
  
        service_name    : Service01
  
        service_provider: FFmpeg
  
      Stream #0:0[0x100]: Video: mpeg1video ([2][0][0][0] / 0x0002), yuv420p(tv), 1920x1080 [SAR 1:1 D
  
  AR 16:9], 104857 kb/s, 50 tbr, 90k tbn, 50 tbc
  
      Stream #0:1[0x101]: Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 384 kb/s
  
  [swscaler @ 01d70060] deprecated pixel format used, make sure you did set range correctly
  
  Output #0, image2, to 'C:\Test\%5d.jpg':
  
    Metadata:
  
      encoder         : Lavf56.4.101
  
      Stream #0:0: Video: mjpeg, yuvj420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25
  
  tbn, 25 tbc
  
      Metadata:
  
        encoder         : Lavc56.1.100 mjpeg
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mpeg1video (native) -> mjpeg (native))
  
  Press [q] to stop, [?] for help
  
  frame=  125 fps= 25 q=24.8 Lsize=N/A time=00:00:05.00 bitrate=N/A dup=1 drop=0
  
  video:9710kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown