Recherche avancée

Sur d’autres sites (9398)

• The 7 GDPR Principles : A Guide to Compliance

  11 août 2023, par Erin — Analytics Tips, GDPR

  We all knew it was coming. It’s all anyone could talk about — the General Data Protection Regulation (GDPR) took effect on 25 May 2018. 

  You might think five years would have been plenty of time for organisations to achieve compliance, yet many have failed to do so. As of 2022, 81% of French businesses and 95% of American companies were still not compliant.

  If you’re one of these organisations still working on compliance, this blog will provide valuable information about the seven GDPR principles and guide you on your way to compliance. It will also explore how web analytics tools can help organisations improve transparency, ensure data security and achieve GDPR compliance.

  What is GDPR ?

  The European Union (EU) created the General Data Protection Regulation (GDPR) to grant individuals greater control over their data and promote transparency in data processing. 

  Known by many other names across Europe (e.g., RGPD, DSGVO, etc.), the GDPR created a set of rules surrounding the handling of personal data of EU citizens and residents, to make sure organisations aren’t being irresponsible with user names, locations, IP addresses, information gleaned from cookies, and so on. 

  Organisations must assume several responsibilities to achieve GDPR compliance, regardless of their physical location. These obligations include :

  • Respecting user rights
  • Implementing documentation and document retention policies
  • Ensuring data security 

  Why is GDPR compliance important ?

  Data has become a valuable asset for businesses worldwide. The collection and use of data is a feature of almost every sector. However, with increased data usage comes a greater responsibility to protect individuals’ privacy and rights. 

  A YouGov study conducted in 17 key markets found that two in three adults worldwide believe tech corporations across all markets have too much control over their data.

  

  GDPR is the most extensive government framework aiming to tackle the increasing concern over data collection and handling. GDPR safeguards personal data from misuse, unauthorised access and data breaches. It ensures that businesses handle information responsibly and with respect for individual privacy. It also provided a foundation for similar laws to be created in other countries, including China, which is among the least concerned regions (56%), along with Sweden (54%) and Indonesia (56%).

  GDPR has been pivotal in safeguarding personal data and empowering individuals with more control over their information. Compliance with GDPR builds trust between businesses and their customers. Currently, 71% of the countries in the world are covered by data protection and privacy legislation.

  What are the risks of non-compliance ?

  We’ve established the siginficance of GDPR, but what about the implications — what does it mean for your business ? The consequences of non-compliance can be severe and are not worth being lax about. 

  According to Article 83 of the GDPR, you can be penalised up to 4% of your annual global revenue or €20 million, whichever is higher, for violations. For smaller businesses, such substantial fines could be devastating. Non-compliance could even result in legal action from individuals or data protection authorities, leading to further financial losses.

  Potential outcomes are not just legal and financial. GDPR violations can significantly damage your reputation as a company. Non-compliance could also cost you business opportunities if your policies and processes do not comply and, therefore, do not align with potential partners. Customers trust businesses that take data protection seriously over those that do not.

  

  Finally, and perhaps the most timid outcome on the surface, individuals have the right to complain to data protection authorities if they believe you violate their data rights. These complaints can trigger an investigation, and if your business is found to be breaking the rules, you could face all of the consequences mentioned above. 

  You may think it couldn’t happen to you, but GDPR fines have collectively reached over €4 billion and are growing at a notable rate. Fines grew 92% from H1 2021 compared with H1 2022. A record-breaking €1.2 billion fine to Meta in 2023 is the biggest we’ve seen, so far. But smaller businesses can be fined, too. A bank in Hungary was fined €1,560 for not erasing and correcting data when the subject requested it. (Individuals can also be fined in flagrant cases, like a police officer fined €1,400 for using police info for private purposes.)

  

  The 7 GDPR principles and how to comply

  You should now have a good understanding of GDPR, why it’s important and the consequences of not being compliant. 

  Your first step to compliance is to identify the personal data your organisation processes and determine the legal basis for processing each type. You then need to review your data processing activities to ensure they align with the GDPR’s purpose and principles.

  There are seven key principles in Article 5 of the GDPR that govern the lawful processing of personal data :

  Lawfulness, fairness and transparency

  This principle ensures you collect and use data in a legal and transparent way. It must be collected with consent, and you must tell your customers why you need their data. Data processing must be conducted fairly and transparently. 

  How to comply

  • Review your data practices and identify if and why you collect personal data from customers.
    • Learn what personally identifiable information (PII) is.
  • Update your website and forms to include a clear and easy-to-understand explanation of why you need their data and what you’ll use it for.
  • Obtain explicit consent from individuals when processing their sensitive data.
  • Add a cookie consent banner to your website, informing users about the cookies you use and why.
    • Website analytics tools like Google Analytics and Matomo offer the ability to create cookie consent banners and integrate with Consent Management Platforms (CMPs) to manage user consent and privacy settings.
    • Matomo also offers a setting without tracking cookies, in which case you would not need to add the cookie consent banner. 
  • Privacy notices must be accessible at all times. 
  • To ensure your cookies are GDPR compliant, you must :
    • Get consent before using any cookies (except strictly necessary cookies). 
    • Clearly explain what each cookie tracks and its purpose.
    • Document and store user consent.
    • Don’t refuse access to services if users do not consent to the use of certain cookies.
    • Make the consent withdrawal process simple. 

  Use tools like Matomo that can be configured to automatically anonymise data so you don’t process any personal data.

  Purpose limitation

  You can only use data for the specific, legitimate purposes you told your visitors, prospects or customers about at the time of collection. You can’t use it for anything else without asking again. 

  How to comply

  • Define the specific purposes for collecting personal data (e.g., processing orders, sending newsletters).
  • Ensure you don’t use the data for any other purposes without getting explicit consent from the individuals.

  Data minimisation

  Data minimisation means you should only collect the data you need, aligned with the stated purpose. You shouldn’t gather or store more data than necessary. Implementing data minimisation practices ensures compliance and protects against data breaches.

  How to comply

  • Identify the minimum data required for each purpose.
  • Conduct a data audit to identify and eliminate unnecessary data collection points.
  • Don’t ask for unnecessary information or store data that’s not essential for your business operations.
  • Implement data retention policies to delete data when it is no longer required.

  Accuracy

  You are responsible for keeping data accurate and up-to-date at all times. You should have processes to promptly erase or correct any data if you have incorrect information for your customers.

  How to comply

  • Implement a process to regularly review and update customer data.
  • Provide an easy way for customers to request corrections to their data if they find any errors.

  Storage limitation

  Data should not be kept longer than necessary. You should only hold onto it for as long as you have a valid reason, which should be the purpose stated and consented to. Securely dispose of data when it is no longer needed. There is no upper time limit on data storage. 

  How to comply

  • Set clear retention periods for the different types of data you collect.
  • Develop data retention policies and adhere to them consistently.
  • Delete data when it’s no longer needed for the purposes you specified.

  Integrity and confidentiality

  You must take measures to protect data from unauthorised or unlawful access, like keeping it locked away and secure.

  How to comply

  • Securely store personal data with encryption and access controls, and keep it either within the EU or somewhere with similar privacy protections. 
  • Train your staff on data protection and restrict access to data only to those who need it for their work.
  • Conduct regular security assessments and address vulnerabilities promptly.

  Accountability

  Accountability means that you are responsible for complying with the other principles. You must demonstrate that you are following the rules and taking data protection seriously.

  How to comply

  • Appoint a Data Protection Officer (DPO) or someone responsible for data privacy in your company.
  • Maintain detailed records of data processing activities and any data breaches.
  • Data breaches must be reported within 72 hours.

  Compliance with GDPR is an ongoing process, and it’s vital to review and update your practices regularly. 

  What are GDPR rights ?

  Individuals are granted various rights under the GDPR. These rights give them more control over their personal data.

  

  The right to be informed : People can ask why their data is required.

  What to do : Explain why personal data is required and how it will be used.

  The right to access : People can request and access the personal data you hold about them.
  What to do : Provide a copy of the data upon request, free of charge and within one month.

  The right to rectification : If data errors or inaccuracies are found, your customers can ask you to correct them.
  What to do : Promptly update any incorrect information to ensure it is accurate and up-to-date.

  The right to object to processing : Your customers have the right to object to processing their data for certain purposes, like direct marketing.
  What to do : Respect this objection unless you have legitimate reasons for processing the data.

  Rights in relation to automated decision-making and profiling : GDPR gives individuals the right not to be subject to decisions based solely on automated processing, including profiling, if it significantly impacts them.
  What to do : Offer individuals the right to human intervention and express their point of view in such cases.

  The right to be forgotten : Individuals can request the deletion of their data under certain circumstances, such as when the data is no longer necessary or when they withdraw consent.
  What to do : Comply with such requests unless you have a legal obligation to keep the data.

  The right to data portability : People can request their personal data in a commonly used and machine-readable format.
  What to do : Provide the data to the individual if they want to transfer it to another service provider.

  The right to restrict processing : Customers can ask you to temporarily stop processing their data, for example, while they verify its accuracy or when they object to its usage.
  What to do : Store the data during this period but do not process it further.

  Are all website analytics tools GDPR compliant ?

  Unfortunately, not all web analytics tools are built the same. No matter where you are located in the world, if you are processing the personal data of European citizens or residents, you need to fulfil GDPR obligations.

  While your web analytics tool helps you gain valuable insights from your user base and web traffic, they don’t all comply with GDPR. No matter how hard you work to adhere to the seven principles and GDPR rights, using a non-compliant tool means that you’ll never be fully GDPR compliant.

  When using website analytics tools and handling data, you should consider the following :

  Collection of data

  Aligned with the lawfulness, fairness and transparency principle, you must collect consent from visitors for tracking if you are using website analytics tools to collect visitor behavioural data — unless you anonymise data entirely with Matomo.

  

  To provide transparency, you should also clarify the types of data you collect, such as IP addresses, device information and browsing behaviour. Note that data collection aims to improve your website’s performance and understand your audience better.

  Storage of data

  Assure your visitors that you securely store their data and only keep it for as long as necessary, following GDPR’s storage limitation principle. Clearly state the retention periods for different data types and specify when you’ll delete or anonymise it.

  Usage of data

  Make it clear that to comply with the purpose limitation principle, the data you collect will not be used for other purposes beyond website analytics. You should also promise not to share data with third parties for marketing or unrelated activities without their explicit consent. 

  Anonymisation and pseudonymisation

  Features like IP anonymisation to protect users’ privacy are available with GA4 (Google Analytics) and Matomo. Describe how you use these tools and mention that you may use pseudonyms or unique identifiers instead of real names to safeguard personal data further.

  Cookies and consent

  Inform visitors that your website uses cookies and other tracking technologies for analytics purposes. Matomo offers customisable cookie banners and opt-out options that allow users to choose their preferences regarding cookies and tracking, along with cookieless options that don’t require consent banners. 

  Right to access and correct data

  Inform visitors of their rights and provide instructions on requesting information. Describe how to correct inaccuracies in their data and update their preferences.

  Security measures

  Assure visitors that you take data security seriously and have implemented measures to protect their data from unauthorised access or breaches. You can also use this opportunity to highlight any encryption or access controls you use to safeguard data.

  Contact information

  Provide contact details for your company’s Data Protection Officer (DPO) and encourage users to reach out if they have any questions or concerns about their data and privacy.

  When selecting web analytics tools, consider how well they align with GDPR principles. Look for features like anonymisation, consent management options, data retention controls, security measures and data storage within the EU or a similarly privacy-protecting jurisdiction. 

  Matomo offers an advanced GDPR Manager. This is to make sure websites are fully GDPR compliant by giving users the ability to access, withdraw consent, object or erase their data, in addition to the anonymizing features. 

  And finally, when you use Matomo, you have 100% data ownership — stored with us in the EU if you’re using Matomo Cloud or on your own servers with Matomo On-Premise — so you can be data-driven and still be compliant with worldwide privacy laws. We are also trusted across industries as we provide accurate data (no trying to fill in the gaps with AI), a robust API that lets you connect your data to your other tools and cookieless tracking options so you don’t need a cookie consent banner. What’s more, our open-source nature allows you to explore the inner workings, offering the assurance of security firsthand. 

  Ready to become GDPR compliant ?

  Whether you’re an established business or just starting out, if you work with data from EU citizens or residents, then achieving GDPR compliance is essential. It doesn’t need to cost you a fortune or five years to get to compliant status. With the right tools and processes, you can be on top of the privacy requirements in no time at all, avoiding any of those hefty penalties or the resulting damage to your reputation. 

  You don’t need to sacrifice powerful data insights to be GDPR compliant. While Google Analytics uses data for its ‘own purposes’, Matomo is an ethical alternative. Using our all-in-one web analytics platform means you own 100% of your data 100% of the time. 

  Start a 21-day free trial of Matomo — no credit card required.

  Disclaimer

  We are not lawyers and don’t claim to be. The information provided here is to help give an introduction to GDPR. We encourage every business and website to take data privacy seriously and discuss these issues with your lawyer if you have any concerns.

• Apostrophe issue with FFmpeg

  1er mars 2024, par Rohan Molinillo

  I'm working on a company's project which is vue.js.
There are also code parts in php.
It uses FFmpeg to create a video from multiple videos.
On each video, there is a text type subtitle.
Each text is retrieved from a .txt file
But I have a problem with apostrophes.

  


  If in the subtitle is stored like this ( I'm here ) in the txt file, on the video there will be written ( Im ).
The apostrophe is removed and the rest of the text too ( here ) will not be displayed.

  


  I'm new to php and ffmpeg, I've been on this problem for almost 3 weeks.

  


  I share the php code with you.

  


  <?php

declare(strict_types=1);

array_shift($argv); // remove script name in $argv[0]

$parameters = array_reduce($argv, function ($carry, $arg) {
    $tokens = explode('=', $arg);
    $carry[$tokens[0]] = $tokens[1];
    return $carry;
}, []);

$projectPath = $parameters['projectPath'];
$musicPath = $parameters['musicPath'];

$fontPath = getcwd() . "/public/fonts/cobol/Cobol-Bold.ttf";
$logoPath = getcwd() . "/public/images/saintex.jpg";
$carnetLogoPath = getcwd() . "/public/images/CarnetTitre.jpg";

// Adding descriptions for each clip and fade in and fade out filters
$clipsToDescribe =  glob("$projectPath/*.webm");
$clipFrameRate = (int) shell_exec("cd $projectPath && ffprobe -v error -select_streams v -of default=noprint_wrappers=1:nokey=1 -show_entries stream=r_frame_rate $clipsToDescribe[0]");
$clipFrameRate = $clipFrameRate > 60 ? 30 : $clipFrameRate;

foreach ($clipsToDescribe as $key => $clipToDescribe) {
    $clipIndex = $key + 1;
    $clipFrames = (int) shell_exec("cd $projectPath && ffprobe -v error -select_streams v:0 -count_packets -show_entries stream=nb_read_packets -of csv=p=0 $clipToDescribe");
    $clipDuration = (float) ($clipFrames / $clipFrameRate) - 0.5;
    file_put_contents("$projectPath/clip{$clipIndex}_desc.txt", addslashes($parameters["clip{$clipIndex}_desc"]));
    shell_exec("cd $projectPath && ffmpeg -i $clipToDescribe -vf 'drawtext=fontfile=$fontPath: textfile=clip{$clipIndex}_desc.txt: fontcolor=white: fontsize=46: box=1: boxcolor=black@0.5: boxborderw=5: x=(w-text_w)/2: y=(h-text_h-50): fix_bounds=true, fade=t=in:st=0:d=0.3,fade=t=out:st=$clipDuration:d=0.3' -b:v 3000K -b:a 192K clip{$clipIndex}.webm");
}
array_map('unlink', glob("$projectPath/*desc.txt"));

shell_exec("cd $projectPath && ffmpeg -t 2 -f lavfi -i color=c=black:s=1280x720 -r 30 blank.webm");
shell_exec("cd $projectPath && ffmpeg -i blank.webm -i $carnetLogoPath -filter_complex '[0:v][1:v] overlay=(main_w/2)-(overlay_w/2):(main_h/2)-(overlay_h/2)' -pix_fmt yuv420p -c:a copy logo.webm");



$workshop = $parameters["workshop_type"];
$title = $parameters["title"];
shell_exec("cd $projectPath && ffmpeg -f lavfi -i color=size=1280x720:duration=3:rate=30:color=black -vf 'drawtext=text=$workshop:fontfile=$fontPath:fontcolor=white:fontsize=46:x=(w-text_w)/2:y=(h-text_h)/2, drawtext=text=$title:fontfile=$fontPath:fontcolor=white:fontsize=46:x=(w-text_w)/2:y=((h-text_h)/2)+lh+5' opening.webm");
unlink("$projectPath/blank.webm");

$videosFile = "file 'logo.webm'\n";
$videosFile .= "file 'opening.webm'\n";{
    file_put_contents("$projectPath/project_desc.txt", $parameters["project_desc"]);
    shell_exec("cd $projectPath && ffmpeg -f lavfi -i color=size=1280x720:duration=3:rate=30:color=black -vf 'drawtext=fontfile=$fontPath:fontsize=46:fontcolor=white:x=(w-text_w)/2:y=(h-text_h)/2:textfile=project_desc.txt' project_desc.webm");
    unlink("$projectPath/project_desc.txt");
    $videosFile .= "file 'project_desc.webm'\n";
} 
if(array_key_exists("participants", $parameters)) {
    file_put_contents("$projectPath/participants.txt", "Participants :\n" . $parameters["participants"]);
    shell_exec("cd $projectPath && ffmpeg -f lavfi -i color=size=1280x720:duration=2:rate=30:color=black -vf 'drawtext=fontfile=$fontPath:fontsize=46:fontcolor=white:x=(w-text_w)/2:y=(h-text_h)/2:textfile=participants.txt' participants.webm");
    unlink("$projectPath/participants.txt");
}

$videos =  glob("$projectPath/clip*.webm");
foreach($videos as $video) {
    if($video != "originalVideos") {
        $videosFile .= "file ". "'{$video}'\n";
    }
} 
if(array_key_exists("participants", $parameters)) {
    $videosFile .= "file '$projectPath/participants.webm'";
}
array_map('unlink', glob("$projectPath/*webm.txt"));
file_put_contents("$projectPath/videosFile.txt", $videosFile);
if($musicPath == "/_musics/") {
    echo(shell_exec("cd $projectPath && ffmpeg -f concat -safe 0 -i videosFile.txt -b:v 10000K -crf 20 -b:a 192K output.webm"));
} else {
    echo(shell_exec("cd $projectPath && ffmpeg -f concat -safe 0 -i videosFile.txt -b:v 10000K -crf 20 -b:a 192K assembled.webm && ffmpeg -i assembled.webm -i ../..$musicPath -filter_complex ' [1:0] apad ' -shortest -y -b:v 3000K -b:a 192K output.webm"));
}


  


  I tried many things but each time there were errors.
I think I didn't implement the code properly.

  


  I share you the error

  


          [09:21:02] RECEIVED EVENT: videoRequest
{ Error: Command failed: php ./public/src/generate.php projectPath='/home/rohan/Documents/dodoc2/_projects/its-a-test' musicPath='/_musics/classic.mp3' clip1_name='' clip2_name='' clip3_name='' clip4_name='' clip5_name='' clip1_desc='It's a first test' clip2_desc='It's a second test' clip3_desc='It's a third test' clip4_desc='It's a fourth' clip5_desc='It's a last test' project_desc='' workshop_type='Atelier Robotique' title='It's a test' participants='Molinillo Rohan
'
PHP Warning:  Undefined array key 1 in /home/rohan/carnet-numerique/public/src/generate.php on line 9
PHP Warning:  Undefined array key 1 in /home/rohan/carnet-numerique/public/src/generate.php on line 9
PHP Warning:  Undefined array key 1 in /home/rohan/carnet-numerique/public/src/generate.php on line 9
PHP Warning:  Undefined array key 1 in /home/rohan/carnet-numerique/public/src/generate.php on line 9
PHP Warning:  Undefined array key 1 in /home/rohan/carnet-numerique/public/src/generate.php on line 9
PHP Warning:  Undefined array key 1 in /home/rohan/carnet-numerique/public/src/generate.php on line 9
ffmpeg version 5.1.1-1ubuntu2.1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12 (Ubuntu 12.2.0-3ubuntu1)
  configuration: --prefix=/usr --extra-version=1ubuntu2.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-shared
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Input #0, matroska,webm, from '/home/rohan/Documents/dodoc2/_projects/its-a-test/video-20230404-091933-682.webm':
  Metadata:
    encoder         : QTmuxingAppLibWebM-0.0.1
  Duration: N/A, start: -0.001000, bitrate: N/A
  Stream #0:0(eng): Video: vp8, yuv420p(progressive), 1280x720, SAR 1:1 DAR 16:9, 1k tbr, 1k tbn (default)
  Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
Stream mapping:
  Stream #0:0 -> #0:0 (vp8 (native) -> vp9 (libvpx-vp9))
  Stream #0:1 -> #0:1 (opus (native) -> opus (libopus))
Press [q] to stop, [?] for help
[libvpx-vp9 @ 0x55952c183000] v1.12.0
Output #0, webm, to 'clip1.webm':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0(eng): Video: vp9, yuv420p(tv, bt470bg/unknown/unknown, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 3000 kb/s, 1k fps, 1k tbn (default)
    Metadata:
      encoder         : Lavc59.37.100 libvpx-vp9
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1(eng): Audio: opus, 48000 Hz, stereo, flt, 192 kb/s (default)
    Metadata:
      encoder         : Lavc59.37.100 libopus
frame=  229 fps= 11 q=12.0 Lsize=    2786kB time=00:00:07.52 bitrate=3034.7kbits/s speed=0.377x    
video:2601kB audio:181kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.175036%
PHP Warning:  Undefined array key "clip2_desc" in /home/rohan/carnet-numerique/public/src/generate.php on line 29
PHP Fatal error:  Uncaught TypeError: addslashes(): Argument #1 ($string) must be of type string, null given in /home/rohan/carnet-numerique/public/src/generate.php:29
Stack trace:
#0 /home/rohan/carnet-numerique/public/src/generate.php(29): addslashes()
#1 {main}
  thrown in /home/rohan/carnet-numerique/public/src/generate.php on line 29

    at ChildProcess.exithandler (child_process.js:275:12)
    at emitTwo (events.js:126:13)
    at ChildProcess.emit (events.js:214:7)
    at maybeClose (internal/child_process.js:925:16)
    at Socket.stream.socket.on (internal/child_process.js:346:11)
    at emitOne (events.js:116:13)
    at Socket.emit (events.js:211:7)
    at Pipe._handle.close [as _onclose] (net.js:554:12)
  killed: false,
  code: 255,
  signal: null,
  cmd: 'php ./public/src/generate.php projectPath=\'/home/rohan/Documents/dodoc2/_projects/its-a-test\' musicPath=\'/_musics/classic.mp3\' clip1_name=\'\' clip2_name=\'\' clip3_name=\'\' clip4_name=\'\' clip5_name=\'\' clip1_desc=\'It\'s a first test\' clip2_desc=\'It\'s a second test\' clip3_desc=\'It\'s a third test\' clip4_desc=\'It\'s a fourth\' clip5_desc=\'It\'s a last test\' project_desc=\'\' workshop_type=\'Atelier Robotique\' title=\'It\'s a test\' participants=\'Molinillo Rohan\n\'' }


  


• Muxing Android MediaCodec encoded H264 packets into RTMP

  31 décembre 2015, par Vadym

  I am coming from a thread Encoding H.264 from camera with Android MediaCodec. My setup is very similar. However, I attempt to write mux the encoded frames and with javacv and broadcast them via rtmp.

  RtmpClient.java

  ...
  
  private volatile BlockingQueue mFrameQueue = new LinkedBlockingQueue(MAXIMUM_VIDEO_FRAME_BACKLOG);
  
  ...
  
  private void startStream() throws FrameRecorder.Exception, IOException {
  
      if (TextUtils.isEmpty(mDestination)) {
  
          throw new IllegalStateException("Cannot start RtmpClient without destination");
  
      }
  
  
  
      if (mCamera == null) {
  
          throw new IllegalStateException("Cannot start RtmpClient without camera.");
  
      }
  
  
  
      Camera.Parameters cameraParams = mCamera.getParameters();
  
  
  
      mRecorder = new FFmpegFrameRecorder(
  
              mDestination,
  
              mVideoQuality.resX,
  
              mVideoQuality.resY,
  
              (mAudioQuality.channelType.equals(AudioQuality.CHANNEL_TYPE_STEREO) ? 2 : 1));
  
  
  
      mRecorder.setFormat("flv");
  
  
  
      mRecorder.setFrameRate(mVideoQuality.frameRate);
  
      mRecorder.setVideoBitrate(mVideoQuality.bitRate);
  
      mRecorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
  
  
  
      mRecorder.setSampleRate(mAudioQuality.samplingRate);
  
      mRecorder.setAudioBitrate(mAudioQuality.bitRate);
  
      mRecorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
  
  
  
      mVideoStream = new VideoStream(mRecorder, mVideoQuality, mFrameQueue, mCamera);
  
      mAudioStream = new AudioStream(mRecorder, mAudioQuality);
  
  
  
      mRecorder.start();
  
  
  
      // Setup a bufferred preview callback
  
      setupCameraCallback(mCamera, mRtmpClient, DEFAULT_PREVIEW_CALLBACK_BUFFERS,
  
              mVideoQuality.resX * mVideoQuality.resY * ImageFormat.getBitsPerPixel(
  
                      cameraParams.getPreviewFormat())/8);
  
  
  
      try {
  
          mVideoStream.start();
  
          mAudioStream.start();
  
      }
  
      catch(Exception e) {
  
          e.printStackTrace();
  
          stopStream();
  
      }
  
  }
  
  ...
  
  @Override
  
  public void onPreviewFrame(byte[] data, Camera camera) {
  
      boolean frameQueued = false;
  
  
  
      if (mRecorder == null || data == null) {
  
          return;
  
      }
  
  
  
      frameQueued = mFrameQueue.offer(data);
  
  
  
      // return the buffer to be reused - done in videostream
  
      //camera.addCallbackBuffer(data);
  
  }
  
  ...

  VideoStream.java

  ...
  
  @Override
  
  public void run() {
  
      try {
  
          mMediaCodec = MediaCodec.createEncoderByType("video/avc");
  
          MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", mVideoQuality.resX, mVideoQuality.resY);
  
          mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, mVideoQuality.bitRate);
  
          mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, mVideoQuality.frameRate);
  
          mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar);
  
          mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);
  
          mMediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
  
          mMediaCodec.start();
  
      }
  
      catch(IOException e) {
  
          e.printStackTrace();
  
      }
  
  
  
      long startTimestamp = System.currentTimeMillis();
  
      long frameTimestamp = 0;
  
      byte[] rawFrame = null;
  
  
  
      try {
  
          while (!Thread.interrupted()) {
  
              rawFrame = mFrameQueue.take();
  
  
  
              frameTimestamp = 1000 * (System.currentTimeMillis() - startTimestamp);
  
  
  
              encodeFrame(rawFrame, frameTimestamp);
  
  
  
              // return the buffer to be reused
  
              mCamera.addCallbackBuffer(rawFrame);
  
          }
  
      }
  
      catch (InterruptedException ignore) {
  
          // ignore interrup while waiting
  
      }
  
  
  
      // Clean up video stream allocations
  
      try {
  
          mMediaCodec.stop();
  
          mMediaCodec.release();
  
          mOutputStream.flush();
  
          mOutputStream.close();
  
      } catch (Exception e){
  
          e.printStackTrace();
  
      }
  
  }
  
  ...
  
  private void encodeFrame(byte[] input, long timestamp) {
  
      try {
  
          ByteBuffer[] inputBuffers = mMediaCodec.getInputBuffers();
  
          ByteBuffer[] outputBuffers = mMediaCodec.getOutputBuffers();
  
  
  
          int inputBufferIndex = mMediaCodec.dequeueInputBuffer(0);
  
  
  
          if (inputBufferIndex >= 0) {
  
              ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
  
              inputBuffer.clear();
  
              inputBuffer.put(input);
  
              mMediaCodec.queueInputBuffer(inputBufferIndex, 0, input.length, timestamp, 0);
  
          }
  
  
  
          MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
  
  
  
          int outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, 0);
  
  
  
          if (outputBufferIndex >= 0) {
  
              while (outputBufferIndex >= 0) {
  
                  ByteBuffer outputBuffer = outputBuffers[outputBufferIndex];
  
  
  
                  // Should this be a direct byte buffer?
  
                  byte[] outData = new byte[bufferInfo.size - bufferInfo.offset];
  
                  outputBuffer.get(outData);
  
  
  
                  mFrameRecorder.record(outData, bufferInfo.offset, outData.length, timestamp);
  
  
  
                  mMediaCodec.releaseOutputBuffer(outputBufferIndex, false);
  
                  outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, 0);
  
              }
  
          }
  
          else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
  
              outputBuffers = mMediaCodec.getOutputBuffers();
  
          } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
  
              // ignore for now
  
          }
  
      } catch (Throwable t) {
  
          t.printStackTrace();
  
      }
  
  
  
  }
  
  ...

  FFmpegFrameRecorder.java

  ...
  
  // Hackish codec copy frame recording function
  
  public boolean record(byte[] encodedData, int offset, int length, long frameCount) throws Exception {
  
      int ret;
  
  
  
      if (encodedData == null) {
  
          return false;
  
      }
  
  
  
      av_init_packet(video_pkt);
  
  
  
      // this is why i wondered whether I should get outputbuffer data into direct byte buffer
  
      video_outbuf.put(encodedData, 0, encodedData.length);
  
  
  
      video_pkt.data(video_outbuf);
  
      video_pkt.size(video_outbuf_size);
  
  
  
      video_pkt.pts(frameCount);
  
      video_pkt.dts(frameCount);
  
  
  
      video_pkt.stream_index(video_st.index());
  
  
  
      synchronized (oc) {
  
          /* write the compressed frame in the media file */
  
          if (interleaved && audio_st != null) {
  
              if ((ret = av_interleaved_write_frame(oc, video_pkt)) < 0) {
  
                  throw new Exception("av_interleaved_write_frame() error " + ret + " while writing interleaved video frame.");
  
              }
  
          } else {
  
              if ((ret = av_write_frame(oc, video_pkt)) < 0) {
  
                  throw new Exception("av_write_frame() error " + ret + " while writing video frame.");
  
              }
  
          }
  
      }
  
      return (video_pkt.flags() & AV_PKT_FLAG_KEY) == 1;
  
  }
  
  ...

  When I try to stream the video and run ffprobe on it, I get the following output :

  ffprobe version 2.5.3 Copyright (c) 2007-2015 the FFmpeg developers
  
    built on Jan 19 2015 12:56:57 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-55)
  
    configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --enable-bzlib --disable-crystalhd --enable-libass --enable-libdc1394 --enable-libfaac --enable-nonfree --disable-indev=jack --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab --enable-avfilter --enable-avresample --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --enable-libcaca --shlibdir=/usr/lib64 --enable-runtime-cpudetect
  
    libavutil      54. 15.100 / 54. 15.100
  
    libavcodec     56. 13.100 / 56. 13.100
  
    libavformat    56. 15.102 / 56. 15.102
  
    libavdevice    56.  3.100 / 56.  3.100
  
    libavfilter     5.  2.103 /  5.  2.103
  
    libavresample   2.  1.  0 /  2.  1.  0
  
    libswscale      3.  1.101 /  3.  1.101
  
    libswresample   1.  1.100 /  1.  1.100
  
    libpostproc    53.  3.100 / 53.  3.100
  
  Metadata:
  
    Server                NGINX RTMP (github.com/arut/nginx-rtmp-module)
  
    width                 320.00
  
    height                240.00
  
    displayWidth          320.00
  
    displayHeight         240.00
  
    duration              0.00
  
    framerate             0.00
  
    fps                   0.00
  
    videodatarate         261.00
  
    videocodecid          7.00
  
    audiodatarate         62.00
  
    audiocodecid          10.00
  
    profile
  
    level
  
  [live_flv @ 0x1edb0820] Could not find codec parameters for stream 0 (Video: none, none, 267 kb/s): unknown codec
  
  Consider increasing the value for the 'analyzeduration' and 'probesize' options
  
  Input #0, live_flv, from 'rtmp://<server>/input/<stream>':
  
    Metadata:
  
      Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
  
      displayWidth    : 320
  
      displayHeight   : 240
  
      fps             : 0
  
      profile         :
  
      level           :
  
    Duration: 00:00:00.00, start: 16.768000, bitrate: N/A
  
      Stream #0:0: Video: none, none, 267 kb/s, 1k tbr, 1k tbn, 1k tbc
  
      Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp, 63 kb/s
  
  Unsupported codec with id 0 for input stream 0
  
  </stream></server>

  I am not, by any means, an expert in H264 or video encoding. I know that the encoded frames that come out from MediaCodec contain SPS NAL, PPS NAL, and frame NAL units. I’ve also written the MediaCodec output into a file and was able to play it back (I did have to specify the format and framerate as otherwise it would play too fast).

  My assumption is that things should work (see how little I know :)). Knowing that SPS and PPS are written out, decoder should know enough. Yet, ffprobe fails to recognize codec, fps, and other video information. Do I need to pass packet flag information to FFmpegFrameRecorder.java:record() function ? Or should I use direct buffer ? Any suggestion will be appreciated ! I should figure things out with a hint.

  PS : I know that some codecs use Planar and other SemiPlanar color formats. That distinction will come later if I get past this. Also, I didn’t go the Surface to MediaCodec way because I need to support API 17 and it requires more changes than this route, which I think helps me understand the more basic flow. Agan, I appreciate any suggestions. Please let me know if something needs to be clarified.

  Update #1

  So having done more testing, I see that my encoder outputs the following frames :

  000000016742800DDA0507E806D0A1350000000168CE06E2
  
  0000000165B840A6F1E7F0EA24000AE73BEB5F51CC7000233A84240...
  
  0000000141E2031364E387FD4F9BB3D67F51CC7000279B9F9CFE811...
  
  0000000141E40304423FFFFF0B7867F89FAFFFFFFFFFFCBE8EF25E6...
  
  0000000141E602899A3512EF8AEAD1379F0650CC3F905131504F839...
  
  ...

  The very first frame contains SPS and PPS. From what I was able to see, these are transmitted only once. The rest are NAL types 1 and 5. So, my assumption is that, for ffprobe to see stream info not only when the stream starts, I should capture SPS and PPS frames and re-transmit them myself periodically, after a certain number of frames, or perhaps before every I-frame. What do you think ?

  Update #2

  Unable to validate that I’m writing frames successfully. After having tried to read back the written packet, I cannot validate written bytes. As strange, on successful write of IPL image and streaming, I also cannot print out bytes of encoded packet after avcodec_encode_video2. Hit the official dead end.