Recherche avancée

Sur d’autres sites (7865)

• Understanding Data Processing Agreements and How They Affect GDPR Compliance

  9 octobre 2023, par Erin — GDPR

  The General Data Protection Regulation (GDPR) impacts international organisations that conduct business or handle personal data in the European Union (EU), and they must know how to stay compliant.

  One way of ensuring GDPR compliance is through implementing a data processing agreement (DPA). Most businesses overlook DPAs when considering ways of maintaining user data security. So, what exactly is a DPA’s role in ensuring GDPR compliance ?

  In this article, we’ll discuss DPAs, their advantages, which data protection laws require them and the clauses that make up a DPA. We’ll also discuss the consequences of non-compliance and how you can maintain GDPR compliance using Matomo.

  What is a data processing agreement ?

  A data processing agreement, data protection agreement or data processing addendum is a contractual agreement between a data controller (a company) and a data processor (a third-party service provider.) It defines each party’s rights and obligations regarding data protection.

  A DPA also defines the responsibilities of the controller and the processor and sets out the terms they’ll use for data processing. For instance, when MHP/Team SI sought the services of Matomo (a data processor) to get reliable and compliant web analytics, a DPA helped to outline their responsibilities and liabilities.

  A DPA is one of the basic requirements for GDPR compliance. The GDPR is an EU regulation concerning personal data protection and security. The GDPR is binding on any company that actively collects data from EU residents or citizens, regardless of their location.

  As a business, you need to know what goes into a DPA to identify possible liabilities that may arise if you don’t comply with European data protection laws. For example, having a recurrent security incident can lead to data breaches as you process customer personal data.

  The average data breach cost for 2023 is $4.45 million. This amount includes regulatory fines, containment costs and business losses. As such, a DPA can help you assess the organisational security measures of your data processing methods and define the protocol for reporting a data breach.

  Why is a DPA essential for your business ?

  If your company processes personal data from your customers, such as contact details, you need a DPA to ensure compliance with data security laws like GDPR. You’ll also need a DPA to hire a third party to process your data, e.g., through web analytics or cloud storage.

  But what are the benefits of having a DPA in place ?

  

  A key benefit of signing a DPA is it outlines business terms with a third-party data processor and guarantees compliance with the relevant data privacy laws. A DPA also helps to create an accountability framework between you and your data processor by establishing contractual obligations.

  Additionally, a DPA helps to minimise the risk of unauthorised access to sensitive data. A DPA defines organisational measures that help protect the rights of individuals and safeguard personal data against unauthorised disclosure. Overall, before choosing a data processor, having a DPA ensures that they are capable, compliant and qualified.

  More than 120 countries have already adopted some form of international data protection laws to protect their citizens and their data better. Hence, knowing which laws require a DPA and how you can better ensure compliance is important.

  Which data protection laws require a DPA ?

  Regulatory bodies enact data protection laws to grant consumers greater control over their data and how businesses use it. These laws ensure transparency in data processing and compliance for businesses.

  

  The following are some of the relevant data privacy laws that require you to have a DPA :

  • UK GDPR
  • Brazil LGPD
  • EU GDPR
  • Dubai PDPA
  • Colorado CPA
  • California CCPA/CPRA
  • Virginia VCDPA
  • Connecticut DPA
  • South African POPIA
  • Thailand PDPA

  Companies that don’t adhere to these data protection obligations usually face liabilities such as fines and penalties. With a DPA, you can set clear expectations regarding data processing between you and your customers.

  Review and update any DPAs with third-party processors to ensure compliance with GDPR and the laws we mentioned above. Additionally, confirm that all the relevant clauses are present for compliance with relevant data privacy laws. 

  So, what key data processing clauses should you have in your DPA ? Let’s take a closer look in the next section.

  Key clauses in a data processing agreement

  GDPR provides some general recommendations for what you should state in a DPA.

  

  Here are the elements you should include :

  Data processing specifications

  Your DPA should address the specific business purposes for data processing, the duration of processing and the categories of data under processing. It should also clearly state the party responsible for maintaining GDPR compliance and who the data subjects are, including their location and nationality.

  Your DPA should also address the data processor and controller’s responsibilities concerning data deletion and contract termination.

  Role of processor

  Your DPA should clearly state what your data processor is responsible for and liable for. Some key responsibilities include record keeping, reporting breaches and maintaining data security.

  Other roles of your data processor include providing you with audit opportunities and cooperating with data protection authorities during inquiries. If you decide to end your contract, the data processor is responsible for deleting or returning data, depending on your agreement.

  Role of controller

  Your DPA should inform the responsibilities of the data controller, which typically include issuing processing instructions to the data processor and directing them on how to handle data processing.

  Your DPA should let you define the lawful data processes the data processor should follow and how you’ll uphold the data protection rights of individuals’ sensitive data.

  Organisational and technical specifications

  Your DPA should define specifications such as how third-party processors encrypt, access and test personal data. It should also include specifications on how the data processor and controller will maintain ongoing data security through various factors such as :

  • State of the technology : Do ‌third-party processors have reliable technology, and can they ensure data security within their systems ?
  • Costs of implementation : Does the data controller’s budget allow them to seek third-party services from industry-leading providers who can guarantee a certain level of security ?
  • Variances in users’ personal freedom : Are there privacy policies and opt-out forms for users to express how they want companies to use their sensitive data ?

  Moreover, your DPA should define how you and your data processor will ensure the confidentiality, availability and integrity of data processing services and systems.

  What are the penalties for DPA GDPR non-compliance ?

  Regulators use GDPR’s stiff fines to encourage data controllers and third-party processors to follow‌ best data security practices. One way of maintaining compliance is through drafting up a DPA with your data processor.

  The DPA should clearly outline the necessary legal requirements and include all the relevant clauses mentioned above. Understand what goes into this agreement since data protection authorities can hold your business accountable for a breach — even if a processor’s error caused it.

  Data protection authorities can issue penalties now that the GDPR is in place. For example, according to Article 83 of the GDPR, penalties for data or privacy breaches or non-compliance can amount to up to €20 million or 4% of your annual revenue.

  There are two tiers of fines : tier one and tier two. Violations related to data processors typically attract fines on the tier-one level. Tier one fines can cost your business €10 million or 2% of your company’s global revenue.

  Tier-two fines result from infringement of the right to forget and the right to privacy of your consumer. Tier-two fines can cost your business up to €20 million or 4% of your company’s global revenue.

  GDPR fines make non-compliance an expensive mistake for businesses of all sizes. As such, signing a DPA with any party that acts as a data processor for your business can help you remain GDPR-compliant.

  How a DPA can help your business remain GDPR compliant

  A DPA can help your business define and adhere to lawful data processes.

  

  So, in what other ways can a DPA help you to remain compliant with GDPR ? Let’s take a look !

  1. Assess data processor’s compliance

  Having a DPA helps ensure that the data processor you are working with is GDPR-compliant. You should check if they have a DPA and confirm the processor’s terms of service and legal basis.

  For example, if you want an alternative to Google Analytics that’s GDPR compliant, then you can opt for Matomo. Matomo features a DPA, which you can agree to when you sign up for web analytics services or later.

  2. Establish lawful data processes

  A DPA can also help you review your data processes to ensure they’re GDPR compliant. For example, by defining lawful data processes, you better understand personally identifiable information (PII) and how it relates to data privacy.

  Further, you can allow users to opt out of sharing their data. As such, Matomo can help you to enable Do Not Track preferences on your website.

  

  With this feature, users are given the option to opt in or out of tracking via a toggle in their respective browsers.

  Indeed, establishing lawful data processes helps you define the specific business purposes for collecting and processing personal data. By doing so, you get to notify your users why you need their data and get their consent to process it by including a GDPR-compliant privacy policy on your website.

  3. Anonymise your data

  Global privacy laws like GDPR and ePrivacy mandate companies to display cookie banners or seek consent before tracking visitors’ data. You can either include a cookie consent banner on your site or stop tracking cookies to follow the applicable regulations.

  Further, you can enable cookie-less tracking or easily let users opt out. For example, you can use Matomo without a cookie consent banner, exempting it from many countries’ privacy rules.

  Additionally, through a DPA, you can define organisational measures that define how you’ll anonymise all your users’ data. Matomo can help you anonymise IP addresses, and we recommend that you at least anonymise the last two bytes.

  

  As one of the few web analytics tools you can use to collect data without tracking consent, Matomo also has the French Data Protection Authority (CNIL) approval.

  4. Assess the processor’s bandwidth

  Having a DPA can help you implement data retention policies that show clear retention periods. Such policies are useful when ending a contract with a third-party service provider and determining how they should handle your data.

  A DPA also helps you ensure the processor has the necessary technology to store personal data securely. You can conduct an audit to understand possible vulnerabilities and your data processor’s technological capacity.

  5. Obtain legal counsel

  When drafting a DPA, it’s important to get a consultation on what is needed to ensure complete compliance. Obtaining legal counsel points you in the right direction so you don’t make any mistakes that may lead to non-compliance.

  Conclusion

  Businesses that process users’ data are subject to several DPA contract requirements under GDPR. One of the most important is having DPAs with every third-party provider that helps them perform data processing.

  It’s important to stay updated on GDPR requirements for compliance. As such, Matomo can help you maintain lawful data processes. Matomo gives you complete control over your data and complies with GDPR requirements.

  To get started with Matomo, you can sign up for a 21-day free trial. No credit card required.

  Disclaimer

  We are not lawyers and don’t claim to be. The information provided here is to help give an introduction to GDPR. We encourage every business and website to take data privacy seriously and discuss these issues with your lawyer if you have any concerns.

• ffmpeg giving Error while decoding stream #0:1 : Invalid data found when processing input

  11 octobre 2023, par user1432181

  I am trying to merge two video files into one using ffmpeg on Windows. The process has been proven to work over and over (with over 100 files merged together at some points) - but I have come across an input file that is causing the process to fail with the errors :

  


  _[aac @ 00000142532f74c0] channel element 1.0 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 00000142532f74c0] channel element 1.0 is not allocated
Error while decoding stream #0:1: Invalid data found when processing input
[aac @ 00000142532f74c0] channel element 1.0 is not allocated
.
.
.


  


  There seems to be 3 command line steps to get here, using a concats-inputs.dat file containing :

  


  file E:/..../snippet A.mp4
file E:/..../snippet B.mp4


  


  (Copies of these files can be found at https://filebin.net/77wbowvh7vbklkey/snippet_A.mp4 and https://filebin.net/77wbowvh7vbklkey/snippet_B.mp4)

  


  Step 1 :

  


  > ffmpeg-6.0-full_build/bin/ffmpeg -y -progress ".Default.mp4.progressinfo.dat" -vsync 0 -f concat -safe 0 -i "E:/...../concat-inputs.dat" -c:v copy -c:a copy -crf 0 -b:v 10M "E:/...../video.Default.mp4"


  


  with the output....

  


  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)

  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint

  libavutil      58.  2.100 / 58.  2.100

  libavcodec     60.  3.100 / 60.  3.100

  libavformat    60.  3.100 / 60.  3.100

  libavdevice    60.  1.100 / 60.  1.100

  libavfilter     9.  3.100 /  9.  3.100

  libswscale      7.  1.100 /  7.  1.100

  libswresample   4. 10.100 /  4. 10.100

  libpostproc    57.  1.100 / 57.  1.100

-vsync is deprecated. Use -fps_mode

Passing a number to -vsync is deprecated, use a string argument as described in the manual.

[mov,mp4,m4a,3gp,3g2,mj2 @ 000001bf88ffe240] Auto-inserting h264_mp4toannexb bitstream filter

Input #0, concat, from 'E:/...../concat-inputs.dat':

  Duration: N/A, start: -0.010667, bitrate: 20382 kb/s

  Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 20043 kb/s, 50 fps, 50 tbr, 12800 tbn

    Metadata:

      handler_name    : VideoHandler

      vendor_id       : [0][0][0][0]

      encoder         : Lavc60.3.100 libx264

  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 96000 Hz, 5.1, fltp, 339 kb/s

    Metadata:

      handler_name    : SoundHandler

      vendor_id       : [0][0][0][0]

Output #0, mp4, to 'E:/...../video.Default.mp4':

  Metadata:

    encoder         : Lavf60.3.100

  Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 10000 kb/s, 50 fps, 50 tbr, 12800 tbn

    Metadata:

      handler_name    : VideoHandler

      vendor_id       : [0][0][0][0]

      encoder         : Lavc60.3.100 libx264

  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 96000 Hz, 5.1, fltp, 339 kb/s

    Metadata:

      handler_name    : SoundHandler

      vendor_id       : [0][0][0][0]

Stream mapping:

  Stream #0:0 -> #0:0 (copy)

  Stream #0:1 -> #0:1 (copy)

Press [q] to stop, [?] for help

frame=    0 fps=0.0 q=-1.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=N/A    
_[mov,mp4,m4a,3gp,3g2,mj2 @ 000001bf890653c0] Auto-inserting h264_mp4toannexb bitstream filter

[mp4 @ 000001bf89000580] Non-monotonous DTS in output stream 0:1; previous: 180224, current: 180192; changing to 180225. This may result in incorrect timestamps in the output file.

frame=  210 fps=0.0 q=-1.0 Lsize=   11537kB time=00:00:04.21 bitrate=22433.7kbits/s speed=41.9x

video:11417kB audio:114kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.053312%


  


  Step 2

  


  > ffmpeg-6.0-full_build/bin/ffmpeg -y -progress ".Default.mp4.progressinfo.dat" -vsync 0 -f concat -safe 0 -i "E:/...../concat-inputs.dat" -c:v copy -c:a copy -crf 0 -b:v 10M "E:/...../audio.Default.wav"


  


  which outputs...

  


  ffmpeg version 6.0-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers

  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)

  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint

  libavutil      58.  2.100 / 58.  2.100

  libavcodec     60.  3.100 / 60.  3.100

  libavformat    60.  3.100 / 60.  3.100

  libavdevice    60.  1.100 / 60.  1.100

  libavfilter     9.  3.100 /  9.  3.100

  libswscale      7.  1.100 /  7.  1.100

  libswresample   4. 10.100 /  4. 10.100

  libpostproc    57.  1.100 / 57.  1.100

-vsync is deprecated. Use -fps_mode

Passing a number to -vsync is deprecated, use a string argument as described in the manual.

[mov,mp4,m4a,3gp,3g2,mj2 @ 00000246d314e240] Auto-inserting h264_mp4toannexb bitstream filter

Input #0, concat, from 'E:/...../concat-inputs.dat':

  Duration: N/A, start: -0.010667, bitrate: 20382 kb/s

  Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 20043 kb/s, 50 fps, 50 tbr, 12800 tbn

    Metadata:

      handler_name    : VideoHandler

      vendor_id       : [0][0][0][0]

      encoder         : Lavc60.3.100 libx264

  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 96000 Hz, 5.1, fltp, 339 kb/s

    Metadata:

      handler_name    : SoundHandler

      vendor_id       : [0][0][0][0]

[out#0/wav @ 00000246d31bd240] Codec AVOption b (set bitrate (in bits/s)) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.

Output #0, wav, to 'E:/...../audio.Default.wav':

  Metadata:

    ISFT            : Lavf60.3.100

  Stream #0:0(und): Audio: aac (LC) ([255][0][0][0] / 0x00FF), 96000 Hz, 5.1, fltp, 339 kb/s

    Metadata:

      handler_name    : SoundHandler

      vendor_id       : [0][0][0][0]

Stream mapping:

  Stream #0:1 -> #0:0 (copy)

Press [q] to stop, [?] for help

size=       0kB time=00:00:00.00 bitrate=N/A speed=N/A    
_[mov,mp4,m4a,3gp,3g2,mj2 @ 00000246d3b009c0] Auto-inserting h264_mp4toannexb bitstream filter

[wav @ 00000246d3150580] Non-monotonous DTS in output stream 0:0; previous: 180224, current: 180192; changing to 180224. This may result in incorrect timestamps in the output file.

size=     114kB time=00:00:04.21 bitrate= 222.4kbits/s speed= 128x

video:0kB audio:114kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.102561%


  


  Step 3

  


  > ffmpeg-6.0-full_build/bin/ffmpeg -y -progress ".Default.mp4.progressinfo.dat" -i "E:/...../video.Default.mp4" -i "E:/...../audio.Default.wav" -crf 0 -c:v copy -c:a aac "E:/...../Default.mp4"


  


  ... which then gives the errors....

  


  ffmpeg version 6.0-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers

  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)

  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint

  libavutil      58.  2.100 / 58.  2.100

  libavcodec     60.  3.100 / 60.  3.100

  libavformat    60.  3.100 / 60.  3.100

  libavdevice    60.  1.100 / 60.  1.100

  libavfilter     9.  3.100 /  9.  3.100

  libswscale      7.  1.100 /  7.  1.100

  libswresample   4. 10.100 /  4. 10.100

  libpostproc    57.  1.100 / 57.  1.100

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'E:/...../video.Default.mp4':

  Metadata:

    major_brand     : isom

    minor_version   : 512

    compatible_brands: isomiso2avc1mp41

    encoder         : Lavf60.3.100

  Duration: 00:00:04.23, start: 0.000000, bitrate: 22359 kb/s

  Stream #0:0[0x1](und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 22178 kb/s, 49.80 fps, 50 tbr, 12800 tbn (default)

    Metadata:

      handler_name    : VideoHandler

      vendor_id       : [0][0][0][0]

      encoder         : Lavc60.3.100 libx264

  Stream #0:1[0x2](und): Audio: aac (LC) (mp4a / 0x6134706D), 96000 Hz, 5.1, fltp, 221 kb/s (default)

    Metadata:

      handler_name    : SoundHandler

      vendor_id       : [0][0][0][0]

[aac @ 000001425315e580] Multiple frames in a packet.

Input #1, wav, from 'E:/...../audio.Default.wav':

  Metadata:

    encoder         : Lavf60.3.100

  Duration: 00:00:04.22, bitrate: 221 kb/s

  Stream #1:0: Audio: aac (LC) ([255][0][0][0] / 0x00FF), 96000 Hz, 5.1, fltp, 339 kb/s

Stream mapping:

  Stream #0:0 -> #0:0 (copy)

  Stream #0:1 -> #0:1 (aac (native) -> aac (native))

Press [q] to stop, [?] for help

Output #0, mp4, to 'E:/...../Default.mp4':

  Metadata:

    major_brand     : isom

    minor_version   : 512

    compatible_brands: isomiso2avc1mp41

    encoder         : Lavf60.3.100

  Stream #0:0(und): Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 22178 kb/s, 49.80 fps, 50 tbr, 12800 tbn (default)

    Metadata:

      handler_name    : VideoHandler

      vendor_id       : [0][0][0][0]

      encoder         : Lavc60.3.100 libx264

  Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 96000 Hz, 5.1, fltp, 341 kb/s (default)

    Metadata:

      handler_name    : SoundHandler

      vendor_id       : [0][0][0][0]

      encoder         : Lavc60.3.100 aac

frame=    0 fps=0.0 q=-1.0 size=       0kB time=-577014:32:22.77 bitrate=  -0.0kbits/s speed=N/A    
_[aac @ 00000142532f74c0] channel element 1.0 is not allocated

Error while decoding stream #0:1: Invalid data found when processing input

[aac @ 00000142532f74c0] channel element 1.0 is not allocated
.
.
.


  


  If I was to do this to merge snippet B with snippet B then it would work - it's something about snippet A that is causing the problem.

  


  Is there any way to get around this... what is it about snippet A that is causing a problem... and is there a way to "normalize" it so that it can be merged as part of the "set".

  


  Note, I just upgraded to ffmpeg6 after a previous version was giving the same problems - so I will also work on the deprecated messages when I can.

  


• FFMPEG changes pixel values when reading and saving png without modification

  25 janvier 2023, par walrus

  This is a toy problem that is the result of my trying to identify a bug within a video pipeline I'm working on. The idea is that I want to take a frame from a YUV420 video, modify it as an RGB24 image, and reinsert it. To do this I convert YUV420 -> YUV444 -> RGB -> YUV444 -> YUV420. Doing this without any modification should result in the same frame however I noticed slight color transformations.

  


  I tried to isolate the problem using a toy 3x3 RGB32 png image. The function read_and_save_image reads the image and then saves it as new file. It returns the read pixel array. I run this function thrice successively using the output of the previous run as the input of the next. This is to demonstrate a perplexing fact. While passing an image through the function once causes the resulting image to have different pixel values, doing it twice does not change anything. Perhaps more confusing is that the pixel values returned by the function are all the same.

  


  tldr ; How can I load and save the toy image below using ffmpeg as a new file such that the pixel values of the new and original files are identical ?

  


  Here is the original image followed by the result from one and two passes through the function. Note that the pixel value displayed by when reading these images with Preview has changed ever so slightly. This becomes noticeable within a video.

  


  Test image (very small) ->&#xA; <-

  &#xA;

  Here are the pixel values read (note that after being loaded and saved there is a change) :

  &#xA;

  original test image

  &#xA;

  test image after one pass

  &#xA;

  test image after two passes

  &#xA;

  Edit : here is an RGB24 frame extracted from a video I am using to test my pipeline. I had the same issue with pixel values changing after loading and saving with ffmpeg.

  &#xA;

  frame from video I was testing pipeline on

  &#xA;

  Here is a screenshot showing how the image is noticeably darker after ffmpeg. Same pixels on the top right corner of the image.

  &#xA;

  

  &#xA;

  Here is the code of the toy problem :

  &#xA;

  import os
import ffmpeg
import numpy as np


def read_and_save_image(in_file, out_file, width, height, pix_fmt='rgb32'):
    input_data, _ = (
        ffmpeg
        .input(in_file)
        .output('pipe:', format='rawvideo', pix_fmt=pix_fmt)
        .run(capture_stdout=True)
    )
  
    frame = np.frombuffer(input_data, np.uint8)
    print(in_file,'\n', frame.reshape((height,width,-1)))
    
    save_data = (
        ffmpeg
            .input('pipe:', format='rawvideo', pix_fmt=pix_fmt, s='{}x{}'.format(width, height))
            .output(out_file, pix_fmt=pix_fmt)
            .overwrite_output()
            .run_async(pipe_stdin=True)
    )
    
    

    save_data.stdin.write(frame.tobytes())
    save_data.stdin.close()
    #save_data.wait()

    return frame

try:
    test_img = "test_image.png"
    test_img_1 = "test_image_1.png"
    test_img_2 = "test_image_2.png"
    test_img_3 = "test_image_3.png"

    width, height, pix_fmt = 3,3,'rgb32'
    #width, height, pix_fmt = video_stream['width'], video_stream['height'],  'rgb24'
    test_img_pxls = read_and_save_image(test_img,test_img_1, width, height, pix_fmt)
    test_img_1_pxls = read_and_save_image(test_img_1,test_img_2, width, height, pix_fmt)
    test_img_2_pxls = read_and_save_image(test_img_2,test_img_3, width, height, pix_fmt)

    print(np.array_equiv(test_img_pxls, test_img_1_pxls))
    print(np.array_equiv(test_img_1_pxls, test_img_2_pxls))

except ffmpeg.Error as e:
    print('stdout:', e.stdout.decode('utf8'))
    print('stderr:', e.stderr.decode('utf8'))
    raise e


!mediainfo --Output=JSON --Full $test_img
!mediainfo --Output=JSON --Full $test_img_1
!mediainfo --Output=JSON --Full $test_img_2


  &#xA;

  Here is the console output of the program that shows that the pixel arrays read by ffmpeg are the same despite the images being different.

  &#xA;

  test_image.png 
 [[[253 218 249 255]
  [252 213 248 255]
  [251 200 244 255]]

 [[253 227 250 255]
  [249 209 236 255]
  [243 169 206 255]]

 [[253 235 251 255]
  [245 195 211 255]
  [226 103 125 255]]]
test_image_1.png 
 [[[253 218 249 255]
  [252 213 248 255]
  [251 200 244 255]]

 [[253 227 250 255]
  [249 209 236 255]
  [243 169 206 255]]

 [[253 235 251 255]
  [245 195 211 255]
  [226 103 125 255]]]
test_image_2.png 
 [[[253 218 249 255]
  [252 213 248 255]
  [251 200 244 255]]

 [[253 227 250 255]
  [249 209 236 255]
  [243 169 206 255]]

 [[253 235 251 255]
  [245 195 211 255]
  [226 103 125 255]]]
True
True
{
"media": {
"@ref": "test_image.png",
"track": [
{
"@type": "General",
"ImageCount": "1",
"FileExtension": "png",
"Format": "PNG",
"FileSize": "4105",
"StreamSize": "0",
"File_Modified_Date": "UTC 2023-01-19 13:49:00",
"File_Modified_Date_Local": "2023-01-19 13:49:00"
},
{
"@type": "Image",
"Format": "PNG",
"Format_Compression": "LZ77",
"Width": "3",
"Height": "3",
"BitDepth": "32",
"Compression_Mode": "Lossless",
"StreamSize": "4105"
}
]
}
}

{
"media": {
"@ref": "test_image_1.png",
"track": [
{
"@type": "General",
"ImageCount": "1",
"FileExtension": "png",
"Format": "PNG",
"FileSize": "128",
"StreamSize": "0",
"File_Modified_Date": "UTC 2023-01-24 15:31:58",
"File_Modified_Date_Local": "2023-01-24 15:31:58"
},
{
"@type": "Image",
"Format": "PNG",
"Format_Compression": "LZ77",
"Width": "3",
"Height": "3",
"BitDepth": "32",
"Compression_Mode": "Lossless",
"StreamSize": "128"
}
]
}
}

{
"media": {
"@ref": "test_image_2.png",
"track": [
{
"@type": "General",
"ImageCount": "1",
"FileExtension": "png",
"Format": "PNG",
"FileSize": "128",
"StreamSize": "0",
"File_Modified_Date": "UTC 2023-01-24 15:31:59",
"File_Modified_Date_Local": "2023-01-24 15:31:59"
},
{
"@type": "Image",
"Format": "PNG",
"Format_Compression": "LZ77",
"Width": "3",
"Height": "3",
"BitDepth": "32",
"Compression_Mode": "Lossless",
"StreamSize": "128"
}
]
}
}



  &#xA;