Recherche avancée

Sur d’autres sites (5948)

• What is PII ? Your introduction to personally identifiable information

  15 janvier 2020, par Joselyn Khor — Analytics Tips, Privacy, Security
  Most websites you visit collect information about you via tools like Google Analytics and Matomo – sometimes collecting personally identifiable information (PII).

  When it comes to PII, people are becoming more concerned about data privacy. Identifiable information can be used for illegal purposes like identity theft and fraud. 

  So how can you protect yourself as an innocent internet browser ? In the case of website owners – how do you protect users and your company from falling prey to privacy breaches ?

  

  As one of the most trusted analytics companies, we feel our readers would benefit from being as informed as possible about data privacy issues and PII. Learn what it means, and what you can do to keep yours or others’ information safe.

  Table of Contents

  What does PII stand for ?

  PII acronym

  PII is an acronym for personally identifiable information.

  PII definition

  Personally identifiable information (PII) is a term used predominantly in the United States.

  The appendix of OMB M-10-23 (Guidance for Agency Use of Third-Party Website and Applications) gives this definition for PII :

  “The term ‘personally identifiable information’ refers to information which can be used to distinguish or trace an individual’s identity, such as their name, social security number, biometric records, etc. alone, or when combined with other personal or identifying information which is linked or linkable to a specific individual, such as date and place of birth, mother’s maiden name, etc.”

  What can be considered personally identifiable information (PII) ? Some PII examples :

  • Full name/usernames
  • Home address/mailing address
  • Email address
  • Credit card numbers
  • Date of birth
  • Phone numbers
  • Login details

  • Precise locations
  • Account numbers
  • Passwords
  • Security codes (including biometric records)
  • Personal identification numbers
  • Driver license number
  • Get a more comprehensive list here

  What’s non-PII ?

  Anonymous information, or information that can’t be traced back to an individual, can be considered non-PII.

  Who is affected by the exploitation of PII ?

  Anyone can be affected by the exploitation of personal data, where you have identity theft, account fraud and account takeovers. When websites resort to illegally selling or sharing your data and compromising your privacy, the fear is falling victim to such fraudulent activity. 

  PII can also be an issue when employees have access to the database and the data is not encrypted. For example, anyone working in a bank can access your accounts ; anyone working at Facebook may be able to read your messages. This shows how privacy breaches can easily happen when employees have access to PII.

  Website owner’s responsibility for data privacy (PII and analytics)

  To respect your website visitor’s privacy, best practice is to avoid collecting PII whenever possible. If you work in an industry which requires people to disclose personal information (e.g. healthcare, security industries, public sector), then you must ensure this data is collected and handled securely. 

  

  The US National Institute of Standards and Technology states : “The likelihood of harm caused by a breach involving PII is greatly reduced if an organisation minimises the amount of PII it uses, collects, and stores. For example, an organisation should only request PII in a new form if the PII is absolutely necessary.” 

  How you’re held accountable remains up to the privacy laws of the country you’re doing business in. Make sure you are fully aware of the privacy and data protection laws that relate specifically to you. 

  To reduce the risk of privacy breaches, try collecting as little PII as you can ; purging it as soon as you can ; and making sure your IT security is updated and protected against security threats. 

  If you’re using data collection tools like web analytics, data may be tracked through features like User ID, custom variables, and custom dimensions. Sometimes they are also harder to identify when they are present, for example, in page URLs, page titles, or referrers URLs. So make sure you’re optimising your web analytics tools’ settings to ensure you’re asking your users for consent and respecting users’ privacy.

  If you’re using a GDPR compliant tool like Matomo, learn how you can stop processing such personal data

  PII, GDPR and businesses in the US/EU

  Because PII is broad, you may run into confusion when considering PII and GDPR (which applies in the EU). The General Data Protection Regulation (GDPR) provides more safeguards for user privacy.

  GDPR grants people in the EU more rights concerning their “personal data” (more on PII vs personal data below). In the EU the GDPR restricts the collection and processing of personal data. The repercussions are severe penalties and fines for privacy infringements. Businesses are required to handle this personal data carefully. You can be fined up to 4% of their yearly revenue for data breaches or non-compliance. 

  

  Although there isn’t an overarching data protection law in the US, there are hundreds of laws on both the federal and state levels to protect the personal data of US residents. US Congress has also enacted industry-specific statutes related to data privacy, and the state of California passed the California Consumer Privacy Act. 

  To be on the safe side, if you are using analytics, follow matters relating to “personal data” in the GDPR. It’s all-encompassing when it comes to protecting user privacy. GDPR rules still apply whenever an EU citizen visits any non EU site (that processes personal data).

  Personally identifiable information (PII) vs personal data

  PII and “personal data” aren’t used interchangeably. All personal data can be PII, but not all PII can be defined as personal data.

  The definition of “personal data” according to the GDPR :

  

  This means “personal data” encompasses a greater number of identifiers which include the online sphere. Examples include : IP addresses and URL names. As well as seemingly “innocent” data like height, job position, company etc. 

  What’s considered personal data depends on the context. If a piece of information can be combined with others to establish someone’s identity then that can be considered personal data. 

  Under GDPR, when processing personal data, you need explicit consent. You need to ensure you’re compliant according to GDPR definitions of “personal data” not just what’s considered “PII”.

  How Matomo deals with PII and personal data

  Although Matomo Analytics is a web analytics software that tracks user activity on your website, we take privacy and PII very seriously – on both our Cloud and On-Premise offerings. 

  If you’re using Matomo and would like to know how you can be fully GDPR compliant and protect user privacy, read more :
  

  • Learn how to not process any personally identifiable information – Anonymise IP addresses, user IDs, and order IDs
  • Matomo protects user privacy by talking the talk and walking the walk
  • Stay ahead of the GDPR with a privacy-respecting analytics platform
  • 11 ways Matomo helps you protect your visitor’s privacy

  Disclaimer

  We are not lawyers and don’t claim to be. The information provided here is to help give an introduction to issues you may encounter when dealing with PII. We encourage every business and website to take data privacy seriously and discuss these issues with your lawyer if you have any concerns. 

  Additional resources :

  • https://www.consumeraffairs.com/finance/identity-theft-statistics.html#identity-theft-trends-in-2019
  • https://ec.europa.eu/info/law/law-topic/data-protection_en
  • https://iclg.com/practice-areas/data-protection-laws-and-regulations/usa
  • https://www.csoonline.com/article/3215864/how-to-protect-personally-identifiable-information-pii-under-gdpr.html
  • https://ico.org.uk/for-organisations/guide-to-data-protection/guide-to-the-general-data-protection-regulation-gdpr/key-definitions/what-is-personal-data/

• Transcoding HEVC 4K HDR using ffmpeg and vaapi

  13 mai 2021, par P. Brand

  I am trying to transcode a 4K HEVC HDR video to a lower bitrate using ffmpeg and VAAPI hardware acceleration for my LG OLED television.

  



  I use the following file :
Video

  



  Using the following command :

  



  ffmpeg -init_hw_device vaapi=va:/dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -hwaccel_device va -filter_hw_device va -i LG\ 4K\ HDR\ Demo\ -\ Daylight.mkv -map 0:0 -t 00:00:20 -c:v hevc_vaapi -sei hdr -qp:v 21 lg_vaapi.mkv


  



  But this file is not recognized as HDR on my TV.

  



  When using the libx265 encoder (which is off course much slower) it works fine :

  



  ffmpeg -i LG\ 4K\ HDR\ Demo\ -\ Daylight.mkv -t 00:00:20 -map 0:0 -c:v libx265 -preset medium -crf 18 -x265-params "colorprim=bt2020:colormatrix=bt2020nc:transfer=smpte2084:colormatrix=bt2020nc:hdr=1:info=1:repeat-headers=1:master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(11000000,40)" lg_libx265.mkv


  



  When comparing the output of mediainfo on both files I can see the following properties are not set for the video track :

  



  

  • Color range

  


  • Color primaries

  


  • Transfer characteristics

  


  



  mediainfo for vaapi encode :

  



  General
Unique ID                                : 168011494166392912924249315217763643529 
(0x7E65D636029A5354FE73998A5ED6B089)
Complete name                            : lg_vaapi.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 118 MiB
Duration                                 : 20 s 4 ms
Overall bit rate                         : 49.6 Mb/s
Writing application                      : Lavf58.20.100
Writing library                          : Lavf58.20.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5@Main
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 20 s 4 ms
Bit rate                                 : 48.6 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 (60000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.098
Stream size                              : 116 MiB (98%)
Writing library                          : Lavc58.35.100 hevc_vaapi
Default                                  : Yes
Forced                                   : No
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0040 cd/m2, max: 1100 cd/m2


  



  mediainfo for the libx265 encode :

  



  General
Unique ID                                : 261605749278874425324960669956061374077 (0xC4CF6F42FE952004772A02D5819DA67D)
Complete name                            : lg_libx265.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 155 MiB
Duration                                 : 47 s 715 ms
Overall bit rate                         : 27.3 Mb/s
Writing application                      : Lavf58.20.100
Writing library                          : Lavf58.20.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L5.1@Main
HDR format                               : SMPTE ST 2086, HDR10 compatible
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 47 s 715 ms
Bit rate                                 : 26.8 Mb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 59.940 (60000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.054
Stream size                              : 152 MiB (98%)
Writing library                          : x265 2.8:[Linux][GCC 9.1.0][64 bit] 10bit
Encoding settings                        : cpuid=1111039 / frame-threads=3 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=3 / no-allow-non-conformance / repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=25 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=20 / lookahead-slices=8 / scenecut=40 / radl=0 / no-intra-refresh / ctu=64 / min-cu-size=8 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=2 / limit-refs=3 / no-limit-modes / me=1 / subme=2 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=3 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=18.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=0 / display-window=0 / master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,50) / max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-mv-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0040 cd/m2, max: 1100 cd/m2


  



  I suspect Color range, Color primaries and Transfer characteristics are necessary to make the file recognized as HDR by my TV. 
How can I add these properties to the VAAPI encoded file ?

  



  Edit :
This patch fixes the problem :
Patch

  


• H.264 to DNxHR 444 issue. Colors are not transcoded correctly (HDR project). Note : issue not solved yet

  22 janvier 2020, par Raulo1985

  I’m having an issue transcoding a H.264 UHD HDR file to a DNxHR file in a mxf container with FFmpeg. The issue is that both files don’t look the same at all, the colors look washed out on the DNxHR video, and I tried to make the transcoding as lossless as possible (DNxHR 444 flavor). The original file is a movie I ripped a while ago, H.264, UHD, HDR, in a mkv container.

  My goal : to create an almost lossless DNxHR file to use it as source file in Adobe Premiere Pro, and use another DNxHR file with less quality as proxy for editing. I wanted to do it that way and not use the original H.264 as the source file because it’s out of sync with the proxy file (I mean, when I toggle the proxy icon on and off, you can tell that there’s a short delay between them, which defeats all purposes for editing). My guess is that it may be because H.264 is compressed and DNxHR isn’t, and since I edit making a lot of fast cuts, I need both the source file and the proxy file to be as synced as possible. When the source file and the proxy file are both DNxHR, no matter the flavor, they are perfectly synced. I don’t want to go with Prores for the proxies, because the sync problem is a lot worse (several seconds of delay between files), maybe because it’s a VBR codec and my original file and DNxHR are CBR (for the record, I always prefer CBR).

  Well, the thing is that when I import the original H.264 file to Premiere Pro, use a DNxHR proxy, edit a little, and export it directly from the original file (H.264 10 bits, with all the settings required for HDR output enabled) the colors look as they should. When I do the same with the high quality DNxHR as source file, with the exact same export settings, the colors look washed out. The same with any DNxHR flavor.

  Then I opened both files (original H.264 and high quality DNxHR transcoded from the H.264 one) with VLC, and I also can tell that the mxf file looks washed out and the H.264 file doesn’t. So it’s not an export issue on Premiere’ side, it’s something that has to do with the original transcoding.

  I understand that DNxHR 444 is as lossless as you can get with that codec, preserving all the HDR required data, and I believe that the mfx container has some advantages over MOV, which is the other container that supports DNxHD/DNxHR. So I don’t know what’s happening really.

  The command I used was :

  ffmpeg -channel_layout 63 -i input.mkv -map 0:0 -c:v dnxhd -vf "scale=in_range=limited:out_range=full" -color_range 2 -profile:v dnxhr_444 -pix_fmt yuv444p10le -acodec pcm_s24le -ar 48000 -ac 6 -channel_layout 63 -map 0:2 -hide_banner output.mxf

  Like I said, after the transcoding, both video files look a lot different from each other, color wise. And after using them in Premiere and exporting with the exact same settings, the output files suffer from the same difference.

  Mediainfo shows the expected data for both files :
   10 bits, main 10, level 5, 4:2:0, CBR, BT.2020 for the original h.264 file.
   10 bits, 4:4:4, CBR for the DNxHR 444 file.

  One thing I noticed in Mediainfo is that both have YUV as color space, but the DNxHR 444 video has an extra field that says ColorSpace_Original : RGB. Honestly, I don’t know what that means, since the original is YUV. Color range is fine, from 0 to 1023 (and chroma range 1023). The other thing is that it says "limited" on the color range field of the H.264 file, but I’ve read that that could be a bug or missinterpretation of the file by Mediainfo.

  Well, that’s it, any help would be appreciated. I’d really like to edit with DNxHR 444 as source file and DNxHR LB for the proxies, so I can edit in a fast pace and without sync issues, but the color is just not acceptable. And I do understand that I’m adding an extra transcoding step (from original to DNxHR), but the sync issue between the original and the DNxHR proxies, even though it may be a delay of a fraction of a second, makes my workflow a lot harder since I’ll have to export many times to see if the cuts are made exactly where I want them to be. Not ideal by any means. And Prores is not an option apparently, the sync issue is a lot worse. For me, it all comes down to being able to get a DNxHR 444 file that looks, well, as close to lossless as it can be, and that goal obviously involves the colors.

  Thanks in advance.

  PS : file size is not an issue for me, so having an entire UHD HDR movie transcoded to DNxHR 444 is not a problem.

  PS2 : I tried with a different chroma subsampling (like DNxHR HQX 10 bits, which is 4:2:2), same result. Haven´t tried with 8 bits yet, but I don’t see the point since this is a HDR project.

  UPDATE :

  I tried to transcode from the H.264 source file to a DNxHR video file in a MXF container using Adobe Media Encoder instead of FFmpeg, and the colors are not transcoded correctly again, but this time they seem to be over saturated instead of washed out. Adobe Media Encoder doesn’t give much room for tweaking, but I made sure to select 444 10 bits profile, same resolution (UHD), same frame rate and render with maximum quality and maximum bit depth. FFprobe output of the resulting file again shows BT709 as the color space (the same thing happens with the output file after transcoding using FFmpeg). Seems to be something not related to FFmpeg, apparently. Any ideas ? It’s like there’s no way I can transcode and retain the colors correctly from H.264 to DNxHR, even using its most high quality flavor and correct command settings (at least they look ok to me). How can I post this so maybe developers or people with lots of experience here can give us a clue to what’s happening ? Thanks.

  PS : More potentially useful info on the comments below.

  EXTRA INFO :

  1) FFprobe output of the MXF DNxHR file (this one is 4:2:2, the only difference with the command used compared to the one stated on the OP is -pix_fmt yuv444p10le being -pix_fmt yuv422p10le) :

    libavutil      56. 31.100 / 56. 31.100
  
    libavcodec     58. 54.100 / 58. 54.100
  
    libavformat    58. 29.100 / 58. 29.100
  
    libavdevice    58.  8.100 / 58.  8.100
  
    libavfilter     7. 57.100 /  7. 57.100
  
    libswscale      5.  5.100 /  5.  5.100
  
    libswresample   3.  5.100 /  3.  5.100
  
    libpostproc    55.  5.100 / 55.  5.100
  
  [mxf @ 000001f4d17fbac0] Stream #0: not enough frames to estimate rate; consider increasing probesize
  
  Input #0, mxf, from 'Interstellar_Master_DNxHR_444_UHD_422_PCM24_5.1.mxf':
  
    Metadata:
  
      operational_pattern_ul: 060e2b34.04010101.0d010201.01010900
  
      uid             : adab4424-2f25-4dc7-92ff-29bd000c0000
  
      generation_uid  : adab4424-2f25-4dc7-92ff-29bd000c0001
  
      company_name    : FFmpeg
  
      product_name    : OP1a Muxer
  
      product_version : 58.29.100
  
      product_uid     : adab4424-2f25-4dc7-92ff-29bd000c0002
  
      material_package_umid: 0x060A2B340101010501010D001393EE79529471348D93EE7900529471348D9300
  
      timecode        : 00:00:00:00
  
    Duration: 02:49:03.97, start: 0.000000, bitrate: 1404833 kb/s
  
      Stream #0:0: Video: dnxhd (DNXHR 444), yuv444p10le(bt709/unknown/unknown, progressive), 3840x2160, SAR 1:1 DAR 16:9, 23.98 tbr, 23.98 tbn, 23.98 tbc
  
      Metadata:
  
        file_package_umid: 0x060A2B340101010501010D001393EE79529471348D93EE7900529471348D9301
  
      Stream #0:1: Audio: pcm_s24le, 48000 Hz, 6 channels, s32 (24 bit), 6912 kb/s
  
      Metadata:
  
        file_package_umid: 0x060A2B340101010501010D001393EE79529471348D93EE7900529471348D9301

  2) FFprobe output of the MP4 H.264 source file (this one is 4:2:0, 10 bits, HDR) :

      Stream #0:0(eng): Video: hevc (Main 10) (hev1 / 0x31766568), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 16:9], 15584 kb/s, 23.98 fps, 23.98 tbr, 16k tbn, 23.98 tbc (default)
  
      Metadata:
  
        handler_name    : VideoHandler
  
      Stream #0:1(eng): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 640 kb/s (default)
  
      Metadata:
  
        handler_name    : SoundHandler
  
      Side data:
  
        audio service type: main
  
      Stream #0:2(eng): Data: bin_data (text / 0x74786574)
  
      Metadata:
  
        handler_name    : SubtitleHandler
  
  Unsupported codec with id 100359 for input stream 2