Sur d’autres sites (7313)

• Reading in pydub AudioSegment from url. BytesIO returning "OSError [Errno 2] No such file or directory" on heroku only ; fine on localhost

  24 octobre 2014, par Mark

  EDIT 1 for anyone with the same error : installing ffmpeg did indeed solve that BytesIO error

  EDIT 1 for anyone still willing to help : my problem is now that when I AudioSegment.export("filename.mp3", format="mp3"), the file is made, but has size 0 bytes — details below (as "EDIT 1")

  ---

  EDIT 2 : All problems now solved.

  • Files can be read in as AudioSegment using BytesIO
  • I found buildpacks to ensure ffmpeg was installed correctly on my app, with lame support for exporting proper mp3 files

  Answer below

  ---

  Original question

  I have pydub working nicely locally to crop a particular mp3 file based on parameters in the url.
  (?start_time=3.8&end_time=5.1)

  When I run foreman start it all looks good on localhost. The html renders nicely.
  The key lines from the views.py include reading in a file from a url using

  url = "https://s3.amazonaws.com/shareducate02/The_giving_tree__by_Alex_Blumberg__sponsored_by_mailchimp-short.mp3"
  
  mp3 = urllib.urlopen(url).read() # inspired by http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
  
  original=AudioSegment.from_mp3(BytesIO(mp3))  # AudioSegment.from_mp3 is a pydub command, see http://pydub.com
  
  section = original[start_time_ms:end_time_ms]

  That all works great... until I push to heroku (django app) and run it online.
  then when I load the same page now on the herokuapp.com, I get this error

  OSError at /path/to/page
  
  [Errno 2] No such file or directory
  
  Request Method: GET
  
  Request URL:    http://my.website.com/path/to/page?start_time=3.8&end_time=5
  
  Django Version: 1.6.5
  
  Exception Type: OSError
  
  Exception Value:    
  
  [Errno 2] No such file or directory
  
  Exception Location: /app/.heroku/python/lib/python2.7/subprocess.py in _execute_child, line 1327
  
  Python Executable:  /app/.heroku/python/bin/python
  
  Python Version: 2.7.8
  
  Python Path:    
  
  ['/app',
  
   '/app/.heroku/python/bin',
  
   '/app/.heroku/python/lib/python2.7/site-packages/setuptools-5.4.1-py2.7.egg',
  
   '/app/.heroku/python/lib/python2.7/site-packages/distribute-0.6.36-py2.7.egg',
  
   '/app/.heroku/python/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg',
  
   '/app',
  
   '/app/.heroku/python/lib/python27.zip',
  
   '/app/.heroku/python/lib/python2.7',
  
   '/app/.heroku/python/lib/python2.7/plat-linux2',
  
   '/app/.heroku/python/lib/python2.7/lib-tk',
  
   '/app/.heroku/python/lib/python2.7/lib-old',
  
   '/app/.heroku/python/lib/python2.7/lib-dynload',
  
   '/app/.heroku/python/lib/python2.7/site-packages',
  
   '/app/.heroku/python/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info']
  
  
  
  
  
  Traceback:
  
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  
    112.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
  
  File "/app/evernote/views.py" in finalize
  
    105.       original=AudioSegment.from_mp3(BytesIO(mp3))
  
  File "/app/.heroku/python/lib/python2.7/site-packages/pydub/audio_segment.py" in from_mp3
  
    318.         return cls.from_file(file, 'mp3')
  
  File "/app/.heroku/python/lib/python2.7/site-packages/pydub/audio_segment.py" in from_file
  
    302.         retcode = subprocess.call(convertion_command, stderr=open(os.devnull))
  
  File "/app/.heroku/python/lib/python2.7/subprocess.py" in call
  
    522.     return Popen(*popenargs, **kwargs).wait()
  
  File "/app/.heroku/python/lib/python2.7/subprocess.py" in __init__
  
    710.                                 errread, errwrite)
  
  File "/app/.heroku/python/lib/python2.7/subprocess.py" in _execute_child
  
    1327.                 raise child_exception

  I have commented out some of the original to convince myself that sure enough the single line original=AudioSegment.from_mp3(BytesIO(mp3)) is where the problem kicks in... but this is not a problem locally

  The full function in views.py starts like this :

  from django.shortcuts import render, get_object_or_404 
  
  from django.http import HttpResponseRedirect #, Http404, HttpResponse
  
  from django.core.urlresolvers import reverse
  
  from django.views import generic
  
  import pydub
  
  # Maybe only need: 
  
  from pydub import AudioSegment # == see below
  
  from time import gmtime, strftime
  
  
  
  import boto
  
  from boto.s3.connection import S3Connection
  
  from boto.s3.key import Key
  
  
  
  # http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
  
  import urllib
  
  from io import BytesIO
  
  # import numpy as np
  
  # import scipy.signal as sg
  
  # import pydub # mentioned above already
  
  # import matplotlib.pyplot as plt
  
  # from IPython.display import Audio, display
  
  # import matplotlib as mpl
  
  # %matplotlib inline
  
  
  
  import os
  
  # from settings import AWS_ACCESS_KEY, AWS_SECRET_KEY, AWS_BUCKET_NAME
  
  AWS_ACCESS_KEY = os.environ.get('AWS_ACCESS_KEY') # there must be a better way?
  
  AWS_SECRET_KEY = os.environ.get('AWS_SECRET_KEY')
  
  AWS_BUCKET_NAME = os.environ.get('S3_BUCKET_NAME')
  
  
  
  # http://stackoverflow.com/questions/415511/how-to-get-current-time-in-python
  
  
  
  boto_conn = S3Connection(AWS_ACCESS_KEY, AWS_SECRET_KEY)
  
  bucket = boto_conn.get_bucket(AWS_BUCKET_NAME)
  
  s3_url_format = 'https://s3.amazonaws.com/shareducate02/{end_path}'

  and specifically the view in views.py that’s called when I visit the page :

  def finalize(request):
  
  
  
      start_time = request.GET.get('start_time')
  
  
  
      end_time = request.GET.get('end_time')
  
  
  
      original_file = "https://s3.amazonaws.com/shareducate02/The_giving_tree__by_Alex_Blumberg__sponsored_by_mailchimp-short.mp3"
  
  
  
  
  
      if start_time:
  
  
  
        # original=AudioSegment.from_mp3(original_file)  #...that didn't work 
  
        # but this works below:
  
  
  
        # next three uncommented lines from http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
  
        # python 2.x
  
        url = original_file
  
        # req = urllib.Request(url, headers={'User-Agent': ''}) # Note: I commented out this because I got error that "Request" did not exist
  
        mp3 = urllib.urlopen(url).read()
  
        # That's for my 2.7
  
  
  
        # If I ever upgrade to python 3.x, would need to change it to:
  
        # req = urllib.request.Request(url, headers={'User-Agent': ''}) 
  
        # mp3 = urllib.request.urlopen(req).read()
  
        # as per instructions on http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
  
  
  
        original=AudioSegment.from_mp3(BytesIO(mp3))
  
        # original=AudioSegment.from_mp3("static/givingtree.mp3") # alternative that works locally (on laptop) but no use for heroku
  
  
  
        start_time_ms = int(float(start_time) * 1000)
  
        if end_time:
  
          end_time_ms = int(float(end_time) * 1000)
  
        else:
  
          end_time_ms = int(float(original.duration_seconds) * 1000)
  
        duration_ms = end_time_ms - start_time_ms
  
        # duration = end_time - start_time
  
        duration = duration_ms/1000
  
  
  
     #   section = original[start_time_ms:end_time_ms]
  
     #   section_with_fading = section.fade_in(100).fade_out(100)
  
  
  
        clip = "demo-"
  
        number = strftime("%Y-%m-%d_%H-%M-%S", gmtime())
  
        clip += number
  
        clip += ".mp3" 
  
  
  
        # DON'T BOTHER writing locally:
  
        # clip_with_path = "evernote/static/"+clip
  
        # section_with_fading.export(clip_with_path, format = "mp3")
  
  
  
     #   tempclip = section_with_fading.export(format = "mp3")
  
  
  
        # commented out while de-bugging, but was working earlier if run on localhost
  
        # c = boto.connect_s3()
  
        # b = c.get_bucket(S3_BUCKET_NAME)  # as defined above
  
        # k = Key(b)
  
        # k.key=clip
  
        # # k.set_contents_from_filename(clip_with_path)
  
        # k.set_contents_from_file(tempclip)
  
        # k.set_acl('public-read')
  
        clip_made = True
  
      else: 
  
        duration = 0.0
  
        clip_made = False
  
        clip = ""
  
      context = {'original_file':original_file, 'new_file':clip, 'start_time': start_time, 'end_time':end_time, 'duration':duration, 'clip_made':clip_made} 
  
      return render(request, 'finalize.html' , context) 

  Any suggestions ?

  Potentially related :
  I have ffmpeg installed locally

  But have been unable to install it onto heroku, due to not understanding buildpacks. I tried just a moment ago (http://stackoverflow.com/questions/14407388/how-to-install-ffmpeg-for-a-django-app-on-heroku and https://github.com/shunjikonishi/heroku-buildpack-ffmpeg) but so far ffmpeg is not working on heroku (ffmpeg is not recognised when I do "heroku run ffmpeg —version")
  ...do you think this is the reason ?

  An answer like any of these would be much appreciated as I’m going round in circles here :

  1. "I think ffmpeg is indeed your problem. Try harder to sort that out, to get it installed on heroku"
  2. "Actually, I think this is why BytesIO is not working for you : ..."
  3. "Your approach is terrible anyway... if you want to read in an audio file to process using pydub, you should just do this instead : ..." (since I’m just hacking my way through pydub for my first time... my approach may be poor)

  ---

  EDIT 1

  ffmpeg is now installed (e.g., I can output wav files)

  However, I can’t create mp3 files, still... or more correctly, I can, but the filesize is zero

  (venv-app)moriartymacbookair13:getstartapp macuser$ heroku config:add BUILDPACK_URL=https://github.com/ddollar/heroku-buildpack-multi.git 
  
  Setting config vars and restarting awe01... done, v93
  
  BUILDPACK_URL: https://github.com/ddollar/heroku-buildpack-multi.git
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ vim .buildpacks 
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ cat .buildpacks 
  
  https://github.com/shunjikonishi/heroku-buildpack-ffmpeg.git
  
  https://github.com/heroku/heroku-buildpack-python.git
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ git add --all
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ git commit -m "need multi, not just ffmpeg, so adding back in multi + shun + heroku, with trailing .git in .buildpacks file"
  
  [master cd99fef] need multi, not just ffmpeg, so adding back in multi + shun + heroku, with trailing .git in .buildpacks file
  
   1 file changed, 2 insertions(+), 2 deletions(-)
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ git push heroku master
  
  Fetching repository, done.
  
  Counting objects: 5, done.
  
  Delta compression using up to 4 threads.
  
  Compressing objects: 100% (3/3), done.
  
  Writing objects: 100% (3/3), 372 bytes | 0 bytes/s, done.
  
  Total 3 (delta 2), reused 0 (delta 0)
  
  
  
  -----> Fetching custom git buildpack... done
  
  -----> Multipack app detected
  
  =====> Downloading Buildpack: https://github.com/shunjikonishi/heroku-buildpack-ffmpeg.git
  
  =====> Detected Framework: ffmpeg
  
  -----> Install ffmpeg
  
         DOWNLOAD_URL =  http://flect.github.io/heroku-binaries/libs/ffmpeg.tar.gz
  
         exporting PATH and LIBRARY_PATH
  
  =====> Downloading Buildpack: https://github.com/heroku/heroku-buildpack-python.git
  
  =====> Detected Framework: Python
  
  -----> Installing dependencies with pip
  
         Cleaning up...
  
  
  
  -----> Preparing static assets
  
         Collectstatic configuration error. To debug, run:
  
         $ heroku run python ./example/manage.py collectstatic --noinput
  
  
  
  Using release configuration from last framework (Python).
  
  -----> Discovering process types
  
         Procfile declares types -> web
  
  
  
  -----> Compressing... done, 198.1MB
  
  -----> Launching... done, v94
  
         http://[redacted].herokuapp.com/ deployed to Heroku
  
  
  
  To git@heroku.com:awe01.git
  
     78d6b68..cd99fef  master -> master
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ heroku run ffmpeg
  
  Running `ffmpeg` attached to terminal... up, run.6408
  
  ffmpeg version git-2013-06-02-5711e4f Copyright (c) 2000-2013 the FFmpeg developers
  
    built on Jun  2 2013 07:38:40 with gcc 4.4.3 (Ubuntu 4.4.3-4ubuntu5.1)
  
    configuration: --enable-shared --disable-asm --prefix=/app/vendor/ffmpeg
  
    libavutil      52. 34.100 / 52. 34.100
  
    libavcodec     55. 13.100 / 55. 13.100
  
    libavformat    55.  8.102 / 55.  8.102
  
    libavdevice    55.  2.100 / 55.  2.100
  
    libavfilter     3. 74.101 /  3. 74.101
  
    libswscale      2.  3.100 /  2.  3.100
  
    libswresample   0. 17.102 /  0. 17.102
  
  Hyper fast Audio and Video encoder
  
  usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
  
  
  
  Use -h to get full help or, even better, run 'man ffmpeg'
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ heroku run bash
  
  Running `bash` attached to terminal... up, run.9660
  
  ~ $ python
  
  Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
  
  [GCC 4.4.3] on linux2
  
  Type "help", "copyright", "credits" or "license" for more information.
  
  >>> import pydub
  
  >>> from pydub import AudioSegment
  
  >>> exit()
  
  ~ $ which ffmpeg
  
  /app/vendor/ffmpeg/bin/ffmpeg
  
  ~ $ python 
  
  
  
  Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
  
  [GCC 4.4.3] on linux2
  
  Type "help", "copyright", "credits" or "license" for more information.
  
  >>> import pydub
  
  >>> from pydub import AudioSegment
  
  >>> AudioSegment.silent(5000).export("/tmp/asdf.mp3", "mp3")
  
  <open file="file"></open>tmp/asdf.mp3', mode 'wb+' at 0x7f9a37d44780>
  
  >>> exit ()
  
  ~ $ cd /tmp/
  
  /tmp $ ls
  
  asdf.mp3
  
  /tmp $ open asdf.mp3
  
  bash: open: command not found
  
  /tmp $ ls -lah
  
  total 8.0K
  
  drwx------  2 u36483 36483 4.0K 2014-10-22 04:14 .
  
  drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3

  Note the file size of 0 above for the mp3 file... when I do the same thing on my macbook, the file size is never zero

  Back to the heroku shell :

  /tmp $ python
  
  Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
  
  [GCC 4.4.3] on linux2
  
  Type "help", "copyright", "credits" or "license" for more information.
  
  >>> import pydub
  
  >>> from pydub import AudioSegment
  
  >>> pydub.AudioSegment.ffmpeg = "/app/vendor/ffmpeg/bin/ffmpeg" 
  
  >>> AudioSegment.silence(1200).export("/tmp/herokuSilence.mp3", format="mp3")
  
  Traceback (most recent call last):
  
    File "<stdin>", line 1, in <module>
  
  AttributeError: type object 'AudioSegment' has no attribute 'silence'
  
  >>> AudioSegment.silent(1200).export("/tmp/herokuSilence.mp3", format="mp3")
  
  <open file="file"></open>tmp/herokuSilence.mp3', mode 'wb+' at 0x7fcc2017c780>
  
  >>> exit()
  
  /tmp $ ls
  
  asdf.mp3  herokuSilence.mp3
  
  /tmp $ ls -lah
  
  total 8.0K
  
  drwx------  2 u36483 36483 4.0K 2014-10-22 04:29 .
  
  drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:29 herokuSilence.mp3
  
  </module></stdin>

  I realised the first time that I had forgotten the pydub.AudioSegment.ffmpeg = "/app/vendor/ffmpeg/bin/ffmpeg" command, but as you can see above, the file is still zero size

  Out of desperation, I even tried adding the ".heroku" into the path to be as verbatim as your example, but that didn’t fix it :

  /tmp $ python
  
  Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
  
  [GCC 4.4.3] on linux2
  
  Type "help", "copyright", "credits" or "license" for more information.
  
  >>> import pydub
  
  >>> from pydub import AudioSegment
  
  >>> pydub.AudioSegment.ffmpeg = "/app/.heroku/vendor/ffmpeg/bin/ffmpeg"
  
  >>> AudioSegment.silent(1200).export("/tmp/herokuSilence03.mp3", format="mp3")
  
  <open file="file"></open>tmp/herokuSilence03.mp3', mode 'wb+' at 0x7fc92aca7780>
  
  >>> exit()
  
  /tmp $ ls -lah
  
  total 8.0K
  
  drwx------  2 u36483 36483 4.0K 2014-10-22 04:31 .
  
  drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:31 herokuSilence03.mp3
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:29 herokuSilence.mp3

  Finally, I tried exporting a .wav file to check pydub was at least working correctly

  /tmp $ python
  
  Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
  
  [GCC 4.4.3] on linux2
  
  Type "help", "copyright", "credits" or "license" for more information.
  
  >>> import pydub
  
  >>> from pydub import AudioSegment
  
  >>> pydub.AudioSegment.ffmpeg = "/app/vendor/ffmpeg/bin/ffmpeg"
  
  >>> AudioSegment.silent(1300).export("/tmp/heroku_wav_silence01.wav", format="wav")
  
  <open file="file"></open>tmp/heroku_wav_silence01.wav', mode 'wb+' at 0x7fa33cbf3780>
  
  >>> exit()
  
  /tmp $ ls
  
  asdf.mp3  herokuSilence03.mp3  herokuSilence.mp3  heroku_wav_silence01.wav
  
  /tmp $ ls -lah
  
  total 40K
  
  drwx------  2 u36483 36483 4.0K 2014-10-22 04:42 .
  
  drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:31 herokuSilence03.mp3
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:29 herokuSilence.mp3
  
  -rw-------  1 u36483 36483  29K 2014-10-22 04:42 heroku_wav_silence01.wav
  
  /tmp $ 

  At least that filesize for .wav is non-zero, so pydub is working

  My current theory is that either I’m still not using ffmpeg correctly, or it’s insufficient... maybe I need an mp3 additional install on top of basic ffmpeg.

  Several sites mention "libavcodec-extra-53" but I’m not sure how to install that on heroku, or to check if I have it ? https://github.com/jiaaro/pydub/issues/36
  Similarly tutorials on libmp3lame seem to be geared towards laptop installation rather than installation on heroku, so I’m at a loss http://superuser.com/questions/196857/how-to-install-libmp3lame-for-ffmpeg

  In case relevant, I also have youtube-dl in my requirements.txt... this also works locally on my macbook, but fails when I run it in the heroku shell :

  ~/ytdl $ youtube-dl --restrict-filenames -x --audio-format mp3 n2anDgdUHic
  
  [youtube] Setting language
  
  [youtube] Confirming age
  
  [youtube] n2anDgdUHic: Downloading webpage
  
  [youtube] n2anDgdUHic: Downloading video info webpage
  
  [youtube] n2anDgdUHic: Extracting video information
  
  [download] Destination: Boyce_Avenue_feat._Megan_Nicole_-_Skyscraper_Patrick_Ebert_Edit-n2anDgdUHic.m4a
  
  [download] 100% of 5.92MiB in 00:00
  
  [ffmpeg] Destination: Boyce_Avenue_feat._Megan_Nicole_-_Skyscraper_Patrick_Ebert_Edit-n2anDgdUHic.mp3
  
  ERROR: audio conversion failed: Unknown encoder 'libmp3lame'
  
  ~/ytdl $ 

  The informative link is that it too specificies an mp3 failure, so perhaps they two issues are related.

  ---

  EDIT 2

  See answer, all problems solved

• trying to make OpenCV 3.2.0 work with virtualenv

  24 juillet 2017, par lollercoaster

  I’m on Ubuntu 16.04 with Python 2.7 and virtualenv & virtualenvwrapper.

  By following this guide I managed to get the following script working with my system Python2.7 which has cv2 globally installed.

  I used this script to install it :

  ######################################
  
  # INSTALL OPENCV ON UBUNTU OR DEBIAN #
  
  ######################################
  
  
  
  # |         THIS SCRIPT IS TESTED CORRECTLY ON         |
  
  # |----------------------------------------------------|
  
  # | OS             | OpenCV       | Test | Last test   |
  
  # |----------------|--------------|------|-------------|
  
  # | Ubuntu 16.04.2 | OpenCV 3.2.0 | OK   | 20 May 2017 |
  
  # | Debian 8.8     | OpenCV 3.2.0 | OK   | 20 May 2017 |
  
  # | Debian 9.0     | OpenCV 3.2.0 | OK   | 25 Jun 2017 |
  
  
  
  # 1. KEEP UBUNTU OR DEBIAN UP TO DATE
  
  
  
  sudo apt-get -y update
  
  sudo apt-get -y upgrade
  
  sudo apt-get -y dist-upgrade
  
  sudo apt-get -y autoremove
  
  
  
  
  
  # 2. INSTALL THE DEPENDENCIES
  
  
  
  # Build tools:
  
  sudo apt-get install -y build-essential cmake
  
  
  
  # GUI (if you want to use GTK instead of Qt, replace 'qt5-default' with 'libgtkglext1-dev' and remove '-DWITH_QT=ON' option in CMake):
  
  sudo apt-get install -y qt5-default libvtk6-dev
  
  
  
  # Media I/O:
  
  sudo apt-get install -y zlib1g-dev libjpeg-dev libwebp-dev libpng-dev libtiff5-dev libjasper-dev libopenexr-dev libgdal-dev
  
  
  
  # Video I/O:
  
  sudo apt-get install -y libdc1394-22-dev libavcodec-dev libavformat-dev libswscale-dev libtheora-dev libvorbis-dev libxvidcore-dev libx264-dev yasm libopencore-amrnb-dev libopencore-amrwb-dev libv4l-dev libxine2-dev
  
  
  
  # Parallelism and linear algebra libraries:
  
  sudo apt-get install -y libtbb-dev libeigen3-dev
  
  
  
  # Python:
  
  sudo apt-get install -y python-dev python-tk python-numpy python3-dev python3-tk python3-numpy
  
  
  
  # Documentation:
  
  sudo apt-get install -y doxygen
  
  
  
  # UI stuff
  
  sudo apt-get install libgtk-3-dev libatlas-base-dev gfortran
  
  
  
  
  
  # 3. INSTALL THE LIBRARY (YOU CAN CHANGE '3.2.0' FOR THE LAST STABLE VERSION)
  
  sudo apt-get install -y unzip wget
  
  
  
  # opencv contrib
  
  wget https://github.com/opencv/opencv_contrib/archive/3.2.0.zip -O opencv_contrib-3.2.0.zip
  
  unzip opencv_contrib-3.2.0.zip
  
  rm opencv_contrib-3.2.0.zip
  
  
  
  # opencv
  
  wget https://github.com/opencv/opencv/archive/3.2.0.zip
  
  unzip 3.2.0.zip
  
  rm 3.2.0.zip
  
  mv opencv-3.2.0 OpenCV-3.2.0
  
  cd OpenCV-3.2.0
  
  
  
  mkdir build
  
  cd build
  
  cmake -D WITH_QT=ON \
  
      -D WITH_OPENGL=ON \
  
      -D FORCE_VTK=ON \
  
      -D WITH_TBB=ON \
  
      -D WITH_GDAL=ON \
  
      -D WITH_XINE=ON \
  
      -D BUILD_EXAMPLES=ON \
  
      -D INSTALL_PYTHON_EXAMPLES=ON \
  
      -D ENABLE_PRECOMPILED_HEADERS=OFF \
  
      -D BUILD_NEW_PYTHON_SUPPORT=ON \
  
      ..
  
  
  
  make -j4
  
  sudo make install
  
  sudo ldconfig
  
  
  
  
  
  # 4. EXECUTE SOME OPENCV EXAMPLES AND COMPILE A DEMONSTRATION
  
  
  
  # To complete this step, please visit 'http://milq.github.io/install-opencv-ubuntu-debian'.

  The following script below works great with that system-wide installation :

  import cv2
  
  
  
  img = cv2.imread('some_img.jpg')

  Though this one doesn’t - even the system Python can’t read videos for some reason...

  import cv2
  
  
  
  video_capture = cv2.VideoCapture(0)
  
  ret, frame = video_capture.read()
  
  print ret  # always False

  but I want it to work with my virtualenv. So I recompiled OpenCV with :

  cmake -D WITH_QT=ON \
  
      -D WITH_OPENGL=ON \
  
      -D FORCE_VTK=ON \
  
      -D WITH_TBB=ON \
  
      -D WITH_GDAL=ON \
  
      -D WITH_XINE=ON \
  
      -D BUILD_EXAMPLES=ON \
  
      -D INSTALL_PYTHON_EXAMPLES=ON \
  
      -D ENABLE_PRECOMPILED_HEADERS=OFF \
  
      -D BUILD_NEW_PYTHON_SUPPORT=ON \
  
      -D OPENCV_EXTRA_MODULES_PATH=/home/me/code/myproject/opencv_contrib-3.2.0/modules \
  
      -D PYTHON_EXECUTABLE=~/.envs/myenv/bin/python \
  
      ..
  
  
  
  make -j4
  
  sudo make install
  
  sudo ldconfig

  Here’s the CMake log :

  -- Found VTK ver. 6.2.0 (usefile: /usr/lib/cmake/vtk-6.2/UseVTK.cmake)
  
  -- Caffe:   NO
  
  -- Protobuf:   YES
  
  -- Glog:   NO
  
  -- freetype2:   YES
  
  -- harfbuzz:    YES
  
  -- Module opencv_sfm disabled because the following dependencies are not found: Glog/Gflags
  
  -- freetype2:   YES
  
  -- harfbuzz:    YES
  
  -- Checking for modules 'tesseract;lept'
  
  --   No package 'tesseract' found
  
  --   No package 'lept' found
  
  -- Tesseract:   NO
  
  -- Check contents of vgg_generated_48.i ...
  
  -- Check contents of vgg_generated_64.i ...
  
  -- Check contents of vgg_generated_80.i ...
  
  -- Check contents of vgg_generated_120.i ...
  
  -- Check contents of boostdesc_bgm.i ...
  
  -- Check contents of boostdesc_bgm_bi.i ...
  
  -- Check contents of boostdesc_bgm_hd.i ...
  
  -- Check contents of boostdesc_binboost_064.i ...
  
  -- Check contents of boostdesc_binboost_128.i ...
  
  -- Check contents of boostdesc_binboost_256.i ...
  
  -- Check contents of boostdesc_lbgm.i ...
  
  -- 
  
  -- General configuration for OpenCV 3.2.0 =====================================
  
  --   Version control:               817bd7b-dirty
  
  -- 
  
  --   Extra modules:
  
  --     Location (extra):            /home/me/code/myproject/opencv_contrib-3.2.0/modules
  
  --     Version control (extra):     817bd7b-dirty
  
  -- 
  
  --   Platform:
  
  --     Timestamp:                   2017-07-20T18:25:26Z
  
  --     Host:                        Linux 4.8.0-58-generic x86_64
  
  --     CMake:                       3.5.1
  
  --     CMake generator:             Unix Makefiles
  
  --     CMake build tool:            /usr/bin/make
  
  --     Configuration:               Release
  
  -- 
  
  --   C/C++:
  
  --     Built as dynamic libs?:      YES
  
  --     C++ Compiler:                /usr/bin/c++  (ver 5.4.0)
  
  --     C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
  
  --     C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
  
  --     C Compiler:                  /usr/bin/cc
  
  --     C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wno-narrowing -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
  
  --     C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wno-narrowing -Wno-comment -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -msse -msse2 -mno-avx -msse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
  
  --     Linker flags (Release):
  
  --     Linker flags (Debug):
  
  --     ccache:                      NO
  
  --     Precompiled headers:         NO
  
  --     Extra dependencies:          Qt5::Test Qt5::Concurrent Qt5::OpenGL /usr/lib/x86_64-linux-gnu/libwebp.so /usr/lib/x86_64-linux-gnu/libjasper.so /usr/lib/x86_64-linux-gnu/libImath.so /usr/lib/x86_64-linux-gnu/libIlmImf.so /usr/lib/x86_64-linux-gnu/libIex.so /usr/lib/x86_64-linux-gnu/libHalf.so /usr/lib/x86_64-linux-gnu/libIlmThread.so /usr/lib/libgdal.so dc1394 xine avcodec-ffmpeg avformat-ffmpeg avutil-ffmpeg swscale-ffmpeg Qt5::Core Qt5::Gui Qt5::Widgets /usr/lib/x86_64-linux-gnu/hdf5/serial/lib/libhdf5.so /usr/lib/x86_64-linux-gnu/libpthread.so /usr/lib/x86_64-linux-gnu/libsz.so /usr/lib/x86_64-linux-gnu/libdl.so /usr/lib/x86_64-linux-gnu/libm.so vtkRenderingOpenGL vtkImagingHybrid vtkIOImage vtkCommonDataModel vtkCommonMath vtkCommonCore vtksys vtkCommonMisc vtkCommonSystem vtkCommonTransforms vtkCommonExecutionModel vtkDICOMParser vtkIOCore /usr/lib/x86_64-linux-gnu/libz.so vtkmetaio /usr/lib/x86_64-linux-gnu/libjpeg.so /usr/lib/x86_64-linux-gnu/libpng.so /usr/lib/x86_64-linux-gnu/libtiff.so vtkImagingCore vtkRenderingCore vtkCommonColor vtkFiltersExtraction vtkFiltersCore vtkFiltersGeneral vtkCommonComputationalGeometry vtkFiltersStatistics vtkImagingFourier vtkalglib vtkFiltersGeometry vtkFiltersSources vtkInteractionStyle vtkRenderingLOD vtkFiltersModeling vtkIOPLY vtkIOGeometry /usr/lib/x86_64-linux-gnu/libjsoncpp.so vtkFiltersTexture vtkRenderingFreeType /usr/lib/x86_64-linux-gnu/libfreetype.so vtkftgl vtkIOExport vtkRenderingAnnotation vtkImagingColor vtkRenderingContext2D vtkRenderingGL2PS vtkRenderingContextOpenGL /usr/lib/libgl2ps.so vtkRenderingLabel dl m pthread rt /usr/lib/x86_64-linux-gnu/libGLU.so /usr/lib/x86_64-linux-gnu/libGL.so tbb
  
  --     3rdparty dependencies:       libprotobuf
  
  -- 
  
  --   OpenCV modules:
  
  --     To be built:                 core flann hdf imgproc ml photo reg surface_matching video viz dnn freetype fuzzy imgcodecs shape videoio highgui objdetect plot superres ts xobjdetect xphoto bgsegm bioinspired dpm face features2d line_descriptor saliency text calib3d ccalib cvv datasets rgbd stereo tracking videostab xfeatures2d ximgproc aruco optflow phase_unwrapping stitching structured_light java python2 python3
  
  --     Disabled:                    world contrib_world
  
  --     Disabled by dependency:      -
  
  --     Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev cnn_3dobj matlab sfm
  
  -- 
  
  --   GUI: 
  
  --     QT 5.x:                      YES (ver 5.5.1)
  
  --     QT OpenGL support:           YES (Qt5::OpenGL 5.5.1)
  
  --     OpenGL support:              YES (/usr/lib/x86_64-linux-gnu/libGLU.so /usr/lib/x86_64-linux-gnu/libGL.so)
  
  --     VTK support:                 YES (ver 6.2.0)
  
  -- 
  
  --   Media I/O: 
  
  --     ZLib:                        /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.8)
  
  --     JPEG:                        /usr/lib/x86_64-linux-gnu/libjpeg.so (ver )
  
  --     WEBP:                        /usr/lib/x86_64-linux-gnu/libwebp.so (ver encoder: 0x0202)
  
  --     PNG:                         /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.2.54)
  
  --     TIFF:                        /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 - 4.0.6)
  
  --     JPEG 2000:                   /usr/lib/x86_64-linux-gnu/libjasper.so (ver 1.900.1)
  
  --     OpenEXR:                     /usr/lib/x86_64-linux-gnu/libImath.so /usr/lib/x86_64-linux-gnu/libIlmImf.so /usr/lib/x86_64-linux-gnu/libIex.so /usr/lib/x86_64-linux-gnu/libHalf.so /usr/lib/x86_64-linux-gnu/libIlmThread.so (ver 2.2.0)
  
  --     GDAL:                        /usr/lib/libgdal.so
  
  --     GDCM:                        NO
  
  -- 
  
  --   Video I/O:
  
  --     DC1394 1.x:                  NO
  
  --     DC1394 2.x:                  YES (ver 2.2.4)
  
  --     FFMPEG:                      YES
  
  --       avcodec:                   YES (ver 56.60.100)
  
  --       avformat:                  YES (ver 56.40.101)
  
  --       avutil:                    YES (ver 54.31.100)
  
  --       swscale:                   YES (ver 3.1.101)
  
  --       avresample:                NO
  
  --     GStreamer:                   NO
  
  --     OpenNI:                      NO
  
  --     OpenNI PrimeSensor Modules:  NO
  
  --     OpenNI2:                     NO
  
  --     PvAPI:                       NO
  
  --     GigEVisionSDK:               NO
  
  --     Aravis SDK:                  NO
  
  --     UniCap:                      NO
  
  --     UniCap ucil:                 NO
  
  --     V4L/V4L2:                    NO/YES
  
  --     XIMEA:                       NO
  
  --     Xine:                        YES (ver 1.2.6)
  
  --     gPhoto2:                     NO
  
  -- 
  
  --   Parallel framework:            TBB (ver 4.4 interface 9002)
  
  -- 
  
  --   Other third-party libraries:
  
  --     Use IPP:                     9.0.1 [9.0.1]
  
  --          at:                     /home/me/code/myproject/OpenCV-3.2.0/build/3rdparty/ippicv/ippicv_lnx
  
  --     Use IPP Async:               NO
  
  --     Use VA:                      NO
  
  --     Use Intel VA-API/OpenCL:     NO
  
  --     Use Lapack:                  NO
  
  --     Use Eigen:                   YES (ver 3.2.92)
  
  --     Use Cuda:                    NO
  
  --     Use OpenCL:                  YES
  
  --     Use OpenVX:                  NO
  
  --     Use custom HAL:              NO
  
  -- 
  
  --   OpenCL:                        <dynamic loading="loading" of="of" opencl="opencl" library="library">
  
  --     Include path:                /home/me/code/myproject/OpenCV-3.2.0/3rdparty/include/opencl/1.2
  
  --     Use AMDFFT:                  NO
  
  --     Use AMDBLAS:                 NO
  
  -- 
  
  --   Python 2:
  
  --     Interpreter:                 /home/me/.envs/myenv/bin/python (ver 2.7.12)
  
  --     Libraries:                   /usr/lib/x86_64-linux-gnu/libpython2.7.so (ver 2.7.12)
  
  --     numpy:                       /home/me/.envs/myenv/local/lib/python2.7/site-packages/numpy/core/include (ver 1.13.1)
  
  --     packages path:               lib/python2.7/site-packages
  
  -- 
  
  --   Python 3:
  
  --     Interpreter:                 /usr/bin/python3 (ver 3.5.2)
  
  --     Libraries:                   /usr/lib/x86_64-linux-gnu/libpython3.5m.so (ver 3.5.2)
  
  --     numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.11.0)
  
  --     packages path:               lib/python3.5/dist-packages
  
  -- 
  
  --   Python (for build):            /home/me/.envs/myenv/bin/python
  
  -- 
  
  --   Java:
  
  --     ant:                         /usr/bin/ant (ver 1.9.6)
  
  --     JNI:                         /usr/lib/jvm/default-java/include /usr/lib/jvm/default-java/include/linux /usr/lib/jvm/default-java/include
  
  --     Java wrappers:               YES
  
  --     Java tests:                  YES
  
  -- 
  
  --   Matlab:                        Matlab not found or implicitly disabled
  
  -- 
  
  --   Documentation:
  
  --     Doxygen:                     /usr/bin/doxygen (ver 1.8.11)
  
  -- 
  
  --   Tests and samples:
  
  --     Tests:                       YES
  
  --     Performance tests:           YES
  
  --     C/C++ Examples:              YES
  
  -- 
  
  --   Install path:                  /usr/local
  
  -- 
  
  --   cvconfig.h is in:              /home/me/code/myproject/OpenCV-3.2.0/build
  
  -- -----------------------------------------------------------------
  
  -- 
  
  </dynamic>

  Unfortunately, while this works and I can import cv2 in the shell, it cannot read video using the above script, probably due to incorrect compilation or linking of ffmpeg ? The confusing part is the system-wide installation of OpenCV works fine, even without ffmpeg installed !

  What am I doing wrong ? How can I get OpenCV working with a virtualenv ?

  ====

  EDIT : Running the C++ video writing example has this result :

  $ cd /home/me/code/myproject/OpenCV-3.2.0/build/bin
  
  $ ./cpp-tutorial-video-write ../../samples/data/vtest.avi R Y
  
  ------------------------------------------------------------------------------
  
  This program shows how to write video files.
  
  You can extract the R or G or B color channel of the input video.
  
  Usage:
  
  ./video-write  [ R | G | B] [Y | N]
  
  ------------------------------------------------------------------------------
  
  
  
  OpenCV: FFMPEG: tag 0xffffffff/'����' is not found (format 'avi / AVI (Audio Video Interleaved)')'
  
  
  
  (cpp-tutorial-video-write:19523): GStreamer-CRITICAL **: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
  
  OpenCV Error: Unsupported format or combination of formats (Gstreamer Opencv backend does not support this codec.) in CvVideoWriter_GStreamer::open, file /home/me/code/myproject/OpenCV-3.2.0/modules/videoio/src/cap_gstreamer.cpp, line 1388
  
  VIDEOIO(cvCreateVideoWriter_GStreamer(filename, fourcc, fps, frameSize, is_color)): raised OpenCV exception:
  
  
  
  /home/me/code/myproject/OpenCV-3.2.0/modules/videoio/src/cap_gstreamer.cpp:1388: error: (-210) Gstreamer Opencv backend does not support this codec. in function CvVideoWriter_GStreamer::open
  
  
  
  Could not open the output video for write: ../../samples/data/vtest.avi

  And the opencv_test_videoio unit test reports the following : https://pastebin.com/q4mf224Q

  However, running the c++ video starter example DOES work, with the following command and output, I can see the webcam working and streaming video in the highgui interface :

  $ ./cpp-example-videocapture_starter 0
  
  VIDEOIO ERROR: V4L: device 0: Unable to query number of channels
  
  (ERROR)icvOpenAVI_XINE(): Unable to initialize video driver.
  
  GStreamer: Error opening bin: no element "0"
  
  press space to save a picture. q or esc to quit
  
  init done
  
  opengl support available

• Understanding Data Processing Agreements and How They Affect GDPR Compliance

  9 octobre 2023, par Erin — GDPR

  The General Data Protection Regulation (GDPR) impacts international organisations that conduct business or handle personal data in the European Union (EU), and they must know how to stay compliant.

  One way of ensuring GDPR compliance is through implementing a data processing agreement (DPA). Most businesses overlook DPAs when considering ways of maintaining user data security. So, what exactly is a DPA’s role in ensuring GDPR compliance ?

  In this article, we’ll discuss DPAs, their advantages, which data protection laws require them and the clauses that make up a DPA. We’ll also discuss the consequences of non-compliance and how you can maintain GDPR compliance using Matomo.

  What is a data processing agreement ?

  A data processing agreement, data protection agreement or data processing addendum is a contractual agreement between a data controller (a company) and a data processor (a third-party service provider.) It defines each party’s rights and obligations regarding data protection.

  A DPA also defines the responsibilities of the controller and the processor and sets out the terms they’ll use for data processing. For instance, when MHP/Team SI sought the services of Matomo (a data processor) to get reliable and compliant web analytics, a DPA helped to outline their responsibilities and liabilities.

  A DPA is one of the basic requirements for GDPR compliance. The GDPR is an EU regulation concerning personal data protection and security. The GDPR is binding on any company that actively collects data from EU residents or citizens, regardless of their location.

  As a business, you need to know what goes into a DPA to identify possible liabilities that may arise if you don’t comply with European data protection laws. For example, having a recurrent security incident can lead to data breaches as you process customer personal data.

  The average data breach cost for 2023 is $4.45 million. This amount includes regulatory fines, containment costs and business losses. As such, a DPA can help you assess the organisational security measures of your data processing methods and define the protocol for reporting a data breach.

  Why is a DPA essential for your business ?

  If your company processes personal data from your customers, such as contact details, you need a DPA to ensure compliance with data security laws like GDPR. You’ll also need a DPA to hire a third party to process your data, e.g., through web analytics or cloud storage.

  But what are the benefits of having a DPA in place ?

  

  A key benefit of signing a DPA is it outlines business terms with a third-party data processor and guarantees compliance with the relevant data privacy laws. A DPA also helps to create an accountability framework between you and your data processor by establishing contractual obligations.

  Additionally, a DPA helps to minimise the risk of unauthorised access to sensitive data. A DPA defines organisational measures that help protect the rights of individuals and safeguard personal data against unauthorised disclosure. Overall, before choosing a data processor, having a DPA ensures that they are capable, compliant and qualified.

  More than 120 countries have already adopted some form of international data protection laws to protect their citizens and their data better. Hence, knowing which laws require a DPA and how you can better ensure compliance is important.

  Which data protection laws require a DPA ?

  Regulatory bodies enact data protection laws to grant consumers greater control over their data and how businesses use it. These laws ensure transparency in data processing and compliance for businesses.

  

  The following are some of the relevant data privacy laws that require you to have a DPA :

  • UK GDPR
  • Brazil LGPD
  • EU GDPR
  • Dubai PDPA
  • Colorado CPA
  • California CCPA/CPRA
  • Virginia VCDPA
  • Connecticut DPA
  • South African POPIA
  • Thailand PDPA

  Companies that don’t adhere to these data protection obligations usually face liabilities such as fines and penalties. With a DPA, you can set clear expectations regarding data processing between you and your customers.

  Review and update any DPAs with third-party processors to ensure compliance with GDPR and the laws we mentioned above. Additionally, confirm that all the relevant clauses are present for compliance with relevant data privacy laws. 

  So, what key data processing clauses should you have in your DPA ? Let’s take a closer look in the next section.

  Key clauses in a data processing agreement

  GDPR provides some general recommendations for what you should state in a DPA.

  

  Here are the elements you should include :

  Data processing specifications

  Your DPA should address the specific business purposes for data processing, the duration of processing and the categories of data under processing. It should also clearly state the party responsible for maintaining GDPR compliance and who the data subjects are, including their location and nationality.

  Your DPA should also address the data processor and controller’s responsibilities concerning data deletion and contract termination.

  Role of processor

  Your DPA should clearly state what your data processor is responsible for and liable for. Some key responsibilities include record keeping, reporting breaches and maintaining data security.

  Other roles of your data processor include providing you with audit opportunities and cooperating with data protection authorities during inquiries. If you decide to end your contract, the data processor is responsible for deleting or returning data, depending on your agreement.

  Role of controller

  Your DPA should inform the responsibilities of the data controller, which typically include issuing processing instructions to the data processor and directing them on how to handle data processing.

  Your DPA should let you define the lawful data processes the data processor should follow and how you’ll uphold the data protection rights of individuals’ sensitive data.

  Organisational and technical specifications

  Your DPA should define specifications such as how third-party processors encrypt, access and test personal data. It should also include specifications on how the data processor and controller will maintain ongoing data security through various factors such as :

  • State of the technology : Do ‌third-party processors have reliable technology, and can they ensure data security within their systems ?
  • Costs of implementation : Does the data controller’s budget allow them to seek third-party services from industry-leading providers who can guarantee a certain level of security ?
  • Variances in users’ personal freedom : Are there privacy policies and opt-out forms for users to express how they want companies to use their sensitive data ?

  Moreover, your DPA should define how you and your data processor will ensure the confidentiality, availability and integrity of data processing services and systems.

  What are the penalties for DPA GDPR non-compliance ?

  Regulators use GDPR’s stiff fines to encourage data controllers and third-party processors to follow‌ best data security practices. One way of maintaining compliance is through drafting up a DPA with your data processor.

  The DPA should clearly outline the necessary legal requirements and include all the relevant clauses mentioned above. Understand what goes into this agreement since data protection authorities can hold your business accountable for a breach — even if a processor’s error caused it.

  Data protection authorities can issue penalties now that the GDPR is in place. For example, according to Article 83 of the GDPR, penalties for data or privacy breaches or non-compliance can amount to up to €20 million or 4% of your annual revenue.

  There are two tiers of fines : tier one and tier two. Violations related to data processors typically attract fines on the tier-one level. Tier one fines can cost your business €10 million or 2% of your company’s global revenue.

  Tier-two fines result from infringement of the right to forget and the right to privacy of your consumer. Tier-two fines can cost your business up to €20 million or 4% of your company’s global revenue.

  GDPR fines make non-compliance an expensive mistake for businesses of all sizes. As such, signing a DPA with any party that acts as a data processor for your business can help you remain GDPR-compliant.

  How a DPA can help your business remain GDPR compliant

  A DPA can help your business define and adhere to lawful data processes.

  

  So, in what other ways can a DPA help you to remain compliant with GDPR ? Let’s take a look !

  1. Assess data processor’s compliance

  Having a DPA helps ensure that the data processor you are working with is GDPR-compliant. You should check if they have a DPA and confirm the processor’s terms of service and legal basis.

  For example, if you want an alternative to Google Analytics that’s GDPR compliant, then you can opt for Matomo. Matomo features a DPA, which you can agree to when you sign up for web analytics services or later.

  2. Establish lawful data processes

  A DPA can also help you review your data processes to ensure they’re GDPR compliant. For example, by defining lawful data processes, you better understand personally identifiable information (PII) and how it relates to data privacy.

  Further, you can allow users to opt out of sharing their data. As such, Matomo can help you to enable Do Not Track preferences on your website.

  

  With this feature, users are given the option to opt in or out of tracking via a toggle in their respective browsers.

  Indeed, establishing lawful data processes helps you define the specific business purposes for collecting and processing personal data. By doing so, you get to notify your users why you need their data and get their consent to process it by including a GDPR-compliant privacy policy on your website.

  3. Anonymise your data

  Global privacy laws like GDPR and ePrivacy mandate companies to display cookie banners or seek consent before tracking visitors’ data. You can either include a cookie consent banner on your site or stop tracking cookies to follow the applicable regulations.

  Further, you can enable cookie-less tracking or easily let users opt out. For example, you can use Matomo without a cookie consent banner, exempting it from many countries’ privacy rules.

  Additionally, through a DPA, you can define organisational measures that define how you’ll anonymise all your users’ data. Matomo can help you anonymise IP addresses, and we recommend that you at least anonymise the last two bytes.

  

  As one of the few web analytics tools you can use to collect data without tracking consent, Matomo also has the French Data Protection Authority (CNIL) approval.

  4. Assess the processor’s bandwidth

  Having a DPA can help you implement data retention policies that show clear retention periods. Such policies are useful when ending a contract with a third-party service provider and determining how they should handle your data.

  A DPA also helps you ensure the processor has the necessary technology to store personal data securely. You can conduct an audit to understand possible vulnerabilities and your data processor’s technological capacity.

  5. Obtain legal counsel

  When drafting a DPA, it’s important to get a consultation on what is needed to ensure complete compliance. Obtaining legal counsel points you in the right direction so you don’t make any mistakes that may lead to non-compliance.

  Conclusion

  Businesses that process users’ data are subject to several DPA contract requirements under GDPR. One of the most important is having DPAs with every third-party provider that helps them perform data processing.

  It’s important to stay updated on GDPR requirements for compliance. As such, Matomo can help you maintain lawful data processes. Matomo gives you complete control over your data and complies with GDPR requirements.

  To get started with Matomo, you can sign up for a 21-day free trial. No credit card required.

  Disclaimer

  We are not lawyers and don’t claim to be. The information provided here is to help give an introduction to GDPR. We encourage every business and website to take data privacy seriously and discuss these issues with your lawyer if you have any concerns.