Recherche avancée

Sur d’autres sites (10173)

• Playing With File

  8 septembre 2011, par Multimedia Mike — General

  I played with the ‘file’ utility a long time ago because I wanted to make it recognize a large number of multimedia formats. I had trouble getting my changes to take. But I’m prepared to try again after many years.

  Aiming at the Corpus
  In my local mirror of the MPlayerHQ samples archive, I find 9853 unique files. So I run all of them through the ‘file’ command :

    ’find /path/to/samples -type f -print0 | xargs -0 file —no-pad’
  

  My Ubuntu installation has file v5.04. I also tested against 5.07 and the latest, 5.08. Here is the number of files each version was unable to identify (generically marking as ‘data’) :

  5.04  1521
  5.07  1405
  5.08  1501
  

  That seems like a regression for v5.08 until I dug into the details and saw quite a few items like this, indicating that the MPEG detection could use some work :

  
   mov/mov-demux-infinite-loop.mpg : DOS-executable (
  +mov/mg-‹demux-infinite-loop.mpg : data
  
   image-samples/UNeedQT4.pntg : DOS-executable (
  +imY- samples/UNeedQT4.pntg : data
  

  Workflow
  These are just notes to myself and perhaps anyone else who wants to add new file formats to be identified by the ‘file’ command.

  First, download either the latest release from the FTP or clone from Github. Do the usual unpack, ‘./configure’, ‘make’ routine. To use this newly-built version and its associated magic file :

    ./src/file —magic-file magic/magic.mgc <file>
  

  To add a new format for ID, first, run the foregoing command to ensure that it’s not already identified. Then, check over the files in magic/Magdir and see which one might pertain to what you’re doing (it’s unlikely that your format will merit a new file in this directory). For example, for this round, I modified animation, audio, iff, and riff. Add or modify existing specs based on the copious examples in the directory and by consulting the appropriate man page (‘man 5 magic’).

  Finally, run ‘make’ again which will regenerate the magic file. Invoke the above command again to use the modified magic file.

  Before and After
  On a selection of formats taken from the samples archive (renamed and cut down to a kilobyte because detection typically only relies on the first few bytes), here is the “before” :

  amv :            RIFF (little-endian) data
  armovie :        data
  bbc-dirac :      data
  interplay-mve :  data
  mtv :            data
  nintendo-thp :   data
  nullsoft-video : data
  redcode :        data
  sega-film :      data
  smacker :        data
  trueaudio :      data
  vqa :            IFF data
  wavpack :        data
  wc3-mve :        IFF data
  wtv :            data
  

  And the “after” :

  amv :            RIFF (little-endian) data, AMV
  armovie :        ARMovie
  bbc-dirac :      BBC Dirac Video
  interplay-mve :  Interplay MVE Movie
  mtv :            MTV Multimedia File
  nintendo-thp :   Nintendo THP Multimedia
  nullsoft-video : Nullsoft Video
  redcode :        REDCode Video
  sega-film :      Sega FILM/CPK Multimedia, 320 x 224
  smacker :        RAD Game Tools Smacker Multimedia version 2, 320 x 200, 100 frames
  trueaudio :      True Audio Lossless Audio
  vqa :            IFF data, Westwood Studios VQA Multimedia, 418 video frames, 320 x 200
  wavpack :        WavPack Lossless Audio
  wc3-mve :        IFF data, Wing Commander III Video, PC version
  wtv :            Windows Television DVR Media
  

  After rerunning ‘file’ on the mphq corpus using the modified magic file, only 1329 files remain unidentified (down from 1501).

  Going Forward
  As mentioned, MPEG detection could probably be strengthened. However, a major weakness is QuickTime/MP4. Many files are not detected, probably owing to the many ways that QuickTime files can begin.

• What’s So Hard About Building ?

  10 septembre 2011, par Multimedia Mike — Programming

  I finally had a revelation as to why so building software can be so difficult– because build systems are typically built on programming languages that you don’t normally use in your day to day programming activities. If the project is simple enough, the build system usually takes care of the complexities. If there are subtle complexities — and there always are — then you can to figure out how to customize the build system to meet your needs.

  First, there’s the Makefile. It’s easy to forget that the syntax which comprises a Makefile pretty well qualifies as a programming language. I wonder if it’s Turing-complete ? But writing and maintaining Makefiles manually is arduous and many systems have been created to generate Makefiles for you. At the end of the day, running ‘make’ still requires the presence of a Makefile and in the worst case scenario, you’re going to have to inspect and debug what was automatically generated for that Makefile.

  So there is the widespread GNU build system, a.k.a., “the autotools”, named due to its principle components such as autoconf and automake. In this situation, you have no fewer than 3 distinct languages at work. You write your general build instructions using a set of m4 macros (language #1). These get processed by the autotools in order to generate a shell script (language #2) called configure. When this is executed by the user, it eventually generates a Makefile (language #3).

  Over the years, a few challengers have attempted to dethrone autotools. One is CMake which configures a project using its own custom programming language that you will need to learn. Configuration generates a standard Makefile. So there are 2 languages involved in this approach.

  Another option is SCons, which is Python-based, top to bottom. Only one programming language is involved in the build system ; there’s no Makefile generated and run. Until I started writing this, I was guessing that the Python component generated a Makefile, but no.

  That actually makes SCons look fairly desirable, at least if your only metric when choosing a build system is to minimize friction against rarely-used programming languages.

  I should also make mention of a few others : Apache Ant is a build system in which the build process is described by an XML file. XML doesn’t qualify as a programming language (though that apparently doesn’t stop some people from using it as such). I see there’s also qmake, related to the Qt system. This system uses its own custom syntax.

• How to Choose a GDPR Compliant Web Analytics Solution

  2 mars 2022, par Matthieu Aubry — Privacy

  Since the launch of GDPR, one big question has lingered around with uncertainty – is Google Analytics GDPR compliant ? The current GDPR enforcement trend happening across the EU is certainly shedding some light on this question.

  Starting with the Austrian Data Protection Authority’s ruling on Google Analytics and more recently, CNIL (the French Data Protection Authority) has followed suit by also ruling Google Analytics illegal to use. Organisations with EU-based web visitors are now scrambling to find a compliant solution.

  The French Data Protection Authority (CNIL) has already started delivering formal notices to websites using Google Analytics, so now is the time to act. According to CNIL, organisations have two options :

  1. Ceasing use of the Google Analytics functionality (under the current conditions) 
  2. Use a compliant web analytics tool that does not transfer data outside the EU

  Getting started 

  For organisations considering migrating to a compliant web analytics tool, I’ve outlined below the things you need to consider when weighing up compliant web analytics tools. Once you’ve made a choice, I’ve also included a step-by-step guide to migrating away from Google Analytics. This guide is useful regardless of which GDPR compliant analytics provider you choose.

  Before getting started, I recommend that you document your findings against the following considerations while reviewing GDPR compliant Google Analytics alternatives. This document can then be shared with your Data Protection Officer (DPO) to get their final recommendation.

  10 key considerations when selecting a GDPR compliant web analytics tools

  Many tools will claim to be GDPR compliant so it’s important that you do your due diligence and review tools against the following considerations. 

  1. Where does the tool store data ? 

  The rulings in France and Austria were based on the fact that Google Analytics stores data in the US, which does not have an adequate level of data protection. Your safest option is to find a tool that legally stores data in the EU.

  You should be able to find out where the data is stored in the organisation’s privacy policy. Generally, data storage information can be found under sections titled “Subprocessors” and “Third-party services”. Check out the Matomo Privacy Policy as an example. 

  If you’re unable to easily find this information or it’s unclear, reach out to the organisation for more information.

  2. Does the tool offer anonymous tracking ?

  Anonymous tracking comes with many benefits, including :

  • The ability to track visitors without a cookie consent screen. Due to the privacy-respecting aspect of cookieless tracking, you don’t need to worry about the extra steps involved with compliant cookie banners.
  • More accurate data. When visitors deny tracking cookies, you lose out on valuable data. With anonymous tracking there is no data lost as you don’t need consent to track.
  • Simplified GDPR compliance. With this enabled, there are fewer steps you need to take to get GDPR compliant and stay GDPR compliant.

  For those reasons, it may be important for you to select a tool that offers anonymous tracking functionalities. The level of anonymous tracking you require will depend on your situation but you should look out for tools that allow you to :

  • Disable fingerprinting 
  • Disable user profiles 
  • Anonymise data
  • Cookieless tracking

  If you want to read more about data anonymization, check out this guide on data anonymization in web analytics.

  3. Does the tool integrate with my existing tech stack ?

  You’ll want to ensure that a new web analytics tool will play well with other tools in your tech stack including things like your CMS (content management system), eCommerce shop, etc. You should list out all the existing tools that currently integrate with your Google Analytics and check that the same integrations can be re-created with the new tool, via integrations or APIs.

  If not, it could become costly trying to connect your existing tech stack to a new solution.

  4. Does the tool offer the same features and insights you are currently using in Google Analytics ? Or more, if necessary ? 

  Just because you are moving to a new web analytics platform, doesn’t mean you have to give up the insights, reports and features you’ve grown accustomed to with Google Analytics. Ensuring that a new platform provides the same features and reports that you value the most will result in a smoother transition away from Google Analytics.

  It’s unlikely that a new tool will have all of the same features as Google Analytics, so I’d recommend listing out and prioritising your business-critical features and reports. 

  If I had to guess, you probably set up Google Analytics years ago because it was the default option. Now is your chance to make the most of this switch from Google Analytics and find a tool that offers additional reports and features that better aligns with your business. If time permits, I’d highly recommend that you consider other features or reports that you might have been missing out on while using Google Analytics.

  Check out this comparison of Google Analytics vs Matomo to see side-by-side feature comparison.

  5. Does the tool accept Google Analytics data imports ? 

  The historical data in Google Analytics is a critical asset for many businesses. Fortunately, some tools accept Google Analytics data imports so you don’t lose all of the data you’ve generated over time.

  However, it’s important to note that any data you import from Google Analytics to a new tool needs to be compliant data. I’ll cover this more below.

  6. Does the tool provide conversion tracking exports ? 

  Do you invest in paid advertising ? If you do, then tracking the conversions from people clicking on these paid ads is critical in assessing your return on investment. Since sending IP addresses or other personal information to the US is illegal under GDPR, we can only assume that this will also apply to advertising pixel/conversion tracking (e.g., Facebook pixel, Google Ads conversion tracking, etc). 

  As an example, Matomo offers conversion tracking exports so you can get a better understanding of ad performance while meeting privacy laws and without requiring consent from users. See how it works with Matomo’s conversion tracking exports. 

  7. How will you train up your in-house team ? Or can you hire a contractor ?

  This is a common concern of many, and rightfully so. You’ll want to confirm what resources are readily available so you can hit the ground running with your new web analytics tool. If you’d prefer to train up your in-house team, check the provider’s site for training resources, videos, guides, etc.

  If you’d rather hire an external contractor, we recommend heading to LinkedIn, reaching out to your community or asking the provider if they have any recommendations for contractors.

  In addition, check that the provider offers technical support or a forum, in case you have specific questions and need help.   

  8. Does the tool offer self-hosting ? (optional)

  For organisations that want full control over their data and storage location, an on-premise web analytics tool will be the preferred option. From a GDPR perspective, this is also the easiest option for compliance.

  Keep in mind that this requires resources, regular maintenance, technical knowledge and/or technical consultants. If you’re unsure which option is best for your organisation, check out our on-premise vs cloud web analytics comparison breakdown.

  Find out more about self-hosting Matomo.

  9. Is the tool approved by the CNIL for tracking without consent ?

  This is an important step for websites with French users. This step will help narrow down your selection of tools. The CNIL offers a programme to identify web analytics solutions that can be used without tracking consent. The CNIL’s list of recommended web analytics tools can act as your starting point for solutions to review.

  While this step is specific to sites with French users, it can also be helpful for websites with visitors from any other EU country.

  Benefits of consent-free tracking

  There are many benefits of tracking without consent.
  

  For one, it simplifies GDPR compliance and reduces the chances of GDPR breaches and fines. Cookie consent screens have recently been the target for EU Data Protection Authorities because many websites are unknowingly serving cookie consent screens that do not meet GDPR requirements. 

  Yet another benefit, and quite possibly the most important is more accurate data. Even if a website displays a user-friendly, lawful consent screen, the majority of users will either ignore or reject cookie consent. Legally website owners can’t track anything unless the visitor gives consent. So not having a cookie consent screen ensures that every visit is tracked and your web analytics data is 100% accurate. 

  Lastly, many visitors have grown fatigued and frustrated with invasive cookie consent screens. Not having one on your site creates a user-friendly experience, which will likely result in longer user sessions and lower bounce rates.

  10. Does the tool offer a Data Processing Agreement (DPA) ? 

  Technically, any GDPR compliant web analytics tool should offer a DPA but for the sake of completeness, I’ve added this as a consideration. Double check that any tools you are looking at provide this legally binding document. This should be located in the Privacy Policy of the web analytics provider, if not reach out to request it.

  As an example, here’s Matomo’s Data Processing Agreement which can be found in our Privacy Policy under Subprocessors. 

  That wraps up the key considerations. When it comes to compliance, privacy and customer data, Matomo leads the way. We are looking forward to helping you achieve GDPR compliance easily. Start your free 21-day trial of Matomo now – no credit card required.

  Try Matomo Cloud

  A step-by-step guide to migrating from Google Analytics

  Once you’ve identified a tool that suits your needs and your Data Protection Officer (DPO) has approved, you’re ready to get started. Here’s a simple step-by-step guide with all the important steps for you to follow :

  1. Before getting started, you should sign or download the Data Processing Agreement (DPA) offered by your new web analytics provider.

  2. Register for the new tool and configure it for compliance. The provider should offer guides on how to configure for GDPR compliance. This will include things like giving your users an easy way to opt-out of all tracking, turning on cookieless tracking or asking users for consent and anonymizing data and IP addresses, for instance.

  3. Inform your organisation about the change. Whether your colleagues use the tool or not, it’s important that you share information about the new tool with your staff. Let them know what the tool will be used for, who will use the tool and how it complies with GDPR. 

  4. Let your DPO know that you’ve removed Google Analytics and have implemented the new tool.

  5. Update your records of processing activities to include the new tool.

  6. Update your privacy policy. You’ll need to include details about the web analytics provider, where the data is stored, what data is being collected, how long the data will be stored and why the data is being collected. The web analytics tool should readily have this information for you.

  As an example, if you decide to use Matomo as your web analytics tool, we provide a Privacy Policy template for you to use on your site and a guide on how to complete your privacy policy under GDPR with Matomo. Note that these are only applicable if you are using Matomo.

  In addition, if the tool has an opt-out feature, you will also need to put the opt-out into the privacy policy (e.g., when using cookieless tracking).

  7. Now, the exciting part. Add the tracking code to your site by following the steps provided by the web analytics tool. 

  If you’re not comfortable with this step, the provider should offer steps to do this and you can share this with your web developer.

  8. Once added, login to your tool and check to see if traffic is being tracked.

  9. If your tool does not offer Google Analytics data imports or you do not need the historical data in your new tool, go to step 11. 

  To plan for your Google Analytics data migration, you’ll first need to establish what historical data is compliant with GDPR.

  For example, you shouldn’t import any data stored beyond the retention period established in your Privacy Policy or any personally identifiable information (PII) like IP addresses that aren’t anonymised. Discuss this further with your DPO.

  10. Once you’ve established what data you can legally import, then you can begin the import. Follow the steps provided by your new web analytics solution provider.

  11. Remove Google Analytics tracking code from your site. This will stop the collection of your visitors data by Google as well as slightly increase the page load speed.

  If you still haven’t made a choice yet, try Matomo free for 21-days and see why over 1 million websites choose Matomo.