Recherche avancée

Sur d’autres sites (9512)

• A Comprehensive Guide to Robust Digital Marketing Analytics

  15 novembre 2023, par Erin — Analytics Tips

  First impressions are everything. This is not only true for dating and job interviews but also for your digital marketing strategy. Like a poorly planned job application getting tossed in the “no thank you” pile, 38% of visitors to your website will stop engaging with your content if they find the layout unpleasant. Thankfully, digital marketers can access data that can be harnessed to optimise websites and turn those “no thank you’s” into “absolutely’s.”

  So, how can we transform raw data into valuable insights that pay off ? The key is web analytics tools that can help you make sense of it all while collecting data ethically. In this article, we’ll equip you with ways to take your digital marketing strategy to the next level with the power of web analytics.

  What are the different types of digital marketing analytics ?

  Digital marketing analytics are like a cipher into the complex behaviour of your buyers. Digital marketing analytics help collect, analyse and interpret data from any touchpoint you interact with your buyers online. Whether you’re trying to gauge the effectiveness of a new email marketing campaign or improve your mobile app layout, there’s a way for you to make use of the insights you gain.

  

  As we go through the eight commonly known types of digital marketing analytics, please note we’ll primarily focus on what falls under the umbrella of web analytics. 

  1. Web analytics help you better understand how users interact with your website. Good web analytics tools will help you understand user behaviour while securely handling user data. 
  2. Learn more about the effectiveness of your organisation’s social media platforms with social media analytics. Social media analytics include user engagement, post reach and audience demographics. 
  3. Email marketing analytics help you see how email campaigns are being engaged with.
  4. Search engine optimisation (SEO) analytics help you understand your website’s visibility in search engine results pages (SERPs). 
  5. Pay-per-click (PPC) or campaign analytics measure the performance of paid advertising campaigns.
  6. Content marketing analytics focus on how your content is performing with your audience. 
  7. Customer analytics helps organisations identify and examine buyer behaviour to retain the biggest spenders. 
  8. Mobile app analytics track user interactions within mobile applications. 

  Choosing which digital marketing analytics tools are the best fit for your organisation is not an easy task. When making these decisions, it’s critical to remember the ethical implications of data collection. Although data insights can be invaluable to your organisation, they won’t be of much use if you lose the trust of your users. 

  Tips and best practices for developing robust digital marketing analytics 

  So, what separates top-notch, robust digital marketing analytics from the rest ? We’ve already touched on it, but a big part involves respecting user privacy and ethically handling data. Data security should be on your list of priorities, alongside conversion rate optimisation when developing a digital marketing strategy. In this section, we will examine best practices for using digital marketing analytics while retaining user trust.

  

  Clear objectives

  Before comparing digital marketing analytics tools, you should define clear and measurable goals. Try asking yourself what you need your digital marketing analytics strategy to accomplish. Do you want to improve conversion rates while remaining data compliant ? Maybe you’ve noticed users are not engaging with your platform and want to fix that. Save yourself time and energy by focusing on the most relevant pain points and areas of improvement.

  Choose the right tools for the job

  Don’t just base your decision on what other people tell you. Take the tool for a test drive — free trials allow you to test features and user interfaces and learn more about the platform before committing. When choosing digital marketing analytics tools, look for ones that ensure data accuracy as well as compliance with privacy laws like GDPR.

  Don’t overlook data compliance

  GDPR ensures organisations prioritise data protection and privacy. You could be fined up to €20 million, or 4% of the previous year’s revenue for violations. Without data compliance practices, you can say goodbye to the time and money spent on digital marketing strategies. 

  Don’t sacrifice data quality and accuracy

  Inaccurate and low-quality data can taint your analysis, making it hard to glean valuable insights from your digital marketing analytics efforts. Many analytics tools only show sampled data or use AI and ML to fill data gaps, potentially compromising the accuracy and completeness of your analytics. 

  When your analytics are based on incomplete or inaccurate data, it’s like trying to assemble a puzzle with missing pieces—you might get a glimpse of the whole picture, but it’s never quite clear. Accurate data isn’t just helpful—it’s the backbone of smart marketing strategies. It lets you make confident decisions and enables precise targeting for greater impact.

  Communicate your findings

  Having insights is one thing ; effectively communicating complex data findings is just as important. Customise dashboards to display key metrics aligned with your objectives. Make sure to automate reports, allowing stakeholders to stay updated without manual intervention. 

  Understand the user journey

  To optimise your conversion rates, you need to understand the user journey. Start by analysing visitors interactions with your website — this will help you identify conversion bottlenecks in your sales or lead generation processes. Implement A/B testing for landing page optimisation, refining elements like call-to-action buttons or copy, and leverage Form Analytics to make informed, data-driven improvements to your forms.

  Continuous improvement

  Learn from the data insights you gain, and iterate your marketing strategies based on the findings. Stay updated with evolving web analytics trends and technologies to leverage new growth opportunities. 

  Why you need web analytics to support your digital marketing analytics toolbox

  You wouldn’t set out on a roadtrip without a map, right ? Digital marketing analytics without insights into how users interact with your website are just as useless. Used ethically, web analytics tools can be an invaluable addition to your digital marketing analytics toolbox. 

  The data collected via web analytics reveals user interactions with your website. These could include anything from how long visitors stay on your page to their actions while browsing your website. Web analytics tools help you gather and understand this data so you can better understand buyer preferences. It’s like a domino effect : the more you understand your buyers and user behaviour, the better you can assess the effectiveness of your digital content and campaigns. 

  Web analytics reveal user behaviour, highlighting navigation patterns and drop-off points. Understanding these patterns helps you refine website layout and content, improving engagement and conversions for a seamless user experience.

  

  Concrete CMS harnessed the power of web analytics, specifically Matomo’s Form Analytics, to uncover crucial insights within their user onboarding process. Their data revealed a significant issue : the “address” input field was causing visitors to drop off and not complete the form, severely impacting the overall onboarding experience and conversion rate.

  Armed with these insights, Concrete CMS made targeted optimisations to the form, resulting in a substantial transformation. By addressing the specific issue identified through Form Analytics, they achieved an impressive outcome – a threefold increase in lead generation.

  This case is a great example of how web analytics can uncover customer needs and preferences and positively impact conversion rates. 

  Ethical implications of digital marketing analytics

  As we’ve touched on, digital marketing analytics are a powerful tool to help better understand online user behaviour. With great power comes great responsibility, however, and it’s a legal and ethical obligation for organisations to protect individual privacy rights. Let’s get into the benefits of practising ethical digital marketing analytics and the potential risks of not respecting user privacy : 

  • If someone uses your digital platform and then opens their email one day to find it filled with random targeted ad campaigns, they won’t be happy. Avoid losing user trust — and facing a potential lawsuit — by informing users what their data will be used for. Give them the option to consent to opt-in or opt-out of letting you use their personal information. If users are also assured you’ll safeguard personal information against unauthorised access, they’ll be more likely to trust you to handle their data securely.
  • Protecting data against breaches means investing in technology that will let you end-to-end encrypt and securely store data. Other important data-security best practices include access control, backing up data regularly and network and physical security of assets.

  A fine line separates digital marketing analytics and misusing user data — many companies have gotten into big trouble for crossing it. (By big trouble, we mean millions of dollars in fines.) When it comes to digital marketing analytics, you should never cut corners when it comes to user privacy and data security. This balance involves understanding what data can be collected and what should be collected and respecting user boundaries and preferences.

  

  Learn more 

  We discussed a lot of facets of digital marketing analytics, namely how to develop a robust digital marketing strategy while prioritising data compliance. With Matomo, you can protect user data and respect user privacy while gaining invaluable insights into user behaviour with 100% accurate data. Save your organisation time and money by investing in a web analytics solution that gives you the best of both worlds. 

  If you’re ready to begin using ethical and robust digital marketing analytics on your website, try Matomo. Start your 21-day free trial now — no credit card required.

• Accessibility Testing : Why It Matters and How to Get Started

  7 mai 2024, par Erin

  Nearly 96% of website homepages had failures with meeting web accessibility criteria in 2024. Aside from not complying with web accessibility laws and regulations, companies are failing a growing number of users with accessibility needs.

  With disabilities, chronic illnesses and ageing populations all rising, brands need to take accessibility more seriously. 

  In this article, we explain why accessibility testing is so important and how you can get started today.

  What is accessibility testing ?

  Accessibility testing optimises digital experiences to make them accessible for users with a range of disabilities and impairments. This includes users with vision impairments, hearing loss, neurodivergence, motor disabilities and cognitive conditions.

  The goal is to create inclusive experiences for everyone by implementing UX principles that address the usability needs of diverse audiences.

  To help developers create accessible experiences, the World Wide Web Consortium (W3C) created the Web Content Accessibility Guidelines (WCAG). The international WCAG standards define the Four Principles of Accessibility :

  • Perceivable : Information and user interface components must be presentable to users in ways they perceive.

  • Operable : User interface components and navigation must be operable.

  • Understandable : Information and the operation of user interfaces must be understandable.
  • Robust : Content must be robust enough to be interpreted reliably by various user agents, including assistive technologies.

  

  The current version of WCAG (2.2) contains 86 success criteria with three grades representing conformance levels :

  • Level A is the minimum conformance rating, indicating that web content is accessible to most users.

  • Level AA is the recommended conformance level to make content accessible to almost everyone, including users with severe disabilities.
  • Level AAA is the highest conformance rating, making content accessible to everyone, regardless of disability.

  

  Why is accessibility testing important ?

  With record numbers of lawsuits over online accessibility cases, it’s clear that companies underestimate the importance of accessibility testing. Here are seven key reasons you should pay more attention to it :

  1. Create inclusive experiences : Above all, accessibility testing creates inclusive experiences for all users.
  2. Adhere to accessibility regulations : Accessibility laws in most major markets — including the EU web accessibility policy — make it illegal for companies to discriminate against users with disabilities.
  3. Social responsibility : Companies have an ethical responsibility to cater to all users and consumers. 57% say they’re more loyal to brands that commit to addressing social inequities.
  4. Accessibility needs are growing : 16% of the world’s population (1 in 6) experience significant disability and the number will continue to grow as ageing populations rise.
  5. Improve experiences for everyone : Accessibility improves experiences for all users — for example, 80% of UK viewers aged 18-25 (2021) watch content with subtitles enabled.
  6. Maximise marketing reach : Platforms like Google prioritise accessibility yearly, making accessible content and experiences more visible.
  7. Accessibility is profitable : Inclusive companies earn 1.6x more revenue, 2.6x more net income and 2x more profit, according to Accenture (PDF).

  

  Who needs inclusive UX ?

  Accessibility testing starts with understanding the usability needs of audiences with disabilities and impairments. Here’s a quick summary of the most common impairments and some of the needs they have in common :

  • Visual impairments : Users may rely on screen readers, magnification software, braille displays, etc. or require certain levels of contrast, text sizes and colour combinations to aid visibility.
  • Hearing impairments : Users may rely on closed captions and subtitles for video content, transcripts for multimedia content and visual alerts/notifications for updates.
  • Motor or mobility impairments : Users might rely on adaptive keyboards, voice recognition and other assistive devices.
  • Cognitive and neurological impairments : Users may rely on technologies like text-to-speech software or require simplified user interfaces, contrast designs, etc., to aid comprehension.
  • Speech impairments : Users may rely on speech recognition and dictation software for any interaction that requires them to speak (e.g., automated customer service machines).

  While accessibility tools can alleviate certain accessibility challenges, inclusive design can remove much of the burden from users. This can involve using plenty of contrast, careful font selection, increasing whitespace and plenty of other design choices.

  Refer to the latest version of the WCAG for further guidance.

  How to run accessibility testing

  Now that we’ve emphasised the importance of accessibility, let’s explain how you can implement your own accessibility testing strategy.

  Create your accessibility testing plan

  Careful planning is crucial for making accessibility testing affordable and profitable. This starts with identifying the assets you need to test and optimise. This may include :

  • Website or web app
  • Mobile app
  • Videos
  • Podcasts and audio
  • PDFs
  • Marketing emails

  Map out all the assets your target audience interacts with and bring them into your accessibility testing plan. Optimising your website for screen readers is great, but you don’t want to forget your marketing emails and exclude vision-impaired users.

  Once you’ve got a complete list of assets, identify the elements and interactions with each one that require accessibility testing. For example, on your website, you should optimise navigation, user interfaces, layouts, web forms, etc.

  You also need to consider the impact of device types. For example, how touchscreens change the experience for motor impairments.

  Now that you know the scope of your testing strategy, it’s time to define your accessibility standards. Use external frameworks like WCAG guidelines and relevant legal requirements to create an internal set of standards.

  Once your accessibility standards are complete, train your staff at every level. This includes designers, developers, and content creators — everyone who works on assets is included in your accessibility testing strategy.

  Implement your accessibility standards throughout the design and development phases. Aim to create the most inclusive experiences possible before the accessibility testing stage.

  Implement accessibility practices at every level

  Treating accessibility as an afterthought is the biggest mistake you can make. Aside from neglecting the importance of accessibility, it’s simply not affordable to create assets and then optimise them for accessibility.

  Instead, you need to implement accessibility standards in every design and development stage. This way, you create inclusive assets from the beginning, and accessibility testing flags minor fixes rather than overhauls.

  By extension, you can take lessons from accessibility tests and update your accessibility standards to improve the quality of future assets.

  Set clear specifications in your accessibility standards for everyone to follow. For example, content publishers should be responsible for adding alt-text to all images. Make designers responsible for following contrast guidelines when optimising elements like CTA buttons.

  

  Next, managers can review assets and check for accessibility standards before anything is signed off. This way, you achieve higher test accessibility scores, and most fixes should be minor.

  This is the key to making accessibility testing manageable and profitable.

  Automate accessibility testing

  Automation is the other big factor in making accessibility efficient. With the right tools, you can run tests periodically without any manual workload, collecting data and flagging potential issues at almost no cost.

  For example, you can run automated accessibility tests on your website every month to check for common issues. This might flag up pages without alt-text for images, colour issues on a new batch of landing pages or a sudden drop in mobile loading times.

  Every automated test you can run reduces the manual workload of optimising accessibility. This frees up more time for the manual tests that require the attention of accessibility experts. 

  • Free up time for accessibility tasks that require manual testing
  • Identify issues with new content, assets, code, etc. faster
  • Run automated accessibility testing on new CRO changes

  Schedule manual accessibility reviews

  While it’s important to automate as much accessibility testing as possible, most accessibility standards require some form of manual testing. If we use the WCAG standards as a guideline, more than 70% of success require manual review and verification, including :

  • Testing websites with a screen reader
  • Navigating apps by only using a keyword
  • Quality assessing closed captions and subtitles
  • Testing web forms for people using speech input
  • Checking conversion actions for users with mobility issues (CTAs, forms, payments, etc.)

  Yes, you can automatically check all images for alt-text, but simply providing alt-text isn’t enough. You also have to review alt-text to make sure they’re descriptive, accurate and informative about the experience.

  Once again, the best way to minimise your time spent on manual testing is to implement accessibility standards throughout design and development. Train your content publishers to create alt-text that meets your criteria and editors to review them before pieces are signed off. 

  This way, you should always have the required alt-text before the content reaches the accessibility testing stage. The same applies to video transcriptions, web forms, website navigation, etc.

  Building a culture of accessibility makes the testing process as efficient as possible.

  What tools do you need for accessibility testing ?

  Now that we’ve covered the key essentials of accessibility testing, let’s look at some of the best accessibility testing tools to help you implement your strategy.

  accessiBe : AI-powered accessibility testing automation

  accessiBe is an accessibility testing automation and management system. It incorporates two core products : accessWidget for automating UI accessibility and accessFlow as an all-in-one solution for developers.

  

  Key features :

  • Automated accessibility testing
  • Accessibility widget for easy optimisation
  • Product accessibility for web, mobile and native apps
  • AI-powered accessibility insights
  • Compliance with WCAG, EAA and more

  As explained earlier, automation is crucial for making accessibility testing efficient and profitable. With accessiBe, you can automate the first line of accessibility checks so testers only need to get involved when manual action is necessary.

  Maze : Intelligent usability testing software

  Maze is a usability testing system that uses AI and automation to enhance traditional qualitative testing. You can run automated tests on live websites, capture survey feedback and recruit users to test experiences with real people.

  

  Key features :

  • Live website testing
  • Feedback surveys
  • Usability interviews
  • Test recruitment
  • Automated analysis

  While traditional usability interviews can provide in-depth insights, they’re expensive, time-consuming and difficult to run at scale. Maze’s solution is a hybrid testing system that automates data capture and analysis while supporting real user testing in one system.

  Matomo : Empowering people with ethical web analytics

  Matomo is a web analytics solution that gives you 100% data ownership while respecting user privacy. Think of this as a Google Analytics alternative that doesn’t use your visitors’ data for advertising purposes.

  

  Key features :

  • Privacy-friendly and GDPR-compliant tracking
  • Conversion rate optimisation features like heatmaps, session recordings, A/B testing and more
  • Accurate, unsampled data – see 40-60% more data than other analytics tools that sample data
  • Open-source

  Accessibility starts with creating quality experiences for everyone. Matomo reliably captures 100% of the data you need to optimise experiences without losing their trust. Instead of handing their personal info to Google or other tech giants, you retain full data ownership — fully compliant with GDPR, CCPA, etc.

  Try Matomo free for 21-days (no credit card required), or speak to our sales team for more info on how Matomo can enhance your site’s user experience and support your accessibility testing strategy.

  Try Matomo for Free

  Get the web insights you need, without compromising data accuracy.

  Start for free

  No credit card required

  UserTesting : Video-based user testing software

  UserTesting is the more traditional system for running usability tests with real people. The platform helps you recruit users and manage usability tests with a series of sessions and video interviews.

  

  Key features :

  • Usability testing
  • Test recruitment
  • Live interviews
  • AI-powered insights
  • Usability services

  UserTesting is a slower, more expensive approach to testing experiences, but its video-based interviews allow you to have meaningful conversations with real users.

  Siteimprove : WCAG compliance testing

  Siteimprove automates website testing, accessibility and optimisation. It includes dedicated tools for checking WCAG and DCI compliance with an automated scoring system. This helps you keep track of scores and identify any accessibility and usability issues faster.

  

  Key features :

  • Automated accessibility checks
  • Inclusivity scores
  • Accessibility recommendations
  • Accessibility tracking
  • Marketing and revenue attribution
  • Usability insights

  Siteimprove provides a first line of accessibility testing with automated checks and practical recommendations. It also tracks accessibility scores, including ratings for all three WCAG compliance levels (A, AA and AAA).

  Find the value in accessibility testing

  Accessibility testing isn’t only a moral obligation ; it’s good business. Aside from avoiding fines and lawsuits, inclusive experiences are increasingly profitable. User bases with accessibility needs are only growing while non-disabled audiences are using accessibility resources like subtitles and transcripts in greater numbers.

  Accessibility improves everyone’s experiences, and this only does good things for conversion rates, revenue and profit.

  Start building your datasets for accessibility testing today with a Matomo 21-day free trial — no credit card required. Gain 100% ownership over your analytics data while complying with GDPR and other data privacy regulations.

  Try Matomo for Free

  21 day free trial. No credit card required.

  
  

  Your email address
  

  Your website address
  

  Your Analytics subdomain will be .matomo.cloud (change)

  Choose your analytics subdomain
  

  .matomo.cloud

  I accept the terms and conditions and data processing agreement (DPA)

  
  Your information will be used to create an account on our cloud service. For more information please consult our privacy policy.
  
  
  

  Start improving your websites now

  

• Developing MobyCAIRO

  26 mai 2021, par Multimedia Mike — General

  I recently published a tool called MobyCAIRO. The ‘CAIRO’ part stands for Computer-Assisted Image ROtation, while the ‘Moby’ prefix refers to its role in helping process artifact image scans to submit to the MobyGames database. The tool is meant to provide an accelerated workflow for rotating and cropping image scans. It works on both Windows and Linux. Hopefully, it can solve similar workflow problems for other people.

  As of this writing, MobyCAIRO has not been tested on Mac OS X yet– I expect some issues there that should be easily solvable if someone cares to test it.

  The rest of this post describes my motivations and how I arrived at the solution.

  Background
  I have scanned well in excess of 2100 images for MobyGames and other purposes in the past 16 years or so. The workflow looks like this :

  
  

  

  Image workflow

  
  

  It should be noted that my original workflow featured me manually rotating the artifact on the scanner bed in order to ensure straightness, because I guess I thought that rotate functions in image editing programs constituted dark, unholy magic or something. So my workflow used to be even more arduous :

  
  

  

  I can’t believe I had the patience to do this for hundreds of scans

  
  

  Sometime last year, I was sitting down to perform some more scanning and found myself dreading the oncoming tedium of straightening and cropping the images. This prompted a pivotal question :

  
  Why can’t a computer do this for me ?
  

  After all, I have always been a huge proponent of making computers handle the most tedious, repetitive, mind-numbing, and error-prone tasks. So I did some web searching to find if there were any solutions that dealt with this. I also consulted with some like-minded folks who have to cope with the same tedious workflow.

  I came up empty-handed. So I endeavored to develop my own solution.

  Problem Statement and Prior Work
  
  I want to develop a workflow that can automatically rotate an image so that it is straight, and also find the most likely crop rectangle, uniformly whitening the area outside of the crop area (in the case of circles).

  As mentioned, I checked to see if any other programs can handle this, starting with my usual workhorse, Photoshop Elements. But I can’t expect the trimmed down version to do everything. I tried to find out if its big brother could handle the task, but couldn’t find a definitive answer on that. Nor could I find any other tools that seem to take an interest in optimizing this particular workflow.

  When I brought this up to some peers, I received some suggestions, including an idea that the venerable GIMP had a feature like this, but I could not find any evidence. Further, I would get responses of “Program XYZ can do image rotation and cropping.” I had to tamp down on the snark to avoid saying “Wow ! An image editor that can perform rotation AND cropping ? What a game-changer !” Rotation and cropping features are table stakes for any halfway competent image editor for the last 25 or so years at least. I am hoping to find or create a program which can lend a bit of programmatic assistance to the task.

  Why can’t other programs handle this ? The answer seems fairly obvious : Image editing tools are general tools and I want a highly customized workflow. It’s not reasonable to expect a turnkey solution to do this.

  Brainstorming An Approach
  I started with the happiest of happy cases— A disc that needed archiving (a marketing/press assets CD-ROM from a video game company, contents described here) which appeared to have some pretty clear straight lines :

  
  
  

  My idea was to try to find straight lines in the image and then rotate the image so that the image is parallel to the horizontal based on the longest single straight line detected.

  I just needed to figure out how to find a straight line inside of an image. Fortunately, I quickly learned that this is very much a solved problem thanks to something called the Hough transform. As a bonus, I read that this is also the tool I would want to use for finding circles, when I got to that part. The nice thing about knowing the formal algorithm to use is being able to find efficient, optimized libraries which already implement it.

  Early Prototype
  A little searching for how to perform a Hough transform in Python led me first to scikit. I was able to rapidly produce a prototype that did some basic image processing. However, running the Hough transform directly on the image and rotating according to the longest line segment discovered turned out not to yield expected results.

  
  
  

  It also took a very long time to chew on the 3300×3300 raw image– certainly longer than I care to wait for an accelerated workflow concept. The key, however, is that you are apparently not supposed to run the Hough transform on a raw image– you need to compute the edges first, and then attempt to determine which edges are ‘straight’. The recommended algorithm for this step is the Canny edge detector. After applying this, I get the expected rotation :

  
  
  

  The algorithm also completes in a few seconds. So this is a good early result and I was feeling pretty confident. But, again– happiest of happy cases. I should also mention at this point that I had originally envisioned a tool that I would simply run against a scanned image and it would automatically/magically make the image straight, followed by a perfect crop.

  Along came my MobyGames comrade Foxhack to disabuse me of the hope of ever developing a fully automated tool. Just try and find a usefully long straight line in this :

  
  

  Darn it, Foxhack…
  

  There are straight edges, to be sure. But my initial brainstorm of rotating according to the longest straight edge looks infeasible. Further, it’s at this point that we start brainstorming that perhaps we could match on ratings badges such as the standard ESRB badges omnipresent on U.S. video games. This gets into feature detection and complicates things.

  This Needs To Be Interactive
  At this point in the effort, I came to terms with the fact that the solution will need to have some element of interactivity. I will also need to get out of my safe Linux haven and figure out how to develop this on a Windows desktop, something I am not experienced with.

  I initially dreamed up an impressive beast of a program written in C++ that leverages Windows desktop GUI frameworks, OpenGL for display and real-time rotation, GPU acceleration for image analysis and processing tricks, and some novel input concepts. I thought GPU acceleration would be crucial since I have a fairly good GPU on my main Windows desktop and I hear that these things are pretty good at image processing.

  I created a list of prototyping tasks on a Trello board and made a decent amount of headway on prototyping all the various pieces that I would need to tie together in order to make this a reality. But it was ultimately slowgoing when you can only grab an hour or 2 here and there to try to get anything done.

  Settling On A Solution
  Recently, I was determined to get a set of old shareware discs archived. I ripped the data a year ago but I was blocked on the scanning task because I knew that would also involve tedious straightening and cropping. So I finally got all the scans done, which was reasonably quick. But I was determined to not manually post-process them.

  This was fairly recent, but I can’t quite recall how I managed to come across the OpenCV library and its Python bindings. OpenCV is an amazing library that provides a significant toolbox for performing image processing tasks. Not only that, it provides “just enough” UI primitives to be able to quickly create a basic GUI for your program, including image display via multiple windows, buttons, and keyboard/mouse input. Furthermore, OpenCV seems to be plenty fast enough to do everything I need in real time, just with (accelerated where appropriate) CPU processing.

  So I went to work porting the ideas from the simple standalone Python/scikit tool. I thought of a refinement to the straight line detector– instead of just finding the longest straight edge, it creates a histogram of 360 rotation angles, and builds a list of lines corresponding to each angle. Then it sorts the angles by cumulative line length and allows the user to iterate through this list, which will hopefully provide the most likely straightened angle up front. Further, the tool allows making fine adjustments by 1/10 of an angle via the keyboard, not the mouse. It does all this while highlighting in red the straight line segments that are parallel to the horizontal axis, per the current candidate angle.

  
  
  

  The tool draws a light-colored grid over the frame to aid the user in visually verifying the straightness of the image. Further, the program has a mode that allows the user to see the algorithm’s detected edges :

  
  
  

  For the cropping phase, the program uses the Hough circle transform in a similar manner, finding the most likely circles (if the image to be processed is supposed to be a circle) and allowing the user to cycle among them while making precise adjustments via the keyboard, again, rather than the mouse.

  
  
  

  Running the Hough circle transform is a significantly more intensive operation than the line transform. When I ran it on a full 3300×3300 image, it ran for a long time. I didn’t let it run longer than a minute before forcibly ending the program. Is this approach unworkable ? Not quite– It turns out that the transform is just as effective when shrinking the image to 400×400, and completes in under 2 seconds on my Core i5 CPU.

  For rectangular cropping, I just settled on using OpenCV’s built-in region-of-interest (ROI) facility. I tried to intelligently find the best candidate rectangle and allow fine adjustments via the keyboard, but I wasn’t having much success, so I took a path of lesser resistance.

  Packaging and Residual Weirdness
  I realized that this tool would be more useful to a broader Windows-using base of digital preservationists if they didn’t have to install Python, establish a virtual environment, and install the prerequisite dependencies. Thus, I made the effort to figure out how to wrap the entire thing up into a monolithic Windows EXE binary. It is available from the project’s Github release page (another thing I figured out for the sake of this project !).

  The binary is pretty heavy, weighing in at a bit over 50 megabytes. You might advise using compression– it IS compressed ! Before I figured out the --onefile command for pyinstaller.exe, the generated dist/ subdirectory was 150 MB. Among other things, there’s a 30 MB FORTRAN BLAS library packaged in !

  Conclusion and Future Directions
  Once I got it all working with a simple tkinter UI up front in order to select between circle and rectangle crop modes, I unleashed the tool on 60 or so scans in bulk, using the Windows forfiles command (another learning experience). I didn’t put a clock on the effort, but it felt faster. Of course, I was livid with proudness the whole time because I was using my own tool. I just wish I had thought of it sooner. But, really, with 2100+ scans under my belt, I’m just getting started– I literally have thousands more artifacts to scan for preservation.

  The tool isn’t perfect, of course. Just tonight, I threw another scan at MobyCAIRO. Just go ahead and try to find straight lines in this specimen :

  
  
  

  I eventually had to use the text left and right of center to line up against the grid with the manual keyboard adjustments. Still, I’m impressed by how these computer vision algorithms can see patterns I can’t, highlighting lines I never would have guessed at.

  I’m eager to play with OpenCV some more, particularly the video processing functions, perhaps even some GPU-accelerated versions.

  The post Developing MobyCAIRO first appeared on Breaking Eggs And Making Omelettes.