Recherche avancée

Sur d’autres sites (4858)

• Privacy-enhancing technologies : Balancing data utility and security

  18 juillet, par Joe

  In the third quarter of 2024, data breaches exposed 422.61 million records, affecting millions of people around the world. This highlights the need for organisations to prioritise user privacy. 

  Privacy-enhancing technologies can help achieve this by protecting sensitive information and enabling safe data sharing. 

  This post explores privacy-enhancing technologies, including their types, benefits, and how our website analytics platform, Matomo, supports them by providing privacy-focused features.

  What are privacy-enhancing technologies ? 

  Privacy Enhancing Technologies (PETs) are tools that protect personal data while allowing organisations to process information responsibly. 

  In industries like healthcare, finance and marketing, businesses often need detailed analytics to improve operations and target audiences effectively. However, collecting and processing personal data can lead to privacy concerns, regulatory challenges, and reputational risks.

  PETs minimise the collection of sensitive information, enhance security and allow users to control how companies use their data. 

  Global privacy laws like the following are making PETs essential for compliance :

  • General Data Protection Regulation (GDPR) in the European Union
  • California Consumer Privacy Act (CCPA) in California
  • Personal Information Protection and Electronic Documents Act (PIPEDA) in Canada
  • Lei Geral de Proteção de Dados (LGPD) in Brazil

  Non-compliance can lead to severe penalties, including hefty fines and reputational damage. For example, under GDPR, organisations may face fines of up to €20 million or 4% of their global annual revenue for serious violations. 

  Types of PETs 

  What are the different types of technologies available for privacy protection ? Let’s take a look at some of them. 

  Homomorphic encryption

  Homomorphic encryption is a cryptographic technique in which users can perform calculations on cipher text without decrypting it first. When the results are decrypted, they match those of the same calculation on plain text. 

  This technique keeps data safe during processing, and users can analyse data without exposing private or personal data. It is most useful in financial services, where analysts need to protect sensitive customer data and secure transactions. 

  Despite these advantages, homomorphic encryption can be complex to compute and take longer than other traditional methods. 

  Secure Multi-Party Computation (SMPC)

  SMPC enables joint computations on private data without revealing the raw data. 

  In 2021, the European Data Protection Board (EDPB) issued technical guidance supporting SMPC as a technology that protects privacy requirements. This highlights the importance of SMPC in healthcare and cybersecurity, where data sharing is necessary but sensitive information must be kept safe. 

  For example, several hospitals can collaborate on research without sharing patient records. They use SMPC to analyse combined data while keeping individual records confidential. 

  Synthetic data

  Synthetic data is artificially generated to mimic real datasets without revealing actual information. It is useful for training models without compromising privacy. 

  Imagine a hospital wants to train an AI model to predict patient outcomes based on medical records. Sharing real patient data, however, poses privacy challenges, so that can be changed with synthetic data. 

  Synthetic data may fail to capture subtle nuances or anomalies in real-world datasets, leading to inaccuracies in AI model predictions.

  Pseudonymisation

  Pseudonymisation replaces personal details with fake names or codes, making it hard to determine who the information belongs to. This helps keep people’s personal information safe. Even if someone gets hold of the data, it’s not easy to connect it back to real individuals. 

  

  Pseudonymisation works differently from synthetic data, though both help protect individual privacy. 

  When we pseudonymise, we take factual information and replace the bits that could identify someone with made-up labels. Synthetic data takes an entirely different approach. It creates new, artificial information that looks and behaves like real data but doesn’t contain any details about real people.

  Differential privacy

  Differential privacy adds random noise to datasets. This noise helps protect individual entries while still allowing for overall analysis of the data. 

  It’s useful in statistical studies where trends need to be understood without accessing personal details.

  For example, imagine a survey about how many hours people watch TV each week. 

  Differential privacy would add random variation to each person’s answer, so users couldn’t tell exactly how long John or Jane watched TV. 

  However, they could still see the average number of hours everyone in the group watched, which helps researchers understand viewing habits without invading anyone’s privacy.

  Zero-Knowledge Proofs (ZKP)

  Zero-knowledge proofs help verify the truth without exposing sensitive details. This cryptographic approach lets someone prove they know something or meet certain conditions without revealing the actual information behind that proof.

  Take ZCash as a real-world example. While Bitcoin publicly displays every financial transaction detail, ZCash offers privacy through specialised proofs called Zero-Knowledge Succinct Non-Interactive Arguments of Knowledge (zk-SNARKs). These mathematical proofs confirm that a transaction follows all the rules without broadcasting who sent money, who received it, or how much changed hands.

  The technology comes with trade-offs, though. 

  Creating and checking these proofs demands substantial computing power, which slows down transactions and drives up costs. Implementing these systems requires deep expertise in advanced cryptography, which keeps many organisations from adopting them despite their benefits.

  Trusted Execution Environment (TEE)

  TEEs create special protected zones inside computer processors where sensitive code runs safely. These secure areas process valuable data while keeping it away from anyone who shouldn’t see it.

  TEEs are widely used in high-security applications, such as mobile payments, digital rights management (DRM), and cloud computing.

  Consider how companies use TEEs in the cloud : A business can run encrypted datasets within a protected area on Microsoft Azure or AWS Nitro Enclaves. Due to this setup, even the cloud provider can’t access the private data or see how the business uses it. 

  TEEs do face limitations. Their isolated design makes them struggle with large or spread-out computing tasks, so they don’t work well for complex calculations across multiple systems.

  Different TEE implementations often lack standardisation, so there can be compatibility issues and dependence on specific vendors. If the vendor stops the product or someone discovers a security flaw, switching to a new solution often proves expensive and complicated.

  Obfuscation (Data masking)

  Data masking involves replacing or obscuring sensitive data to prevent unauthorised access. 

  It replaces sensitive data with fictitious but realistic values. For example, a customer’s credit card number might be masked as “1234-XXXX-XXXX-5678.” 

  The original data is permanently altered or hidden, and the masked data can’t be reversed to reveal the original values.

  Federated learning

  Federated learning is a machine learning approach that trains algorithms across multiple devices without centralising the data. This method allows organisations to leverage insights from distributed data sources while maintaining user privacy.

  For example, NVIDIA’s Clara platform uses federated learning to train AI models for medical imaging (e.g., detecting tumours in MRI scans). 

  Hospitals worldwide contribute model updates from their local datasets to build a global model without sharing patient scans. This approach may be used to classify stroke types and improve cancer diagnosis accuracy.

  Now that we have explored the various types of PETs, it’s essential to understand the principles that guide their development and use. 

  Key principles of PET (+ How to enable them with Matomo) 

  PETs are based on several core principles that aim to balance data utility with privacy protection. These principles include :

  Data minimisation

  Data minimisation is a core PET principle focusing on collecting and retaining only essential data.

  Matomo, an open-source web analytics platform, helps organisations to gather insights about their website traffic and user behaviour while prioritising privacy and data protection. 

  Recognising the importance of data minimisation, Matomo offers several features that actively support this principle :

  • Cookieless tracking : Eliminates reliance on cookies, reducing unnecessary data collection.
  • IP anonymisation : Automatically anonymises IP addresses, preventing identification of individual users.

  

  (Image Source)

  • Custom data retention policies : Allows organisations to define how long user data is stored before automatic deletion.

  7Assets, a fintech company, was using Google Analytics and Plausible as their web analytics tools. 

  However, with Google Analytics, they faced a problem of unnecessary data tracking, which created legal work overhead. Plausible didn’t have the features for the kind of analysis they wanted. 

  They switched to Matomo to enjoy the balance of privacy yet detailed analytics. With Matomo, they had full control over their data collection while also aligning with privacy and compliance requirements.

  Transparency and User Control

  Transparency and user control are important for trust and compliance. 

  Matomo enables these principles through :

  • Consent management : Offers integration with Consent Mangers (CMPs), like Cookiebot and Osano, for collecting and managing user consent.
  • Respect for DoNotTrack settings : Honours browser-based privacy preferences by default, empowering users with control over their data.

  

  (Image Source)

  • Opt-out mechanisms : These include iframe features that allow visitors to opt out of tracking. 

  Security and Confidentiality

  Security and confidentiality protect sensitive data against inappropriate access. 

  Matomo achieves this through :

  • On-premise hosting : Gives organisations the ability to host analytics data on-site for complete data control.
  • Data security : Protects stored information through access controls, audit logs, two-factor authentication and SSL encryption.
  • Open source code : Enables community reviews for better security and transparency.

  Purpose Limitation

  Purpose limitation means organisations use data solely for the intended purpose and don’t share or sell it to third parties. 

  Matomo adheres to this principle by using first-party cookies by default, so there’s no third-party involvement. Matomo offers 100% data ownership, meaning all the data organisations get from our web analytics is of the organisation, and we don’t sell it to any external parties. 

  Compliance with Privacy Regulations

  Matomo aligns with global privacy laws such as GDPR, CCPA, HIPAA, LGPD and PECR. Its compliance features include :

  • Configurable data protection : Matomo can be configured to avoid tracking personally identifiable information (PII).
  • Data subject request tools : These provide mechanisms for handling requests like data deletion or access in accordance with legal frameworks.
  • GDPR manager : Matomo provides a GDPR Manager that helps businesses manage compliance by offering features like visitor log deletion and audit trails to support accountability.

  

  (Image Source)

  Mandarine Academy is a French-based e-learning company. It found that complying with GDPR regulations was difficult with Google Analytics and thought GA4 was hard to use. Therefore, it was searching for a web analytics solution that could help it get detailed feedback on its site’s strengths and friction points while respecting privacy and GDPR compliance. With Matomo, it checked all the boxes.

  Data collaboration : A key use case of PETs

  One specific area where PETs are quite useful is data collaboration. Data collaboration is important for organisations for research and innovation. However, data privacy is at stake. 

  This is where tools like data clean rooms and walled gardens play a significant role. These use one or more types of PETs (they aren’t PETs themselves) to allow for secure data analysis. 

  Walled gardens restrict data access but allow analysis within their platforms. Data clean rooms provide a secure space for data analysis without sharing raw data, often using PETs like encryption. 

  Tackling privacy issues with PETs 

  Amidst data breaches and privacy concerns, organisations must find ways to protect sensitive information while still getting useful insights from their data. Using PETs is a key step in solving these problems as they help protect data and build customer trust. 

  Tools like Matomo help organisations comply with privacy laws while keeping data secure. They also allow individuals to have more control over their personal information, which is why 1 million websites use Matomo.

  In addition to all the nice features, switching to Matomo is easy :

  “We just followed the help guides, and the setup was simple,” said Rob Jones. “When we needed help improving our reporting, the support team responded quickly and solved everything in one step.” 

  To experience Matomo, sign up for our 21-day free trial, no credit card details needed. 

• A pragmatic strategy to merge multiple video files

  19 juin 2021, par saurav

  I currently am working on recording a multiparty video conference which supports up to 6 participants. I am recording the conference using a media server and storing audio/video streams individually for every participant.

  


  Next, I need to merge those individual recordings into a single video file and upload it to a cloud storage like aws s3. For this I am considering 2 options, either Gstreamer or FFMPEG. I am leaning towards FFMPEG as I have used FFMPEG previously. I currently am playing with FFMPEG things like the hstack and vstack filters etc.

  


  Here is the FFMPEG command I recently used to join 2 webm videos of 2 mins 40sec and 1min 40sec to create a mp4 video file for upload. Both the videos are 1280x720 in this case but I have included the scale part because in real life scenario different participants joining with different cameras produces video files of different resolution which is a problem for the hstack/vstack filter. Therefore, to make the video resolutions of all participant consistent, I have included the scale property.

  


  ffmpeg -i 1.webm -i 2.webm -filter_complex "[0:v]scale=1280:720,setsar=1[l];[1:v]scale=1280:720,setsar=1[r];[l][r]hstack;[0][1]amix" output-1280x720.mp4


  


  Currently I am facing 2 issues with this command.

  


  


  1. The output mp4 file is very big, in this case 140Mb (approx) for a less than 3 minutes video.

     


  


  2. How do I add delay to any video before starting to merge ?
     
Currently the videos are going out of sync if all the participants don't join at the same time which is highly unlikely to happen in a real world scenario.

     


  


  


  Any pointer in the right direction will be highly appreciated.

  


  Here is a log sample from FFmpeg (or see the full log link) :

  


  ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, matroska,webm, from '3.webm':
  Metadata:
    title           : FFmpeg
    ENCODER         : Lavf58.29.100
  Duration: 00:01:39.63, start: 0.000000, bitrate: 707 kb/s
    Stream #0:0: Video: vp8, yuv420p(tv, bt470bg/unknown/unknown, progressive), 1280x720, SAR 1:1 DAR 16:9, 1k tbr, 1k tbn, 1k tbc (default)
    Metadata:
      DURATION        : 00:01:39.618000000
    Stream #0:1: Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 00:01:39.629000000
Input #1, matroska,webm, from '4.webm':
  Metadata:
    title           : FFmpeg
    ENCODER         : Lavf58.29.100
  Duration: 00:02:39.07, start: 0.000000, bitrate: 708 kb/s
    Stream #1:0: Video: vp8, yuv420p(tv, bt470bg/unknown/unknown, progressive), 1280x720, SAR 1:1 DAR 16:9, 1k tbr, 1k tbn, 1k tbc (default)
    Metadata:
      DURATION        : 00:02:39.050000000
    Stream #1:1: Audio: opus, 48000 Hz, stereo, fltp (default)
    Metadata:
      DURATION        : 00:02:39.068000000
Stream mapping:
  Stream #0:0 (vp8) -> scale
  Stream #0:1 (opus) -> amix:input0
  Stream #1:0 (vp8) -> scale
  Stream #1:1 (opus) -> amix:input1
  hstack -> Stream #0:0 (libx264)
  amix -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
[libx264 @ 0x562b4842a500] using SAR=1/1
[libx264 @ 0x562b4842a500] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x562b4842a500] profile High, level 6.1
[libx264 @ 0x562b4842a500] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'output-new.mp4':
  Metadata:
    title           : FFmpeg
    encoder         : Lavf58.29.100
    Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuv420p(progressive), 2560x720 [SAR 1:1 DAR 32:9], q=-1--1, 1k fps, 16k tbn, 1k tbc (default)
    Metadata:
      encoder         : Lavc58.54.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      encoder         : Lavc58.54.100 aac

frame=  129 fps=0.0 q=33.0 size=       0kB time=00:00:00.23 bitrate=   1.6kbits/s dup=123 drop=0 speed=0.44x    
frame=  257 fps=228 q=33.0 size=       0kB time=00:00:00.51 bitrate=   0.8kbits/s dup=243 drop=0 speed=0.455x    
frame=  379 fps=224 q=33.0 size=     256kB time=00:00:00.73 bitrate=2855.1kbits/s dup=358 drop=0 speed=0.434x    
frame=  497 fps=222 q=33.0 size=     256kB time=00:00:00.86 bitrate=2431.5kbits/s dup=469 drop=0 speed=0.386x    
 
...
More than 1000 frames duplicated
...
  
frame=158751 fps=196 q=33.0 size=  134656kB time=00:02:39.00 bitrate=6937.4kbits/s dup=151385 drop=0 speed=0.196x    
frame=158851 fps=196 q=33.0 size=  134912kB time=00:02:39.00 bitrate=6950.6kbits/s dup=151482 drop=0 speed=0.196x    
frame=158983 fps=196 q=33.0 size=  134912kB time=00:02:39.00 bitrate=6950.6kbits/s dup=151610 drop=0 speed=0.196x    
frame=159081 fps=196 q=-1.0 Lsize=  137197kB time=00:02:39.07 bitrate=7065.2kbits/s dup=151706 drop=0 speed=0.196x    

video:132693kB audio:2494kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.486001%

[libx264 @ 0x562b4842a500] frame I:637   Avg QP:17.73  size:123895
[libx264 @ 0x562b4842a500] frame P:40088 Avg QP:19.73  size:  1134
[libx264 @ 0x562b4842a500] frame B:118356 Avg QP:27.54  size:    97
[libx264 @ 0x562b4842a500] consecutive B-frames:  0.8%  0.0%  0.0% 99.2%
[libx264 @ 0x562b4842a500] mb I  I16..4: 11.1% 67.3% 21.6%
[libx264 @ 0x562b4842a500] mb P  I16..4:  0.1%  0.1%  0.0%  P16..4:  2.6%  0.4%  0.3%  0.0%  0.0%    skip:96.5%
[libx264 @ 0x562b4842a500] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  0.7%  0.0%  0.0%  direct: 0.0%  skip:99.3%  L0:38.7% L1:61.3% BI: 0.0%
[libx264 @ 0x562b4842a500] 8x8 transform intra:66.8% inter:71.4%
[libx264 @ 0x562b4842a500] coded y,uvDC,uvAC intra: 81.8% 89.5% 72.3% inter: 0.2% 0.4% 0.0%
[libx264 @ 0x562b4842a500] i16 v,h,dc,p: 25% 21% 17% 37%
[libx264 @ 0x562b4842a500] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 33% 22% 12%  4%  5%  6%  6%  6%  6%
[libx264 @ 0x562b4842a500] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 42% 24%  6%  4%  5%  5%  6%  4%  4%
[libx264 @ 0x562b4842a500] i8c dc,h,v,p: 42% 24% 26%  9%
[libx264 @ 0x562b4842a500] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0x562b4842a500] ref P L0: 82.4% 11.5%  5.3%  0.8%
[libx264 @ 0x562b4842a500] ref B L0: 83.0% 16.9%  0.1%
[libx264 @ 0x562b4842a500] ref B L1: 94.9%  5.1%
[libx264 @ 0x562b4842a500] kb/s:6833.11
[aac @ 0x562b4842b540] Qavg: 239.393


  


• Long Overdue MediaWiki Upgrade

  5 février 2014, par Multimedia Mike — General

  What do I do ? What I do ? This library book is 42 years overdue !
  I admit that it’s mine, yet I can’t pay the fine,
  Should I turn it in or should I hide it again ?
  What do I do ? What do I do ?

  I internalized the forgoing paean to the perils of procrastination by Shel Silverstein in my formative years. It’s probably why I’ve never paid a single cent in late fees in my entire life.

  However, I have been woefully negligent as the steward of the MediaWiki software that drives the world famous MultimediaWiki, the internet’s central repository of obscure technical knowledge related to multimedia. It is currently running of version 1.6 software. The latest version is 1.22.

  The Story So Far
  According to my records, I first set up the wiki late in 2005. I don’t know which MediaWiki release I was using at the time. I probably conducted a few upgrades in the early days, but that went by the wayside perhaps in 2007. My web host stopped allowing shell access and the MediaWiki upgrade process pretty much requires running a PHP script from a command line. Upgrade time came around and I put off the project. Weeks turned into months turned into years until, according to some notes, the wiki abruptly stopped working in July, 2011. Suddenly, there were PHP errors about “Namespace” being a reserved word.

  While I finally laid out a plan to upgrade the wiki after all these years, I eventually found that the problem had been caused when my webhost upgraded from PHP 5.2 -> 5.3. I also learned of a small number of code changes that caused the problem to go away, thus kicking the can down the road once more.

  Then a new problem showed up last week. I think it might be related to a new version of PHP again. This time, a few other things on my site broke, and I learned that my webhost now allows me to select a PHP version to use (with the version then set to “auto”, which didn’t yield much information). Rolling back to an earlier version of PHP might have solved the problem easily.

  But NO ! I made the determination that this goes no further. I want this wiki upgraded.

  The Arduous Upgrade Path
  There are 2 general upgrade paths I can think of :
  

  1. Upgrade in place on the server
  2. Upgrade offline and put the site back on the server

  Approach #1 is problematic since I don’t have direct shell access, though I considered using something like PHP Shell. Approach #2 involves getting the entire set of wiki files and a backup of the MySQL tables. This is workable since I keep automated backups of these items anyway.

  In fairly short order, I was able to set up a working copy of the MultimediaWiki hosted on a local Linux machine. Now what’s the move ? The MediaWiki software I’m running is 1.6.10. The very latest, as of this upgrade project is 1.22.2. I suppose it’s way too much to hope that the software will upgrade cleanly from 1.6.x straight to 1.22.x, but I guess it’s worth a shot…

  HA ! No chance. Okay, next idea is to march through the various versions and upgrade each in turn. MediaWiki has all their historic releases online, all the way back to the 1.3 lineage. I decided that the latest of each lineage should upgrade cleanly from anything in the previous version of lineage. E.g., 1.6.10 should upgrade cleanly to 1.7.3 (last in the 1.7 series). This seemed to be a workable strategy. So I downloaded the latest of each series, unpacked, and copied all the wiki files over the working installation and ran ‘php update.php’ in the maintenance/ directory.

  The process is tedious and not without its obstacles. I consider this penance for my years of wiki neglect. First, I run into the “PHP Parse error : syntax error, unexpected T_NAMESPACE, expecting T_STRING” issue, the same that I saw years ago after the webhost transitioned from PHP 5.2 -> 5.3. I could solve this by editing assorted files and changing “Namespace” -> “MWNamespace” (which is what MediaWiki did by version 1.13). But I would prefer not to.

  Instead, I downloaded the source for PHP 5.2 and compiled it in a separate directory, then called ‘/path/to/php/5.2/bin/php update.php’. Problem solved.

  The next problem is that a bunch of the database update scripts are specifying “Type=InnoDB”. This isn’t supported by modern MySQL databases. Now, it’s “Engine=InnoDB”. A quick search & replace at the command line fixes this for 1.6.x… and 1.7.x… and 1.8 through 1.12. Finally, at 1.13, it was no longer necessary. As a bonus, at 1.13, I was able to test the installation since Namespace had been renamed to MWNamespace. I would later learn that the table type modifications probably could have been simplified in by changing “$wgDBmysql4 = true ;” to “$wgDBmysql5 = true ;” somewhere in LocalSettings.php.

  Command line upgrading worked smoothly up through 1.18 series when I got a new syntax error :

  <br />
PHP Fatal error:  Call to a member function addMessages() on a non-object in /mnt/sdb1/archive/wiki/extensions/Cite.php on line 68<br />

  Best I could do was comment out that line. I hope that doesn’t break anything important.

  In the home stretch, the very last transition (1.21 -> 1.22) failed :

  PHP Fatal error :  Cannot redeclare wfProfileIn() (previously declared in 
  /mnt/sdb1/archive/wiki/includes/profiler/Profiler.php:33) in 
  /mnt/sdb1/archive/wiki/includes/ProfilerStub.php on line 25
  

  Apparently, this problem arises occasionally since 1.18. I found a way around it thanks to this page : Deleted the file StartProfiler.php. Who am I to argue ?

  Upon completing the transition to 1.22, the wiki doesn’t look correct– the pictures aren’t showing up. The solution was to fix the temporary directory via LocalSettings.php.

  Back To Production
  Okay, it all works again ! Locally, that is. How to get it back to the server ? My first idea was that, knowing that this upgrade process can succeed, try stepping through the upgrade process again, but tell the update.php scripts to access the database tables on multimedia.cx. This seemed to be working for awhile, even though the database update phase often took 4-5 minutes. However, the transition from 1.8.5 -> 1.9.6 took 75 minutes and then timed out. According to my notes, “This isn’t going to work.”

  The new process :

  1. Dump the database tables from the local database.
  2. Create a new database remotely (melanson_wiki_ng).
  3. Dump the database table into melanson_wiki_ng.
  4. Move the index.php file out of the wiki files directory temporarily (or rename).
  5. Modify the LocalSettings.php to talk to the new database.
  6. Perform a lftp mirror operation in order to send all the files up to the server.
  7. Send the index.php file and hope beyond hope that everything magically works.

  And that’s the story of how the updated MultimediaWiki came back online. Despite the database dump file being over 110 MB, it only tool MySQL 1m45s to transmit it all to the remote server (let’s hear it for the ‘–compress’ option). For comparison, inserting the tables back into a fresh local database took 1m07s.

  When the MultimediaWiki was first live again, it loaded, but ever so slowly. This is when I finally looked into optimization and found that I was lacking any caching. So as a bonus, the MultimediaWiki should be much faster now.

  Going Forward
  For all I know, I did everything described here in the hardest way possible. But at least I got it done. Unless I learn of a better process, future upgrades will probably look similar to this.

  Additionally, I should probably take some time to figure out what new features are part of the standard MediaWiki distribution nowadays.