Recherche avancée

Sur d’autres sites (11949)

• How to fix laggy ffmpeg screen and audio capture ?

  26 juillet 2022, par Wh0r00t

  I am using ffmpeg to capture the screen along with audio.

  


  The ffmpeg command that i tried is

  


  ffmpeg -y \
    -f x11grab \
    -framerate 60 \
    -s 1366x768 \
    -i :0.0 \
    -f alsa -i default -ac 2 \
    -r 30 \
    -c:v h264 -crf 0 -preset ultrafast -c:a vorbis -strict experimental  \
    "$HOME/Videos/$fname-$(date '+%y%m%d-%H%M-%S').mkv"


  


  The stdout of the ffmpeg https://pastebin.com/Qmi5TMKv

  


  ffmpeg version n5.0.1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12.1.0 (GCC)
  configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-amf --enable-avisynth --enable-cuda-llvm --enable-lto --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libjack --enable-libmfx --enable-libmodplug --enable-libmp3lame --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librav1e --enable-librsvg --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libv4l2 --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid --enable-libzimg --enable-nvdec --enable-nvenc --enable-shared --enable-version3
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
[x11grab @ 0x561faf77eb00] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, x11grab, from ':0.0':
  Duration: N/A, start: 1658814267.169414, bitrate: 2014248 kb/s
  Stream #0:0: Video: rawvideo (BGR[0] / 0x524742), bgr0, 1366x768, 2014248 kb/s, 60 fps, 1000k tbr, 1000k tbn
Guessed Channel Layout for Input Stream #1.0 : stereo
Input #1, alsa, from 'default':
  Duration: N/A, start: 1658814267.230653, bitrate: 1536 kb/s
  Stream #1:0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
  Stream #1:0 -> #0:1 (pcm_s16le (native) -> vorbis (native))
Press [q] to stop, [?] for help
[libx264 @ 0x561faf7d4300] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX XOP FMA3 BMI1
[libx264 @ 0x561faf7d4300] profile High 4:4:4 Predictive, level 3.2, 4:4:4, 8-bit
[libx264 @ 0x561faf7d4300] 264 - core 164 r3081 19856cc - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=0 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=0 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=cqp mbtree=0 qp=0
[alsa @ 0x561faf78a940] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
Output #0, matroska, to '/home/earth/Videos/-220726-1114-27.mkv':
  Metadata:
    encoder         : Lavf59.16.100
  Stream #0:0: Video: h264 (H264 / 0x34363248), yuv444p(tv, progressive), 1366x768, q=2-31, 30 fps, 1k tbn
    Metadata:
      encoder         : Lavc59.18.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: vorbis (oV[0][0] / 0x566F), 48000 Hz, stereo, fltp
    Metadata:
      encoder         : Lavc59.18.100 vorbis
[vorbis @ 0x561faf7d5500] Queue input is backward in time0 bitrate=N/A speed=   0x
frame=  153 fps= 31 q=-1.0 Lsize=    2295kB time=00:00:05.06 bitrate=3709.5kbits/s dup=0 drop=150 speed=1.01x
video:2282kB audio:7kB subtitle:0kB other streams:0kB global headers:3kB muxing overhead: 0.281689%
[libx264 @ 0x561faf7d4300] frame I:1     Avg QP: 0.00  size:381729
[libx264 @ 0x561faf7d4300] frame P:152   Avg QP: 0.00  size: 12857
[libx264 @ 0x561faf7d4300] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x561faf7d4300] mb P  I16..4: 56.3%  0.0%  0.0%  P16..4:  0.1%  0.0%  0.0%  0.0%  0.0%    skip:43.6%
[libx264 @ 0x561faf7d4300] coded y,u,v intra: 1.6% 1.6% 1.6% inter: 0.2% 0.2% 0.2%
[libx264 @ 0x561faf7d4300] i16 v,h,dc,p: 99%  1%  0%  0%
[libx264 @ 0x561faf7d4300] kb/s:3664.27
Exiting normally, received signal 15.


  


  I am using the preset ultrafast because I read that it helps not to compress the video too much.
The output of the recorded test file using ffmpeg is as below.

  


   (+) Video --vid=1 (h264 1366x768 30.000fps)
 (+) Audio --aid=1 (vorbis 2ch 48000Hz)
AO: [pulse] 48000Hz stereo 2ch float
VO: [gpu] 1366x768 yuv444p
AV: 00:00:03 / 00:00:19 (17%) A-V:  0.000
[mkv] Discarding potentially broken or useless index.
AV: 00:00:14 / 00:00:19 (73%) A-V:  0.000

Exiting... (Quit)


  


  The recording works but there is a audio lag. If I record the same using simplescreenrecorder with the same settings like,

  


  audio backend - alsa

  


  source - default

  


  audio codec - vorbis

  


  video codec - h.264

  


  container - matroska

  


  preset - superfast

  


  The simplescreenrecorder log https://pastebin.com/83hMMRQF

  


  [PageRecord::StartPage] Starting page ...
[PageRecord::StartPage] Started page.
[PageRecord::StartOutput] Starting output ...
[PageRecord::StartOutput] Output file: /home/earth/Videos/simplescreenrecorder-2022-07-26_11.18.13.mkv
[Muxer::Init] Using format matroska (Matroska).
[Muxer::AddStream] Using codec libx264 (libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10).
[VideoEncoder::PrepareStream] Using pixel format nv12.
[libx264 @ 0x563436cbfd40] using SAR=1/1
[libx264 @ 0x563436cbfd40] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX XOP FMA3 BMI1
[libx264 @ 0x563436cbfd40] profile High, level 3.2, 4:2:0, 8-bit
[libx264 @ 0x563436cbfd40] 264 - core 164 r3081 19856cc - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x3 me=dia subme=1 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=4 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 pb_ratio=1.30 aq=1:1.00
[Muxer::AddStream] Using codec libvorbis (libvorbis).
[BaseEncoder::EncoderThread] Encoder thread started.
[AudioEncoder::PrepareStream] Using sample format f32p.
[BaseEncoder::EncoderThread] Encoder thread started.
[Muxer::MuxerThread] Muxer thread started.
[PageRecord::StartOutput] Started output.
[Synchronizer::SynchronizerThread] Synchronizer thread started.
[PageRecord::StartInput] Starting input ...
[X11Input::Init] Using X11 shared memory.
[X11Input::Init] Detecting screen configuration ...
[X11Input::Init] Screen 0: x1 = 0, y1 = 0, x2 = 1366, y2 = 768
[X11Input::InputThread] Input thread started.
[ALSAInput::InputThread] Using sample format s16.
[PageRecord::StartInput] Started input.
[ALSAInput::InputThread] Input thread started.
[FastResampler::Resample] Resample ratio is 1.0000 (was 0.0000).
[PageRecord::StopOutput] Stopping output ...
[PageRecord::StopOutput] Stopped output.
[PageRecord::StopInput] Stopping input ...
[X11Input::~X11Input] Stopping input thread ...
[X11Input::InputThread] Input thread stopped.
[ALSAInput::~ALSAInput] Stopping input thread ...
[ALSAInput::InputThread] Input thread stopped.
[PageRecord::StopInput] Stopped input.


  


  It works perfectly without any lag whatsoever. The output of the recorded test file using simplescreenrecorder is as below.

  


   (+) Video --vid=1 (h264 1366x768)
 (+) Audio --aid=1 (vorbis 2ch 48000Hz)
AO: [pulse] 48000Hz stereo 2ch float
VO: [gpu] 1366x768 yuv420p
AV: 00:00:01 / 00:00:17 (7%) A-V:  0.000
[mkv] Discarding potentially broken or useless index.
AV: 00:00:08 / 00:00:17 (47%) A-V:  0.000

Exiting... (Quit)


  


  The only difference that I saw between these two recordings is VO: [gpu] 1366x768 yuv444p
VO: [gpu] 1366x768 yuv420p for ffmpeg and simplescreenrecorder receptively.
I do not know if this matters but is there something that I could tweak to make ffmpeg to capture the screen and audio without any lag.
Like answered here https://unix.stackexchange.com/questions/675436/ffmpeg-recording-slows-down-when-audio-inputs-are-added
I do open pavucontrol but its not much of a help.

  


  The reason that I going with ffmpeg is because I can kill the process using pid at a particular time using cronjobs.
These are my system information, in case if it helps

  


  System:
  Host: taco Kernel: 5.18.12-arch1-1 arch: x86_64 bits: 64 Desktop: dwm
    v: 6.2 Distro: Arch Linux
Machine:
  Type: Desktop Mobo: Acer model: A75F2-M v: P21-A1 serial: N/A BIOS: Acer
    v: P21-A1 date: 02/07/2014
CPU:
  Info: quad core model: AMD A8-5500B APU with Radeon HD Graphics bits: 64
    type: MT MCP cache: L2: 4 MiB
  Speed (MHz): avg: 1400 min/max: 1400/3200 cores: 1: 1400 2: 1400 3: 1400
    4: 1400
Graphics:
  Device-1: AMD Trinity [Radeon HD 7560D] driver: radeon v: kernel
  Display: server: X.Org v: 21.1.4 driver: X: loaded: modesetting
    gpu: radeon resolution: 1366x768~60Hz
  OpenGL: renderer: AMD ARUBA (DRM 2.50.0 / 5.18.12-arch1-1 LLVM 14.0.6)
    v: 4.3 Mesa 22.1.3
Audio:
  Device-1: AMD FCH Azalia driver: snd_hda_intel
  Sound Server-1: ALSA v: k5.18.12-arch1-1 running: yes
  Sound Server-2: PulseAudio v: 16.1 running: yes
  Sound Server-3: PipeWire v: 0.3.56 running: yes


  


  Any help is much appreciated.

  


• Google Analytics 4 and GDPR : Everything You Need to Know

  17 mai 2022, par Erin

  Four years have passed since the European General Data Protection Regulation (GDPR, also known as DSGVO in German, and RGPD in French) took effect.

  That’s ample time to get compliant, especially for an organisation as big and innovative as Google. Or is it ? 

  If you are wondering how GDPR affects Google Analytics 4 and what the compliance status is at present, here’s the lowdown. 

  Is Google Analytics 4 GDPR Compliant ?

  No. As of mid-2022, Google Analytics 4 (GA4) isn’t fully GDPR compliant. Despite adding extra privacy-focused features, GA4 still has murky status with the European regulators. After the invalidation of the Privacy Shield framework in 2020, Google is yet to regulate EU-US data protection. At present, the company doesn’t sufficiently protect EU citizens’ and residents’ data against US surveillance laws. This is a direct breach of GDPR.

  Google Analytics and GDPR : a Complex Relationship 

  European regulators have scrutinised Google since GDPR came into effect in 2018.

  While the company took steps to prepare for GDPR provisions, it didn’t fully comply with important regulations around user data storage, transfer and security.

  The relationship between Google and EU regulators got more heated after the Court of Justice of the European Union (CJEU) invalidated the Privacy Shield — a leeway Google used for EU-US data transfers. After 2020, GDPR litigation against Google followed. 

  This post summarises the main milestones in this story and explains the consequences for Google Analytics users. 

  

  2018 : Google Analytics Meets GDPR 

  In 2018, the EU adopted the General Data Protection Regulation (GDPR) — a set of privacy and data security laws, covering all member states. Every business interacting with EU citizens and/or residents had to comply.

  GDPR harmonised data protection laws across member states and put down extra provisions for what constitutes sensitive personal information (or PII). Broadly, PII includes any data about the person’s :

  • Racial or ethnic origin 
  • Employment status 
  • Religious or political beliefs
  • State of health 
  • Genetic or biometric data 
  • Financial records (such as payment method data)
  • Address and phone numbers 

  Businesses were barred from collecting this information without explicit consent (and even with it in some cases). If collected, such sensitive information is also subject to strict requirements on how it should be stored, secured, transferred and used. 

  7 Main GDPR Principles Explained 

  Article 5 of the GDPR lays out seven main GDPR principles for personal data and privacy protection : 

  • Lawfulness, fairness and transparency — data must be obtained legally, collected with consent and in adherence to laws. 
  • Purpose limitation — all personal information must be collected for specified, explicit and legal purposes. 
  • Data minimisation — companies must collect only necessary and adequate data, aligned with the stated purpose. 
  • Accuracy — data accuracy must be ensured at all times. Companies must have mechanisms to erase or correct inaccurate data without delays. 
  • Storage limitation — data must be stored only for as long as the stated purpose suggests. Though there’s no upper time limit on data storage. 
  • Integrity and confidentiality (security) — companies must take measures to ensure secure data storage and prevent unlawful or unauthorised access to it. 
  • Accountability — companies must be able to demonstrate adherence to the above principles. 

  Google claimed to have taken steps to make all of their products GDPR compliant ahead of the deadline. But in practice, this wasn’t always the case.

  In March 2018, a group of publishers admonished Google for not providing them with enough tools for GDPR compliance :

  “[Y]ou refuse to provide publishers with any specific information about how you will collect, share and use the data. Placing the full burden of obtaining new consent on the publisher is untenable without providing the publisher with the specific information needed to provide sufficient transparency or to obtain the requisite specific, granular and informed consent under the GDPR.”

  The proposed Google Analytics GDPR consent form was hard to implement and lacked customisation options. In fact, Google “makes unilateral decisions” on how the collected data is stored and used. 

  Users had no way to learn about or control all intended uses of people’s data — which made compliance with the second clause impossible. 

  Unsurprisingly, Google was among the first companies to face a GDPR lawsuit (together with Facebook). 

  By 2019, French data regulator CNIL, successfully argued that Google wasn’t sufficiently disclosing its data collection across products — and hence in breach of GDPR. After a failed appeal, Google had to pay a €50 million fine and promise to do better. 

  2019 : Google Analytics 4 Announcement 

  Throughout 2019, Google rightfully attempted to resolve some of its GDPR shortcomings across all products, Google Universal Analytics (UA) included. 

  They added a more visible consent mechanism for online tracking and provided extra compliance tips for users to follow. In the background, Google also made tech changes to its data processing mechanism to get on the good side of regulations.

  Though Google addressed some of the issues, they missed others. A 2019 independent investigation found that Google real-time-bidding (RTB) ad auctions still used EU citizens’ and residents’ data without consent, thanks to a loophole called “Push Pages”. But they managed to quickly patch this up before the allegations had made it to court. 

  In November 2019, Google released a beta version of the new product version — Google Analytics 4, due to replace Universal Analytics. 

  GA4 came with a set of new privacy-focused features for ticking GDPR boxes such as :

  • Data deletion mechanism. Users can now request to surgically extract certain data from the Analytics servers via a new interface. 
  • Shorter data retention period. You can now shorten the default retention period to 2 months by default (instead of 14 months) or add a custom limit.  
  • IP Anonymisation. GA4 doesn’t log or store IP addresses by default. 

  Google Analytics also updated its data processing terms and made changes to its privacy policy. 

  Though Google made some progress, Google Analytics 4 still has many limitations — and isn’t GDPR compliant. 

  2020 : Privacy Shield Invalidation Ruling 

  As part of the 2018 GDPR preparations, Google named its Irish entity (Google Ireland Limited) as the “data controller” legally responsible for EEA and Swiss users’ information. 

  The company announcement says : 

  

  Initially, Google assumed that this legal change would help them ensure GDPR compliance as “legally speaking” a European entity was set in charge of European data. 

  Practically, however, EEA consumers’ data was still primarily transferred and processed in the US — where most Google data centres are located. Until 2020, such cross-border data transfers were considered legal thanks to the Privacy Shield framework. 

  But in July 2020, The EU Court of Justice ruled that this framework doesn’t provide adequate data protection to digitally transmitted data against US surveillance laws. Hence, companies like Google can no longer use it. The Swiss Federal Data Protection and Information Commissioner (FDPIC) reached the same conclusion in September 2020. 

  The invalidation of the Privacy Shield framework put Google in a tough position.

   Article 14. f of the GDPR explicitly states : 

  “The controller (the company) that intends to carry out a transfer of personal data to a recipient (Analytics solution) in a third country or an international organisation must provide its users with information on the place of processing and storage of its data”.

  Invalidation of the Privacy Shield framework prohibited Google from moving data to the US. At the same time, GDPR provisions mandated that they must disclose proper data location. 

  But Google Analytics (like many other products) had no a mechanism for : 

  • Guaranteeing intra-EU data storage 
  • Selecting a designated regional storage location 
  • Informing users about data storage location or data transfers outside of the EU 

  And these factors made Google Analytics in direct breach of GDPR — a territory, where they remain as of 2022.

  2020-2022 : Google GDPR Breaches and Fines 

  The 2020 ruling opened Google to GDPR lawsuits from country-specific data regulators.

  Google Analytics in particular was under a heavy cease-fire. 

  • Sweden first fined Google for violating GDPR for no not fulfilling its obligations to request data delisting in 2020. 
  • France rejected Google Analytics 4 IP address anonymisation function as a sufficient measure for protecting cross-border data transfers. Even with it, US intelligence services can still access user IPs and other PII. France declared Google Analytics illegal and pressed a €150 million fine. 
  • Austria also found Google Analytics GDPR non-compliant and proclaimed the service as “illegal”. The authority now seeks a fine too. 

  The Dutch Data Protection Authority and  Norwegian Data Protection Authority also found Google Analytics guilty of a GDPR breach and seek to limit Google Analytics usage. 

  New privacy controls in Google Analytics 4 do not resolve the underlying issue — unregulated, non-consensual EU-US data transfer. 

  Google Analytics GDPR non-compliance effectively opens any website tracking or analysing European visitors to legal persecution.

  In fact, this is already happening. noyb, a European privacy-focused NGO, has already filed over 100 lawsuits against European websites using Google Analytics.

  2022 : Privacy Shield 2.0. Negotiations

  Google isn’t the only US company affected by the Privacy Shield framework invalidation. The ruling puts thousands of digital companies at risk of non-compliance.

  To settle the matter, US and EU authorities started “peace talks” in spring 2022.

  European Commission President Ursula von der Leyen said that they are working with the Biden administration on the new agreement that will “enable predictable and trustworthy data flows between the EU and US, safeguarding the privacy and civil liberties.” 

  However, it’s just the beginning of a lengthy negotiation process. The matter is far from being settled and contentious issues remain as we discussed on Twitter (come say hi !).

  

  For one, the US isn’t eager to modify its surveillance laws and is mostly willing to make them “proportional” to those in place in the EU. These modifications may still not satisfy CJEU — which has the power to block the agreement vetting or invalidate it once again. 

  While these matters are getting hashed out, Google Analytics users, collecting data about EU citizens and/or residents, remain on slippery grounds. As long as they use GA4, they can be subject to GDPR-related lawsuits. 

  To Sum It Up 

  • Google Analytics 4 and Google Universal Analytics are not GDPR compliant because of Privacy Shield invalidation in 2020. 
  • French and Austrian data watchdogs named Google Analytics operations “illegal”. Swedish, Dutch and Norwegian authorities also claim it’s in breach of GDPR. 
  • Any website using GA for collecting data about European citizens and/or residents can be taken to court for GDPR violations (which is already happening). 
  • Privacy Shield 2.0 Framework discussions to regulate EU-US data transfers have only begun and may take years. Even if accepted, the new framework(s) may once again be invalidated by local data regulators as has already happened in the past. 

  Time to Get a GDPR Compliant Google Analytics Alternative 

  Retaining 100% data ownership is the optimal path to GDPR compliance.

  By selecting a transparent web analytics solution that offers 100% data ownership, you can rest assured that no “behind the scenes” data collection, processing or transfers take place. 

  Unlike Google Analytics 4, Matomo offers all of the features you need to be GDPR compliant : 

  • Full data anonymisation 
  • Single-purpose data usage 
  • Easy consent and an opt-out mechanism 
  • First-party cookies usage by default 
  • Simple access to collect data 
  • Fast data removals 
  • EU-based data storage for Matomo Cloud (or storage in the country of your choice with Matomo On-Premise)

  Learn about your audiences in a privacy-centred way and protect your business against unnecessary legal exposure. 

  Start your 21-day free trial (no credit card required) to see how fully GDPR-compliant website analytics works ! 

  21 day free trial. No credit card required.

  Your email address
  

  Your website address
  

  Your Analytics subdomain will be .matomo.cloud (change)

  Choose your analytics subdomain
  

  .matomo.cloud

  I hereby accept the terms and conditions and the data processing agreeement (DPA).

  
  Your information will be used to create an account on our cloud service. For more information please consult our privacy policy.
  
  

  » Start improving your websites now

  

• ISO-9660 Compromise, Part 2 : Finding Root

  25 octobre 2021, par Multimedia Mike — General

  A long time ago, I dashed off a quick blog post with a curious finding after studying the ISO-9660 spec : The format stores multi-byte numbers in a format I termed “omni-endian”– the committee developing the format apparently couldn’t come to an agreement on this basic point regarding big- vs. little-endian encoding (I’m envisioning something along the lines of “tastes great ! … less filling !” in the committee meetings).

  I recently discovered another bit of compromise in the ISO-9660 spec : It seems that there are 2 different methods for processing the directory structure. That means it’s incumbent upon ISO-9660 creation software to fill in the data structures to support both methods, because some ISO-reading programs out there rely on one set of data structures while the rest prefer to read the other set.

  Background

  As a refresher, the “ISO” extension of an ISO file refers to the ISO-9660 specification. This is a type of read-only filesystem (i.e, the filesystem is created once and never updated after initial creation) for the purpose of storing on a read-only medium, often an optical disc (CD-ROM, DVD-ROM). The level of nostalgic interest I display for the ISO-9660 filesystem reminds me of my computer science curriculum professors from the mid-90s reminiscing about ye olden days of punchcard programming, but such is my lot. I’m probably also alone in my frustration of seeing rips of, e.g., GameCube or Xbox or 3DO games being tagged with the extension .ISO since those systems use different read-only filesystems.

  I recently fell in with an odd bunch called the eXoDOS project and was trying to help fill in a few gaps. One request was a 1994 game called Power Drive for DOS.

  
  
  

  
  

  My usual CD-ROM ripping method (for the data track) is a simple ‘dd’ command from a Linux command line to copy the string of raw sectors. However, it turned out to be unusually difficult to open the resulting ISO. A few of the the options I know of worked but most didn’t. What’s the difference ?

  Methods that work :

  • Mounting the file with the Linux iso9660 kernel module, i.e.,

    mount -t iso9660 /dev/optical-drive /mnt

    or

    mount -t iso9660 -o loop /path/to/Power-Drive.iso /mnt

  • Directory Opus
  • Windows 10 can read the filesystem when reading the physical disc
  • Windows 10 can burn the ISO image to a new CD (“right click” -> “Burn disc image”) ; this method does not modify any of the existing sectors but did append 149 additional empty sectors

  Methods that don’t work :

  • fuseiso
  • Dosbox
  • Winrar
  • 7zip
  • Daemon Tools
  • Imgburn
  • Internet Archive’s ISO lister (“View contents” on the ISO file)

  Understanding The Difference

  I think I might have a handle on why some tools are able to process this disc while most can’t. There appears to be 2 sets of data structures to describe the base of the filesystem : A root directory, and a path table. These both occur in the first substantive sector of the ISO-9660 filesystem, usually sector 16.

  A compact disc can be abstractly visualized as a long string of sectors, each one 2,352 bytes long. (See my Grand Unified Theory of Compact Disc post for deeper discussion.) A CD-ROM data track will contain 2048 bytes of data. Thus, sector 16 appears at 0x8000 of an ISO filesystem. I like the clarity of this description of the ISO-9660 spec. It shows that the path table is defined at byte 140 (little-endian ; big comes later) and location of the root directory is at byte 158. Thus, these locations generally occur at 0x808c and 0x809e.

  
  
  Primary Volume Descriptor
  

  The path table is highlighted in green and the root directory record is highlighted in red. These absolute locations are specified in sectors. So the path table is located at sector 0x12 = offset 0x9000 in the image, while the root directory record is supposed to be at sector 0x62 = 0x31000. Checking into those sectors, it turns out that the path table is valid while the root directory record is invalid. Thus, any tool that relies on the path table will be successful in interpreting the disc, while tools that attempt to recursively traverse starting from root directory record are gonna have a bad time.

  Since I was able to view the filesystem with a few different tools, I know what the root directory contains. Searching for those filenames reveals that the root directory was supposed to point to the next sector, number 0x63. So this was a bizarre off-by-1 error on the part of the ISO creation tool. Maybe. I manually corrected 0x62 -> 0x63 and that fixed the interaction with fuseiso, but not with other tools. So there may have been some other errors. Note that a quick spot-check of another, functional ISO revealed that this root directory sector is supposed to be exact, not 1-indexed.

  Upon further inspection, I noticed that, while fuseiso appeared to work with that one patch, none of the files returned correct data, and none of the directories contained anything. That’s when I noticed that ALL of the sector locations described in the various directory and file records are off by 1 !

  Further Investigation

  I have occasionally run across ISO images on the Internet Archive that return the error about not being able to read the contents when trying to “View contents” (error text : “failed to obtain file list from xyz.iso”, as seen with this ISO). Too bad I didn’t make a record of them because I would be interested to see if they have the same corruption.

  Eventually, I’ll probably be able to compile an archive of deviant ISO-9660 images. A few months ago, I was processing a large collection from IA and found a corrupted ISO which had a cycle, i.e., the subdirectory pointed to a parent directory, which caused various ISO tools to loop forever. Just one of those things that is “never supposed to happen”, so why write code to deal with it gracefully ?

  See Also

  • The disc archived at The Internet Archive
  • MobyGames Entry
  • Wikipedia Entry

  The post ISO-9660 Compromise, Part 2 : Finding Root first appeared on Breaking Eggs And Making Omelettes.