Recherche avancée

Sur d’autres sites (4664)

• ISO-9660 Compromise, Part 2 : Finding Root

  25 octobre 2021, par Multimedia Mike — General

  A long time ago, I dashed off a quick blog post with a curious finding after studying the ISO-9660 spec : The format stores multi-byte numbers in a format I termed “omni-endian”– the committee developing the format apparently couldn’t come to an agreement on this basic point regarding big- vs. little-endian encoding (I’m envisioning something along the lines of “tastes great ! … less filling !” in the committee meetings).

  I recently discovered another bit of compromise in the ISO-9660 spec : It seems that there are 2 different methods for processing the directory structure. That means it’s incumbent upon ISO-9660 creation software to fill in the data structures to support both methods, because some ISO-reading programs out there rely on one set of data structures while the rest prefer to read the other set.

  Background

  As a refresher, the “ISO” extension of an ISO file refers to the ISO-9660 specification. This is a type of read-only filesystem (i.e, the filesystem is created once and never updated after initial creation) for the purpose of storing on a read-only medium, often an optical disc (CD-ROM, DVD-ROM). The level of nostalgic interest I display for the ISO-9660 filesystem reminds me of my computer science curriculum professors from the mid-90s reminiscing about ye olden days of punchcard programming, but such is my lot. I’m probably also alone in my frustration of seeing rips of, e.g., GameCube or Xbox or 3DO games being tagged with the extension .ISO since those systems use different read-only filesystems.

  I recently fell in with an odd bunch called the eXoDOS project and was trying to help fill in a few gaps. One request was a 1994 game called Power Drive for DOS.

  
  
  

  
  

  My usual CD-ROM ripping method (for the data track) is a simple ‘dd’ command from a Linux command line to copy the string of raw sectors. However, it turned out to be unusually difficult to open the resulting ISO. A few of the the options I know of worked but most didn’t. What’s the difference ?

  Methods that work :

  • Mounting the file with the Linux iso9660 kernel module, i.e.,

    mount -t iso9660 /dev/optical-drive /mnt

    or

    mount -t iso9660 -o loop /path/to/Power-Drive.iso /mnt

  • Directory Opus
  • Windows 10 can read the filesystem when reading the physical disc
  • Windows 10 can burn the ISO image to a new CD (“right click” -> “Burn disc image”) ; this method does not modify any of the existing sectors but did append 149 additional empty sectors

  Methods that don’t work :

  • fuseiso
  • Dosbox
  • Winrar
  • 7zip
  • Daemon Tools
  • Imgburn
  • Internet Archive’s ISO lister (“View contents” on the ISO file)

  Understanding The Difference

  I think I might have a handle on why some tools are able to process this disc while most can’t. There appears to be 2 sets of data structures to describe the base of the filesystem : A root directory, and a path table. These both occur in the first substantive sector of the ISO-9660 filesystem, usually sector 16.

  A compact disc can be abstractly visualized as a long string of sectors, each one 2,352 bytes long. (See my Grand Unified Theory of Compact Disc post for deeper discussion.) A CD-ROM data track will contain 2048 bytes of data. Thus, sector 16 appears at 0x8000 of an ISO filesystem. I like the clarity of this description of the ISO-9660 spec. It shows that the path table is defined at byte 140 (little-endian ; big comes later) and location of the root directory is at byte 158. Thus, these locations generally occur at 0x808c and 0x809e.

  
  
  Primary Volume Descriptor
  

  The path table is highlighted in green and the root directory record is highlighted in red. These absolute locations are specified in sectors. So the path table is located at sector 0x12 = offset 0x9000 in the image, while the root directory record is supposed to be at sector 0x62 = 0x31000. Checking into those sectors, it turns out that the path table is valid while the root directory record is invalid. Thus, any tool that relies on the path table will be successful in interpreting the disc, while tools that attempt to recursively traverse starting from root directory record are gonna have a bad time.

  Since I was able to view the filesystem with a few different tools, I know what the root directory contains. Searching for those filenames reveals that the root directory was supposed to point to the next sector, number 0x63. So this was a bizarre off-by-1 error on the part of the ISO creation tool. Maybe. I manually corrected 0x62 -> 0x63 and that fixed the interaction with fuseiso, but not with other tools. So there may have been some other errors. Note that a quick spot-check of another, functional ISO revealed that this root directory sector is supposed to be exact, not 1-indexed.

  Upon further inspection, I noticed that, while fuseiso appeared to work with that one patch, none of the files returned correct data, and none of the directories contained anything. That’s when I noticed that ALL of the sector locations described in the various directory and file records are off by 1 !

  Further Investigation

  I have occasionally run across ISO images on the Internet Archive that return the error about not being able to read the contents when trying to “View contents” (error text : “failed to obtain file list from xyz.iso”, as seen with this ISO). Too bad I didn’t make a record of them because I would be interested to see if they have the same corruption.

  Eventually, I’ll probably be able to compile an archive of deviant ISO-9660 images. A few months ago, I was processing a large collection from IA and found a corrupted ISO which had a cycle, i.e., the subdirectory pointed to a parent directory, which caused various ISO tools to loop forever. Just one of those things that is “never supposed to happen”, so why write code to deal with it gracefully ?

  See Also

  • The disc archived at The Internet Archive
  • MobyGames Entry
  • Wikipedia Entry

  The post ISO-9660 Compromise, Part 2 : Finding Root first appeared on Breaking Eggs And Making Omelettes.

• Dreamcast Finds

  15 avril 2022, par Multimedia Mike — Sega Dreamcast

  Pursuant to my recent post about finally understanding how Sega Dreamcast GD-ROM rips are structured, I was able to prepare the contents of various demo discs in a manner that makes exploration easy via the Internet Archive. This is due to the way that IA makes it easy to browse archives such as ZIP or ISO files (anything that 7zip knows how to unpack), and also presents the audio tracks for native playback directly through the web browser.

  These are some of the interesting things I have found while perusing the various Dreamcast sampler discs.

  Multimedia Formats
  First and foremost : Multimedia-wise, SFD and ADX files abound on all the discs. SFD files are Sofdec, a middleware format used for a lot of FMV on Dreamcast games. These were little more than MPEG video files with a non-MPEG (ADPCM instead) audio codec. VLC will usually play the video portions of these files but has trouble detecting the audio. It’s not for lack of audio codec support because it can play the ADX files just fine.
  

  It should be noted that Dreamcast Magazine Disc 11 has an actual .mpg file (as opposed to a .sfd file) that has proper MPEG audio instead instead of ADX ADPCM.

  The only other multimedia format I know of that was used in any Dreamcast games was 4XM, used on Alone In The Dark : The New Nightmare. I wrote a simple C tool a long time to recover these files from a disc image I extracted myself. Rather than interpreting the ISO-9660 filesystem, the tool just crawled through the binary blob searching for ‘4XMV’ file signatures and using length data within the files for extraction.

  Also, there are plentiful PVR files (in reference to the PowerVR2 GPU hardware that the DC uses) which ‘file’ dutifully identifies as “Sega PVR image”. There are probably tools to view them. It doesn’t appear to be a complicated format.

  Scripting
  I was fascinated to see Lua files on at least one of the discs. It turns out that MDK 2 leverages the language, as several other games do. But it was still interesting to see the .lua files show up in the Dreamcast version as well.

  That Windows CE Logo
  Every Sega Dreamcast is famously emblazoned with a logo mentioning Microsoft Windows CE :

  
  
  

  It has confused many folks. It also confused me until this exploratory exercise. Many would wonder if the Dreamcast booted up into some Windows CE OS environment that then ran the game, but that certainly wasn’t it. Indeed, Dreamcast was one of the last consoles that really didn’t have any kind of hypervisor operating system managing everything.

  I found a file called rt2dc.exe on one sampler disc. At first, I suspected that this was a development utility for Windows to convert some “RT” graphical format into a format more suitable for the Dreamcast. Then, ‘file’ told me that it was actually a Windows EXE but compiled for the Hitachi SH-4 CPU (the brain inside the DC). Does the conversion utility run on the Dreamcast itself ? Then I analyzed the strings inside the binary and saw references to train stations. That’s when it started to click for me that this was the binary executable for the demo version of Railroad Tycoon 2 : Gold Edition, hence “rt2dc.exe”. Still, this provides some insight about whether Dreamcast “runs” Windows. This binary was built against a series of Windows CE libraries. The symbols also imply DirectX compatibility.

  Here is a page with more info about the WinCE/DirectX variant for the Sega Dreamcast. It seems that this was useful for closing the gap between PC and DC ports of games (i.e., being able to re-use more code between the 2 platforms). I guess this was part of what made Dreamcast a dry run for the DirectXbox (later Xbox).

  Here is a list of all the Dreamcast games that are known to use Windows CE.

  Suddenly, I am curious if tools such as IDA Pro or Ghidra can possibly open up Windows CE binaries that contain SH-4 code. Not that I’m particularly interested in reverse engineering any algorithms locked up in Dreamcast land.

  Tomb Raider Easter Egg
  The volume 6 sampler disc has a demo of Tomb Raider : The Last Revelation. While inspecting the strings, I found an Easter egg. I was far from the first person to discover it, though, as seen on this The Cutting Room Floor wiki page (look under “Developer Message”). It looks like I am the first person to notice it on the Dreamcast version. It shows up at offset 0xE3978 in the Dreamcast (demo version) binary, if anyone with permissions wants to update the page.

  Web Browser
  Then there’s the Web Browser for Sega Dreamcast. It seemed to be included on a lot of these sampler discs. But only mentioning the web browser undersells it– the thing also bundled an email client and an IRC client. It’s important to remember that the Dreamcast also had a keyboard peripheral.

  I need to check the timeline for when the web browser first became available vs. when the MIL-CD hack became known. My thinking is that there is no way that the web browser program didn’t have some security issues– buffer overflows and the like. It seems like this would have been a good method of breaking the security of the system.

  Ironically, I suddenly can think of a reason why one might want to use advanced reverse engineering tools on Dreamcast binaries, something I struggled with just a few paragraphs ago.

  Odds ‘n Ends
  It’s always fun to find plain text files among video game assets and speculating on the precise meaning… while also marveling how long people have been struggling to correctly spell “length”.

  Internationalization via plain text files.

  Another game (Slave Zero) saw fit to zip its assets. Maybe this was to save space in order to fit everything on the magazine sampler disc. Quizzically, this didn’t really save an appreciable amount of space.

  Finally, all the discs have an audio track 2 that advises that the disc must be played in a Dreamcast console. Not unusual. However, volume 4 also has a Japanese lady saying the same thing on track 4. This is odd because track 4 is one of the GD area audio tracks and is not accessible with normal CD hardware. Further, she identifies the disc as a “Windows CE disc”.

  The post Dreamcast Finds first appeared on Breaking Eggs And Making Omelettes.

• ffmpeg command never work in lambda function using nodejs [closed]

  4 décembre 2022, par Santosh swain

  I am trying to implement FFmpeg video streaming functionality such as Instagram countdown functionality. In this code, first of all, I get records(URLs) from the s3 bucket and then split them according to my need, and then create the command and execute it with exec() belonging to childe_process. in this, I am trying to store the out in some specific folder in lambda function but it was never stored. I thought lambda does allow to write files locally so I am trying to do the direct upload on the s3 bucket by using the stdout parameter of exec()'s callback. guys, please help to do that. I have a question lambda does allow to write content in its local folder ? or if not allow then whats the way to do that thing ? I just share my code please guide me.

  


  
    // dependencies
var AWS = require('aws-sdk');
var s3 = new AWS.S3();
var { exec } = require('child_process');
var path = require('path')
var AWS_ACCESS_KEY = '';
var AWS_SECRET_ACCESS_KEY = '';
var fs = require('fs')

s3 = new AWS.S3({
    accessKeyId: AWS_ACCESS_KEY,
    secretAccessKey: AWS_SECRET_ACCESS_KEY
});

exports.handler = async function (event, context) {

    var bucket_name = "sycu-game";
    var bucketName = "sycu-test";

    //CREATE OVERLAY AND BG_VALUE PATH TO GET VALUE FROM S3
    const bgValue = (event.Records[0].bg_value).split('/');
    const overlayImage = (event.Records[0].overlay_image_url).split('/');


    var s3_bg_value = bgValue[3] + "/" + bgValue[4];
    var s3_overlay_image = overlayImage[4] + "/" + overlayImage[5] + "/" + overlayImage[6];
    const signedUrlExpireSeconds = 60 * 5;


    //RETREIVE BG_VALUE FROM S3 AND CREATE URL FOR FFMPEG INPUT VALUE
    var bg_value_url = s3.getSignedUrl('getObject', {
        Bucket: bucket_name,
        Key: s3_bg_value,
        Expires: signedUrlExpireSeconds
    });
    bg_value_url = bg_value_url.split("?");
    bg_value_url = bg_value_url[0];


    //RETREIVE OVERLAY IMAGE FROM S3 AND CREATE URL FOR FFMPEG INPUT VALUE   
    var overlay_image_url = s3.getSignedUrl('getObject', {
        Bucket: bucket_name,
        Key: s3_overlay_image,
        Expires: signedUrlExpireSeconds
    });
    overlay_image_url = overlay_image_url.split("?");
    overlay_image_url = overlay_image_url[0];


    //MANUAL ASSIGN VARIABLE FOR FFMPEG COMMAND 
    var command,
        ExtraTimerSec = event.Records[0].timer_seconds + 5,
        TimerSec = event.Records[0].timer_seconds + 1,
        BackgroundWidth = 1080,
        BackgroundHeight = 1920,
        videoPath = (__dirname + '/tmp/' + event.Records[0].name);
    console.log("path", videoPath)
    //TEMP DIRECTORY

    var videoPath = '/media/volume-d/generatedCountdownS3/tmp/' + event.Records[0].name
    var tmpFile = fs.createWriteStream(videoPath)
    //FFMPEG COMMAND 
    if (event.Records[0].bg_type == 2) {
        if (event.Records[0].is_rotate) {
            command = ' -stream_loop -1 -t ' + ExtraTimerSec + ' -i ' + bg_value_url + ' -i ' + overlay_image_url + ' -filter_complex "color=color=0x000000@0.0:s= ' + event.Records[0].resized_box_width + 'x' + event.Records[0].resized_box_height + ',drawtext=fontcolor=' + event.Records[0].time_text_color + ':fontsize=' + event.Records[0].time_text_size + ':x=' + event.Records[0].minute_x + ':y=' + event.Records[0].minute_y + ':text=\'%{eif\\:trunc(mod(((' + TimerSec + '-if(between(t, 0, 1),1,if(gte(t,' + TimerSec + '),' + TimerSec + ',t)))/60),60))\\:d\\:2}\',drawtext=fontcolor=' + event.Records[0].time_text_color + ':fontsize=' + event.Records[0].time_text_size + ':x=' + event.Records[0].second_x + ':y=' + event.Records[0].second_y + ':text=\'%{eif\\:trunc(mod(' + TimerSec + '-if(between(t, 0, 1),1,if(gte(t,' + TimerSec + '),' + TimerSec + ',t))\,60))\\:d\\:2}\'[txt]; [txt] rotate=' + event.Records[0].box_angle + '*PI/180:fillcolor=#00000000 [rotated];[0] scale=w=' + BackgroundWidth + ':h=' + BackgroundHeight + '[t];[1] scale=w=' + BackgroundWidth + ':h=' + BackgroundHeight + '[ot];[t][ot] overlay = :x=0 :y=0 [m1];[m1][rotated]overlay = :x=' + event.Records[0].flat_box_coordinate_x + ' :y=' + event.Records[0].flat_box_coordinate_x + ' [m2]" -map "[m2]" -pix_fmt yuv420p -t ' +
                ExtraTimerSec + ' -r 24 -c:a copy ' + videoPath + "";
        }
        else {
            command = ' -stream_loop -1 -t ' + ExtraTimerSec + ' -i ' + bg_value_url + ' -i ' + overlay_image_url + ' -filter_complex "color=color=0x000000@0.0:s= ' + event.Records[0].resized_box_width + 'x' + event.Records[0].resized_box_height + ',drawtext=fontcolor=' + event.Records[0].time_text_color + ':fontsize=' + event.Records[0].time_text_size + ':x=' + event.Records[0].minute_x + ':y=' + event.Records[0].minute_y + ':text=\'%{eif\\:trunc(mod(((' + TimerSec + '-if(between(t, 0, 1),1,if(gte(t,' + TimerSec + '),' + TimerSec + ',t)))/60),60))\\:d\\:2}\',drawtext=fontcolor=' + event.Records[0].time_text_color + ':fontsize=' + event.Records[0].time_text_size + ':x=' + event.Records[0].second_x + ':y=' + event.Records[0].second_y + ':text=\'%{eif\\:trunc(mod(' + TimerSec + '-if(between(t, 0, 1),1,if(gte(t,' + TimerSec + '),' + TimerSec + ',t))\,60))\\:d\\:2}\'[txt]; [txt] rotate=' + event.Records[0].box_angle + '*PI/180:fillcolor=#00000000 [rotated];[0] scale=w=' + BackgroundWidth + ':h=' + BackgroundHeight + '[t];[1] scale=w=' + BackgroundWidth + ':h=' + BackgroundHeight + '[ot];[t][ot] overlay = :x=0 :y=0 [m1];[m1][rotated]overlay = :x=' + event.Records[0].flat_box_coordinate_x + ' :y=' + event.Records[0].flat_box_coordinate_x + ' [m2]" -map "[m2]" -pix_fmt yuv420p -t ' +
                ExtraTimerSec + ' -r 24 -c:a copy ' + videoPath + "";
        }
    }
    var final_command = '/usr/bin/ffmpeg' + command;


    //COMMAND EXECUTE HERE

    await exec(final_command, function (err, stdout, stderr) {
        console.log("data is here")
        console.log('err:', err);
        console.log('stdout:', stdout);
        console.log('stderr:', stderr);
        const params = {
            Bucket: bucketName,
            Key: "countdown/output.mp4",
            Body: stdout,
        }
        s3.upload(params).promise().then(data => {
            console.log("data is here -->", data)
        });
    });
    var tmpFile = fs.createReadStream(videoPath)
    console.log('temp file data:', tmpFile.toString())
};