Recherche avancée

Sur d’autres sites (12754)

• Creating A Lossless SMC Encoder

  26 avril 2011, par Multimedia Mike — General

  Look, I can’t explain how or why I come up with this stuff. For some reason, I thought it would be interesting to write a new encoder for the Apple SMC video codec. I can’t even remember why. I just sat down the other day, started writing, and now I have a lossless SMC encoder that I’m not sure what to do with. Maybe this is to be my new thing— writing encoders for marginal multimedia formats.

  Introduction
  SMC is a vector quantizer (a lossy method) but I decided to attack it from the angle of lossless encoding. A.k.a. Apple Graphics Codec, SMC operates on 4x4 blocks in an 8-bit paletted colorspace. Each 4x4 block can be encoded with 1, 2, 4, 8, or 16 colors. Blocks can also be skipped (copied from previous frame) or copied from blocks rendered immediately prior within the same frame.

  Step 1 : Validating Infrastructure
  The goal of this step is to encode the most braindead SMC frame possible and see if FFmpeg/libav’s QuickTime muxer can create a valid file. I think the simplest frame would be one in which each vector is encoded with the single-color mode, starting with color 0 and incrementing through the palette.

  Status : Successful. The only ’trick’ was to set avctx->bits_per_coded_sample to 8. (For fun, this can also be set to 40 (8 | 0x20) to specify a grayscale palette.)
  

  
  
  

  Step 2 : Preprocessing
  The video frames will arrive at the encoder as 32-bit RGB. These will need to be converted to a paletted colorspace before encoding. I don’t want to use FFmpeg’s default dithering approach as this will result in a substantial loss of quality as described in this post. I would rather maintain a palette built from observed colors throughout successive frames. If the total number of unique observed colors ever exceeds 256, error out.

  That’s what I would like to do. However, I noticed that FFmpeg/libav’s QuickTime muxer has never taken into account the possibility of encoding palettes. The path of least resistance in this case is to dither the input to match QuickTime’s default 8-bit palette (if a paletted QuickTime file does not specify a palette, a default 1-, 2-, 4-, or 8-bit palette is selected).

  Status : Successful, if slow. I definitely need to optimize this step later.

  Step 3 : Most Naive Encoding
  The most basic encoding is to "encode" each block as a 16-color block. This will actually result in a slightly larger frame size than a raw encoding since each 4x4 block will be prepended by a byte opcode (0xE0 in this case) to indicate encoding mode. This should demonstrate that the encoder is functioning at the most basic level.

  Status : Successful. Try not to laugh too hard at the Big Buck Bunny dithered to an 8-bit palette :

  
  
  

  Step 4 : Better Representation
  It seems to me that encoding this format (losslessly) will entail performing vector operations on lots of 16-element (4x4-pixel) vectors. These could be done on the frame as-is, but it strikes me as more efficient and perhaps less error prone to rearrange the input images into a vector of vectors (or array of arrays if you prefer) :

    0  1  2  3  w ...
    4  5  6  7  x ...
    8  9  A  B  y ...
    C  D  E  F  z ...
  

    0 : [0 1 2 3 4 5 6 7 8 9 A B C D E F]
    1 : [...]
  

  Status : Successful.

  Step 5 : Add Interframe Skip Codes
  Time to add a bit of brainpower to the proceedings : On non-keyframes, compare the current vector to the vector at the same position from the previous frame.

  Test this by encoding a pair of identical frames. Ideally, all codes should be skip codes.

  Status : Successful, though my vector matching function could probably be improved.

  Step 6 : Analyze Blocks For Optimal Color Coding
  This is where things get potentially interesting, algorithmically. At least, I need to figure out (or look up) an algorithm to count the unique elements in a vector.

  Naive algorithm (i.e., first thing I can think of) :

  • initialize a count variable to 0
  • initialize an array of 256 flags to false
  • for each 8-bit element in vector :
    • if flag array[element] is 0, set array[element] to true and increment count

  Status : Successful. Here is the distribution for the 640x360 Big Buck Bunny title :

  1194 4636 4113 2140 1138 568 325 154 80 36 9 5 2 0 0 0

  Or, in pretty graph form, demonstrating that vectors with few distinct elements dominate :
  

  
  
  

  Step 7 : Encode Monochrome Blocks
  At this point, the structure is starting to come together pretty well. This phase involves encoding a 0x60 opcode and a palette index when the count_distinct() function returns 1.

  Status : Absolutely no problem.

  Step 8 : Encode 2-, 4-, and 8-color Modes
  This step is a little more involved. This is where SMC’s 2-, 4-, and 8-color circular palette caches come into play. E.g., when the first 2-color block is encoded, the pair of colors it uses will be inserted into entry 0 of the 2-color cache. During the next 2-color block encoding, if the block uses a pair of colors that already occurs in the cache, the encoding can reference that cache entry. Otherwise, it adds the pair to the next available cache entry, looping back around to 0 as necessary.

  I think I should modify the count_distinct() function to also return a 16-byte array that contains a sorted list of the palette indicies used in the vector. The color pair cache will contain 256 16-bit, 32-bit ints for the quads and 64-bit ints for the octets. This will allow a slightly faster linear cache search.

  Status : The 2-color encoding wasn’t too much trouble and I was able to adapt it to the 4-color mode pretty quickly afterward. I’m still having trouble with the insane 8-color coding mode, though. So that’s commented out for the time being.

  Step 9 : Run Encoding and Putting It All Together
  For each frame, convert the input pixels to a paletted format via one method or another (match to default QuickTime palette for first pass). Then, preprocess each vector to determine the minimum number of elements that can be used to represent it, storing the sorted list of distinct colors in a separate array. The number of elements can either be 0 (only for interframes and indicates a skip block), 1, 2, 4, 8, or 16. Also during this phase, for each vector after the first, test if the vector is the same as the previous vector. If it is, denote this fact in the preprocessed encoding (set the high bit of the element count number).

  Finally, pack it into the bytestream. Iterate through the element count array and search for the longest runs of elements that are encoded with the same mode (up to 256 for skip modes, up to 16 for other modes). If the high bit of an element count is set, that indicates that a copy mode can be encoded. Look for the longest run of element counts with the high bit set and encode a copy mode.

  Status : In-process. Will finish this as motivation strikes.

• Translating Return To Ringworld

  17 août 2016, par Multimedia Mike — Game Hacking

  As indicated in my previous post, the Translator has expressed interest in applying his hobby towards another DOS adventure game from the mid 1990s : Return to Ringworld (henceforth R2RW) by Tsunami Media. This represents significantly more work than the previous outing, Phantasmagoria.

  
  
  Return to Ringworld Title Screen
  

  I have been largely successful thus far in crafting translation tools. I have pushed the fruits of these labors to a Github repository named improved-spoon (named using Github’s random name generator because I wanted something more interesting than ‘game-hacking-tools’).

  Further, I have recorded everything I have learned about the game’s resource format (named RLB) at the XentaxWiki.

  New Challenges
  The previous project mostly involved scribbling subtitle text on an endless series of video files by leveraging a separate software library which took care of rendering fonts. In contrast, R2RW has at least 30k words of English text contained in various blocks which require translation. Further, the game encodes its own fonts (9 of them) which stubbornly refuse to be useful for rendering text in nearly any other language.

  Thus, the immediate 2 challenges are :
  

  1. Translating volumes of text to Spanish
  2. Expanding the fonts to represent Spanish characters

  Normally, “figuring out the file format data structures involved” is on the list as well. Thankfully, understanding the formats is not a huge challenge since the folks at the ScummVM project already did all the heavy lifting of reverse engineering the file formats.

  The Pitch
  Here was the plan :

  • Create a tool that can dump out the interesting data from the game’s master resource file.
  • Create a tool that can perform the elaborate file copy described in the previous post. The new file should be bit for bit compatible with the original file.
  • Modify the rewriting tool to repack some modified strings into the new resource file.
  • Unpack the fonts and figure out a way to add new characters.
  • Repack the new fonts into the resource file.
  • Repack message strings with Spanish characters.

  Showing The Work : Modifying Strings
  First, I created the tool to unpack blocks of message string resources. I elected to dump the strings to disk as JSON data since it’s easy to write and read JSON using Python, and it’s quick to check if any mistakes have crept in.

  The next step is to find a string to focus on. So I started the game and looked for the first string I could trigger :

  
  
  

  This shows up in the JSON string dump as :

    
      "Spanish" : " !0205Your quarters on the Lance of Truth are spartan, in accord with your mercenary lifestyle.",
      "English" : " !0205Your quarters on the Lance of Truth are spartan, in accord with your mercenary lifestyle."
    ,
  

  As you can see, many of the strings are encoded with an ID key as part of the string which should probably be left unmodified. I changed the Spanish string :

    
      "Spanish" : " !0205Hey, is this thing on ?",
      "English" : " !0205Your quarters on the Lance of Truth are spartan, in accord with your mercenary lifestyle."
    ,
  

  And then I wrote the repacking tool to substitute this message block for the original one. Look ! The engine liked it !

  
  
  

  Little steps, little steps.

  Showing The Work : Modifying Fonts
  The next little step is to find a place to put the new characters. First, a problem definition : The immediate goal is to translate the game into Spanish. The current fonts encoded in the game resource only support 128 characters, corresponding to 7-bit ASCII. In order to properly express Spanish, 16 new characters are required : á, é, í, ó, ú, ü, ñ (each in upper and lower case for a total of 14 characters) as well as the inverted punctuation symbols : ¿, ¡.

  Again, ScummVM already documents (via code) the font coding format. So I quickly determined that each of the 9 fonts is comprised of 128 individual bitmaps with either 1 or 2 bits per pixel. I wrote a tool to unpack each character into an individual portable grey map (PGM) image. These can be edited with graphics editors or with text editors since they are just text files.

  Where to put the 16 new Spanish characters ? ASCII characters 1-31 are non-printable, so my first theory was that these characters would be empty and could be repurposed. However, after dumping and inspecting, I learned that they represent the same set of characters as seen in DOS Code Page 437. So that’s a no-go (so I assumed ; I didn’t check if any existing strings leveraged those characters).

  My next plan was hope that I could extend the font beyond index 127 and use positions 128-143. This worked superbly. This is the new example string :

    
      "Spanish" : " !0205¿Ves esto ? ¡La puntuacion se hace girar !",
      "English" : " !0205Your quarters on the Lance of Truth are spartan, in accord with your mercenary lifestyle."
    ,
  

  Fortunately, JSON understands UTF-8 and after mapping the 16 necessary characters down to the numeric range of 128-143, I repacked the new fonts and the new string :

  
  
  Translation : “See this ? The punctuation is rotated !”
  

  Another victory. Notice that there are no diacritics in this string. None are required for this translation (according to Google Translate). But adding the diacritics to the 14 characters isn’t my department. My tool does help by prepopulating [aeiounAEIOUN] into the right positions to make editing easier for the Translator. But the tool does make the effort to rotate the punctuation since that is easy to automate.

  Next Steps and Residual Weirdness
  There is another method for storing ASCII text inside the R2RW resource called strip resources. These store conversation scripts. There are plenty of fields in the data structures that I don’t fully understand. So, following the lessons I learned from my previous translation outing, I was determined to modify as little as possible. This means copying over most of the original data structures intact, but changing the field representing the relative offset that points to the corresponding string. This works well since the strings are invariably stored NULL-terminated in a concatenated manner.

  I wanted to document for the record that the format that R2RW uses has some weirdness in they way it handles residual bytes in a resource. The variant of the resource format that R2RW uses requires every block to be aligned on a 16-byte boundary. If there is space between the logical end of the resource and the start of the next resource, there are random bytes in that space. This leads me to believe that these bytes were originally recorded from stale/uninitialized memory. This frustrates me because when I write the initial file copy tool which unpacks and repacks each block, I want the new file to be identical to the original. However, these apparent nonsense bytes at the end thwart that effort.

  But leaving those bytes as 0 produces an acceptable resource file.

  Text On Static Images
  There is one last resource type we are working on translating. There are various bits of text that are rendered as images. For example, from the intro :

  
  
  

  It’s possible to locate and extract the exact image that is overlaid on this scene, though without the colors :

  
  
  

  The palettes are stored in a separate resource type. So it seems the challenge is to figure out the palette in use for these frames and render a transparent image that uses the same palette, then repack the new text-image into the new resource file.

• PHP Foreach query only displays final result for slideshow ?

  22 janvier 2023, par ShaneRibz

  Using a query I pull the most liked video's on a localhosted stream webpage, and use FFMPEG to create a thumbnail if one isnt already existing and the video length.

  


  The query succesfully pulls the 4 most liked video's and displays them in a carousel. FFMPEG succesfully creates each thumbnail, and gets the length of each video.

  


  Somewhere, for whatever reason I cant figure out, only the final slide in the carousel displays the thumbnail as its background.

  


  Sorry if my code is messy, I'm learning how to better optimize.

  


  &lt;?php&#xA;                $sql = "use info";&#xA;                $pdo->exec($sql);&#xA;                $stmt = $pdo->query("SELECT * FROM `files` ORDER BY `like` DESC LIMIT 4");&#xA;                $filelist = $stmt->fetchAll(PDO::FETCH_ASSOC);&#xA;                    $h = 0;&#xA;                  foreach ($filelist as $row) {&#xA;                        if ($h &lt; 25){&#xA;                        $h&#x2B;&#x2B;;&#xA;                &#xA;                            $name = $row[&#x27;name&#x27;];&#xA;                            $location = $row[&#x27;location&#x27;];&#xA;                            $type = $row[&#x27;type&#x27;];&#xA;                            $uploadby = $row[&#x27;uploadby&#x27;];&#xA;                            $like = $row[&#x27;like&#x27;];&#xA;                            $id = $row[&#x27;fileid&#x27;];&#xA;                            $cat1 = $row[&#x27;Cat1&#x27;];&#xA;                            $cat2 = $row[&#x27;Cat2&#x27;];&#xA;            &#xA;                $ffmpeg = &#x27;F:\\xampp\ffmpeg.exe&#x27;;  &#xA;                $location = str_replace("/","\\", $location);&#xA;                $video = &#x27;F:\xampp\htdocs\\&#x27;. $location;  &#xA;                $bigloc = "F:\\xampp\htdocs\\thumb\big";&#xA;                $smallloc = "F:\\xampp\htdocs\\thumb\small";&#xA;                $image = $bigloc . "\\" . $name . ".jpg"; &#xA;                $image2 = $smallloc . "\\" . $name. ".jpg"; &#xA;                            &#xA;                if (file_exists($image) and (file_exists($image2))) {&#xA;                    &#xA;                } else {&#xA;                echo $image . " does not exist";&#xA;                $cmd="$ffmpeg -ss 00:25:00 -i ". "\"" . $video . "\"". " -s 1920x540 -vframes 1 ". "\"" .$image. "\"" . " -report";  &#xA;                $cmd2="$ffmpeg -ss 00:25:00 -i ". "\"" . $video . "\""." -s 270x370 -vframes 1 ". "\"" .$image2. "\"";&#xA;                echo "<br />". $cmd . "<br />".$cmd2."<br />";&#xA;                exec($cmd);&#xA;                exec($cmd2);&#xA;                }&#xA;                &#xA;                $file = "\"". $video. "\"";&#xA;                $result = shell_exec(&#x27;ffmpeg -i &#x27; . escapeshellcmd($file) . &#x27; 2>&amp;1&#x27;);&#xA;                preg_match(&#x27;/(?&lt;=Duration: )(\d{2}:\d{2}:\d{2})\.\d{2}/&#x27;, $result, $match);&#xA;                $time = $match[1];&#xA;                $image = "thumb/big/" . $name . ".jpg"; &#xA;                $image2 = "thumb/small/" . $name. ".jpg";&#xA;&#xA;            ?>&#xA;&#xA;            <div class="single-hero-slider-wrap single-animation-wrap slider-height-hm4 bg-image-hm4 slider-bg-color-black d-flex align-items-center slider-bg-position-1 bg-black" style="&amp;lt;?php echo &amp;#x27;background-image:url(&amp;#x27;. $image.&amp;#x27;);&amp;#x27;;?>">&#xA;                <div class="slider-content-hm4 slider-animated">&#xA;                    <h1 class="title animated">&lt;?php echo $name;?></h1>&#xA;                    <div class="sub-title-time-wrap">&#xA;                        <span class="sub-title animated">&lt;?php if ($cat2 != ""){ echo $cat1 .", ". $cat2; }else{ echo $cat1;}?></span>&#xA;                        <span class="time animated">&lt;?php echo $time;?></span>&#xA;                    </div>&#xA;                    <div class="slider-button">&#xA;                        <a href="http://stackoverflow.com/feeds/tag/movie-details.html" class="btn-style-hm4 animated">Watch Now</a>&#xA;                    </div>&#xA;                </div>&#xA;            </div>&#xA;&#xA;            &lt;?php    &#xA;                        }&#xA;                  }              &#xA;            ?>&#xA;

  


  When inspecting the source code on my web browser ;
"style="<?php echo 'background-image:url('. $image.');';?>"", this will only appear on the final slide. The first 3 slides wont have "background-image" at all.

  


  I feel as though I'm making a silly mistake and just cant see it.

  


  EDIT
I should note, the "$name" and "$time" for each displays properly, its just the background-image that does not.

  


  EDIT 2
Here is the source code :

  


  <div class="single-hero-slider-wrap single-animation-wrap slider-height-hm4 bg-image-hm4 slider-bg-color-black d-flex align-items-center slider-bg-position-1 bg-black" style="background-image: url(&amp;#x27;thumb/big/Alita Battle Angel.png&amp;#x27;);">&#xA;</div>

  


  The "background-image" shows in source, but during inspect element it does not. No errors populate either.