Sur d’autres sites (10184)

• Creating A Lossless SMC Encoder

  26 avril 2011, par Multimedia Mike — General

  Look, I can’t explain how or why I come up with this stuff. For some reason, I thought it would be interesting to write a new encoder for the Apple SMC video codec. I can’t even remember why. I just sat down the other day, started writing, and now I have a lossless SMC encoder that I’m not sure what to do with. Maybe this is to be my new thing— writing encoders for marginal multimedia formats.

  Introduction
  SMC is a vector quantizer (a lossy method) but I decided to attack it from the angle of lossless encoding. A.k.a. Apple Graphics Codec, SMC operates on 4x4 blocks in an 8-bit paletted colorspace. Each 4x4 block can be encoded with 1, 2, 4, 8, or 16 colors. Blocks can also be skipped (copied from previous frame) or copied from blocks rendered immediately prior within the same frame.

  Step 1 : Validating Infrastructure
  The goal of this step is to encode the most braindead SMC frame possible and see if FFmpeg/libav’s QuickTime muxer can create a valid file. I think the simplest frame would be one in which each vector is encoded with the single-color mode, starting with color 0 and incrementing through the palette.

  Status : Successful. The only ’trick’ was to set avctx->bits_per_coded_sample to 8. (For fun, this can also be set to 40 (8 | 0x20) to specify a grayscale palette.)
  

  
  
  

  Step 2 : Preprocessing
  The video frames will arrive at the encoder as 32-bit RGB. These will need to be converted to a paletted colorspace before encoding. I don’t want to use FFmpeg’s default dithering approach as this will result in a substantial loss of quality as described in this post. I would rather maintain a palette built from observed colors throughout successive frames. If the total number of unique observed colors ever exceeds 256, error out.

  That’s what I would like to do. However, I noticed that FFmpeg/libav’s QuickTime muxer has never taken into account the possibility of encoding palettes. The path of least resistance in this case is to dither the input to match QuickTime’s default 8-bit palette (if a paletted QuickTime file does not specify a palette, a default 1-, 2-, 4-, or 8-bit palette is selected).

  Status : Successful, if slow. I definitely need to optimize this step later.

  Step 3 : Most Naive Encoding
  The most basic encoding is to "encode" each block as a 16-color block. This will actually result in a slightly larger frame size than a raw encoding since each 4x4 block will be prepended by a byte opcode (0xE0 in this case) to indicate encoding mode. This should demonstrate that the encoder is functioning at the most basic level.

  Status : Successful. Try not to laugh too hard at the Big Buck Bunny dithered to an 8-bit palette :

  
  
  

  Step 4 : Better Representation
  It seems to me that encoding this format (losslessly) will entail performing vector operations on lots of 16-element (4x4-pixel) vectors. These could be done on the frame as-is, but it strikes me as more efficient and perhaps less error prone to rearrange the input images into a vector of vectors (or array of arrays if you prefer) :

    0  1  2  3  w ...
    4  5  6  7  x ...
    8  9  A  B  y ...
    C  D  E  F  z ...
  

    0 : [0 1 2 3 4 5 6 7 8 9 A B C D E F]
    1 : [...]
  

  Status : Successful.

  Step 5 : Add Interframe Skip Codes
  Time to add a bit of brainpower to the proceedings : On non-keyframes, compare the current vector to the vector at the same position from the previous frame.

  Test this by encoding a pair of identical frames. Ideally, all codes should be skip codes.

  Status : Successful, though my vector matching function could probably be improved.

  Step 6 : Analyze Blocks For Optimal Color Coding
  This is where things get potentially interesting, algorithmically. At least, I need to figure out (or look up) an algorithm to count the unique elements in a vector.

  Naive algorithm (i.e., first thing I can think of) :

  • initialize a count variable to 0
  • initialize an array of 256 flags to false
  • for each 8-bit element in vector :
    • if flag array[element] is 0, set array[element] to true and increment count

  Status : Successful. Here is the distribution for the 640x360 Big Buck Bunny title :

  1194 4636 4113 2140 1138 568 325 154 80 36 9 5 2 0 0 0

  Or, in pretty graph form, demonstrating that vectors with few distinct elements dominate :
  

  
  
  

  Step 7 : Encode Monochrome Blocks
  At this point, the structure is starting to come together pretty well. This phase involves encoding a 0x60 opcode and a palette index when the count_distinct() function returns 1.

  Status : Absolutely no problem.

  Step 8 : Encode 2-, 4-, and 8-color Modes
  This step is a little more involved. This is where SMC’s 2-, 4-, and 8-color circular palette caches come into play. E.g., when the first 2-color block is encoded, the pair of colors it uses will be inserted into entry 0 of the 2-color cache. During the next 2-color block encoding, if the block uses a pair of colors that already occurs in the cache, the encoding can reference that cache entry. Otherwise, it adds the pair to the next available cache entry, looping back around to 0 as necessary.

  I think I should modify the count_distinct() function to also return a 16-byte array that contains a sorted list of the palette indicies used in the vector. The color pair cache will contain 256 16-bit, 32-bit ints for the quads and 64-bit ints for the octets. This will allow a slightly faster linear cache search.

  Status : The 2-color encoding wasn’t too much trouble and I was able to adapt it to the 4-color mode pretty quickly afterward. I’m still having trouble with the insane 8-color coding mode, though. So that’s commented out for the time being.

  Step 9 : Run Encoding and Putting It All Together
  For each frame, convert the input pixels to a paletted format via one method or another (match to default QuickTime palette for first pass). Then, preprocess each vector to determine the minimum number of elements that can be used to represent it, storing the sorted list of distinct colors in a separate array. The number of elements can either be 0 (only for interframes and indicates a skip block), 1, 2, 4, 8, or 16. Also during this phase, for each vector after the first, test if the vector is the same as the previous vector. If it is, denote this fact in the preprocessed encoding (set the high bit of the element count number).

  Finally, pack it into the bytestream. Iterate through the element count array and search for the longest runs of elements that are encoded with the same mode (up to 256 for skip modes, up to 16 for other modes). If the high bit of an element count is set, that indicates that a copy mode can be encoded. Look for the longest run of element counts with the high bit set and encode a copy mode.

  Status : In-process. Will finish this as motivation strikes.

• Nexus One

  19 mars 2010, par Mans — Uncategorized

  I have had a Nexus One for about a week (thanks Google), and naturally I have an opinion or two about it.
  

  Hardware

  With the front side dominated by a touch-screen and a lone, round button, the Nexus One appearance is similar to that of most contemporary smartphones. The reverse sports a 5 megapixel camera with LED flash, a Google logo, and a smaller HTC logo. Power button, volume control, and headphone and micro-USB sockets are found along the edges. It is with appreciation I note the lack of a front-facing camera ; the silly idea of video calls is finally put to rest.

  Powering up the phone (I’m beginning to question the applicability of that word), I am immediately enamoured with the display. At 800×480 pixels, the AMOLED display is crystal-clear and easily viewable even in bright light. In a darker environment, the display automatically dims. The display does have one quirk in that the subpixel pattern doesn’t actually have a full RGB triplet for each pixel. The close-up photo below shows the pattern seen when displaying a solid white colour.

  

  The result of this is that fine vertical lines, particularly red or blue ones, look a bit jagged. Most of the time this is not much of a problem, and I find it an acceptable compromise for the higher effective resolution it provides.

  Basic interaction

  The Android system is by now familiar, and the Nexus offers no surprises in basic usage. All the usual applications come pre-installed : browser, email, calendar, contacts, maps, and even voice calls. Many of the applications integrate with a Google account, which is nice. Calendar entries, map placemarks, etc. are automatically shared between desktop and mobile. Gone is the need for the bug-ridden custom synchronisation software with which mobile phones of the past were plagued.

  Launching applications is mostly speedy, and recently used apps are kept loaded as long as memory needs allow. Although this garbage-collection-style of application management, where you are never quite sure whether an app is still running, takes a few moments of acclimatisation, it works reasonably well in day to day use. Most of the applications are well-behaved and save their data before terminating.

  Email

  Two email applications are included out of the box : one generic and one Gmail-only. As I do not use Gmail, I cannot comment on this application. The generic email client supports IMAP, but is rather limited in functionality. Fortunately, a much-enhanced version, K-9, is available for download. The main feature I find lacking here is threaded message view.

  The features, or lack thereof, in the email applications is not, however, of huge importance, as composing email, or any longer piece of text, is something one rather avoids on a system like this. The on-screen keyboard, while falling among the better of its kind, is still slow to use. Lack of tactile feedback means accidentally tapping the wrong key is easily done, and entering numbers or punctuation is an outright chore.

  Browser

  Whatever the Nexus lacks in email abilities, it makes up for with the browser. Surfing the web on a phone has never been this pleasant. Page rendering is quick, and zooming is fast and simple. Even pages not designed for mobile viewing are easy to read with smart reformatting almost entirely eliminating the sideways scrolling which hampered many a mobile browser of old.

  Calls and messaging

  Being a phone, the Nexus One is obviously able to make and receive calls, and it does so with ease. Entering a number or locating a stored contact are both straight-forward operations. During a call, audio is clear and of adequate loudness, although I have yet to use the phone in really noisy surroundings.

  The other traditional task of a mobile phone, messaging, is also well-supported. There isn’t really much to say about this.

  Multimedia

  Having a bit of an interest in most things multimedia, I obviously tested the capabilities of the Nexus by throwing some assorted samples at it, revealing ample space for improvement. With video limited to H.264 and MPEG4, and the only supported audio codecs being AAC, MP3, Vorbis, and AMR, there are many files which will not play.

  To make matters worse, only selected combinations of audio and video will play together. Several video files I tested played without sound, yet when presented with the very same audio data alone, it was correctly decoded. As for container formats, it appears restricted to MP4/MOV, and Ogg (for Vorbis). AVI files are recognised as media files, but I was unable to find an AVI file which would play.

  With a device clearly capable of so much more, the poor multimedia support is nothing short of embarrassing.

  The Market

  Much of the hype surrounding Android revolves around the Market, Google’s virtual marketplace for app authors to sell or give away their creations. The thousands of available applications are broadly categorised, and a search function is available.

  The categorised lists are divided into free and paid sections, while search results, disappointingly, are not. To aid the decision, ratings and comments are displayed alongside the summary and screenshots of each application. Overall, the process of finding and installing an application is mostly painless. While it could certainly be improved, it could also have been much worse.

  The applications themselves are, as hinted above, beyond numerous. Sadly, quality does not quite match up to quantity. The vast majority of the apps are pointless, though occasionally mildly amusing, gimmicks of no practical value. The really good ones, and they do exist, are very hard to find unless one knows precisely what to look for.

  Battery

  Packing great performance into a pocket-size device comes with a price in battery life. The battery in the Nexus lasts considerably shorter time than that in my older, less feature-packed Nokia phone. To some extent this is probably a result of me actually using it a lot more, yet the end result is the same : more frequent recharging. I should probably get used to the idea of recharging the phone every other night.

  Verdict

  The Nexus One is a capable hardware platform running an OS with plenty of potential. The applications are still somewhat lacking (or very hard to find), although the basic features work reasonably well. Hopefully future Android updates will see more and better core applications integrated, and I imagine that over time, I will find third-party apps to solve my problems in a way I like. I am not putting this phone on the shelf just yet.

• Hacking the Popcorn Hour C-200

  3 mai 2010, par Mans — Hardware, MIPS

  Update : A new firmware version has been released since the publication of this article. I do not know if the procedure described below will work with the new version.

  The Popcorn Hour C-200 is a Linux-based media player with impressive specifications. At its heart is a Sigma Designs SMP8643 system on chip with a 667MHz MIPS 74Kf as main CPU, several co-processors, and 512MB of DRAM attached. Gigabit Ethernet, SATA, and USB provide connectivity with the world around it. With a modest $299 on the price tag, the temptation to repurpose the unit as a low-power server or cheap development board is hard to resist. This article shows how such a conversion can be achieved.
  

  Kernel

  The PCH runs a patched Linux 2.6.22.19 kernel. A source tarball is available from the manufacturer. This contains the sources with Sigma support patches, Con Kolivas’ patch set (scheduler tweaks), and assorted unrelated changes. Properly split patches are unfortunately not available. I have created a reduced patch against vanilla 2.6.22.19 with only Sigma-specific changes, available here.

  The installed kernel has a number of features disabled, notably PTY support and oprofile. We will use kexec to load a more friendly one.

  As might be expected, the PCH kernel does not have kexec support enabled. It does however, by virtue of using closed-source components, support module loading. This lets us turn kexec into a module and load it. A patch for this is available here. To build the module, apply the patch to the PCH sources and build using this configuration. This will produce two modules, kexec.ko and mips_kexec.ko. No other products of this build will be needed.

  The replacement kernel can be built from the PCH sources or, if one prefers, from vanilla 2.6.22.19 with the Sigma-only patch. For the latter case, this config provides a minimal starting point suitable for NFS-root.

  When configuring the kernel, make sure CONFIG_TANGOX_IGNORE_CMDLINE is enabled. Otherwise the command line will be overridden by a useless one stored in flash. A good command line can be set with CONFIG_CMDLINE (under “Kernel hacking” in menuconfig) or passed from kexec.

  Taking control

  In order to load our kexec module, we must first gain root privileges on the PCH, and here a few features of the system are working to our advantage :

  1. The PCH allows mounting any NFS export to access media files stored there.
  2. There is an HTTP server running. As root.
  3. This HTTP server can be readily instructed to fetch files from an NFS mount.
  4. Files with a name ending in .cgi are executed. As root.

  All we need do to profit from this is place the kexec modules, the kexec userspace tools, and a simple script on an NFS export. Once this is done, and the mount point configured on the PCH, a simple HTTP request will send the old kernel screaming to /dev/null, our shiny new kernel taking its place.

  The rootfs

  A kernel is mostly useless without a root filesystem containing tools and applications. A number of tools for cross-compiling a full system exist, each with its strengths and weaknesses. The only thing to look out for is the version of kernel headers used (usually a linux-headers package). As we will be running an old kernel, chances are the default version is too recent. Other than this, everything should be by the book.

  Assembling the parts

  Having gathered all the pieces, it is now time to assemble the hack. The following steps are suitable for an NFS-root system. Adaptation to a disk-based system is left as an exercise.

  1. Build a rootfs for MIPS 74Kf little endian. Make sure kernel headers used are no more recent than 2.6.22.x. Include a recent version of the kexec userspace tools.
  2. Fetch and unpack the PCH kernel sources.
  3. Apply the modular kexec patch.
  4. Using this config, build the modules and install them as usual to the rootfs. The version string must be 2.6.22.19-19-4.
  5. From either the same kernel sources or plain 2.6.22.19 with Sigma patches, build a vmlinux and (optionally) modules using this config. Modify the compiled-in command line to point to the correct rootfs. Set the version string to something other than in the previous step.
  6. Copy vmlinux to any directory in the rootfs.
  7. Copy kexec.sh and kexec.cgi to the same directory as vmlinux.
  8. Export the rootfs over NFS with full read/write permissions for the PCH.
  9. Power on the PCH, and update to latest firmware.
  10. Configure an NFS mount of the rootfs.
  11. Navigate to the rootfs in the PCH UI. A directory listing of bin, dev, etc. should be displayed.
  12. On the host system, run the kexec.sh script with the target hostname or IP address as argument.
  13. If all goes well, the new kernel will boot and mount the rootfs.

  Serial console

  A serial console is indispensable for solving boot problems. The PCH board has two UART connectors. We will use the one labeled UART0. The pinout is as follows (not standard PC pinout).

          +-----------+
         2| * * * * * |10
         1| * * * * * |9
          -----------+
            J7 UART0
      /---------------------/ board edge
  

  Pin	Function
  1	+5V
  5	Rx
  6	Tx
  10	GND

  The signals are 3.3V so a converter, e.g. MAX202, is required for connecting this to a PC serial port. The default port settings are 115200 bps 8n1.