Recherche avancée

Sur d’autres sites (8338)

• On WebP and Academic Exercises

  2 octobre 2010, par Multimedia Mike — General

  Yesterday, Google released a new still image format called WebP. To those skilled in the art, this new format will be recognizable as a single VP8 golden frame with a 20-byte header slapped on the front (and maybe a little metadata thrown in for good measure). We have a MultimediaWiki page and a sample ready to go.

  Further, I submitted a patch to ffmpeg-devel for FFmpeg’s img2 handling system to decode these files. FFmpeg should support processing these files soon… if anyone cares. This leads into…

  The Point, or Lack Thereof
  Since yesterday’s release, I have read a whirlwind of commentary about this format, much of it critical and of the “what’s the point ?” variety. For my part, I can respect academic exercises, a.k.a., just trying random stuff to see if you can make it work. That’s pretty much this blog’s entire raison d’être. But WebP transcends mere academic exercise ; Google seems to be trying to push it as a new web standard. I don’t see how the format can go anywhere based on criticisms raised elsewhere — e.g., see Dark Shikari’s thoughtful write-up — which basically boil down to WebP not solving any real problems, technical, legal, or otherwise.

  How did WebP come to be ? I strongly suspect some engineers noticed that JPEG is roughly the same as an MPEG-1 intraframe, so why not create a new still frame format based on VP8 intraframes ? Again, I can respect that thinking– I have pondered how a still image format would perform if based on VP3/Theora or Sorenson Video 1.

  Technically
  Google claims a significant size savings for WebP vs. standard JPEG. Assuming that’s true (and there will be no shortage of blog posts to the contrary), it will still be some time before WebP support will find its way into the majority of the web browser population.

  But this got me thinking about possible interim solutions. A website could store images compressed in both formats if it so chose. Then it could serve up a WebM image if the browser could support it, as indicated by the ‘Accept’ header in the HTTP request. It seems that a website might have to reference a generic image name such as <img src="some-picture.image"> ; the web server would have to recognize the .image extension and map it to either a .jpg or a .webp image depending on what the browser claims it is capable of displaying.

  Leftovers
  I appreciate that Dark Shikari has once again stuck his neck out and made a valiant — though often futile — effort to educate the internet’s masses. I long ago resigned myself to the fact that many people aren’t going to understand many of the most basic issues surrounding multimedia technology (i.e., moving pictures synchronized with audio). But apparently, this extends to still image formats as well. It was simultaneously humorous and disheartening to see commenters who don’t even understand the application of, e.g., PNG vs. JPEG : Ahem, “We already have a great replacement for jpg : .PNG”. Coupled with the typical accusations of MPEG tribalism, I remain impressed D. Shikari finds the will to bother.

  Still, I appreciate that the discussion has introduced me to some new image formats of which I was previously unaware, such as PGF and JPEG XR.

• Texai Remote Presence System Using VP8 Video

  19 novembre 2010, par noreply@blogger.com (John Luther)

  Guest blogger Josh Tyler is a member of the Texai team at Willow Garage.

  Willow Garage is busy building the next version of its Texai remote presence platform with VP8, the video codec used in WebM.

  In short, Texai is a two-way videoconferencing app on a tele-operated robotic platform (for more details, see the piece about Texai in the New York Times). Video and audio quality are critical to providing the best user experience on Texai. We’ve evaluated several video codecs and found VP8’s image quality, low latency and tolerance to packet loss far better than anything else we tested.

  The video below shows VP8 running on one of our systems.

  (If you have a WebM-enabled browser and are enrolled in the YouTube HTML5 beta the video will play in WebM HTML5, otherwise it will play in Flash Player.)

  We’re also looking for help ! If you’re interested in helping us create an incredible, high-fidelity user experience, either by joining our team, partnering on development, or by being added to our early customer interest list, please email us at texai-info@willowgarage.com.

  

• Adventures in Unicode

  29 novembre 2012, par Multimedia Mike — Programming, php, Python, sqlite3, unicode

  Tangential to multimedia hacking is proper metadata handling. Recently, I have gathered an interest in processing a large corpus of multimedia files which are likely to contain metadata strings which do not fall into the lower ASCII set. This is significant because the lower ASCII set intersects perfectly with my own programming comfort zone. Indeed, all of my programming life, I have insisted on covering my ears and loudly asserting “LA LA LA LA LA ! ALL TEXT EVERYWHERE IS ASCII !” I suspect I’m not alone in this.

  Thus, I took this as an opportunity to conquer my longstanding fear of Unicode. I developed a self-learning course comprised of a series of exercises which add up to this diagram :

  
  
  

  Part 1 : Understanding Text Encoding
  Python has regular strings by default and then it has Unicode strings. The latter are prefixed by the letter ‘u’. This is what ‘ö’ looks like encoded in each type.

  python

  < view plain text >

  1. >>> ’ö’, u’ö’
  2. (’\xc3\xb6’, u’\xf6’)

  A large part of my frustration with Unicode comes from Python yelling at me about UnicodeDecodeErrors and an inability to handle the number 0xc3 for some reason. This usually comes when I’m trying to wrap my head around an unrelated problem and don’t care to get sidetracked by text encoding issues. However, when I studied the above output, I finally understood where the 0xc3 comes from. I just didn’t understand what the encoding represents exactly.
  

  I can see from assorted tables that ‘ö’ is character 0xF6 in various encodings (in Unicode and Latin-1), so u’\xf6′ makes sense. But what does ‘\xc3\xb6′ mean ? It’s my style to excavate straight down to the lowest levels, and I wanted to understand exactly how characters are represented in memory. The UTF-8 encoding tables inform us that any Unicode code point above 0x7F but less than 0×800 will be encoded with 2 bytes :

   110xxxxx 10xxxxxx
  

  Applying this pattern to the \xc3\xb6 encoding :

              hex : 0xc3      0xb6
             bits : 11000011  10110110
   important bits : ---00011  —110110
        assembled : 00011110110
       code point : 0xf6
  

  I was elated when I drew that out and made the connection. Maybe I’m the last programmer to figure this stuff out. But I’m still happy that I actually understand those Python errors pertaining to the number 0xc3 and that I won’t have to apply canned solutions without understanding the core problem.

  I’m cheating on this part of this exercise just a little bit since the diagram implied that the Unicode text needs to come from a binary file. I’ll return to that in a bit. For now, I’ll just contrive the following Unicode string from the Python REPL :

  python

  < view plain text >

  1. >>> u = u’Üñìçôđé’
  2. >>> u
  3. u’\xdc\xf1\xec\xe7\xf4\u0111\xe9’

  Part 2 : From Python To SQLite3
  The next step is to see what happens when I use Python’s SQLite3 module to dump the string into a new database. Will the Unicode encoding be preserved on disk ? What will UTF-8 look like on disk anyway ?

  python

  < view plain text >

  1. >>> import sqlite3
  2. >>> conn = sqlite3.connect(’unicode.db’)
  3. >>> conn.execute("CREATE TABLE t (t text)")
  4. >>> conn.execute("INSERT INTO t VALUES (?)", (u, ))
  5. >>> conn.commit()
  6. >>> conn.close()

  Next, I manually view the resulting database file (unicode.db) using a hex editor and look for strings. Here we go :

  000007F0   02 29 C3 9C  C3 B1 C3 AC  C3 A7 C3 B4  C4 91 C3 A9
  

  Look at that ! It’s just like the \xc3\xf6 encoding we see in the regular Python strings.

  Part 3 : From SQLite3 To A Web Page Via PHP
  Finally, use PHP (love it or hate it, but it’s what’s most convenient on my hosting provider) to query the string from the database and display it on a web page, completing the outlined processing pipeline.

  php

  < view plain text >

  1. < ?php
  2. $dbh = new PDO("sqlite:unicode.db") ;
  3. foreach ($dbh->query("SELECT t from t") as $row) ;
  4. $unicode_string = $row[’t’] ;
  5.  ?>
  6.  
  7. <html>
  8. <head><meta http-equiv="Content-Type" content="text/html ; charset=utf-8"></meta></head>
  9. <body><h1>< ?=$unicode_string ?></h1></body>
  10. </html>

  I tested the foregoing PHP script on 3 separate browsers that I had handy (Firefox, Internet Explorer, and Chrome) :

  
  
  

  I’d say that counts as success ! It’s important to note that the “meta http-equiv” tag is absolutely necessary. Omit and see something like this :

  
  
  

  Since we know what the UTF-8 stream looks like, it’s pretty obvious how the mapping is operating here : 0xc3 and 0xc4 correspond to ‘Ã’ and ‘Ä’, respectively. This corresponds to an encoding named ISO/IEC 8859-1, a.k.a. Latin-1. Speaking of which…

  Part 4 : Converting Binary Data To Unicode
  At the start of the experiment, I was trying to extract metadata strings from these binary multimedia files and I noticed characters like our friend ‘ö’ from above. In the bytestream, this was represented simply with 0xf6. I mistakenly believed that this was the on-disk representation of UTF-8. Wrong. Turns out it’s Latin-1.

  However, I still need to solve the problem of transforming such strings into Unicode to be shoved through the pipeline diagrammed above. For this experiment, I created a 9-byte file with the Latin-1 string ‘Üñìçôdé’ couched by 0′s, to simulate yanking a string out of a binary file. Here’s unicode.file :

  00000000   00 DC F1 EC  E7 F4 64 E9  00         ......d..
  

  (Aside : this experiment uses plain ‘d’ since the ‘đ’ with a bar through it doesn’t occur in Latin-1 ; shows up all over the place in Vietnamese, at least.)

  I’ve been mashing around Python code via the REPL, trying to get this string into a Unicode-friendly format. This is a successful method but it’s probably not the best :

  python

  < view plain text >

  1. >>> import struct
  2. >>> f = open(’unicode.file’, ’r’).read()
  3. >>> u = u’’
  4. >>> for c in struct.unpack("B"*7, f[1 :8]) :
  5. ... u += unichr(c)
  6. ...
  7. >>> u
  8. u’\xdc\xf1\xec\xe7\xf4d\xe9’
  9. >>> print u
  10. Üñìçôdé

  Conclusion
  Dealing with text encoding matters reminds me of dealing with integer endian-ness concerns. When you’re just dealing with one system, you probably don’t need to think too much about it because the system is usually handling everything consistently underneath the covers.

  However, when the data leaves one system and will be interpreted by another system, that’s when a programmer needs to be cognizant of matters such as integer endianness or text encoding.