Recherche avancée

Sur d’autres sites (10702)

• Decoding the h.264 stream from a serial port

  18 mars, par Peter

  I would like to know if there is a reliable way to decode an H.264 NAL stream coming through a serial port using software.

  


  So far, I have managed to decode a single frame using a python script. In this script, I first write the incoming data to a file, and when the end-of-frame marker 00_00_00_01 appears, I display the frame using ffplay.

  


  import serial
import subprocess
import os
import time

ser = serial.Serial('COM3', 115200, timeout=1)
output_file = "output.264"

# Variable to store the ffplay process
ffplay_process = None

# Open the file for writing in binary mode
with open(output_file, "wb") as file:

    print("Writing bytes to output.264. Waiting for the end-of-frame marker 0x00000001.")

    buffer = bytearray()
    marker = b'\x00\x00\x00\x01'

    try:
        while True:
            if ser.in_waiting:  # If there is data in the buffer
                data = ser.read(ser.in_waiting)  # Read all available bytes
                buffer.extend(data)

                # Check if the end-of-frame marker is in the buffer
                while marker in buffer:
                    index = buffer.index(marker) + len(marker)  # Position after the marker
                    frame = buffer[:index]  # Extract the frame
                    buffer = buffer[index:]  # Keep the remaining data

                    print(f"Frame recorded: {len(frame)} bytes")
                    file.write(frame)  # Write the frame to the file
                    file.flush()  # Force writing to disk

                    # Close the ffplay window if it is already open
                    if ffplay_process and ffplay_process.poll() is None:
                        ffplay_process.terminate()
                        ffplay_process.wait()  # Wait for the process to terminate

                    # Play the recorded frame, reopening the window
                    ffplay_process = subprocess.Popen(["ffplay", "-f", "h264", "-i", output_file])

    except KeyboardInterrupt:
        print("\nRecording stopped.")
    finally:
        # Close the serial port and the ffplay process
        ser.close()


  


  However, each time a new end-of-frame marker is detected, the ffplay window closes and reopens to show the next frame. It will flicker when transferring the video. Is there a way to display the frames in the same window for seamless playback when streaming video ?

  


  Or is there a better approach or software that is more suited for this task ? I do not know where to start, so I will be glad for any hints.

  


• Vedanti and Max Sound vs. Google

  14 août 2014, par Multimedia Mike — Legal/Ethical

  Vedanti Systems Limited (VSL) and Max Sound Coporation filed a lawsuit against Google recently. Ordinarily, I wouldn’t care about corporate legal battles. However, this one interests me because it’s multimedia-related. I’m curious to know how coding technology patents might hold up in a real court case.

  Here’s the most entertaining complaint in the lawsuit :

  Despite Google’s well-publicized Code of Conduct — “Don’t be Evil” — which it explains is “about doing the right thing,” “following the law,” and “acting honorably,” Google, in fact, has an established pattern of conduct which is the exact opposite of its claimed piety.

  I wonder if this is the first known case in which Google has been sued over its long-obsoleted “Don’t be evil” mantra ?

  Researching The Plaintiffs
  
  I think I made a mistake by assuming this lawsuit might have merit. My first order of business was to see what the plaintiff organizations have produced. I have a strong feeling that these might be run of the mill patent trolls.

  VSL currently has a blank web page. Further, the Wayback Machine only has pages reaching back to 2011. The earliest page lists these claims against a plain black background (I’ve highlighted some of the more boisterous claims and the passages that make it appear that Vedanti doesn’t actually produce anything but is strictly an IP organization) :

  The inventions key :
  The patent and software reduced any data content, without compressing, up to a 97% total reduction of the data which also produces a lossless result. This physics based invention is often called the Holy Grail.

  Vedanti Systems Intellectual Property
  Our strategic IP portfolio is granted in all of the world’s largest technology development and use countries. A major value indemnification of our licensee products is the early date of invention filing and subsequent Issue. Vedanti IP has an intrinsic 20 year patent protection and valuation in royalties and licensing. The original data transmission art has no prior art against it.

  Vedanti Systems invented among other firsts, The Slice and Partitioning of Macroblocks within a RGB Tri level region in a frame to select or not, the pixel.

  Vedanti Systems invention is used in nearly every wireless chipset and handset in the world

  Our original pixel selection system revolutionized wireless handset communications. An example of this system “Slice” and “Macroblock Partitioning” is used throughout Satellite channel expansion, Wireless partitioning, Telecom – Video Conferencing, Surveillance Cameras, and 2010 developing Media applications.

  Vedanti Systems is a Semiconductor based software, applications, and IP Continuations Intellectual Property company.

  Let’s move onto the other plaintiff, Max Sound. They have a significantly more substantive website. They also have an Android app named Spins HD Audio, which appears to be little more than a music player based on the screenshots.

  Max Sound also has a stock ticker symbol : MAXD. Something clicked into place when I looked up their ticker symbol : While worth only a few pennies, it was worth a few more pennies after this lawsuit was announced, which might be one of the motivations behind the lawsuit.

  Here’s a trick I learned when I was looking for a new tech job last year : When I first look at a company’s website and am trying to figure out what they really do, I head straight to their jobs/careers page. A lot of corporate websites have way too much blathering corporatese that can be tough to cut through. But when I see what mix of talent and specific skills they are hoping to hire, that gives me a much better portrait of what the company does.

  The reason I bring this up is because this tech company doesn’t seem to have jobs/careers page.

  The Lawsuit
  The core complaint centers around Patent 7974339 : Optimized data transmission system and method. It was filed in July 2004 (or possibly as early as January 2002), issued in July 2011, and assigned (purchased ?) by Vedanti in May 2012. The lawsuit alleges that nearly everything Google has ever produced (or, more accurately, purchased) leverages the patented technology.

  The patent itself has 5 drawings. If you’ve ever seen a multimedia codec patent, or any whitepaper on a multimedia codec, you’ve seen these graphs before. E.g., “Raw pixels come in here -> some analysis happens here -> more analysis happens over here -> entropy coding -> final bitstream”. The text of a patent document isn’t meant to be particularly useful. I’ve tried to understand this stuff before and it never goes well. Skimming the text, I just see a blur of the words data, transmission, pixel, and matrix.

  So I read the complaint to try to figure out what this is all about. To summarize the storyline as narrated by the lawsuit, some inventors were unhappy with the state of video compression in 2001 and endeavored to create something better. So they did, and called it the VSL codec. This codec is so far undocumented on the MultimediaWiki, so it probably has yet to be seen “in the wild”. Good luck finding hard technical data on it now since searches for “VSL codec” are overwhelmed by articles about this lawsuit. Also, the original codec probably wasn’t called VSL because VSL is apparently an IP organization formed much later.

  Then, the protagonists of the lawsuit patented the codec. Then, years later, Google wanted to purchase a video codec that they could open source and use to supplant H.264.

  The complaint goes on to allege that in 2010, Google specifically contacted VSL to possibly license or acquire this mysterious VSL technology. Google was allegedly allowed to study the technology, eventually decided not to continue discussions, and shipped back the proprietary materials.

  Here’s where things get weird. When Google shipped back the materials, they allegedly shipped back a bunch of Post-It notes. The notes are alleged to contain a ton of incriminating evidence. The lawsuit claims that the notes contained such tidbits as :

  • Google was concerned that its infringement could be considered “recklessness” (the standard applicable to willful infringement) ;
  • Google personnel should “try” to destroy incriminating emails ;
  • Google should consider a “design around” because it was facing a “risk of litigation.”

  Actually, given Google’s acquisition of On2, I can totally believe that last one (On2’s codecs have famously contained a lot of weirdness which is commonly suspected to be attributable to designing around known patents).

  Anyway, a lot of this case seems to hinge on the authenticity of these Post-It notes :

  “65. The Post-It notes are unequivocal evidence of Google’s knowledge of the ’339 Patent and infringement by Defendants”

  I wish I could find a stock photo of a stack of Post-It notes in an evidence bag.

  I’ve worked at big technology companies. Big tech companies these days are very diligent about indoctrinating employees about IP liability issues. The reason this Post-It situation strikes me as odd is because the alleged contents of the notes basically outline everything the corporate lawyers tell you NOT to do.

  Analysis
  I’m trying to determine what specific algorithms and coding techniques. I guess I was expecting to see a specific claim that, “Our patent outlines this specific coding technique and here is unequivocal proof that Google A) uses the same technique, and B) specifically did so after looking at our patent.” I didn’t find that (well, a bit of part B, c.f., the Post-It note debacle), but maybe that’s not how these patent lawsuits operate. I’ve never kept up before.

  Maybe it’s just a patent troll. Maybe it’s for the stock bump. I’m expecting to see pump-n-dump stock spam featuring the stock symbol MAXD anytime now.

  I’ve never been interested in following a lawsuit case carefully before. I suddenly find myself wondering if I can subscribe to the RSS feed for this case ? Too much to hope for. But I found this item through Pando and maybe they’ll stay on top of it.

• Long Overdue MediaWiki Upgrade

  5 février 2014, par Multimedia Mike — General

  What do I do ? What I do ? This library book is 42 years overdue !
  I admit that it’s mine, yet I can’t pay the fine,
  Should I turn it in or should I hide it again ?
  What do I do ? What do I do ?

  I internalized the forgoing paean to the perils of procrastination by Shel Silverstein in my formative years. It’s probably why I’ve never paid a single cent in late fees in my entire life.

  However, I have been woefully negligent as the steward of the MediaWiki software that drives the world famous MultimediaWiki, the internet’s central repository of obscure technical knowledge related to multimedia. It is currently running of version 1.6 software. The latest version is 1.22.

  The Story So Far
  According to my records, I first set up the wiki late in 2005. I don’t know which MediaWiki release I was using at the time. I probably conducted a few upgrades in the early days, but that went by the wayside perhaps in 2007. My web host stopped allowing shell access and the MediaWiki upgrade process pretty much requires running a PHP script from a command line. Upgrade time came around and I put off the project. Weeks turned into months turned into years until, according to some notes, the wiki abruptly stopped working in July, 2011. Suddenly, there were PHP errors about “Namespace” being a reserved word.

  While I finally laid out a plan to upgrade the wiki after all these years, I eventually found that the problem had been caused when my webhost upgraded from PHP 5.2 -> 5.3. I also learned of a small number of code changes that caused the problem to go away, thus kicking the can down the road once more.

  Then a new problem showed up last week. I think it might be related to a new version of PHP again. This time, a few other things on my site broke, and I learned that my webhost now allows me to select a PHP version to use (with the version then set to “auto”, which didn’t yield much information). Rolling back to an earlier version of PHP might have solved the problem easily.

  But NO ! I made the determination that this goes no further. I want this wiki upgraded.

  The Arduous Upgrade Path
  There are 2 general upgrade paths I can think of :
  

  1. Upgrade in place on the server
  2. Upgrade offline and put the site back on the server

  Approach #1 is problematic since I don’t have direct shell access, though I considered using something like PHP Shell. Approach #2 involves getting the entire set of wiki files and a backup of the MySQL tables. This is workable since I keep automated backups of these items anyway.

  In fairly short order, I was able to set up a working copy of the MultimediaWiki hosted on a local Linux machine. Now what’s the move ? The MediaWiki software I’m running is 1.6.10. The very latest, as of this upgrade project is 1.22.2. I suppose it’s way too much to hope that the software will upgrade cleanly from 1.6.x straight to 1.22.x, but I guess it’s worth a shot…

  HA ! No chance. Okay, next idea is to march through the various versions and upgrade each in turn. MediaWiki has all their historic releases online, all the way back to the 1.3 lineage. I decided that the latest of each lineage should upgrade cleanly from anything in the previous version of lineage. E.g., 1.6.10 should upgrade cleanly to 1.7.3 (last in the 1.7 series). This seemed to be a workable strategy. So I downloaded the latest of each series, unpacked, and copied all the wiki files over the working installation and ran ‘php update.php’ in the maintenance/ directory.

  The process is tedious and not without its obstacles. I consider this penance for my years of wiki neglect. First, I run into the “PHP Parse error : syntax error, unexpected T_NAMESPACE, expecting T_STRING” issue, the same that I saw years ago after the webhost transitioned from PHP 5.2 -> 5.3. I could solve this by editing assorted files and changing “Namespace” -> “MWNamespace” (which is what MediaWiki did by version 1.13). But I would prefer not to.

  Instead, I downloaded the source for PHP 5.2 and compiled it in a separate directory, then called ‘/path/to/php/5.2/bin/php update.php’. Problem solved.

  The next problem is that a bunch of the database update scripts are specifying “Type=InnoDB”. This isn’t supported by modern MySQL databases. Now, it’s “Engine=InnoDB”. A quick search & replace at the command line fixes this for 1.6.x… and 1.7.x… and 1.8 through 1.12. Finally, at 1.13, it was no longer necessary. As a bonus, at 1.13, I was able to test the installation since Namespace had been renamed to MWNamespace. I would later learn that the table type modifications probably could have been simplified in by changing “$wgDBmysql4 = true ;” to “$wgDBmysql5 = true ;” somewhere in LocalSettings.php.

  Command line upgrading worked smoothly up through 1.18 series when I got a new syntax error :

  <br />
PHP Fatal error:  Call to a member function addMessages() on a non-object in /mnt/sdb1/archive/wiki/extensions/Cite.php on line 68<br />

  Best I could do was comment out that line. I hope that doesn’t break anything important.

  In the home stretch, the very last transition (1.21 -> 1.22) failed :

  PHP Fatal error :  Cannot redeclare wfProfileIn() (previously declared in 
  /mnt/sdb1/archive/wiki/includes/profiler/Profiler.php:33) in 
  /mnt/sdb1/archive/wiki/includes/ProfilerStub.php on line 25
  

  Apparently, this problem arises occasionally since 1.18. I found a way around it thanks to this page : Deleted the file StartProfiler.php. Who am I to argue ?

  Upon completing the transition to 1.22, the wiki doesn’t look correct– the pictures aren’t showing up. The solution was to fix the temporary directory via LocalSettings.php.

  Back To Production
  Okay, it all works again ! Locally, that is. How to get it back to the server ? My first idea was that, knowing that this upgrade process can succeed, try stepping through the upgrade process again, but tell the update.php scripts to access the database tables on multimedia.cx. This seemed to be working for awhile, even though the database update phase often took 4-5 minutes. However, the transition from 1.8.5 -> 1.9.6 took 75 minutes and then timed out. According to my notes, “This isn’t going to work.”

  The new process :

  1. Dump the database tables from the local database.
  2. Create a new database remotely (melanson_wiki_ng).
  3. Dump the database table into melanson_wiki_ng.
  4. Move the index.php file out of the wiki files directory temporarily (or rename).
  5. Modify the LocalSettings.php to talk to the new database.
  6. Perform a lftp mirror operation in order to send all the files up to the server.
  7. Send the index.php file and hope beyond hope that everything magically works.

  And that’s the story of how the updated MultimediaWiki came back online. Despite the database dump file being over 110 MB, it only tool MySQL 1m45s to transmit it all to the remote server (let’s hear it for the ‘–compress’ option). For comparison, inserting the tables back into a fresh local database took 1m07s.

  When the MultimediaWiki was first live again, it loaded, but ever so slowly. This is when I finally looked into optimization and found that I was lacking any caching. So as a bonus, the MultimediaWiki should be much faster now.

  Going Forward
  For all I know, I did everything described here in the hardest way possible. But at least I got it done. Unless I learn of a better process, future upgrades will probably look similar to this.

  Additionally, I should probably take some time to figure out what new features are part of the standard MediaWiki distribution nowadays.