Recherche avancée

Sur d’autres sites (9965)

• Adventures in Unicode

  29 novembre 2012, par Multimedia Mike — Programming, php, Python, sqlite3, unicode

  Tangential to multimedia hacking is proper metadata handling. Recently, I have gathered an interest in processing a large corpus of multimedia files which are likely to contain metadata strings which do not fall into the lower ASCII set. This is significant because the lower ASCII set intersects perfectly with my own programming comfort zone. Indeed, all of my programming life, I have insisted on covering my ears and loudly asserting “LA LA LA LA LA ! ALL TEXT EVERYWHERE IS ASCII !” I suspect I’m not alone in this.

  Thus, I took this as an opportunity to conquer my longstanding fear of Unicode. I developed a self-learning course comprised of a series of exercises which add up to this diagram :

  
  
  

  Part 1 : Understanding Text Encoding
  Python has regular strings by default and then it has Unicode strings. The latter are prefixed by the letter ‘u’. This is what ‘ö’ looks like encoded in each type.

  python

  < view plain text >

  1. >>> ’ö’, u’ö’
  2. (’\xc3\xb6’, u’\xf6’)

  A large part of my frustration with Unicode comes from Python yelling at me about UnicodeDecodeErrors and an inability to handle the number 0xc3 for some reason. This usually comes when I’m trying to wrap my head around an unrelated problem and don’t care to get sidetracked by text encoding issues. However, when I studied the above output, I finally understood where the 0xc3 comes from. I just didn’t understand what the encoding represents exactly.
  

  I can see from assorted tables that ‘ö’ is character 0xF6 in various encodings (in Unicode and Latin-1), so u’\xf6′ makes sense. But what does ‘\xc3\xb6′ mean ? It’s my style to excavate straight down to the lowest levels, and I wanted to understand exactly how characters are represented in memory. The UTF-8 encoding tables inform us that any Unicode code point above 0x7F but less than 0×800 will be encoded with 2 bytes :

   110xxxxx 10xxxxxx
  

  Applying this pattern to the \xc3\xb6 encoding :

              hex : 0xc3      0xb6
             bits : 11000011  10110110
   important bits : ---00011  —110110
        assembled : 00011110110
       code point : 0xf6
  

  I was elated when I drew that out and made the connection. Maybe I’m the last programmer to figure this stuff out. But I’m still happy that I actually understand those Python errors pertaining to the number 0xc3 and that I won’t have to apply canned solutions without understanding the core problem.

  I’m cheating on this part of this exercise just a little bit since the diagram implied that the Unicode text needs to come from a binary file. I’ll return to that in a bit. For now, I’ll just contrive the following Unicode string from the Python REPL :

  python

  < view plain text >

  1. >>> u = u’Üñìçôđé’
  2. >>> u
  3. u’\xdc\xf1\xec\xe7\xf4\u0111\xe9’

  Part 2 : From Python To SQLite3
  The next step is to see what happens when I use Python’s SQLite3 module to dump the string into a new database. Will the Unicode encoding be preserved on disk ? What will UTF-8 look like on disk anyway ?

  python

  < view plain text >

  1. >>> import sqlite3
  2. >>> conn = sqlite3.connect(’unicode.db’)
  3. >>> conn.execute("CREATE TABLE t (t text)")
  4. >>> conn.execute("INSERT INTO t VALUES (?)", (u, ))
  5. >>> conn.commit()
  6. >>> conn.close()

  Next, I manually view the resulting database file (unicode.db) using a hex editor and look for strings. Here we go :

  000007F0   02 29 C3 9C  C3 B1 C3 AC  C3 A7 C3 B4  C4 91 C3 A9
  

  Look at that ! It’s just like the \xc3\xf6 encoding we see in the regular Python strings.

  Part 3 : From SQLite3 To A Web Page Via PHP
  Finally, use PHP (love it or hate it, but it’s what’s most convenient on my hosting provider) to query the string from the database and display it on a web page, completing the outlined processing pipeline.

  php

  < view plain text >

  1. < ?php
  2. $dbh = new PDO("sqlite:unicode.db") ;
  3. foreach ($dbh->query("SELECT t from t") as $row) ;
  4. $unicode_string = $row[’t’] ;
  5.  ?>
  6.  
  7. <html>
  8. <head><meta http-equiv="Content-Type" content="text/html ; charset=utf-8"></meta></head>
  9. <body><h1>< ?=$unicode_string ?></h1></body>
  10. </html>

  I tested the foregoing PHP script on 3 separate browsers that I had handy (Firefox, Internet Explorer, and Chrome) :

  
  
  

  I’d say that counts as success ! It’s important to note that the “meta http-equiv” tag is absolutely necessary. Omit and see something like this :

  
  
  

  Since we know what the UTF-8 stream looks like, it’s pretty obvious how the mapping is operating here : 0xc3 and 0xc4 correspond to ‘Ã’ and ‘Ä’, respectively. This corresponds to an encoding named ISO/IEC 8859-1, a.k.a. Latin-1. Speaking of which…

  Part 4 : Converting Binary Data To Unicode
  At the start of the experiment, I was trying to extract metadata strings from these binary multimedia files and I noticed characters like our friend ‘ö’ from above. In the bytestream, this was represented simply with 0xf6. I mistakenly believed that this was the on-disk representation of UTF-8. Wrong. Turns out it’s Latin-1.

  However, I still need to solve the problem of transforming such strings into Unicode to be shoved through the pipeline diagrammed above. For this experiment, I created a 9-byte file with the Latin-1 string ‘Üñìçôdé’ couched by 0′s, to simulate yanking a string out of a binary file. Here’s unicode.file :

  00000000   00 DC F1 EC  E7 F4 64 E9  00         ......d..
  

  (Aside : this experiment uses plain ‘d’ since the ‘đ’ with a bar through it doesn’t occur in Latin-1 ; shows up all over the place in Vietnamese, at least.)

  I’ve been mashing around Python code via the REPL, trying to get this string into a Unicode-friendly format. This is a successful method but it’s probably not the best :

  python

  < view plain text >

  1. >>> import struct
  2. >>> f = open(’unicode.file’, ’r’).read()
  3. >>> u = u’’
  4. >>> for c in struct.unpack("B"*7, f[1 :8]) :
  5. ... u += unichr(c)
  6. ...
  7. >>> u
  8. u’\xdc\xf1\xec\xe7\xf4d\xe9’
  9. >>> print u
  10. Üñìçôdé

  Conclusion
  Dealing with text encoding matters reminds me of dealing with integer endian-ness concerns. When you’re just dealing with one system, you probably don’t need to think too much about it because the system is usually handling everything consistently underneath the covers.

  However, when the data leaves one system and will be interpreted by another system, that’s when a programmer needs to be cognizant of matters such as integer endianness or text encoding.

• How to install a Matomo premium feature

  31 janvier 2018, par InnoCraft

  You may have noticed over the last few months that many fantastic new features have been launched on the Matomo Marketplace. As some of them are paid premium features, you may wonder if the process to install them is straightforward, if you can test them before, and whether there is any support behind it. No worries – we’ve got you covered ! This blog post will answer some questions you may have about getting your first premium plugin.

  So why are there some premium features ?

  Researching, building, documenting, testing and maintaining quality products take years of experience and months of hard work by the team behind the scenes. When you purchase a premium plugin, you get a fully working product and you directly help the Matomo core engineers to grow and fund the new Free Matomo versions and cool features.

  However, it is important for us to mention that Matomo will always be free, it is a Free software under GPLv3 license and it will always be the same.
  Want to know more about this ? Check out our FAQ about why there are premium features.

  Can I test a premium feature before a purchase ?

  Absolutely. There are two ways in order to do that :

  1. InnoCraft Matomo Cloud
  2. Matomo Marketplace

  1. InnoCraft Matomo Cloud

  The easiest way is to create a free trial account (one minute of your time) on our Matomo cloud service. You will then have the possibility to test all the premium features during a 30-day trial period. No credit card is required.

  

  Every premium feature can be trialled for free on the Matomo Cloud

  2. Matomo Marketplace

  The second way is to get the premium feature from the Matomo Marketplace. We have an easy and hassle-free 30-day money back guarantee period on each feature. This means that if you are not happy with a premium feature and you are within the 30-day period, then you will get a full refund for it. Guaranteed !

  How to purchase and install a premium feature ?

  Step 1 : Purchasing the feature

  In order to get a premium feature, just add it to the cart :

  

  Once done, go to your cart and complete the checkout process to confirm the order.

  When the order is confirmed, you immediately get your license key on the order confirmation page. You also receive the license key by email.

  Step 2 : Activating the feature in your Matomo

  Now that you have received the license key, it is time to activate the plugin in your Matomo :

  • Log in to your Matomo and go to “Administration => Marketplace
  • Copy / paste the license key into the license field at the top of the page and click “Activate”

  

  • The key will now be activated and you will see a couple of new buttons.

  

  • To install the premium feature(s) you just purchased in one click, simply click on “Install purchased plugins”. Alternatively, you can scroll down in the Marketplace to the premium feature you purchased and click on “Install”. You can also download the ZIP file on https://shop.matomo.org/my-account/downloads

  And that’s it. The installation of a premium feature is as easy as copy/pasting the license key and clicking a button. Because one license key is linked to all purchased premium features, you only need to enter the license key once. The next time you purchase a premium feature, you simply click on “Install” to have it up and running.

  Updating a premium feature

  Updates for premium features work just like regular plugin updates. When there is a new update available, you will see a notification in the Matomo UI and also receive an email (if enabled under “General Settings”). To upgrade the feature simply click on “Update” and you’re done.

  Which support is provided for each of those premium features ?

  Premium features represent most of our day to day activity, so you can be 100% sure that we will do our maximum in order to answer any of your questions regarding them. To be 100% transparent, we often receive answers from our customers telling us how impressed they are by the quality of the service we are offering.

  Have any questions ?

  We are happy to answer any questions you may have so feel free to get in touch with us.

  Thanks !

  The post How to install a Matomo premium feature appeared first on Analytics Platform - Matomo.

• ffmpeg error while converting to mp4 Error while opening encoder for output stream #0.0

  20 mars 2016, par Josh

  I am trying to convert various file types to mp4 to be displayed using ffmpeg, but i keep getting the error :

  Error while opening encoder for output stream #0.0 - maybe incorrect parameters such as bit_rate, rate, width or height

  Another piece that looks important is :

  [libx264 @ 0x93caef0] broken ffmpeg default settings detected

  [libx264 @ 0x93caef0] use an encoding preset (e.g. -vpre medium)

  [libx264 @ 0x93caef0] preset usage : -vpre -vpre

  [libx264 @ 0x93caef0] speed presets are listed in x264 —help

  [libx264 @ 0x93caef0] profile is optional ; x264 defaults to high

  The latest code I am running is :

  ffmpeg -i source -s 320x240 -r 30000/1001 -b 200k -bt 240k -vcodec libx264 -coder 0 -bf 0 -refs 1 -flags2 -wpred-dct8x8 -level 30 -maxrate 10M -bufsize 10M -acodec libfaac -ac 2 -ar 48000 -ab 192k destination

  I have seen a few other people with this issue, but their fixes didn’t work for some reason.

  In case it matters : ultimately this will be used in php, though I am trying to get it working first via putty

  EDIT: : Here is the full thing as requested(using a wmv, have tested wmv and flv) :

       ~ >> ffmpeg -i path.wmv -s 320x240 -r 30000/1001 -b 200k -r 29.97 -bt 240k -vcodec libx264 -coder 0 -bf 0 -refs 1 -flags2 -wpred-dct8x8 -level 30 -maxrate 10M -bufsize 10M -acodec libfaac -ac 2 -ar 48000 -ab 192k path.mp4
  
  FFmpeg version SVN-r26076, Copyright (c) 2000-2011 the FFmpeg developers
  
    built on Aug 28 2012 17:55:47 with gcc 4.1.2 20080704 (Red Hat 4.1.2-52)
  
    configuration: --enable-version3 --enable-gpl --enable-nonfree --enable-shared --enable-postproc --enable-avfilter --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvpx --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libx264 --enable-libxvid --disable-ffplay --disable-indevs --disable-outdevs --disable-demuxer=v4l --disable-demuxer=v4l2 --disable-mmx
  
    libavutil     50.36. 0 / 50.36. 0
  
    libavcore      0.16. 1 /  0.16. 1
  
    libavcodec    52.108. 0 / 52.108. 0
  
    libavformat   52.93. 0 / 52.93. 0
  
    libavdevice   52. 2. 3 / 52. 2. 3
  
    libavfilter    1.74. 0 /  1.74. 0
  
    libswscale     0.12. 0 /  0.12. 0
  
    libpostproc   51. 2. 0 / 51. 2. 0
  
  
  
  Seems stream 1 codec frame rate differs from container frame rate: 1000.00 (1000/1) -> 29.97 (30000/1001)
  
  Input #0, asf, from 'path.wmv':
  
    Metadata:
  
      SfOriginalFPS   : 299
  
      WMFSDKVersion   : 11.0.6001.7000
  
      WMFSDKNeeded    : 0.0.0.0000
  
      IsVBR           : 0
  
      title           : Wildlife in HD
  
      artist          :
  
      copyright       : © 2008 Microsoft Corporation
  
      comment         : Footage: Small World Productions, Inc; Tourism New Zealand | Producer: Gary F. Spradling | Music: Steve Ball
  
    Duration: 00:00:30.09, start: 8.000000, bitrate: 6977 kb/s
  
      Stream #0.0(eng): Audio: wmav2, 44100 Hz, 2 channels, s16, 192 kb/s
  
      Stream #0.1(eng): Video: vc1, yuv420p, 1280x720, 5942 kb/s, 29.97 tbr, 1k tbn, 1k tbc
  
  File 'path.mp4' already exists. Overwrite ? [y/N] y
  
  [buffer @ 0x9ce9eb0] w:1280 h:720 pixfmt:yuv420p
  
  [scale @ 0x9ce8f70] w:1280 h:720 fmt:yuv420p -> w:320 h:240 fmt:yuv420p flags:0x4
  
  [libx264 @ 0x9ce9ef0] broken ffmpeg default settings detected
  
  [libx264 @ 0x9ce9ef0] use an encoding preset (e.g. -vpre medium)
  
  [libx264 @ 0x9ce9ef0] preset usage: -vpre <speed> -vpre <profile>
  
  [libx264 @ 0x9ce9ef0] speed presets are listed in x264 --help
  
  [libx264 @ 0x9ce9ef0] profile is optional; x264 defaults to high
  
  Output #0, mp4, to 'path.mp4':
  
      Stream #0.0(eng): Video: libx264, yuv420p, 320x240, q=2-31, 200 kb/s, 90k tbn, 29.97 tbc
  
      Stream #0.1(eng): Audio: libfaac, 48000 Hz, 2 channels, s16, 192 kb/s
  
  Stream mapping:
  
    Stream #0.1 -> #0.0
  
    Stream #0.0 -> #0.1
  
  Error while opening encoder for output stream #0.0 - maybe incorrect parameters such as bit_rate, rate, width or height
  
  </profile></speed>

  Thanks for any help