Recherche avancée

Sur d’autres sites (8971)

• Subtitling Sierra RBT Files

  2 juin 2016, par Multimedia Mike — Game Hacking

  This is part 2 of the adventure started in my Subtitling Sierra VMD Files post. After I completed the VMD subtitling, The Translator discovered a wealth of animation files in a format called RBT (this apparently stands for “Robot” but I think “Ribbit” format could be more fun). What are we going to do ? We had come so far by solving the VMD subtitling problem for Phantasmagoria. It would be a shame if the effort ground to a halt due to this.

  Fortunately, the folks behind the ScummVM project already figured out enough of the format to be able to decode the RBT files in Phantasmagoria.

  In the end, I was successful in creating a completely standalone tool that can take a Robot file and a subtitle file and create a new Robot file with subtitles. The source code is here (subtitle-rbt.c). Here’s what the final result looks like :

  
  
  “What’s in the refrigerator ?” I should note at this juncture that I am not sure if this particular Robot file even has sound or dialogue since I was conducting these experiments on a computer with non-working audio.
  

  The RBT Format
  I have created a new MultimediaWiki page describing the Robot Animation format based on the ScummVM source code. I have not worked with a format quite like this before. These are paletted animations which consist of a sequence of independent frames that are designed to be overlaid on top of static background. Because of these characteristics, each frame encodes its own unique dimensions and origin coordinate within the frame. While the Phantasmagoria VMD files are usually 288×144 (which are usually double-sized for the benefit of a 640×400 Super VGA canvas), these frames are meant to be plotted on a game field that was roughly 576×288 (288×144 doublesized).
  

  For example, 2 minimalist animation frames from a desk investigation Robot file :

  
  
  100×147

  
  101×149
  

  As for compression, my first impression was that the algorithm was the same as VMD. This is wrong. It evidently uses an unmodified version of a standard algorithm called Lempel-Ziv-Stac (LZS). It shows up in several RFCs and was apparently used in MS-DOS’s transparent disk compression scheme.

  Approach
  Thankfully, many of the lessons I learned from the previous project are applicable to this project, including : subtitle library interfacing, subtitling in the paletted colorspace, and replacing encoded frames from the original file instead of trying to create a new file.

  Here is the pitch for this project :

  • Create a C program that can traverse through an input file, piece by piece, and generate an output file. The result of this should be a bitwise identical file.
  • Adapt the LZS compression decoding algorithm from ScummVM into the new tool. Make the tool dump raw Portable NetMap (PNM) files of varying dimensions and ensure that they look correct.
  • Compress using LZS.
  • Stretch the frames and draw subtitles.
  • More compression. Find the minimum window for each frame.

  Compression
  Normally, my first goal is to decompress the video and store the data in a raw form. However, this turned out to be mathematically intractable. While the format does support both compressed and uncompressed frames (even though ScummVM indicates that the uncompressed path is yet unexercised), the goal of this project requires making the frames so large that they overflow certain parameters of the file.

  A Robot file has a sequence of frames and 2 tables describing the size of each frame. One table describes the entire frame size (audio + video) while the second table describes just the video frame size. Since these tables only use 16 bits to specify a size, the maximum frame size is 65536 bytes. Leaving space for the audio portion of the frame, this only leaves a per-frame byte budget of about 63000 bytes for the video. Expanding the frame to 576×288 (165,888 pixels) would overflow this limit.

  Anyway, the upshot is that I needed to compress the data up front.

  Fortunately, the LZS compressor is pretty straightforward, at least if you have experience writing VLC-oriented codecs. While the algorithm revolves around back references, my approach was to essentially write an RLE encoder. My compressor would search for runs of data (plentiful when I started to stretch the frame for subtitling purposes). When a run length of n=3 or more of the same pixel is found, encode the pixel by itself, and then store a back reference of offset -1 and length (n-1). It took a little while to iron out a few problems, but I eventually got it to work perfectly.

  I have to say, however, that the format is a little bit weird in how it codes very large numbers. The length encoding is somewhat Golomb-like, i.e., smaller values are encoded with fewer bits. However, when it gets to large numbers, it starts encoding counts of 15 as blocks of 1111. For example, 24 is bigger than 7. Thus, emit 1111 into the bitstream and subtract 8 from 23 -> 16. Still bigger than 15, so stuff another 1111 into the bitstream and subtract 15. Now we’re at 1, so stuff 0001. So 24 is 11111111 0001. 12 bits is not too horrible. But the total number of bytes (value / 30). So a value of 300 takes around 10 bytes (80 bits) to encode.

  Palette Slices
  As in the VMD subtitling project, I took the subtitle color offered in the subtitle spec file as a suggestion and used Euclidean distance to match to the closest available color in the palette. One problem, however, is that the palette is a lot smaller in these animations. According to my notes, for the set of animations I scanned, only about 80 colors were specified, starting at palette index 55. I hypothesize that different slices of the palette are reserved for different uses. E.g., animation, background, and user interface. Thus, there is a smaller number of colors to draw upon for subtitling purposes.

  Scaling
  One bit of residual weirdness in this format is the presence of a per-frame scale factor. While most frames set this to 100 (100% scale), I have observed 70%, 80%, and 90%. ScummVM is a bit unsure about how to handle these, so I am as well. However, I eventually realized I didn’t really need to care, at least not when decoding and re-encoding the frame. Just preserve the scale factor. I intend to modify the tool further to take scale factor into account when creating the subtitle.

  The Final Resolution
  Right around the time that I was composing this post, The Translator emailed me and notified me that he had found a better way to subtitle the Robot files by modifying the scripts, rendering my entire approach moot. The result is much cleaner :

  
  
  Turns out that the engine supported subtitles all along
  

  It’s a good thing that I enjoyed the challenge or I might be annoyed at this point.

  See Also

  • Subtitling Sierra VMD Files : My effort to subtitle the main FMV files found in Sierra games.

• What’s new in Piwik 2.16.0 ?

  1er février 2016, par Matthieu Aubry — About, Development, Uncategorized

  In this blog post we showcase what is new in Piwik 2.16.0 and why you should upgrade your Piwik to this latest release !

  Piwik 2.16.0 is currently in Release Candidate and you can already use it by following these simple instructions. Piwik 2.16.0 will be officially released in a few days.

  To see the list of all 250+ changes, read the Piwik 2.16.0 changelog.

  Long Term Support for Piwik 2.X

  Piwik 2.16.0 is our Long Term Support version for Piwik 2. Learn more about Piwik LTS.

  New : Custom Dimensions feature

  With Custom Dimensions you can assign any custom data to your visitors or actions (like pages, events, …) and then visualize the reports of how many visits, conversions, pageviews, etc. there were for each Custom Dimension. They are similar to Custom Variables see the comparison of Custom Dimensions and Custom Variables.

  The new Custom Dimensions plugin is available on the Piwik Marketplace.

  Learn more about creating, tracking, reporting, managing, segmenting custom dimensions in the Custom Dimensions user guide.

  

  

  

  View the list of all visitors who reached a specific goal

  All Goal reports now include a link below the Goal report summary, that lets you see in one click all users that have converted any of your Goals :

  

  Events reports : option to view all columns : min, max and avg values

  By default in Actions > Events report, the columns displayed are “Total events” (Total number of events) and “Total value” (the sum of event values). It is now possible to click on the footer icon to view more detailed columns about event values : minimum event value, maximum event value, and average event value.

  

  Allow zoom to country in realtime visitor map

  In the real time map in Piwik (which displays your users activity appearing in real time !), it is now possible to zoom on a given country :

  

  Export all Records : new ‘all’ option in the Row limit selector

  It is now possible to export all of the data in your reports directly from the user interface ! Select ‘all’ in the row limit selector before exporting your data reports :

  

  New themes on the Marketplace !

  Feel like a change of colors ? Try out the new community submitted Piwik Themes on the Marketplace ! Learn how to install a new theme in Piwik (in only a few clicks).

  

  Let Super User view and edit segments created by other users

  As a Super User of Piwik, it can be very useful to view all Custom Segments created by all users. Starting in Piwik 2.16.0 any Super User can now view all Segments for a given website in the Segment selector :

  • Segments that are shared with All users will now appear below a section “Shared with you :” (a segment can be shared to all users by any admin user by selecting “This segment is visible to All Users” in the Segment editor)
  • Segments that are not shared with anyone are now also visible under the section “Visible to you because you have Super User access :”

  

  New segment ‘deviceBrand’ to segment by Device Brand

  You can now segment by device brands such as Apple, Samsung, LG, Google, Nokia, Sony, Lenovo, Alcatel, etc. The complete list of device brands you can use is listed on the Segmentation developer guide.

  

  New segment ‘actionType’ to view only particular Actions types (pagesviews, downloads, outlinks, events…)

  Use the new “Action type” segment to view only particular actions such as : pageviews, contents, sitesearches, events, outlinks and downloads.

  

  New segment : ‘actionUrl’ to view any actions that matches a given URL

  Use the new “Action URL” segment to view any action that matches a given URL, whether they are Pageviews, Site searches, Contents, Downloads or Events.

  

  New segment operators : “Starts with” and “Ends with”

  The new segment operators “Starts with” and “Ends with” can come handy when creating Custom Segments, and complement well the existing segment operators : Contains, Does not contain, Equals, Not equals, Greater than or equal to, Less than or equal to.

  

  Learn more about Segmentation in our user guides or in the Segmentation developer guide.

  Making Piwik more accessible

  Web accessibility refers to the inclusive practice of removing barriers that prevent interaction with, or access to applications or websites, by people with disabilities. When sites are correctly designed, developed and edited, all users have equal access to information and functionality.

  In Piwik 2.16.0, and with the help of high school students from the Catalyst Academy, we have made several improvements that make Piwik more accessible :

  • Menu now include the ARIA metadata (to make menus easier to navigate and use)
  • Page titles and various key tooltips have been improved (to give a better context to the data)
  • Keyboard access has been much improvement and Piwik is starting to be usable with the keyboard ! (this is very useful as many users are not able to use a mouse, and they should be able to view Piwik reports with the keyboard only)
  • ‘Skip to main content’ link (when pressing the TAB key) now skips the cursor to the page content.
  • Search bar was further improved : try it by pressing the ‘f’ key (useful to open any Piwik report or page without using the mouse)

  New user preference : display times either as 24h format or AM/PM format

  Users can now select whether times should be displayed in 24h format (16:20) or in AM/PM format (4:20pm)

  

  Other admins and Super Users are not visible to users with ‘admin’ permissions anymore

  Piwik 2.16.0 just became even more respectful of your privacy : any user you created and assigned an “admin” permission will not be able to view all other usernames anymore. This change was requested by many Piwik users who rightfully preferred not to disclose all usernames to any ‘admin’ user.

  As a result of this change, when a user with ‘admin’ permission (admin user) wants to give additional permission to another user (target user), the admin user will now need to know the target username and enter it in the Manage Users interface :

  

  New Config file viewer

  A new menu entry Administration > Diagnostics > Config file, lets Super Users view all config values in the UI :

  This new screen serves several purposes :

  • help Piwik administrators discover and learn about all the possible config file settings that Piwik offers.
  • let Super Users and Admins easily view any overridden config.ini.php INI setting without having to open the config file on the server.

  

  New report ‘Personal settings > Custom Variables’ displays custom variable usage

  A new report visible to Super Users lets you see how your Custom Variables slots are being used, which ones are still unused, and how many times a given custom variable name was used.

  

  Learn more about Custom Variables.

  Improvements to Heartbeat feature to accurately count time spent on the last page

  The heartbeat feature which lets you accurately track how long do visitors spend on your website, has been improved in 2.16.0. If you would like to measure how long do people spend on your pages (even if they don’t interact with your website or app), then consider using the heartbeat feature. Learn more in our guide Accurately measure the time spent on each page.

  

  New AnonymousPiwikUsageMeasurement plugin
  

  AnonymousPiwikUsageMeasurement plugin was created for those who would like to measure how their Piwik services is being used. This plugin also sends anonymised tracking data to our demo-anonymous.piwik.org which will help us learn how your Piwik is used and how we could improve the application. Learn more about AnonymousPiwikUsageMeasurement plugin.

  

  Show a warning when Piwik is used as a SuperUser via HTTP

  It is highly recommended to use HTTPS (SSL) for your Piwik server. If you are logged in as a Super User and viewing any Administration pages, you will now see a warning notification message at the top of the screen inviting you to use HTTPS :

  

  MySQL 5.7+ support

  The awesome Piwik community has helped us identify several issues with MySQL 5.7 support and Piwik is now fully compatible with the latest MySQL version 5.7+. Piwik is officially compatible with MySQL >= 4.1 and up to Mysql 5.7 ! Piwik is also compatible with MariaDB.

  

  Better detection of new devices and robots

  Our Device Detector library can now detect many new types of devices : smart TVs, new tablets and mobile phones, and more. New bots were added to our list of bots to keep your analytics report meaningful.

  Learn more : What Device Detector is able to detect, About Device Detector.

  New Referrer Spammers

  Every week we keep updating our community-list of referrer spammers. More than 50 new spammers were added since the last Piwik release !

  Note that the list of referrer spammer is automatically updated every week in your Piwik, so you need to wait at most one week to benefit from the very latest list.

  Learn more about Stopping Referrer Spam in Analytics reports.

  

  Ability to support new SMS providers to send Scheduled Reports by text message

  When you create scheduled reports in Piwik, you can specify one or several email addresses (to send reports by email) and you can also configure phone numbers (so your reports are sent to your phone via text message). So far we only support one SMS Provider “Clockwork SMS” but it is now possible for anyone to create a very simple plugin that adds a new SMS Provider to Piwik.

  A new SMS Provider plugin was released on the Marketplace : FreeMobileMessaging. If you have a favorite SMS Provider, we encourage you to create a new plugin for it !

  Piwik Consulting mentions

  A few new mentions of Piwik Consultants were added into Piwik to let users know that they can get Professional Help, and additional services and products. For example a new widget was added to the default dashboard. If you provide your Piwik service to your own clients, maybe you would like to remove the Professional services widget and other ads, see this FAQ : How do I remove from Piwik the links to “Piwik Consultants” or to “Professional Services” ?

  Stability, Reliability and Performance

  Dozens of other bugs were fixed (including several Major bugs) and many small improvements are included in this release. All these changes make Piwik more stable and reliable, as well as easier to troubleshoot and maintain. We are proud to be able to offer Piwik 2.16.0 as our Long Term Support release !

  If you have any feedback, questions, suggestion, or bug to report, please join our Piwik Community Forums.

  We hope you will love this 2.16.0 release as much as we loved making it.

  Happy Analytics !

• What’s new in Piwik 2.16.0 ?

  1er février 2016, par Matthieu Aubry — About, Development, Uncategorized

  In this blog post we showcase what is new in Piwik 2.16.0 and why you should upgrade your Piwik to this latest release !

  Piwik 2.16.0 is currently in Release Candidate and you can already use it by following these simple instructions. Piwik 2.16.0 will be officially released in a few days.

  To see the list of all 250+ changes, read the Piwik 2.16.0 changelog.

  Long Term Support for Piwik 2.X

  Piwik 2.16.0 is our Long Term Support version for Piwik 2. Learn more about Piwik LTS.

  New : Custom Dimensions feature

  With Custom Dimensions you can assign any custom data to your visitors or actions (like pages, events, …) and then visualize the reports of how many visits, conversions, pageviews, etc. there were for each Custom Dimension. They are similar to Custom Variables see the comparison of Custom Dimensions and Custom Variables.

  The new Custom Dimensions plugin is available on the Piwik Marketplace.

  Learn more about creating, tracking, reporting, managing, segmenting custom dimensions in the Custom Dimensions user guide.

  

  

  

  View the list of all visitors who reached a specific goal

  All Goal reports now include a link below the Goal report summary, that lets you see in one click all users that have converted any of your Goals :

  

  Events reports : option to view all columns : min, max and avg values

  By default in Actions > Events report, the columns displayed are “Total events” (Total number of events) and “Total value” (the sum of event values). It is now possible to click on the footer icon to view more detailed columns about event values : minimum event value, maximum event value, and average event value.

  

  Allow zoom to country in realtime visitor map

  In the real time map in Piwik (which displays your users activity appearing in real time !), it is now possible to zoom on a given country :

  

  Export all Records : new ‘all’ option in the Row limit selector

  It is now possible to export all of the data in your reports directly from the user interface ! Select ‘all’ in the row limit selector before exporting your data reports :

  

  New themes on the Marketplace !

  Feel like a change of colors ? Try out the new community submitted Piwik Themes on the Marketplace ! Learn how to install a new theme in Piwik (in only a few clicks).

  

  Let Super User view and edit segments created by other users

  As a Super User of Piwik, it can be very useful to view all Custom Segments created by all users. Starting in Piwik 2.16.0 any Super User can now view all Segments for a given website in the Segment selector :

  • Segments that are shared with All users will now appear below a section “Shared with you :” (a segment can be shared to all users by any admin user by selecting “This segment is visible to All Users” in the Segment editor)
  • Segments that are not shared with anyone are now also visible under the section “Visible to you because you have Super User access :”

  

  New segment ‘deviceBrand’ to segment by Device Brand

  You can now segment by device brands such as Apple, Samsung, LG, Google, Nokia, Sony, Lenovo, Alcatel, etc. The complete list of device brands you can use is listed on the Segmentation developer guide.

  

  New segment ‘actionType’ to view only particular Actions types (pagesviews, downloads, outlinks, events…)

  Use the new “Action type” segment to view only particular actions such as : pageviews, contents, sitesearches, events, outlinks and downloads.

  

  New segment : ‘actionUrl’ to view any actions that matches a given URL

  Use the new “Action URL” segment to view any action that matches a given URL, whether they are Pageviews, Site searches, Contents, Downloads or Events.

  

  New segment operators : “Starts with” and “Ends with”

  The new segment operators “Starts with” and “Ends with” can come handy when creating Custom Segments, and complement well the existing segment operators : Contains, Does not contain, Equals, Not equals, Greater than or equal to, Less than or equal to.

  

  Learn more about Segmentation in our user guides or in the Segmentation developer guide.

  Making Piwik more accessible

  Web accessibility refers to the inclusive practice of removing barriers that prevent interaction with, or access to applications or websites, by people with disabilities. When sites are correctly designed, developed and edited, all users have equal access to information and functionality.

  In Piwik 2.16.0, and with the help of high school students from the Catalyst Academy, we have made several improvements that make Piwik more accessible :

  • Menu now include the ARIA metadata (to make menus easier to navigate and use)
  • Page titles and various key tooltips have been improved (to give a better context to the data)
  • Keyboard access has been much improvement and Piwik is starting to be usable with the keyboard ! (this is very useful as many users are not able to use a mouse, and they should be able to view Piwik reports with the keyboard only)
  • ‘Skip to main content’ link (when pressing the TAB key) now skips the cursor to the page content.
  • Search bar was further improved : try it by pressing the ‘f’ key (useful to open any Piwik report or page without using the mouse)

  New user preference : display times either as 24h format or AM/PM format

  Users can now select whether times should be displayed in 24h format (16:20) or in AM/PM format (4:20pm)

  

  Other admins and Super Users are not visible to users with ‘admin’ permissions anymore

  Piwik 2.16.0 just became even more respectful of your privacy : any user you created and assigned an “admin” permission will not be able to view all other usernames anymore. This change was requested by many Piwik users who rightfully preferred not to disclose all usernames to any ‘admin’ user.

  As a result of this change, when a user with ‘admin’ permission (admin user) wants to give additional permission to another user (target user), the admin user will now need to know the target username and enter it in the Manage Users interface :

  

  New Config file viewer

  A new menu entry Administration > Diagnostics > Config file, lets Super Users view all config values in the UI :

  This new screen serves several purposes :

  • help Piwik administrators discover and learn about all the possible config file settings that Piwik offers.
  • let Super Users and Admins easily view any overridden config.ini.php INI setting without having to open the config file on the server.

  

  New report ‘Personal settings > Custom Variables’ displays custom variable usage

  A new report visible to Super Users lets you see how your Custom Variables slots are being used, which ones are still unused, and how many times a given custom variable name was used.

  

  Learn more about Custom Variables.

  Improvements to Heartbeat feature to accurately count time spent on the last page

  The heartbeat feature which lets you accurately track how long do visitors spend on your website, has been improved in 2.16.0. If you would like to measure how long do people spend on your pages (even if they don’t interact with your website or app), then consider using the heartbeat feature. Learn more in our guide Accurately measure the time spent on each page.

  

  New AnonymousPiwikUsageMeasurement plugin
  

  AnonymousPiwikUsageMeasurement plugin was created for those who would like to measure how their Piwik services is being used. This plugin also sends anonymised tracking data to our demo-anonymous.piwik.org which will help us learn how your Piwik is used and how we could improve the application. Learn more about AnonymousPiwikUsageMeasurement plugin.

  

  Show a warning when Piwik is used as a SuperUser via HTTP

  It is highly recommended to use HTTPS (SSL) for your Piwik server. If you are logged in as a Super User and viewing any Administration pages, you will now see a warning notification message at the top of the screen inviting you to use HTTPS :

  

  MySQL 5.7+ support

  The awesome Piwik community has helped us identify several issues with MySQL 5.7 support and Piwik is now fully compatible with the latest MySQL version 5.7+. Piwik is officially compatible with MySQL >= 4.1 and up to Mysql 5.7 ! Piwik is also compatible with MariaDB.

  

  Better detection of new devices and robots

  Our Device Detector library can now detect many new types of devices : smart TVs, new tablets and mobile phones, and more. New bots were added to our list of bots to keep your analytics report meaningful.

  Learn more : What Device Detector is able to detect, About Device Detector.

  New Referrer Spammers

  Every week we keep updating our community-list of referrer spammers. More than 50 new spammers were added since the last Piwik release !

  Note that the list of referrer spammer is automatically updated every week in your Piwik, so you need to wait at most one week to benefit from the very latest list.

  Learn more about Stopping Referrer Spam in Analytics reports.

  

  Ability to support new SMS providers to send Scheduled Reports by text message

  When you create scheduled reports in Piwik, you can specify one or several email addresses (to send reports by email) and you can also configure phone numbers (so your reports are sent to your phone via text message). So far we only support one SMS Provider “Clockwork SMS” but it is now possible for anyone to create a very simple plugin that adds a new SMS Provider to Piwik.

  A new SMS Provider plugin was released on the Marketplace : FreeMobileMessaging. If you have a favorite SMS Provider, we encourage you to create a new plugin for it !

  Piwik PRO mentions

  A few new mentions of Piwik PRO were added into Piwik to let users know that they can get Professional Help, and additional services and products. For example a new widget was added to the default dashboard. If you provide your Piwik service to your own clients, maybe you would like to remove the Piwik PRO widget and other ads, see this FAQ : How do I remove from Piwik the links to “Piwik PRO” or to “Professional Services” ?

  Stability, Reliability and Performance

  Dozens of other bugs were fixed (including several Major bugs) and many small improvements are included in this release. All these changes make Piwik more stable and reliable, as well as easier to troubleshoot and maintain. We are proud to be able to offer Piwik 2.16.0 as our Long Term Support release !

  If you have any feedback, questions, suggestion, or bug to report, please join our Piwik Community Forums.

  We hope you will love this 2.16.0 release as much as we loved making it.

  Happy Analytics !