Recherche avancée

Sur d’autres sites (6716)

• Managing Music Playback Channels

  30 juin 2013, par Multimedia Mike — General

  My Game Music Appreciation site allows users to interact with old video game music by toggling various channels, as long as the underlying synthesizer engine supports it.

  
  
  

  Users often find their way to the Nintendo DS section pretty quickly. This is when they notice an obnoxious quirk with the channel toggling feature : specifically, one channel doesn’t seem to map to a particular instrument or track.

  When it comes to computer music playback methodologies, I have long observed that there are 2 general strategies : Fixed channel and dynamic channel allocation.

  Fixed Channel Approach
  One of my primary sources of computer-based entertainment used to be watching music. Sure I listened to it as well. But for things like Amiga MOD files and related tracker formats, there was a rich ecosystem of fun music playback programs that visualized the music. There exist music visualization modes in various music players these days (such as iTunes and Windows Media Player), but those largely just show you a single wave form. These files were real time syntheses based on multiple audio channels and usually showed some form of analysis for each channel. My personal favorite was Cubic Player :

  
  
  

  Most of these players supported the concept of masking individual channels. In doing so, the user could isolate, study, and enjoy different components of the song. For many 4-channel Amiga MOD files, I observed that the common arrangement was to use the 4 channels for beat (percussion track), bass line, chords, and melody. Thus, it was easy to just listen to, e.g., the bass line in isolation.

  MODs and similar formats specified precisely which digital audio sample to play at what time and on which specific audio channel. To view the internals of one of these formats, one gets the impression that they contain an extremely computer-centric view of music.

  Dynamic Channel Allocation Algorithm
  MODs et al. enjoyed a lot of popularity, but the standard for computer music is MIDI. While MOD and friends took a computer-centric view of music, MIDI takes, well, a music-centric view of music.

  There are MIDI visualization programs as well. The one that came with my Gravis Ultrasound was called PLAYMIDI.EXE. It looked like this…

  
  
  

  … and it confused me. There are 16 distinct channels being visualized but some channels are shown playing multiple notes. When I dug into the technical details, I learned that MIDI just specifies what notes need to be played, at what times and frequencies and using which instrument samples, and it was the MIDI playback program’s job to make it happen.

  Thus, if a MIDI file specifies that track 1 should play a C major chord consisting of notes C, E, and G, it would transmit events “key-on C ; delta time 0 ; key-on E ; delta time 0 ; key-on G ; delta time … ; [other commands]“. If the playback program has access to multiple channels (say, up to 32, in the case of the GUS), the intuitive approach would be to maintain a pool of all available channels. Then, when it’s time to process the “key-on C” event, fetch the first available channel from the pool, mark it as in-use, play C on the channel, and return that channel to the pool when either the sample runs its course or the corresponding “key-off C” event is encountered in the MIDI command stream.

  About That Game Music
  Circling back around to my game music website, numerous supported systems use the fixed channel approach for playback while others use dynamic channel allocation approach, including evey Nintendo DS game I have so far analyzed.

  Which approach is better ? As in many technical matters, there are trade-offs either way. For many systems, the fixed channel approach is necessary because for many older audio synthesis systems, different channels had very specific purposes. The 8-bit NES had 5 channels : 2 square wave generators (used musically for melody/treble), 1 triangle wave generator (usually used for bass line), a noise generator (subverted for all manner of percussive sounds), and a limited digital channel (was sometimes assigned richer percussive sounds). Dynamic channel allocation wouldn’t work here.

  But the dynamic approach works great on hardware with 16 digital channels available like, for example, the Nintendo DS. Digital channels are very general-purpose. What about the SNES, with its 8 digital channels ? Either approach could work. In practice, most games used a fixed channel approach : Games might use 4-6 channels for music while reserving the remainder for various in-game sound effects. Some notable exceptions to this pattern were David Wise’s compositions for Rare’s SNES games (think Battletoads and the various Donkey Kong Country titles). These clearly use some dynamic channel approach since masking all but one channel will give you a variety of instrument sounds.

  Epilogue
  There ! That took a long time to explain but I find it fascinating for some reason. I need to distill it down to far fewer words because I want to make it a FAQ on my website for “Why can’t I isolate specific tracks for Nintendo DS games ?”

  Actually, perhaps I should remove the ability to toggle Nintendo DS channels in the first place. Here’s a funny tale of needless work : I found the Vio2sf engine for synthesizing Nintendo DS music and incorporated it into the program. It didn’t support toggling of individual channels so I figured out a way to add that feature to the engine. And then I noticed that most Nintendo DS games render that feature moot. After I released the webapp, I learned that I was out of date on the Vio2sf engine. The final insult was that the latest version already supports channel toggling. So I did the work for nothing. But then again, since I want to remove that feature from the UI, doubly so.

• Q&A : An interview with Matomo founder, Matthieu Aubry

  20 novembre 2018, par Joselyn Khor — About, Community

  Hey everyone ! Joselyn here. As always the views of our community remain top of mind. So to make sure you guys know the thinking behind these new projects, we reached out to Matomo’s founder, Matthieu, to ask questions you might want answered. Please check it out below !

  Hi guys, it’s Matthieu ! Here to answer some questions about the rebrand and the future of Matomo and Innocraft.

  What’s upcoming ?

  We’ve been busy implementing our rebrand into all aspects of Matomo and there’s also our new website, which is launching today ! The new website will help people better understand what Matomo is and how they can benefit from using modern web analytics.

  Why was Matomo and Innocraft brought onto one website ?

  In the past the separation caused a bit of confusion so we’re taking this as a chance to unite both the business brand, Innocraft and community brand, Matomo, on one website. Putting our focus on one brand, Matomo, makes it easier for people to see us with fresh eyes. We have a community side as well as a business side and while the community is still incredibly important to us, we find we have a powerful analytics tool that is capable of helping businesses too.

  Is Matomo becoming commercial or turning corporate ?

  No. nothing is changing. Matomo is still an open-source project and community. Although we’ll have a pricing page and “start free trial” on the new website brought over from Innocraft.cloud, the Matomo community will still play the biggest part on the Matomo website. We have dedicated sections focused on Community and On-Premise.

  The rebrand exercise helped us gain a refreshed perspective. After reflecting on how far we’ve come, we can feel more confident about Matomo Analytics itself as a platform. We believe it’s a great chance to bring that confidence into the brand and vision. We are proud that it’s an awesome open-source platform and at the same time it’s also powerful as a tool for businesses.

  Why is there no ‘download for free’ button on the homepage ?

   

  
  We feel many users coming to the site will get confused about our hosting options (Cloud and On-Premise) which is something you don’t usually consider when choosing an analytics tool.

  The reason for us to not have that button is when people see a “download for free” button on the homepage next to a “try it for free” button, it creates confusion. For those who do choose to download Matomo often become confused when they are left with a .zip file unaware how to install it and the technical requirements of self-hosting. We feel presenting our users with the simplest installation option first will give them the best chance possible to try Matomo to its full potential, without cost.

  And you can still find the link to Download Matomo in the footer of each page.
  

   

  Is Matomo still free to download and have forever ?

  Absolutely. The free open-source download can be found on the On-Premise section of the website, or download Matomo here.

  Why is it important to have a business behind the project ?

  There’s the reality that we have to make money in order for the Matomo project to survive … and thrive. The reason we still need a business side (Innocraft) is to fund and sustain the Matomo project. Whenever people purchase premium features, this helps finance the development of Matomo for our community.

  Because of the business we’re able to continually maintain and develop Matomo for you guys as well as future users. For example, the next release Matomo 3.8.0 is already mostly developed and will bring lots of interesting features too, like the two-factor authentication, Brute Force Protection, failed tracking requests reporting, lots of JavaScript tracker improvements, a new total summary row below reports, and many more security fixes, bug fixes, and other new features.

  So we see a business being very helpful in supporting our open-source community. Without a business side, our free, open-source project would not be able to survive.

  How will you protect the Matomo project ?

  We’ve ensured the Matomo project will be protected for the future as we wish to turn it into a not-for-profit foundation.

  We’ve also got a safeguard where the open-source code will stay under a GPL license forever. This is so we can guarantee, that no matter what happens, the Matomo project itself will stay completely free software.

  Is there a way for people to help ?

  There are heaps of ways to help ! You can help other Matomo users in the forums, contribute to fixes on GitHub, leave a great review (e.g. alternativeTo), help look for bugs with our Security Bounty Programme or participate and spread the word about Matomo in our community social media pages – Mastodon, Facebook, Twitter. Telling your friends about us would be very helpful too !

  What’s planned for the future ?
  We’ve worked hard to become the #1 open-source analytics platform (1.4 million websites use Matomo today), but now we need to empower even more individuals and businesses to take back control of their own data.

  Showing our community that we have a powerful platform is crucial, but alongside that our values are what define us. User privacy is still of utmost importance and we’re here to make it known that power needs to rest in the hands of people and not large corporations.

  You can rest easy knowing you’re doing your part in using trustworthy and dependable tools. By joining many other companies who are growing this movement to decentralise the Internet, we can build a safer, online world together.

  Join this analytics revolution and let us know what you think about Matomo !

• Creating A Lossless SMC Encoder

  26 avril 2011, par Multimedia Mike — General

  Look, I can’t explain how or why I come up with this stuff. For some reason, I thought it would be interesting to write a new encoder for the Apple SMC video codec. I can’t even remember why. I just sat down the other day, started writing, and now I have a lossless SMC encoder that I’m not sure what to do with. Maybe this is to be my new thing— writing encoders for marginal multimedia formats.

  Introduction
  SMC is a vector quantizer (a lossy method) but I decided to attack it from the angle of lossless encoding. A.k.a. Apple Graphics Codec, SMC operates on 4x4 blocks in an 8-bit paletted colorspace. Each 4x4 block can be encoded with 1, 2, 4, 8, or 16 colors. Blocks can also be skipped (copied from previous frame) or copied from blocks rendered immediately prior within the same frame.

  Step 1 : Validating Infrastructure
  The goal of this step is to encode the most braindead SMC frame possible and see if FFmpeg/libav’s QuickTime muxer can create a valid file. I think the simplest frame would be one in which each vector is encoded with the single-color mode, starting with color 0 and incrementing through the palette.

  Status : Successful. The only ’trick’ was to set avctx->bits_per_coded_sample to 8. (For fun, this can also be set to 40 (8 | 0x20) to specify a grayscale palette.)
  

  
  
  

  Step 2 : Preprocessing
  The video frames will arrive at the encoder as 32-bit RGB. These will need to be converted to a paletted colorspace before encoding. I don’t want to use FFmpeg’s default dithering approach as this will result in a substantial loss of quality as described in this post. I would rather maintain a palette built from observed colors throughout successive frames. If the total number of unique observed colors ever exceeds 256, error out.

  That’s what I would like to do. However, I noticed that FFmpeg/libav’s QuickTime muxer has never taken into account the possibility of encoding palettes. The path of least resistance in this case is to dither the input to match QuickTime’s default 8-bit palette (if a paletted QuickTime file does not specify a palette, a default 1-, 2-, 4-, or 8-bit palette is selected).

  Status : Successful, if slow. I definitely need to optimize this step later.

  Step 3 : Most Naive Encoding
  The most basic encoding is to "encode" each block as a 16-color block. This will actually result in a slightly larger frame size than a raw encoding since each 4x4 block will be prepended by a byte opcode (0xE0 in this case) to indicate encoding mode. This should demonstrate that the encoder is functioning at the most basic level.

  Status : Successful. Try not to laugh too hard at the Big Buck Bunny dithered to an 8-bit palette :

  
  
  

  Step 4 : Better Representation
  It seems to me that encoding this format (losslessly) will entail performing vector operations on lots of 16-element (4x4-pixel) vectors. These could be done on the frame as-is, but it strikes me as more efficient and perhaps less error prone to rearrange the input images into a vector of vectors (or array of arrays if you prefer) :

    0  1  2  3  w ...
    4  5  6  7  x ...
    8  9  A  B  y ...
    C  D  E  F  z ...
  

    0 : [0 1 2 3 4 5 6 7 8 9 A B C D E F]
    1 : [...]
  

  Status : Successful.

  Step 5 : Add Interframe Skip Codes
  Time to add a bit of brainpower to the proceedings : On non-keyframes, compare the current vector to the vector at the same position from the previous frame.

  Test this by encoding a pair of identical frames. Ideally, all codes should be skip codes.

  Status : Successful, though my vector matching function could probably be improved.

  Step 6 : Analyze Blocks For Optimal Color Coding
  This is where things get potentially interesting, algorithmically. At least, I need to figure out (or look up) an algorithm to count the unique elements in a vector.

  Naive algorithm (i.e., first thing I can think of) :

  • initialize a count variable to 0
  • initialize an array of 256 flags to false
  • for each 8-bit element in vector :
    • if flag array[element] is 0, set array[element] to true and increment count

  Status : Successful. Here is the distribution for the 640x360 Big Buck Bunny title :

  1194 4636 4113 2140 1138 568 325 154 80 36 9 5 2 0 0 0

  Or, in pretty graph form, demonstrating that vectors with few distinct elements dominate :
  

  
  
  

  Step 7 : Encode Monochrome Blocks
  At this point, the structure is starting to come together pretty well. This phase involves encoding a 0x60 opcode and a palette index when the count_distinct() function returns 1.

  Status : Absolutely no problem.

  Step 8 : Encode 2-, 4-, and 8-color Modes
  This step is a little more involved. This is where SMC’s 2-, 4-, and 8-color circular palette caches come into play. E.g., when the first 2-color block is encoded, the pair of colors it uses will be inserted into entry 0 of the 2-color cache. During the next 2-color block encoding, if the block uses a pair of colors that already occurs in the cache, the encoding can reference that cache entry. Otherwise, it adds the pair to the next available cache entry, looping back around to 0 as necessary.

  I think I should modify the count_distinct() function to also return a 16-byte array that contains a sorted list of the palette indicies used in the vector. The color pair cache will contain 256 16-bit, 32-bit ints for the quads and 64-bit ints for the octets. This will allow a slightly faster linear cache search.

  Status : The 2-color encoding wasn’t too much trouble and I was able to adapt it to the 4-color mode pretty quickly afterward. I’m still having trouble with the insane 8-color coding mode, though. So that’s commented out for the time being.

  Step 9 : Run Encoding and Putting It All Together
  For each frame, convert the input pixels to a paletted format via one method or another (match to default QuickTime palette for first pass). Then, preprocess each vector to determine the minimum number of elements that can be used to represent it, storing the sorted list of distinct colors in a separate array. The number of elements can either be 0 (only for interframes and indicates a skip block), 1, 2, 4, 8, or 16. Also during this phase, for each vector after the first, test if the vector is the same as the previous vector. If it is, denote this fact in the preprocessed encoding (set the high bit of the element count number).

  Finally, pack it into the bytestream. Iterate through the element count array and search for the longest runs of elements that are encoded with the same mode (up to 256 for skip modes, up to 16 for other modes). If the high bit of an element count is set, that indicates that a copy mode can be encoded. Look for the longest run of element counts with the high bit set and encode a copy mode.

  Status : In-process. Will finish this as motivation strikes.