Recherche avancée

Sur d’autres sites (8423)

• Managing Music Playback Channels

  30 juin 2013, par Multimedia Mike — General

  My Game Music Appreciation site allows users to interact with old video game music by toggling various channels, as long as the underlying synthesizer engine supports it.

  
  
  

  Users often find their way to the Nintendo DS section pretty quickly. This is when they notice an obnoxious quirk with the channel toggling feature : specifically, one channel doesn’t seem to map to a particular instrument or track.

  When it comes to computer music playback methodologies, I have long observed that there are 2 general strategies : Fixed channel and dynamic channel allocation.

  Fixed Channel Approach
  One of my primary sources of computer-based entertainment used to be watching music. Sure I listened to it as well. But for things like Amiga MOD files and related tracker formats, there was a rich ecosystem of fun music playback programs that visualized the music. There exist music visualization modes in various music players these days (such as iTunes and Windows Media Player), but those largely just show you a single wave form. These files were real time syntheses based on multiple audio channels and usually showed some form of analysis for each channel. My personal favorite was Cubic Player :

  
  
  

  Most of these players supported the concept of masking individual channels. In doing so, the user could isolate, study, and enjoy different components of the song. For many 4-channel Amiga MOD files, I observed that the common arrangement was to use the 4 channels for beat (percussion track), bass line, chords, and melody. Thus, it was easy to just listen to, e.g., the bass line in isolation.

  MODs and similar formats specified precisely which digital audio sample to play at what time and on which specific audio channel. To view the internals of one of these formats, one gets the impression that they contain an extremely computer-centric view of music.

  Dynamic Channel Allocation Algorithm
  MODs et al. enjoyed a lot of popularity, but the standard for computer music is MIDI. While MOD and friends took a computer-centric view of music, MIDI takes, well, a music-centric view of music.

  There are MIDI visualization programs as well. The one that came with my Gravis Ultrasound was called PLAYMIDI.EXE. It looked like this…

  
  
  

  … and it confused me. There are 16 distinct channels being visualized but some channels are shown playing multiple notes. When I dug into the technical details, I learned that MIDI just specifies what notes need to be played, at what times and frequencies and using which instrument samples, and it was the MIDI playback program’s job to make it happen.

  Thus, if a MIDI file specifies that track 1 should play a C major chord consisting of notes C, E, and G, it would transmit events “key-on C ; delta time 0 ; key-on E ; delta time 0 ; key-on G ; delta time … ; [other commands]“. If the playback program has access to multiple channels (say, up to 32, in the case of the GUS), the intuitive approach would be to maintain a pool of all available channels. Then, when it’s time to process the “key-on C” event, fetch the first available channel from the pool, mark it as in-use, play C on the channel, and return that channel to the pool when either the sample runs its course or the corresponding “key-off C” event is encountered in the MIDI command stream.

  About That Game Music
  Circling back around to my game music website, numerous supported systems use the fixed channel approach for playback while others use dynamic channel allocation approach, including evey Nintendo DS game I have so far analyzed.

  Which approach is better ? As in many technical matters, there are trade-offs either way. For many systems, the fixed channel approach is necessary because for many older audio synthesis systems, different channels had very specific purposes. The 8-bit NES had 5 channels : 2 square wave generators (used musically for melody/treble), 1 triangle wave generator (usually used for bass line), a noise generator (subverted for all manner of percussive sounds), and a limited digital channel (was sometimes assigned richer percussive sounds). Dynamic channel allocation wouldn’t work here.

  But the dynamic approach works great on hardware with 16 digital channels available like, for example, the Nintendo DS. Digital channels are very general-purpose. What about the SNES, with its 8 digital channels ? Either approach could work. In practice, most games used a fixed channel approach : Games might use 4-6 channels for music while reserving the remainder for various in-game sound effects. Some notable exceptions to this pattern were David Wise’s compositions for Rare’s SNES games (think Battletoads and the various Donkey Kong Country titles). These clearly use some dynamic channel approach since masking all but one channel will give you a variety of instrument sounds.

  Epilogue
  There ! That took a long time to explain but I find it fascinating for some reason. I need to distill it down to far fewer words because I want to make it a FAQ on my website for “Why can’t I isolate specific tracks for Nintendo DS games ?”

  Actually, perhaps I should remove the ability to toggle Nintendo DS channels in the first place. Here’s a funny tale of needless work : I found the Vio2sf engine for synthesizing Nintendo DS music and incorporated it into the program. It didn’t support toggling of individual channels so I figured out a way to add that feature to the engine. And then I noticed that most Nintendo DS games render that feature moot. After I released the webapp, I learned that I was out of date on the Vio2sf engine. The final insult was that the latest version already supports channel toggling. So I did the work for nothing. But then again, since I want to remove that feature from the UI, doubly so.

• lavc : prefer the mp3float decoder to the mp3 decoder

  1er avril 2018, par Rostislav Pehlivanov

  lavc : prefer the mp3float decoder to the mp3 decoder
  On modern x86 systems its around 2x faster. For systems without
  
  FPUs it'll be slower, but our policy is to prefer floating point
  
  implementations and to let users decide what's best (or just not
  
  compile them on systems without FPUs).
  Signed-off-by : Rostislav Pehlivanov <atomnuker@gmail.com>
  

  • [DH] libavcodec/allcodecs.c
  • [DH] tests/fate/gapless.mak
  • [DH] tests/ref/fate/exif-image-embedded

• How to join webcam FLVs

  18 mars 2015, par Marc-André Lafortune

  I want my website to join some webcam recordings in FLV files (like this one). This needs to be done on Linux without user input. How do I do this ? For simplicity’s sake, I’ll use the same flv as both inputs in hope of getting a flv that plays the same thing twice in a row.

  That should be easy enough, right ? There’s even a full code example in the ffmpeg FAQ.

  Well, pipes seem to be giving me problems (both on my mac running Leopard and on Ubuntu 8.04) so let’s keep it simple and use normal files. Also, if I don’t specify a rate of 15 fps, the visual part plays extremely fast. The example script thus becomes :

  ffmpeg -i input.flv -vn -f u16le -acodec pcm_s16le -ac 2 -ar 44100 \
  
    - > temp.a < /dev/null
  
  ffmpeg -i input.flv -an -f yuv4mpegpipe - > temp.v < /dev/null
  
  cat temp.v temp.v > all.v
  
  cat temp.a temp.a > all.a
  
  ffmpeg -f u16le -acodec pcm_s16le -ac 2 -ar 44100 -i all.a \
  
    -f yuv4mpegpipe -i all.v -sameq -y output.flv

  Well, using this will work for the audio, but I only get the video the first time around. This seems to be the case for any flv I throw as input.flv, including the movie teasers that come with red5.

  a) Why doesn’t the example script work as advertised, in particular why do I not get all the video I’m expecting ?

  b) Why do I have to specify a framerate while Wimpy player can play the flv at the right speed ?

  The only way I found to join two flvs was to use mencoder. Problem is, mencoder doesn’t seem to join flvs :

  mencoder input.flv input.flv -o output.flv -of lavf -oac copy \
  
  -ovc lavc -lavcopts vcodec=flv

  I get a Floating point exception...

  MEncoder 1.0rc2-4.0.1 (C) 2000-2007 MPlayer Team
  
  CPU: Intel(R) Xeon(R) CPU 5150 @ 2.66GHz (Family: 6, Model: 15, Stepping: 6)
  
  CPUflags: Type: 6 MMX: 1 MMX2: 1 3DNow: 0 3DNow2: 0 SSE: 1 SSE2: 1
  
  Compiled for x86 CPU with extensions: MMX MMX2 SSE SSE2
  
  
  
  success: format: 0 data: 0x0 - 0x45b2f
  
  libavformat file format detected.
  
  [flv @ 0x697160]Unsupported audio codec (6)
  
  [flv @ 0x697160]Could not find codec parameters (Audio: 0x0006, 22050 Hz, mono)
  
  [lavf] Video stream found, -vid 0
  
  [lavf] Audio stream found, -aid 1
  
  VIDEO: [FLV1] 240x180 0bpp 1000.000 fps 0.0 kbps ( 0.0 kbyte/s)
  
  [V] filefmt:44 fourcc:0x31564C46 size:240x180 fps:1000.00 ftime:=0.0010
  
  ** MUXER_LAVF *****************************************************************
  
  REMEMBER: MEncoder's libavformat muxing is presently broken and can generate
  
  INCORRECT files in the presence of B frames. Moreover, due to bugs MPlayer
  
  will play these INCORRECT files as if nothing were wrong!
  
  *******************************************************************************
  
  OK, exit
  
  Opening video filter: [expand osd=1]
  
  Expand: -1 x -1, -1 ; -1, osd: 1, aspect: 0.000000, round: 1
  
  ==========================================================================
  
  Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
  
  Selected video codec: [ffflv] vfm: ffmpeg (FFmpeg Flash video)
  
  ==========================================================================
  
  audiocodec: framecopy (format=6 chans=1 rate=22050 bits=16 B/s=0 sample-0)
  
  VDec: vo config request - 240 x 180 (preferred colorspace: Planar YV12)
  
  VDec: using Planar YV12 as output csp (no 0)
  
  Movie-Aspect is undefined - no prescaling applied.
  
  videocodec: libavcodec (240x180 fourcc=31564c46 [FLV1])
  
  VIDEO CODEC ID: 22
  
  AUDIO CODEC ID: 10007, TAG: 0
  
  Writing header...
  
  [NULL @ 0x67d110]codec not compatible with flv
  
  Floating point exception

  c) Is there a way for mencoder to decode and encode flvs correctly ?

  So the only way I’ve found so far to join flvs, is to use ffmpeg to go back and forth between flv and avi, and use mencoder to join the avis :

  ffmpeg -i input.flv -vcodec rawvideo -acodec pcm_s16le -r 15 file.avi
  
  mencoder -o output.avi -oac copy -ovc copy -noskip file.avi file.avi
  
  ffmpeg -i output.avi output.flv

  d) There must be a better way to achieve this... Which one ?

  e) Because of the problem of the framerate, though, only flvs with constant framerate (like the one I recorded through facebook) will be converted correctly to avis, but this won’t work for the flvs I seem to be recording (like this one or this one). Is there a way to do this for these flvs too ?

  Any help would be very appreciated.