Recherche avancée

Sur d’autres sites (12219)

• Managing Music Playback Channels

  30 juin 2013, par Multimedia Mike — General

  My Game Music Appreciation site allows users to interact with old video game music by toggling various channels, as long as the underlying synthesizer engine supports it.

  
  
  

  Users often find their way to the Nintendo DS section pretty quickly. This is when they notice an obnoxious quirk with the channel toggling feature : specifically, one channel doesn’t seem to map to a particular instrument or track.

  When it comes to computer music playback methodologies, I have long observed that there are 2 general strategies : Fixed channel and dynamic channel allocation.

  Fixed Channel Approach
  One of my primary sources of computer-based entertainment used to be watching music. Sure I listened to it as well. But for things like Amiga MOD files and related tracker formats, there was a rich ecosystem of fun music playback programs that visualized the music. There exist music visualization modes in various music players these days (such as iTunes and Windows Media Player), but those largely just show you a single wave form. These files were real time syntheses based on multiple audio channels and usually showed some form of analysis for each channel. My personal favorite was Cubic Player :

  
  
  

  Most of these players supported the concept of masking individual channels. In doing so, the user could isolate, study, and enjoy different components of the song. For many 4-channel Amiga MOD files, I observed that the common arrangement was to use the 4 channels for beat (percussion track), bass line, chords, and melody. Thus, it was easy to just listen to, e.g., the bass line in isolation.

  MODs and similar formats specified precisely which digital audio sample to play at what time and on which specific audio channel. To view the internals of one of these formats, one gets the impression that they contain an extremely computer-centric view of music.

  Dynamic Channel Allocation Algorithm
  MODs et al. enjoyed a lot of popularity, but the standard for computer music is MIDI. While MOD and friends took a computer-centric view of music, MIDI takes, well, a music-centric view of music.

  There are MIDI visualization programs as well. The one that came with my Gravis Ultrasound was called PLAYMIDI.EXE. It looked like this…

  
  
  

  … and it confused me. There are 16 distinct channels being visualized but some channels are shown playing multiple notes. When I dug into the technical details, I learned that MIDI just specifies what notes need to be played, at what times and frequencies and using which instrument samples, and it was the MIDI playback program’s job to make it happen.

  Thus, if a MIDI file specifies that track 1 should play a C major chord consisting of notes C, E, and G, it would transmit events “key-on C ; delta time 0 ; key-on E ; delta time 0 ; key-on G ; delta time … ; [other commands]“. If the playback program has access to multiple channels (say, up to 32, in the case of the GUS), the intuitive approach would be to maintain a pool of all available channels. Then, when it’s time to process the “key-on C” event, fetch the first available channel from the pool, mark it as in-use, play C on the channel, and return that channel to the pool when either the sample runs its course or the corresponding “key-off C” event is encountered in the MIDI command stream.

  About That Game Music
  Circling back around to my game music website, numerous supported systems use the fixed channel approach for playback while others use dynamic channel allocation approach, including evey Nintendo DS game I have so far analyzed.

  Which approach is better ? As in many technical matters, there are trade-offs either way. For many systems, the fixed channel approach is necessary because for many older audio synthesis systems, different channels had very specific purposes. The 8-bit NES had 5 channels : 2 square wave generators (used musically for melody/treble), 1 triangle wave generator (usually used for bass line), a noise generator (subverted for all manner of percussive sounds), and a limited digital channel (was sometimes assigned richer percussive sounds). Dynamic channel allocation wouldn’t work here.

  But the dynamic approach works great on hardware with 16 digital channels available like, for example, the Nintendo DS. Digital channels are very general-purpose. What about the SNES, with its 8 digital channels ? Either approach could work. In practice, most games used a fixed channel approach : Games might use 4-6 channels for music while reserving the remainder for various in-game sound effects. Some notable exceptions to this pattern were David Wise’s compositions for Rare’s SNES games (think Battletoads and the various Donkey Kong Country titles). These clearly use some dynamic channel approach since masking all but one channel will give you a variety of instrument sounds.

  Epilogue
  There ! That took a long time to explain but I find it fascinating for some reason. I need to distill it down to far fewer words because I want to make it a FAQ on my website for “Why can’t I isolate specific tracks for Nintendo DS games ?”

  Actually, perhaps I should remove the ability to toggle Nintendo DS channels in the first place. Here’s a funny tale of needless work : I found the Vio2sf engine for synthesizing Nintendo DS music and incorporated it into the program. It didn’t support toggling of individual channels so I figured out a way to add that feature to the engine. And then I noticed that most Nintendo DS games render that feature moot. After I released the webapp, I learned that I was out of date on the Vio2sf engine. The final insult was that the latest version already supports channel toggling. So I did the work for nothing. But then again, since I want to remove that feature from the UI, doubly so.

• Statically built FFMPEG binary segmentation fault

  12 février 2020, par stevendesu

  I want to create a custom build of FFMPEG which rips out everything except for the ability to transmux HLS videos to MP4, and I need this build to be 100% static with no external dependencies

  I tried using the following configuration :

  ./configure \
  
      --extra-cflags='-static -static-libstdc++ -static-libgcc' \
  
      --extra-cxxflags='-static -static-libstdc++ -static-libgcc' \
  
      --extra-ldflags='-static -static-libstdc++ -static-libgcc' \
  
      --pkg-config-flags='--static' \
  
      --enable-static \
  
      --disable-shared \
  
      --disable-runtime-cpudetect \
  
      --disable-autodetect \
  
      --disable-ffplay \
  
      --disable-ffprobe \
  
      --disable-doc \
  
      --disable-avdevice \
  
      --disable-swresample \
  
      --disable-swscale \
  
      --disable-postproc \
  
      --disable-pthreads \
  
      --disable-w32threads \
  
      --disable-os2threads \
  
      --enable-network \
  
      --disable-dct \
  
      --disable-dwt \
  
      --disable-error-resilience \
  
      --disable-lsp \
  
      --disable-lzo \
  
      --disable-mdct \
  
      --disable-rdft \
  
      --disable-fft \
  
      --disable-faan \
  
      --disable-pixelutils \
  
      --disable-encoders \
  
      --disable-decoders \
  
      --disable-hwaccels \
  
      --disable-muxers \
  
      --enable-muxer=mov \
  
      --enable-muxer=mp4 \
  
      --disable-demuxers \
  
      --enable-demuxer=hls \
  
      --enable-demuxer=mpegts \
  
      --enable-demuxer=h264 \
  
      --enable-demuxer=aac \
  
      --disable-parsers \
  
      --enable-parser=h264 \
  
      --enable-parser=aac \
  
      --disable-bsfs \
  
      --disable-protocols \
  
      --enable-protocol=tcp \
  
      --enable-protocol=tls \
  
      --enable-protocol=http \
  
      --enable-protocol=https \
  
      --enable-protocol=hls \
  
      --disable-indevs \
  
      --disable-outdevs \
  
      --disable-devices \
  
      --disable-filters \
  
      --disable-alsa \
  
      --disable-appkit \
  
      --disable-avfoundation \
  
      --disable-bzlib \
  
      --disable-coreimage \
  
      --disable-iconv \
  
      --disable-lzma \
  
      --enable-openssl \
  
      --disable-sndio \
  
      --disable-sdl2 \
  
      --disable-securetransport \
  
      --disable-xlib \
  
      --disable-zlib \
  
      --disable-amf \
  
      --disable-audiotoolbox \
  
      --disable-cuda-llvm \
  
      --disable-cuvid \
  
      --disable-d3d11va \
  
      --disable-dxva2 \
  
      --disable-ffnvcodec \
  
      --disable-nvdec \
  
      --disable-nvenc \
  
      --disable-v4l2-m2m \
  
      --disable-vaapi \
  
      --disable-vdpau \
  
      --disable-videotoolbox \
  
      --disable-debug

  This looked about like what I wanted :

  install prefix            /usr/local
  
  source path               .
  
  C compiler                gcc
  
  C library                 glibc
  
  ARCH                      x86 (generic)
  
  big-endian                no
  
  runtime cpu detection     no
  
  standalone assembly       yes
  
  x86 assembler             nasm
  
  MMX enabled               yes
  
  MMXEXT enabled            yes
  
  3DNow! enabled            yes
  
  3DNow! extended enabled   yes
  
  SSE enabled               yes
  
  SSSE3 enabled             yes
  
  AESNI enabled             yes
  
  AVX enabled               yes
  
  AVX2 enabled              yes
  
  AVX-512 enabled           yes
  
  XOP enabled               yes
  
  FMA3 enabled              yes
  
  FMA4 enabled              yes
  
  i686 features enabled     yes
  
  CMOV is fast              yes
  
  EBX available             yes
  
  EBP available             yes
  
  debug symbols             no
  
  strip symbols             yes
  
  optimize for size         no
  
  optimizations             yes
  
  static                    yes
  
  shared                    no
  
  postprocessing support    no
  
  network support           yes
  
  threading support         no
  
  safe bitstream reader     yes
  
  texi2html enabled         no
  
  perl enabled              yes
  
  pod2man enabled           yes
  
  makeinfo enabled          no
  
  makeinfo supports HTML    no
  
  
  
  External libraries:
  
  openssl
  
  
  
  External libraries providing hardware acceleration:
  
  
  
  Libraries:
  
  avcodec                 avfilter                avformat                avutil
  
  
  
  Programs:
  
  ffmpeg
  
  
  
  Enabled decoders:
  
  
  
  Enabled encoders:
  
  
  
  Enabled hwaccels:
  
  
  
  Enabled parsers:
  
  aac                     h264
  
  
  
  Enabled demuxers:
  
  aac                     h264                    hls                     mpegts
  
  
  
  Enabled muxers:
  
  mov                     mp4
  
  
  
  Enabled protocols:
  
  hls                     http                    https                   tcp                     tls
  
  
  
  Enabled filters:
  
  aformat                 anull                   atrim                   format                  hflip                   null                    transpose               trim                    vflip
  
  
  
  Enabled bsfs:
  
  null
  
  
  
  Enabled indevs:
  
  
  
  Enabled outdevs:
  
  
  
  License: LGPL version 2.1 or later

  It included several filters which I won’t ever need or use, but these filters are pulled in automatically if you don’t specify --disable-avfilter, and specifying --disable-avfilter prevents the ffmpeg binary from being produced. So I’m stuck with those.

  Using these parameters and then running make, I received a binary that was about 5.9 MB in size and looked right :

  $> ldd ffmpeg
  
          not a dynamic executable

  But when I try to run it :

  $> ./ffmpeg -version
  
  Segmentation fault

  Using valgrind to try and inspect the cause of the segmentation fault :

  $> valgrind ./ffmpeg -version
  
  .... lots of stuff ...
  
  ==61362== Jump to the invalid address stated on the next line
  
  ==61362==    at 0x0: ???
  
  ==61362==    by 0x70BB1B: ??? (in /src/FFmpeg/ffmpeg)
  
  ==61362==    by 0x70B2E6: ??? (in /src/FFmpeg/ffmpeg)
  
  ==61362==    by 0x4033F9: ??? (in /src/FFmpeg/ffmpeg)
  
  ==61362==    by 0x1FFF000677: ???
  
  ==61362==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
  
  ==61362==
  
  ==61362==
  
  ==61362== Process terminating with default action of signal 11 (SIGSEGV)
  
  ==61362==  Bad permissions for mapped region at address 0x0
  
  ==61362==    at 0x0: ???
  
  ==61362==    by 0x70BB1B: ??? (in /src/FFmpeg/ffmpeg)
  
  ==61362==    by 0x70B2E6: ??? (in /src/FFmpeg/ffmpeg)
  
  ==61362==    by 0x4033F9: ??? (in /src/FFmpeg/ffmpeg)
  
  ==61362==    by 0x1FFF000677: ???
  
  ==61362==
  
  ==61362== HEAP SUMMARY:
  
  ==61362==     in use at exit: 0 bytes in 0 blocks
  
  ==61362==   total heap usage: 0 allocs, 0 frees, 0 bytes allocated
  
  ==61362==
  
  ==61362== All heap blocks were freed -- no leaks are possible
  
  ==61362==
  
  ==61362== For counts of detected and suppressed errors, rerun with: -v
  
  ==61362== Use --track-origins=yes to see where uninitialised values come from
  
  ==61362== ERROR SUMMARY: 93 errors from 90 contexts (suppressed: 0 from 0)
  
  Segmentation fault

  Attempting to access memory at location 0x0 sounds like trying to follow a null pointer. But I’m not sure how to fix this.

  gdb backtrace

  When I first ran gdb ./ffmpeg gdb immediately gave me a segmentation fault and I wasn’t kicked into the gdb REPL, so I couldn’t investigate

  After rebuilding ffmpeg I was able to get in this time :

  $> gdb ./ffmpeg
  
  
  
  GNU gdb (Ubuntu 8.1-0ubuntu3.2) 8.1.0.20180409-git
  
  Copyright (C) 2018 Free Software Foundation, Inc.
  
  License GPLv3+: GNU GPL version 3 or later /gnu.org/licenses/gpl.html>
  
  This is free software: you are free to change and redistribute it.
  
  There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
  
  and "show warranty" for details.
  
  This GDB was configured as "x86_64-linux-gnu".
  
  Type "show configuration" for configuration details.
  
  For bug reporting instructions, please see:
  
  /www.gnu.org/software/gdb/bugs/>.
  
  Find the GDB manual and other documentation resources online at:
  
  /www.gnu.org/software/gdb/documentation/>.
  
  For help, type "help".
  
  Type "apropos word" to search for commands related to "word"...
  
  Reading symbols from ffmpeg...done.
  
  (gdb) r
  
  Starting program: /src/FFmpeg/ffmpeg
  
  warning: Error disabling address space randomization: Operation not permitted
  
  
  
  Program received signal SIGSEGV, Segmentation fault.
  
  0x0000000000000000 in ?? ()
  
  (gdb) bt
  
  #0  0x0000000000000000 in ?? ()
  
  #1  0x0000000000f9a8d5 in __register_frame_info_bases.part.6 ()
  
  #2  0x00000000004445fd in frame_dummy ()
  
  #3  0x0000000000000001 in ?? ()
  
  #4  0x0000000000ebd20c in __libc_csu_init ()
  
  #5  0x0000000000ebc9d7 in __libc_start_main ()
  
  #6  0x000000000044451a in _start ()
  
  (gdb)

  I tried grep’ing the code base for __register_frame_info_bases and found nothing. So I’m not really sure where to go from here

  A fix, but not an explanation

  By randomly removing configuration parameters and rebuilding I discovered that --disable-pthreads was causing the segmentation fault. When I remove this, ffmpeg runs just fine

  I don’t know why this is the case, though. Why would they make it possible to remove something that you need to run ?

• Python - why do I get this ffprobe error ?

  4 juillet 2022, par gasper101

  I am on mac and am working on a project in which my program shoud convert a .mp3 file into a .wav file and than get text from that file (basic speech recognition). For converting a .mp3 file into a .wav file I am using ffmpeg library. I have installed both ffmpeg and ffprobe libraryies using pip and than using homebrew so I have them installed in a folder.

  


  This is my code :

  


  import speech_recognition
from pydub import AudioSegment
from os import path


#get text from original voice file------------------------------------------------- 

sound = AudioSegment.from_mp3("/path/to/file.mp3")
sound.export("/output/path/file.wav", format="wav")

file_audio = speech_recognition.AudioFile('/output/path/file.wav')

r = speech_recognition.Recognizer()
with file_audio as source:
    audio_text = r.record(source)

word = r.recognize_google(audio_text)


  


  And this is the error i get :
FileNotFoundError : [Errno 2] No such file or directory : 'ffprobe'

  


  Full error :
Warning (from warnings module) :
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pydub/utils.py", line 170
warn("Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work", RuntimeWarning)
RuntimeWarning : Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work

  


  Warning (from warnings module) :
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pydub/utils.py", line 198
warn("Couldn't find ffprobe or avprobe - defaulting to ffprobe, but may not work", RuntimeWarning)

  


  RuntimeWarning : Couldn't find ffprobe or avprobe - defaulting to ffprobe, but may not work
Traceback (most recent call last) :
File "/Users/marcus/Downloads/pronaunciation_checker/pronaunciation_check.py", line 12, in 
sound = AudioSegment.from_mp3("/Users/marcus/Desktop/think.mp3")
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pydub/audio_segment.py", line 796, in from_mp3
return cls.from_file(file, 'mp3', parameters=parameters)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pydub/audio_segment.py", line 728, in from_file
info = mediainfo_json(orig_file, read_ahead_limit=read_ahead_limit)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pydub/utils.py", line 274, in mediainfo_json
res = Popen(command, stdin=stdin_parameter, stdout=PIPE, stderr=PIPE)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 951, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/subprocess.py", line 1821, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError : [Errno 2] No such file or directory : 'ffprobe'