Sur d’autres sites (5624)

• Adding A New System To The Game Music Website

  1er août 2012, par Multimedia Mike — General

  At first, I was planning to just make a little website where users could install a Chrome browser extension and play music from old 8-bit NES games. But, like many software projects, the goal sort of ballooned. I created a website where users can easily play old video game music. It doesn’t cover too many systems yet, but I have had individual requests to add just about every system you can think of.

  The craziest part is that I know it’s possible to represent most of the systems. Eventually, it would be great to reach Chipamp parity (a combination plugin for Winamp that packages together plugins for many of these chiptunes). But there is a process to all of this. I have taken to defining a number of phases that are required to get a new system covered.

  Phase 0 informally involves marveling at the obscurity of some of the console systems for which chiptune collections have evolved. WonderSwan ? Sharp X68000 ? PC-88 ? I may be viewing this through a terribly Ameri-centric lens. I’ve at least heard of the ZX Spectrum and the Amstrad CPC even if I’ve never seen either.

  No matter. The goal is to get all their chiptunes cataloged and playable.

  Phase 1 : Finding A Player
  The first step is to find a bit of open source code that can play a particular format. If it’s a library that can handle many formats, like Game Music Emu or Audio Overload SDK, even better (probably). The specific open source license isn’t a big concern for me. I’m almost certain that some of the libraries that SaltyGME currently mixes are somehow incompatible, license-wise. I’ll worry about it when I encounter someone who A) cares, and B) is in a position to do something about it. Historical preservation comes first, and these software libraries aren’t getting any younger (I’m finding some that haven’t been touched in a decade).

  Phase 2 : Test Program
  The next phase is to create a basic test bench program that sends a music file into the library, generates a buffer of audio, and shoves it out to the speakers via PulseAudio’s simple API (people like to rip on PulseAudio, but its simple API really lives up to its name and requires pages less boilerplate code to play a few samples than ALSA).

  Phase 3 : Plug Into Web Player
  After successfully creating the test bench and understanding exactly which source files need to be built, the next phase is to hook it up to the main SaltyGME program via the ad-hoc plugin API I developed. This API requires that a player backend can, at the very least, initialize itself based on a buffer of bytes and generate audio samples into an array of 16-bit numbers. The API also provides functions for managing files with multiple tracks and toggling individual voices/channels if the library supports such a feature. Having the test bench application written beforehand usually smooths out this step.

  But really, I’m just getting started.

  Phase 4 : Collecting A Song Corpus
  Then there is the matter of staging a collection of songs for a given system. It seems like it would just be a matter of finding a large collection of songs for a given format, downloading them in bulk, and mirroring them. Honestly, that’s the easy part. People who are interested in this stuff have been lovingly curating massive collections of these songs for years (see SNESmusic.org for one of the best examples, and they also host a torrent of all their music for really quick and easy hoarding).
  
  In my drive to make this game music website more useful for normal people, the goal is to extract as much metadata as possible to make searching better, and to package the data so that it’s as convenient as possible for users. Whenever I seek to add a new format to the collection, this is the phase where I invariably find that I have to fundamentally modify some of the assumptions I originally made in the player.

  First, there were the NES Sound Format (NSF) files, the original format I wanted to play. These are files that have any number of songs packed into a single file. Playback libraries expose APIs to jump to individual tracks. So the player was designed around that. Game Boy GBS files also fall into this category but present a different challenge vis-à-vis metadata, addressed in the next phase.

  Then, there were the SPC files. Each SPC file is its own song and multiple SPC files are commonly bundled as RAR files. Not wanting to deal with RAR, or any format where I interacted with a general compression API to pull a few files out, I created a custom resource format (inspired by so many I have studied and documented) and compressed it with a simpler compression API. I also had to modify some of the player’s assumptions to deal with this archive format. Genesis VGMs, bundled either in .zip or .7z, followed the same model as SPC in RAR.

  Then it was suggested that I attempt to bring SaltyGME closer to feature parity with Chipamp, rather than just being a Chrome browser frontend for Game Music Emu. When I studied the Portable Sound Format (PSF), I realized it didn’t fit into the player model I already had. PSF uses a sort of shared library model for code execution and I developed another resource archive format to cope with it. So that covers quite a few formats.

  One more architecture challenge arose when I started to study one of the prevailing metadata formats, explained in the next phase.

  Phase 5 : Metadata
  Finally, for the collections to really be useful, I need to harvest that juicy metadata for search and presentation.

  I have created a series of programs and scripts to scrape metadata out of these music files and store it all in a database that drives the website and search engine. I recognize that it’s no good to have a large corpus of songs with minimal metadata and while importing bulk quantities of music, the scripts harshly reject songs that have too little metadata.

  Again, challenges abound. One of the biggest challenges I’m facing is the peculiar quasi-freeform metadata format that emerged as .m3u that takes a form similar to :

  #################################################################
  #
  # GRADIUS2
  # (c) KONAMI  by Furukawa Motoaki, IKACHAN
  #
  #################################################################
  nemesis2.kss::KSS,62,[Nemesis2] (Opening),2:23,,0
  
  nemesis2.kss::KSS,61,[Nemesis2] (Start),7,,0
  
  nemesis2.kss::KSS,43,[Nemesis2] (Air Battle),34,0-
  
  nemesis2.kss::KSS,44,[Nemesis2] (1st. BGM),51,0-
  
  [...]
  

  A lot of file formats (including Game Boy GBS mentioned earlier) store their metadata separately using this format. I have some ideas about tools I can use to help me process this data but I’m pretty sure each one will require some manual intervention.

  As alluded to in phase 4, .m3u presents another architectural challenge : Notice the second field in the CSV .m3u data. That’s a track number. A player can’t expect every track in a bundled chiptune file to be valid, nor to be in any particular order. Thus, I needed to alter the architecture once more to take this into account. However, instead of modifying the SaltyGME player, I simply extended the metadata database to include a playback order which, by default, is the same as the track order but can also accommodate this new issue. This also has the bonus of providing a facility to exclude playback of certain tracks. This comes in handy for many PSF archives which tend to include files that only provide support for other files and aren’t meant to be played on their own.

  Bright Side
  The reward for all of this effort is that the data lands in a proper database in the end. None of it goes back into the chiptune files themselves. This makes further modification easier as all of the data that is indexed and presented on the site comes from the database. Somewhere down the road, I should probably create an API for accessing this metadata.

• Reading in pydub AudioSegment from url. BytesIO returning "OSError [Errno 2] No such file or directory" on heroku only ; fine on localhost

  24 octobre 2014, par Mark

  EDIT 1 for anyone with the same error : installing ffmpeg did indeed solve that BytesIO error

  EDIT 1 for anyone still willing to help : my problem is now that when I AudioSegment.export("filename.mp3", format="mp3"), the file is made, but has size 0 bytes — details below (as "EDIT 1")

  ---

  EDIT 2 : All problems now solved.

  • Files can be read in as AudioSegment using BytesIO
  • I found buildpacks to ensure ffmpeg was installed correctly on my app, with lame support for exporting proper mp3 files

  Answer below

  ---

  Original question

  I have pydub working nicely locally to crop a particular mp3 file based on parameters in the url.
  (?start_time=3.8&end_time=5.1)

  When I run foreman start it all looks good on localhost. The html renders nicely.
  The key lines from the views.py include reading in a file from a url using

  url = "https://s3.amazonaws.com/shareducate02/The_giving_tree__by_Alex_Blumberg__sponsored_by_mailchimp-short.mp3"
  
  mp3 = urllib.urlopen(url).read() # inspired by http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
  
  original=AudioSegment.from_mp3(BytesIO(mp3))  # AudioSegment.from_mp3 is a pydub command, see http://pydub.com
  
  section = original[start_time_ms:end_time_ms]

  That all works great... until I push to heroku (django app) and run it online.
  then when I load the same page now on the herokuapp.com, I get this error

  OSError at /path/to/page
  
  [Errno 2] No such file or directory
  
  Request Method: GET
  
  Request URL:    http://my.website.com/path/to/page?start_time=3.8&end_time=5
  
  Django Version: 1.6.5
  
  Exception Type: OSError
  
  Exception Value:    
  
  [Errno 2] No such file or directory
  
  Exception Location: /app/.heroku/python/lib/python2.7/subprocess.py in _execute_child, line 1327
  
  Python Executable:  /app/.heroku/python/bin/python
  
  Python Version: 2.7.8
  
  Python Path:    
  
  ['/app',
  
   '/app/.heroku/python/bin',
  
   '/app/.heroku/python/lib/python2.7/site-packages/setuptools-5.4.1-py2.7.egg',
  
   '/app/.heroku/python/lib/python2.7/site-packages/distribute-0.6.36-py2.7.egg',
  
   '/app/.heroku/python/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg',
  
   '/app',
  
   '/app/.heroku/python/lib/python27.zip',
  
   '/app/.heroku/python/lib/python2.7',
  
   '/app/.heroku/python/lib/python2.7/plat-linux2',
  
   '/app/.heroku/python/lib/python2.7/lib-tk',
  
   '/app/.heroku/python/lib/python2.7/lib-old',
  
   '/app/.heroku/python/lib/python2.7/lib-dynload',
  
   '/app/.heroku/python/lib/python2.7/site-packages',
  
   '/app/.heroku/python/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info']
  
  
  
  
  
  Traceback:
  
  File "/app/.heroku/python/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
  
    112.                     response = wrapped_callback(request, *callback_args, **callback_kwargs)
  
  File "/app/evernote/views.py" in finalize
  
    105.       original=AudioSegment.from_mp3(BytesIO(mp3))
  
  File "/app/.heroku/python/lib/python2.7/site-packages/pydub/audio_segment.py" in from_mp3
  
    318.         return cls.from_file(file, 'mp3')
  
  File "/app/.heroku/python/lib/python2.7/site-packages/pydub/audio_segment.py" in from_file
  
    302.         retcode = subprocess.call(convertion_command, stderr=open(os.devnull))
  
  File "/app/.heroku/python/lib/python2.7/subprocess.py" in call
  
    522.     return Popen(*popenargs, **kwargs).wait()
  
  File "/app/.heroku/python/lib/python2.7/subprocess.py" in __init__
  
    710.                                 errread, errwrite)
  
  File "/app/.heroku/python/lib/python2.7/subprocess.py" in _execute_child
  
    1327.                 raise child_exception

  I have commented out some of the original to convince myself that sure enough the single line original=AudioSegment.from_mp3(BytesIO(mp3)) is where the problem kicks in... but this is not a problem locally

  The full function in views.py starts like this :

  from django.shortcuts import render, get_object_or_404 
  
  from django.http import HttpResponseRedirect #, Http404, HttpResponse
  
  from django.core.urlresolvers import reverse
  
  from django.views import generic
  
  import pydub
  
  # Maybe only need: 
  
  from pydub import AudioSegment # == see below
  
  from time import gmtime, strftime
  
  
  
  import boto
  
  from boto.s3.connection import S3Connection
  
  from boto.s3.key import Key
  
  
  
  # http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
  
  import urllib
  
  from io import BytesIO
  
  # import numpy as np
  
  # import scipy.signal as sg
  
  # import pydub # mentioned above already
  
  # import matplotlib.pyplot as plt
  
  # from IPython.display import Audio, display
  
  # import matplotlib as mpl
  
  # %matplotlib inline
  
  
  
  import os
  
  # from settings import AWS_ACCESS_KEY, AWS_SECRET_KEY, AWS_BUCKET_NAME
  
  AWS_ACCESS_KEY = os.environ.get('AWS_ACCESS_KEY') # there must be a better way?
  
  AWS_SECRET_KEY = os.environ.get('AWS_SECRET_KEY')
  
  AWS_BUCKET_NAME = os.environ.get('S3_BUCKET_NAME')
  
  
  
  # http://stackoverflow.com/questions/415511/how-to-get-current-time-in-python
  
  
  
  boto_conn = S3Connection(AWS_ACCESS_KEY, AWS_SECRET_KEY)
  
  bucket = boto_conn.get_bucket(AWS_BUCKET_NAME)
  
  s3_url_format = 'https://s3.amazonaws.com/shareducate02/{end_path}'

  and specifically the view in views.py that’s called when I visit the page :

  def finalize(request):
  
  
  
      start_time = request.GET.get('start_time')
  
  
  
      end_time = request.GET.get('end_time')
  
  
  
      original_file = "https://s3.amazonaws.com/shareducate02/The_giving_tree__by_Alex_Blumberg__sponsored_by_mailchimp-short.mp3"
  
  
  
  
  
      if start_time:
  
  
  
        # original=AudioSegment.from_mp3(original_file)  #...that didn't work 
  
        # but this works below:
  
  
  
        # next three uncommented lines from http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
  
        # python 2.x
  
        url = original_file
  
        # req = urllib.Request(url, headers={'User-Agent': ''}) # Note: I commented out this because I got error that "Request" did not exist
  
        mp3 = urllib.urlopen(url).read()
  
        # That's for my 2.7
  
  
  
        # If I ever upgrade to python 3.x, would need to change it to:
  
        # req = urllib.request.Request(url, headers={'User-Agent': ''}) 
  
        # mp3 = urllib.request.urlopen(req).read()
  
        # as per instructions on http://nbviewer.ipython.org/github/ipython-books/cookbook-code/blob/master/notebooks/chapter11_image/06_speech.ipynb
  
  
  
        original=AudioSegment.from_mp3(BytesIO(mp3))
  
        # original=AudioSegment.from_mp3("static/givingtree.mp3") # alternative that works locally (on laptop) but no use for heroku
  
  
  
        start_time_ms = int(float(start_time) * 1000)
  
        if end_time:
  
          end_time_ms = int(float(end_time) * 1000)
  
        else:
  
          end_time_ms = int(float(original.duration_seconds) * 1000)
  
        duration_ms = end_time_ms - start_time_ms
  
        # duration = end_time - start_time
  
        duration = duration_ms/1000
  
  
  
     #   section = original[start_time_ms:end_time_ms]
  
     #   section_with_fading = section.fade_in(100).fade_out(100)
  
  
  
        clip = "demo-"
  
        number = strftime("%Y-%m-%d_%H-%M-%S", gmtime())
  
        clip += number
  
        clip += ".mp3" 
  
  
  
        # DON'T BOTHER writing locally:
  
        # clip_with_path = "evernote/static/"+clip
  
        # section_with_fading.export(clip_with_path, format = "mp3")
  
  
  
     #   tempclip = section_with_fading.export(format = "mp3")
  
  
  
        # commented out while de-bugging, but was working earlier if run on localhost
  
        # c = boto.connect_s3()
  
        # b = c.get_bucket(S3_BUCKET_NAME)  # as defined above
  
        # k = Key(b)
  
        # k.key=clip
  
        # # k.set_contents_from_filename(clip_with_path)
  
        # k.set_contents_from_file(tempclip)
  
        # k.set_acl('public-read')
  
        clip_made = True
  
      else: 
  
        duration = 0.0
  
        clip_made = False
  
        clip = ""
  
      context = {'original_file':original_file, 'new_file':clip, 'start_time': start_time, 'end_time':end_time, 'duration':duration, 'clip_made':clip_made} 
  
      return render(request, 'finalize.html' , context) 

  Any suggestions ?

  Potentially related :
  I have ffmpeg installed locally

  But have been unable to install it onto heroku, due to not understanding buildpacks. I tried just a moment ago (http://stackoverflow.com/questions/14407388/how-to-install-ffmpeg-for-a-django-app-on-heroku and https://github.com/shunjikonishi/heroku-buildpack-ffmpeg) but so far ffmpeg is not working on heroku (ffmpeg is not recognised when I do "heroku run ffmpeg —version")
  ...do you think this is the reason ?

  An answer like any of these would be much appreciated as I’m going round in circles here :

  1. "I think ffmpeg is indeed your problem. Try harder to sort that out, to get it installed on heroku"
  2. "Actually, I think this is why BytesIO is not working for you : ..."
  3. "Your approach is terrible anyway... if you want to read in an audio file to process using pydub, you should just do this instead : ..." (since I’m just hacking my way through pydub for my first time... my approach may be poor)

  ---

  EDIT 1

  ffmpeg is now installed (e.g., I can output wav files)

  However, I can’t create mp3 files, still... or more correctly, I can, but the filesize is zero

  (venv-app)moriartymacbookair13:getstartapp macuser$ heroku config:add BUILDPACK_URL=https://github.com/ddollar/heroku-buildpack-multi.git 
  
  Setting config vars and restarting awe01... done, v93
  
  BUILDPACK_URL: https://github.com/ddollar/heroku-buildpack-multi.git
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ vim .buildpacks 
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ cat .buildpacks 
  
  https://github.com/shunjikonishi/heroku-buildpack-ffmpeg.git
  
  https://github.com/heroku/heroku-buildpack-python.git
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ git add --all
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ git commit -m "need multi, not just ffmpeg, so adding back in multi + shun + heroku, with trailing .git in .buildpacks file"
  
  [master cd99fef] need multi, not just ffmpeg, so adding back in multi + shun + heroku, with trailing .git in .buildpacks file
  
   1 file changed, 2 insertions(+), 2 deletions(-)
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ git push heroku master
  
  Fetching repository, done.
  
  Counting objects: 5, done.
  
  Delta compression using up to 4 threads.
  
  Compressing objects: 100% (3/3), done.
  
  Writing objects: 100% (3/3), 372 bytes | 0 bytes/s, done.
  
  Total 3 (delta 2), reused 0 (delta 0)
  
  
  
  -----> Fetching custom git buildpack... done
  
  -----> Multipack app detected
  
  =====> Downloading Buildpack: https://github.com/shunjikonishi/heroku-buildpack-ffmpeg.git
  
  =====> Detected Framework: ffmpeg
  
  -----> Install ffmpeg
  
         DOWNLOAD_URL =  http://flect.github.io/heroku-binaries/libs/ffmpeg.tar.gz
  
         exporting PATH and LIBRARY_PATH
  
  =====> Downloading Buildpack: https://github.com/heroku/heroku-buildpack-python.git
  
  =====> Detected Framework: Python
  
  -----> Installing dependencies with pip
  
         Cleaning up...
  
  
  
  -----> Preparing static assets
  
         Collectstatic configuration error. To debug, run:
  
         $ heroku run python ./example/manage.py collectstatic --noinput
  
  
  
  Using release configuration from last framework (Python).
  
  -----> Discovering process types
  
         Procfile declares types -> web
  
  
  
  -----> Compressing... done, 198.1MB
  
  -----> Launching... done, v94
  
         http://[redacted].herokuapp.com/ deployed to Heroku
  
  
  
  To git@heroku.com:awe01.git
  
     78d6b68..cd99fef  master -> master
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ heroku run ffmpeg
  
  Running `ffmpeg` attached to terminal... up, run.6408
  
  ffmpeg version git-2013-06-02-5711e4f Copyright (c) 2000-2013 the FFmpeg developers
  
    built on Jun  2 2013 07:38:40 with gcc 4.4.3 (Ubuntu 4.4.3-4ubuntu5.1)
  
    configuration: --enable-shared --disable-asm --prefix=/app/vendor/ffmpeg
  
    libavutil      52. 34.100 / 52. 34.100
  
    libavcodec     55. 13.100 / 55. 13.100
  
    libavformat    55.  8.102 / 55.  8.102
  
    libavdevice    55.  2.100 / 55.  2.100
  
    libavfilter     3. 74.101 /  3. 74.101
  
    libswscale      2.  3.100 /  2.  3.100
  
    libswresample   0. 17.102 /  0. 17.102
  
  Hyper fast Audio and Video encoder
  
  usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...
  
  
  
  Use -h to get full help or, even better, run 'man ffmpeg'
  
  (venv-app)moriartymacbookair13:getstartapp macuser$ heroku run bash
  
  Running `bash` attached to terminal... up, run.9660
  
  ~ $ python
  
  Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
  
  [GCC 4.4.3] on linux2
  
  Type "help", "copyright", "credits" or "license" for more information.
  
  >>> import pydub
  
  >>> from pydub import AudioSegment
  
  >>> exit()
  
  ~ $ which ffmpeg
  
  /app/vendor/ffmpeg/bin/ffmpeg
  
  ~ $ python 
  
  
  
  Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
  
  [GCC 4.4.3] on linux2
  
  Type "help", "copyright", "credits" or "license" for more information.
  
  >>> import pydub
  
  >>> from pydub import AudioSegment
  
  >>> AudioSegment.silent(5000).export("/tmp/asdf.mp3", "mp3")
  
  <open file="file"></open>tmp/asdf.mp3', mode 'wb+' at 0x7f9a37d44780>
  
  >>> exit ()
  
  ~ $ cd /tmp/
  
  /tmp $ ls
  
  asdf.mp3
  
  /tmp $ open asdf.mp3
  
  bash: open: command not found
  
  /tmp $ ls -lah
  
  total 8.0K
  
  drwx------  2 u36483 36483 4.0K 2014-10-22 04:14 .
  
  drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3

  Note the file size of 0 above for the mp3 file... when I do the same thing on my macbook, the file size is never zero

  Back to the heroku shell :

  /tmp $ python
  
  Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
  
  [GCC 4.4.3] on linux2
  
  Type "help", "copyright", "credits" or "license" for more information.
  
  >>> import pydub
  
  >>> from pydub import AudioSegment
  
  >>> pydub.AudioSegment.ffmpeg = "/app/vendor/ffmpeg/bin/ffmpeg" 
  
  >>> AudioSegment.silence(1200).export("/tmp/herokuSilence.mp3", format="mp3")
  
  Traceback (most recent call last):
  
    File "<stdin>", line 1, in <module>
  
  AttributeError: type object 'AudioSegment' has no attribute 'silence'
  
  >>> AudioSegment.silent(1200).export("/tmp/herokuSilence.mp3", format="mp3")
  
  <open file="file"></open>tmp/herokuSilence.mp3', mode 'wb+' at 0x7fcc2017c780>
  
  >>> exit()
  
  /tmp $ ls
  
  asdf.mp3  herokuSilence.mp3
  
  /tmp $ ls -lah
  
  total 8.0K
  
  drwx------  2 u36483 36483 4.0K 2014-10-22 04:29 .
  
  drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:29 herokuSilence.mp3
  
  </module></stdin>

  I realised the first time that I had forgotten the pydub.AudioSegment.ffmpeg = "/app/vendor/ffmpeg/bin/ffmpeg" command, but as you can see above, the file is still zero size

  Out of desperation, I even tried adding the ".heroku" into the path to be as verbatim as your example, but that didn’t fix it :

  /tmp $ python
  
  Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
  
  [GCC 4.4.3] on linux2
  
  Type "help", "copyright", "credits" or "license" for more information.
  
  >>> import pydub
  
  >>> from pydub import AudioSegment
  
  >>> pydub.AudioSegment.ffmpeg = "/app/.heroku/vendor/ffmpeg/bin/ffmpeg"
  
  >>> AudioSegment.silent(1200).export("/tmp/herokuSilence03.mp3", format="mp3")
  
  <open file="file"></open>tmp/herokuSilence03.mp3', mode 'wb+' at 0x7fc92aca7780>
  
  >>> exit()
  
  /tmp $ ls -lah
  
  total 8.0K
  
  drwx------  2 u36483 36483 4.0K 2014-10-22 04:31 .
  
  drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:31 herokuSilence03.mp3
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:29 herokuSilence.mp3

  Finally, I tried exporting a .wav file to check pydub was at least working correctly

  /tmp $ python
  
  Python 2.7.8 (default, Jul  9 2014, 20:47:08) 
  
  [GCC 4.4.3] on linux2
  
  Type "help", "copyright", "credits" or "license" for more information.
  
  >>> import pydub
  
  >>> from pydub import AudioSegment
  
  >>> pydub.AudioSegment.ffmpeg = "/app/vendor/ffmpeg/bin/ffmpeg"
  
  >>> AudioSegment.silent(1300).export("/tmp/heroku_wav_silence01.wav", format="wav")
  
  <open file="file"></open>tmp/heroku_wav_silence01.wav', mode 'wb+' at 0x7fa33cbf3780>
  
  >>> exit()
  
  /tmp $ ls
  
  asdf.mp3  herokuSilence03.mp3  herokuSilence.mp3  heroku_wav_silence01.wav
  
  /tmp $ ls -lah
  
  total 40K
  
  drwx------  2 u36483 36483 4.0K 2014-10-22 04:42 .
  
  drwxr-xr-x 14 root   root  4.0K 2014-09-26 07:08 ..
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:14 asdf.mp3
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:31 herokuSilence03.mp3
  
  -rw-------  1 u36483 36483    0 2014-10-22 04:29 herokuSilence.mp3
  
  -rw-------  1 u36483 36483  29K 2014-10-22 04:42 heroku_wav_silence01.wav
  
  /tmp $ 

  At least that filesize for .wav is non-zero, so pydub is working

  My current theory is that either I’m still not using ffmpeg correctly, or it’s insufficient... maybe I need an mp3 additional install on top of basic ffmpeg.

  Several sites mention "libavcodec-extra-53" but I’m not sure how to install that on heroku, or to check if I have it ? https://github.com/jiaaro/pydub/issues/36
  Similarly tutorials on libmp3lame seem to be geared towards laptop installation rather than installation on heroku, so I’m at a loss http://superuser.com/questions/196857/how-to-install-libmp3lame-for-ffmpeg

  In case relevant, I also have youtube-dl in my requirements.txt... this also works locally on my macbook, but fails when I run it in the heroku shell :

  ~/ytdl $ youtube-dl --restrict-filenames -x --audio-format mp3 n2anDgdUHic
  
  [youtube] Setting language
  
  [youtube] Confirming age
  
  [youtube] n2anDgdUHic: Downloading webpage
  
  [youtube] n2anDgdUHic: Downloading video info webpage
  
  [youtube] n2anDgdUHic: Extracting video information
  
  [download] Destination: Boyce_Avenue_feat._Megan_Nicole_-_Skyscraper_Patrick_Ebert_Edit-n2anDgdUHic.m4a
  
  [download] 100% of 5.92MiB in 00:00
  
  [ffmpeg] Destination: Boyce_Avenue_feat._Megan_Nicole_-_Skyscraper_Patrick_Ebert_Edit-n2anDgdUHic.mp3
  
  ERROR: audio conversion failed: Unknown encoder 'libmp3lame'
  
  ~/ytdl $ 

  The informative link is that it too specificies an mp3 failure, so perhaps they two issues are related.

  ---

  EDIT 2

  See answer, all problems solved

• ffmpeg Transcoding Stops After Few Seconds [migrated]

  15 avril 2018, par Salem F

  I’m trying to do this over week now with no success,
  What’s I’m trying to do is transcoding video from live streaming source and downscale it with FFmpeg, but every time I start transcoding it broadcasting for 11 sec and stop.

  The last command I tried :

  ffmpeg  -re  -i 'http://source.com/1034.ts' -preset ultrafast http://localhost:2052/feed1.ffm

  I tried to download the .ts file with IDM and it finish downloading the file on the exact 12 Sec that FFmpeg stop trans coding on it.

  Does that means that FFmpeg download that file as one segment and not continued reading the source video As what video players does usually. By the way, I tried with source with VLC player and it didn’t stop playing the the same source video.

  I decided to pass FFmpeg command via FFserver config file ffserver.conf

  Launch ffmpeg -i 'http://source.com/1.ts' -copyinkf -codec copy   

  The stream works fine for a while but after testing couple sources I notice it’s struggle to trans-coding HD videos.

  I guess the issue with my VPS KVM server being very limited CPU and RAM ( 128MB only) ! Since I tried using ultrafast preset but din’t solve the issue, another thing, I notice when I enable AVOptionVideo crf setting on ffserver.conf trans-coding runs bit smoothly without frame-rate dropping.
  Las my server uses Xeon L5520 CPU which is outdated CPU specially I gout 1/4 power of V single core (if they count HT it will be 1/8 of the real core) : (

  # vlc -I dummy 'https://source.com/1034.ts' --sout '#standard{access=http,mux=flv,dst=localhost:2052}'
  
  VLC media player 2.2.8 Weatherwax (revision 2.2.7-14-g3cc1d8cba9)
  
  [09d3fdf0] pulse audio output error: PulseAudio server connection failure: Connection refused
  
  [09d279c0] core interface error: no suitable interface module
  
  [09c9b8f8] core libvlc error: interface "globalhotkeys,none" initialization failed
  
  [09d279c0] dbus interface error: Failed to connect to the D-Bus session daemon: Unable to autolaunch a dbus-daemon without a $DISPLAY for X11
  
  [09d279c0] core interface error: no suitable interface module
  
  [09c9b8f8] core libvlc error: interface "dbus,none" initialization failed
  
  [09d279c0] dummy interface: using the dummy interface module...
  
  [b5e04ae0] access_output_http access out: Consider passing --http-host=IP on the command line instead.
  
  [b5e38ab8] ts demux: MPEG-4 descriptor not found for pid 0x101 type 0xf
  
  [b5e90ae0] packetizer_mpeg4audio decoder: AAC channels: 2 samplerate: 48000
  
  [flv @ 0xb5e33b40] dimensions not set
  
  [b5e06360] avformat mux error: could not write header: Invalid argument
  
  [b5e88ef0] core decoder error: cannot continue streaming due to errors
  
  [b5e90ae0] core decoder error: cannot continue streaming due to errors

  Here output with -loglevel verbose

  :~# ffmpeg -i http://source.com/1.ts -copyinkf -codec copy  -loglevel verbose  http://127.0.0.1:8090/feed1.ffm
  
  ffmpeg version 2.6.9 Copyright (c) 2000-2016 the FFmpeg developers
  
    built with gcc 4.9.2 (Debian 4.9.2-10)
  
    configuration: --prefix=/usr --extra-cflags='-g -O2 -fstack-protector-strong -Wformat -Werror=format-security ' --extra-ldflags='-Wl,-z,relro' --cc='ccache cc' --enable-shared --enable-libmp3lame --enable-gpl --enable-nonfree --enable-libvorbis --enable-pthreads --enable-libfaac --enable-libxvid --enable-postproc --enable-x11grab --enable-libgsm --enable-libtheora --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libx264 --enable-libspeex --enable-nonfree --disable-stripping --enable-libvpx --enable-libschroedinger --disable-encoder=libschroedinger --enable-version3 --enable-libopenjpeg --enable-librtmp --enable-avfilter --enable-libfreetype --enable-libvo-aacenc --disable-decoder=amrnb --enable-libvo-amrwbenc --enable-libaacplus --libdir=/usr/lib/i386-linux-gnu --disable-vda --enable-libbluray --enable-libcdio --enable-gnutls --enable-frei0r --enable-openssl --enable-libass --enable-libopus --enable-fontconfig --enable-libpulse --disable-mips32r2 --disable-mipsdspr1 --disable-mipsdspr2 --enable-libvidstab --enable-libzvbi --enable-avresample --disable-htmlpages --disable-podpages --enable-libutvideo --enable-libfdk-aac --enable-libx265 --enable-libiec61883 --enable-vaapi --enable-libdc1394 --disable-altivec --shlibdir=/usr/lib/i386-linux-gnu
  
    libavutil      54. 20.100 / 54. 20.100
  
    libavcodec     56. 26.100 / 56. 26.100
  
    libavformat    56. 25.101 / 56. 25.101
  
    libavdevice    56.  4.100 / 56.  4.100
  
    libavfilter     5. 11.102 /  5. 11.102
  
    libavresample   2.  1.  0 /  2.  1.  0
  
    libswscale      3.  1.101 /  3.  1.101
  
    libswresample   1.  1.100 /  1.  1.100
  
    libpostproc    53.  3.100 / 53.  3.100
  
  Invalid UE golomb code
  
      Last message repeated 2 times
  
  Input #0, mpegts, from 'http://source.com/1.ts':
  
    Duration: N/A, start: 30472.768167, bitrate: N/A
  
    Program 1
  
      Metadata:
  
        service_name    : Service01
  
        service_provider: FFmpeg
  
      Stream #0:0[0x100]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 960x540 (960x544) [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
  
      Stream #0:1[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 105 kb/s
  
  [graph 0 input from stream 0:1 @ 0x971f2c0] tb:1/48000 samplefmt:fltp samplerate:48000 chlayout:0x3
  
  [audio format for output stream 0:0 @ 0x9844de0] auto-inserting filter 'auto-inserted resampler 0' between the filter 'Parsed_anull_0' and the filter 'audio format for output stream 0:0'
  
  [auto-inserted resampler 0 @ 0x97115e0] ch:2 chl:stereo fmt:fltp r:48000Hz -> ch:1 chl:mono fmt:fltp r:22050Hz
  
  [graph 1 input from stream 0:0 @ 0x96f5d00] w:960 h:540 pixfmt:yuv420p tb:1/90000 fr:50/1 sar:1/1 sws_param:flags=2
  
  [scaler for output stream 0:1 @ 0x96f5e80] w:352 h:240 flags:'0x4' interl:0
  
  [scaler for output stream 0:1 @ 0x96f5e80] w:960 h:540 fmt:yuv420p sar:1/1 -> w:352 h:240 fmt:yuv420p sar:40/33 flags:0x4
  
  Output #0, ffm, to 'http://127.0.0.1:8090/feed1.ffm':
  
    Metadata:
  
      creation_time   : now
  
      encoder         : Lavf56.25.101
  
      Stream #0:0: Audio: wmav2, 22050 Hz, mono, fltp, 64 kb/s
  
      Metadata:
  
        encoder         : Lavc56.26.100 wmav2
  
      Stream #0:1: Video: msmpeg4v3 (msmpeg4), yuv420p, 352x240 [SAR 40:33 DAR 16:9], q=2-31, 256 kb/s, 50 fps, 1000k tbn, 15 tbc
  
      Metadata:
  
        encoder         : Lavc56.26.100 msmpeg4
  
  Stream mapping:
  
    Stream #0:1 -> #0:0 (aac (native) -> wmav2 (native))
  
    Stream #0:0 -> #0:1 (h264 (native) -> msmpeg4v3 (msmpeg4))
  
  Press [q] to stop, [?] for help
  
  Invalid UE golomb code
  
  *** dropping frame 3 from stream 1 at ts 1
  
      Last message repeated 1 times
  
  [msmpeg4 @ 0x970f060] warning, clipping 1 dct coefficients to -127..127
  
  *** dropping frame 4 from stream 1 at ts 2
  
      Last message repeated 1 times
  
  *** dropping frame 5 from stream 1 at ts 3
  
      Last message repeated 1 times
  
  *** dropping frame 5 from stream 1 at ts 4
  
  *** dropping frame 6 from stream 1 at ts 4
  
      Last message repeated 1 times
  
  *** dropping frame 7 from stream 1 at ts 5
  
      Last message repeated 1 times
  
  [msmpeg4 @ 0x970f060] warning, clipping 1 dct coefficients to -127..127
  
  *** dropping frame 8 from stream 1 at ts 6
  
      Last message repeated 1 times
  
  *** dropping frame 8 from stream 1 at ts 7
  
  *** dropping frame 9 from stream 1 at ts 7
  
      Last message repeated 1 times
  
  *** dropping frame 10 from stream 1 at ts 8
  
      Last message repeated 1 times
  
  *** dropping frame 11 from stream 1 at ts 9
  
      Last message repeated 1 times
  
  *** dropping frame 11 from stream 1 at ts 10
  
  *** dropping frame 12 from stream 1 at ts 10
  
      Last message repeated 1 times
  
  *** dropping frame 13 from stream 1 at ts 11
  
      Last message repeated 1 times
  
  *** dropping frame 14 from stream 1 at ts 12
  
      Last message repeated 1 times
  
  *** dropping frame 14 from stream 1 at ts 13
  
  *** dropping frame 15 from stream 1 at ts 13
  
      Last message repeated 1 times
  
  *** dropping frame 16 from stream 1 at ts 14
  
      Last message repeated 1 times
  
  *** dropping frame 17 from stream 1 at ts 15
  
      Last message repeated 1 times
  
  *** dropping frame 17 from stream 1 at ts 16
  
  *** dropping frame 18 from stream 1 at ts 16
  
      Last message repeated 1 times
  
  *** dropping frame 19 from stream 1 at ts 17
  
      Last message repeated 1 times
  
  *** dropping frame 20 from stream 1 at ts 18me=00:00:01.33 bitrate= 270.3kbits/s dup=0 drop=39
  
      Last message repeated 1 times
  
  *** dropping frame 20 from stream 1 at ts 19
  
  *** dropping frame 21 from stream 1 at ts 19
  
      Last message repeated 1 times
  
  *** dropping frame 22 from stream 1 at ts 20
  
      Last message repeated 1 times
  
  *** dropping frame 23 from stream 1 at ts 21
  
      Last message repeated 1 times
  
  *** dropping frame 23 from stream 1 at ts 22
  
  *** dropping frame 24 from stream 1 at ts 22
  
      Last message repeated 1 times
  
  *** dropping frame 25 from stream 1 at ts 23
  
      Last message repeated 1 times
  
  *** dropping frame 26 from stream 1 at ts 24
  
      Last message repeated 1 times
  
  *** dropping frame 26 from stream 1 at ts 25
  
  *** dropping frame 27 from stream 1 at ts 25
  
      Last message repeated 1 times
  
  *** dropping frame 28 from stream 1 at ts 26
  
      Last message repeated 1 times
  
  *** dropping frame 29 from stream 1 at ts 27
  
      Last message repeated 1 times
  
  *** dropping frame 29 from stream 1 at ts 28
  
  *** dropping frame 30 from stream 1 at ts 28
  
      Last message repeated 1 times
  
  *** dropping frame 31 from stream 1 at ts 29
  
      Last message repeated 1 times
  
  *** dropping frame 32 from stream 1 at ts 30
  
      Last message repeated 1 times
  
  *** dropping frame 32 from stream 1 at ts 31
  
  *** dropping frame 33 from stream 1 at ts 31
  
      Last message repeated 1 times
  
  *** dropping frame 34 from stream 1 at ts 32
  
      Last message repeated 1 times
  
  *** dropping frame 34 from stream 1 at ts 33
  
  *** dropping frame 35 from stream 1 at ts 33
  
  *** dropping frame 35 from stream 1 at ts 34
  
  *** dropping frame 36 from stream 1 at ts 34
  
      Last message repeated 1 times
  
  *** dropping frame 37 from stream 1 at ts 35
  
      Last message repeated 1 times
  
  Invalid UE golomb code
  
  *** dropping frame 38 from stream 1 at ts 36
  
      Last message repeated 1 times
  
  *** dropping frame 38 from stream 1 at ts 37
  
  *** dropping frame 39 from stream 1 at ts 37
  
      Last message repeated 1 times
  
  *** dropping frame 40 from stream 1 at ts 38
  
      Last message repeated 1 times
  
  *** dropping frame 41 from stream 1 at ts 39me=00:00:02.73 bitrate= 311.7kbits/s dup=0 drop=88
  
      Last message repeated 1 times
  
  *** dropping frame 41 from stream 1 at ts 40
  
  *** dropping frame 42 from stream 1 at ts 40
  
      Last message repeated 1 times
  
  *** dropping frame 43 from stream 1 at ts 41
  
      Last message repeated 1 times
  
  *** dropping frame 44 from stream 1 at ts 42
  
      Last message repeated 1 times
  
  *** dropping frame 44 from stream 1 at ts 43
  
  *** dropping frame 45 from stream 1 at ts 43
  
      Last message repeated 1 times
  
  *** dropping frame 46 from stream 1 at ts 44
  
      Last message repeated 1 times
  
  *** dropping frame 47 from stream 1 at ts 45
  
      Last message repeated 1 times
  
  *** dropping frame 47 from stream 1 at ts 46
  
  *** dropping frame 48 from stream 1 at ts 46
  
      Last message repeated 1 times
  
  *** dropping frame 49 from stream 1 at ts 47
  
      Last message repeated 1 times
  
  *** dropping frame 50 from stream 1 at ts 48
  
      Last message repeated 1 times
  
  *** dropping frame 50 from stream 1 at ts 49
  
  *** dropping frame 51 from stream 1 at ts 49
  
      Last message repeated 1 times
  
  *** dropping frame 52 from stream 1 at ts 50
  
      Last message repeated 1 times
  
  *** dropping frame 53 from stream 1 at ts 51
  
      Last message repeated 1 times
  
  [h264 @ 0x9844a00] error while decoding MB 58 12, bytestream -5
  
  [h264 @ 0x9844a00] concealing 1311 DC, 1311 AC, 1311 MV errors in B frame
  
  *** dropping frame 53 from stream 1 at ts 52
  
  No more output streams to write to, finishing.
  
  frame=   55 fps= 42 q=4.3 Lsize=     152kB time=00:00:03.66 bitrate= 339.6kbits/s dup=0 drop=119
  
  video:116kB audio:26kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 6.760316%
  
  Input file #0 (http://source.com/1.ts):
  
    Input stream #0:0 (video): 174 packets read (220322 bytes); 174 frames decoded;
  
    Input stream #0:1 (audio): 156 packets read (36657 bytes); 156 frames decoded (159744 samples);
  
    Total: 330 packets (256979 bytes) demuxed
  
  Output file #0 (http://127.0.0.1:8090/feed1.ffm):
  
    Output stream #0:0 (audio): 72 frames encoded (73383 samples); 72 packets muxed (26712 bytes);
  
    Output stream #0:1 (video): 55 frames encoded; 55 packets muxed (119080 bytes);
  
    Total: 127 packets (145792 bytes) muxed

  Here input URL file info After I download it to my PC with IDM

  General
  
  ID                             : 1 (0x1)
  
  Complete name                  : D:\1.ts
  
  Format                         : MPEG-TS
  
  File size                      : 256 KiB
  
  Duration                       : 2 s 520 ms
  
  Overall bit rate mode          : Variable
  
  Overall bit rate               : 788 kb/s
  
  
  
  Video
  
  ID                             : 256 (0x100)
  
  Menu ID                        : 1 (0x1)
  
  Format                         : AVC
  
  Format/Info                    : Advanced Video Codec
  
  Format profile                 : High@L3.1
  
  Format settings, CABAC         : Yes
  
  Format settings, RefFrames     : 2 frames
  
  Codec ID                       : 27
  
  Duration                       : 2 s 680 ms
  
  Width                          : 960 pixels
  
  Height                         : 540 pixels
  
  Display aspect ratio           : 16:9
  
  Frame rate                     : 50.000 FPS
  
  Color space                    : YUV
  
  Chroma subsampling             : 4:2:0
  
  Bit depth                      : 8 bits
  
  Scan type                      : Progressive
  
  
  
  Audio
  
  ID                             : 257 (0x101)
  
  Menu ID                        : 1 (0x1)
  
  Format                         : AAC
  
  Format/Info                    : Advanced Audio Codec
  
  Format version                 : Version 4
  
  Format profile                 : LC
  
  Muxing mode                    : ADTS
  
  Codec ID                       : 15
  
  Duration                       : 2 s 69 ms
  
  Bit rate mode                  : Variable
  
  Channel(s)                     : 2 channels
  
  Channel positions              : Front: L R
  
  Sampling rate                  : 48.0 kHz
  
  Frame rate                     : 46.875 FPS (1024 SPF)
  
  Compression mode               : Lossy
  
  Delay relative to video        : -12 ms
  
  
  
  Menu
  
  ID                             : 4096 (0x1000)
  
  Menu ID                        : 1 (0x1)
  
  Duration                       : 2 s 520 ms
  
  List                           : 256 (0x100) (AVC) / 257 (0x101) (AAC)
  
  Service name                   : Service01
  
  Service provider               : FFmpeg
  
  Service type                   : digital television