Recherche avancée

Sur d’autres sites (9123)

• Salty Game Music

  31 mai 2011, par Multimedia Mike — General

  Have you heard of Google’s Native Client (NaCl) project ? Probably not. Basically, it allows native code modules to run inside a browser (where ‘browser’ is defined pretty narrowly as ‘Google Chrome’ in this case). Programs are sandboxed so they aren’t a security menace (or so the whitepapers claim) but are allowed to access a variety of APIs including video and audio. The latter API is significant because sound tends to be forgotten in all the hullabaloo surrounding non-Flash web technologies. At any rate, enjoy NaCl while you can because I suspect it won’t be around much longer.

  After my recent work upgrading some old music synthesis programs to user more modern audio APIs, I got the idea to try porting the same code to run under NaCl in Chrome (first Nosefart, then Game Music Emu/GME). In this exercise, I met with very limited success. This blog post documents some of the pitfalls in my excursion.

  
  
  

  Infrastructure
  People who know me know that I’m rather partial — to put it gently — to straight-up C vs. C++. The NaCl SDK is heavily skewed towards C++. However, it does provide a Python tool called init_project.py which can create the skeleton of a project and can do so in C with the '-c' option :

  ./init_project.py -c -n saltynosefart
  

  This generates something that can be built using a simple ‘make’. When I added Nosefart’s C files, I learned that the project Makefile has places for project-necessary CFLAGS but does not honor them. The problem is that the generated Makefile includes a broader system Makefile that overrides the CFLAGS in the project Makefile. Going into the system Makefile and changing "CFLAGS =" -> "CFLAGS +=" solves this problem.

  Still, maybe I’m the first person to attempt building something in Native Client so I’m the first person to notice this ?

  Basic Playback
  At least the process to create an audio-enabled NaCl app is well-documented. Too bad it doesn’t seem to compile as advertised. According to my notes on the matter, I filled in PPP_InitializeModule() with the appropriate boilerplate as outlined in the docs but got a linker error concerning get_browser_interface().

  Plan B : C++
  Obviously, the straight C stuff is very much a second-class citizen in this NaCl setup. Fortunately, there is already that fully functional tone generator example program in the limited samples suite. Plan B is to copy that project and edit it until it accepts Nosefart/GME audio instead of a sine wave.

  The build system assumes all C++ files should have .cc extensions. I have to make some fixes so that it will accept .cpp files (either that, or rename all .cpp to .cc, but that’s not very clean).

  Making Noise
  You’ll be happy to know that I did successfully swap out the tone generator for either Nosefart or GME. Nosefart has a slightly fickle API that requires revving the emulator frame by frame and generating a certain number of audio samples. GME’s API is much easier to work with in this situation — just tell it how many samples it needs to generate and give it a pointer to a buffer. I played NES and SNES music play through this ad-hoc browser plugin, and I’m confident all the other supported formats would have worked if I went through the bother of converting the music data files into C headers to be included in the NaCl executable binaries (dynamically loading data via the network promised to be a far more challenging prospect reserved for phase 3 of the project).

  Portable ?
  I wouldn’t say so. I developed it on Linux and things ran fine there. I tried to run the same binaries on the Windows version of Chrome to no avail. It looks like it wasn’t even loading the .nexe files (NaCl executables).

  Thinking About The (Lack Of A) Future
  As I was working on this project, I noticed that the online NaCl documentation materialized explicit banners warning that my NaCl binaries compiled for Chrome 11 won’t work for Chrome 12 and that I need to code to the newly-released 0.3 SDK version. Not a fuzzy feeling. I also don’t feel good that I’m working from examples using bleeding edge APIs that feature deprecation as part of their naming convention, e.g., pp::deprecated::ScriptableObject().

  Ever-changing API + minimal API documentation + API that only works in one browser brand + requiring end user to explicitly enable feature = … well, that’s why I didn’t bother to release any showcase pertaining to this little experiment. Would have been neat, but I strongly suspect that this is yet another one of those APIs that Google decides to deprecate soon.

  See Also :

  • I eventually got a second wind and made this work using the C interface.

• discord.py music bot slowing down for longer audio queries

  1er janvier 2023, par Bobluge

  So I'm trying to make a music bot with discord.py. Shown below is a minimum working example of the bot with the problematic functions :

  


  import os

import discord
from discord.ext import commands
from discord import player as p

import yt_dlp as youtube_dl

intents = discord.Intents.default()
intents.members = True

bot = commands.Bot(command_prefix=';')

class Music(commands.Cog):
    def __init__(self, bot):
        self.bot = bot
        self.yt-dlp_opts = {
            'format': 'bestaudio/best',
            'outtmpl': '%(extractor)s-%(id)s-%(title)s.%(ext)s',
            'restrictfilenames': True,
            'noplaylist': True,
            'playlistend': 1,
            'nocheckcertificate': True,
            'ignoreerrors': False,
            'logtostderr': False,
            'quiet': True,
            'no_warnings': True,
            'default_search': 'auto',
            'source_address': '0.0.0.0', # bind to ipv4 since ipv6 addresses cause issues sometimes
        }
        self.ffmpeg_opts = {
            'options': '-vn',
            # Source: https://stackoverflow.com/questions/66070749/
            "before_options": "-reconnect 1 -reconnect_streamed 1 -reconnect_delay_max 5",
        }
        self.cur_stream = None
        self.cur_link = None

    @commands.command(aliases=["p"])
    async def play(self, ctx, url):
        yt-dlp = youtube_dl.YoutubeDL(self.ytdl_opts)
        data = yt-dlp.extract_info(url, download=False)
        filename = data['url']  # So far only works with links
        print(filename)
        audio = p.FFmpegPCMAudio(filename, **self.ffmpeg_opts)
        self.cur_stream = audio
        self.cur_link = filename

        # You must be connected to a voice channel first
        await ctx.author.voice.channel.connect()
        ctx.voice_client.play(audio)
        await ctx.send(f"now playing")

    @commands.command(aliases=["ff"])
    async def seek(self, ctx):
        """
        Fast forwards 10 seconds
        """
        ctx.voice_client.pause()
        for _ in range(500):
            self.cur_stream.read()  # 500*20ms of audio = 10000ms = 10s
        ctx.voice_client.resume()

        await ctx.send(f"fast forwarded 10 seconds")

    @commands.command(aliases=["j"])
    async def jump(self, ctx, time):
        """
        Jumps to a time in the song, input in the format of HH:MM:SS
        """
        ctx.voice_client.stop()
        temp_ffempg = {
            'options': '-vn',
            # Keyframe skipping when passed as an input option (fast)
            "before_options": f"-ss {time} -reconnect 1 -reconnect_streamed 1 -reconnect_delay_max 5",
        }
        new_audio = p.FFmpegPCMAudio(self.cur_link, **temp_ffempg)
        self.cur_stream = new_audio
        ctx.voice_client.play(new_audio)
        await ctx.send(f"skipped to {time}")


bot.add_cog(Music(bot))
bot.run(os.environ["BOT_TOKEN"])


  


  My requirements.txt file :

  


  discord.py[voice]==1.7.3
yt-dlp==2021.9.2


  


  To play a song in Discord the following format is used :

  


  ;p 


  


  Where is any link that yt-dlp supports. Under normal circumstances, the ;p command is used with songs that are relatively short, to which seek() and jump() work extremely quickly to do what they are supposed to do. For example if I execute these sequence of commands in Discord :

  


  ;p https://www.youtube.com/watch?v=n8X9_MgEdCg  <- 4 min song


  


  And when the bot starts playing, spam the following :

  


  ;ff
;ff
;ff
;ff
;ff


  


  The bot is able to almost instantly seek five 10-second increments of the song. Additionally, I can jump to the three minute mark very quickly with :

  


  ;j 00:03:00


  


  From some experimentation, the seek() and jump() functions seem to work quickly for songs that are under 10 minutes. If I try the exact same sequence of commands but with a 15 minute song like https://www.youtube.com/watch?v=Ks9Ck5LfGWE or longer https://www.youtube.com/watch?v=VThrx5MRJXA (10 hours classical music), there is an evident slowdown when running the ;ff command. However, when I include a few seconds of delay between firings of the ;ff command, the seeking is just as fast as previously mentioned. I'm not exactly sure what is going on with yt-dlp/FFmpeg behind the scenes when streaming, but I speculate that there is some sort of internal buffer, and songs that pass a certain length threshold are processed differently.

  


  For longer songs, the seek() command takes longer to get to the desired position, which makes sense since this site specifies that -ss used as an input option loops through keyframes (as there must be more keyframes in longer songs). However, if the following commands are run in Discord :

  


  ;p https://www.youtube.com/watch?v=VThrx5MRJXA  <- 10 hour classical music
;j 09:00:00                                     <- jump to 9 hour mark
;j 00:03:00                                     <- jump to 3 minute mark


  


  The first seek command takes around 5 to 10 seconds to perform a successful seek, which isn't bad, but it could be better. The second seek command takes around the same time as the first command, which doesn't make sense to me, because I thought less keyframes were skipped in order to reach the 3 minute mark.

  


  So I'm wondering what's going on, and how to potentially solve the following :

  


  

  • What is actually going on with the seek() command ? My implementation of seek() uses discord.py's discord.player.FFmpegPCMAudio.read() method, which apparently runs slower if the song's length is longer ? Why ?

  


  • Why does input seeking for long YouTube videos take almost the same time no matter where I seek to ?

  


  • How the yt-dlp and FFmpeg commands work behind the scenes to stream a video from YouTube (or any other website that YTDL supports). Does yt-dlp and FFmpeg behave differently for audio streams above a certain length threshold ?

  


  • Potential ways to speed up seek() and jump() for long songs. I recall some well-known discord music bots were able to do this very quickly.

  


  


• ffmpeg - background music to video with sount

  25 août 2016, par Nir Diamant

  Hey im trying to add background music(mp3 file) to video that contain sounds(mpg file).

  im using the next code to do it :

  ffmpeg -i input.mpg -i bgmusic.mp3 -c:v copy  -filter_
  
  complex \"[0:a]aformat = fltp:44100:stereo,apad[0a];[1]aformat=fltp:44100:stereo
  
  ,volume=0.1[1a];[0a][1a]amerge[a]\" -map 0:v -map \"[a]\" -ac 2 -t 14 finaloutpu
  
  t2.mpg

  but i get the next error message :

  [AVFilterGraph @ 000000000050bd80] No such filter: '"'
  
  Error initializing complex filters.
  
  Invalid argument

  here is the all console log :

  ffmpeg version N-78949-g6f5048f Copyright (c) 2000-2016 the FFmpeg developers
  
    built with gcc 5.3.0 (GCC)
  
    configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-a
  
  isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --ena
  
  le-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca -
  
  enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-
  
  ibilbc --enable-libmodplug --enable-libmfx --enable-libmp3lame --enable-libopen
  
  ore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --en
  
  ble-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enabl
  
  -libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --en
  
  ble-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-lib
  
  264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enabl
  
  -lzma --enable-decklink --enable-zlib
  
    libavutil      55. 19.100 / 55. 19.100
  
    libavcodec     57. 27.101 / 57. 27.101
  
    libavformat    57. 28.100 / 57. 28.100
  
    libavdevice    57.  0.101 / 57.  0.101
  
    libavfilter     6. 39.100 /  6. 39.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  Input #0, mpeg, from 'finaloutput1.mpg':
  
    Duration: 00:00:16.13, start: 0.540000, bitrate: 28775 kb/s
  
      Stream #0:0[0x1e0]: Video: mpeg2video (Main), yuv420p(tv), 1920x1080 [SAR 1
  
  1 DAR 16:9], max. 104857 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
  
  [mp3 @ 0000000002a99020] Skipping 0 bytes of junk at 253.
  
  Input #1, mp3, from 'bgmusic.mp3':
  
    Metadata:
  
      encoder         : Lavf57.28.100
  
    Duration: 00:00:37.04, start: 0.025057, bitrate: 128 kb/s
  
      Stream #1:0: Audio: mp3, 44100 Hz, stereo, s16p, 128 kb/s
  
      Metadata:
  
        encoder         : Lavc57.27
  
  [AVFilterGraph @ 000000000050bd80] No such filter: '"'
  
  Error initializing complex filters.
  
  Invalid argument

  please help.