Recherche avancée

Sur d’autres sites (7362)

• Discord Bot Not Playing Audio When Using YouTube-dl and FFmpeg

  5 novembre 2020, par John Henry 5

  I'm trying to get a bot to join a voice chat and then play the audio in a youtube url. This is the code I have :

  


  @client.command() async def play(ctx):
    channel = ctx.message.author.voice.channel
    voice_client = await channel.connect()

    opts = {'format': 'bestaudio'}
    FFMPEG_OPTIONS = {'before_options': '-reconnect 1 -reconnect_streamed 1 -reconnect_delay_max 5', 'options': '-vn'}
    with youtube_dl.YoutubeDL(opts) as ydl:
        song_info = ydl.extract_info('video', download=False)
        URL = song_info['formats'][0]['url']
    voice_client.play(FFmpegPCMAudio(URL, **FFMPEG_OPTIONS))


  


  No errors are thrown but it just says [youtube] video : Downloading webpage

  


• Detect volume via mic, start recording, end on silence, transcribe and sent to endpoint

  15 juin 2023, par alphadmon

  I have been attempting to get this to work in many ways but I can't seem to get it right. Most of the time I get a part of it to work and then when I try to make other parts work, I generally break other things.

  


  I am intercepting the volume coming from the mic and if it is louder than 50, I start a recording. I then keep recording until there is a silence, if the silence is equal to 5 seconds I then stop the recording.

  


  I then send the recording to be transcribed by whisper using OpenAI API.

  


  Once that is returned, I then want to send it to the open ai chat end point and get the response.

  


  After that, I would like to start listening again.

  


  Here is what I have that is sort of working so far, but the recording is an empty file always :

  


  // DETECT SPEECH
const recorder = require('node-record-lpcm16');

// TRANSCRIBE
const fs = require("fs");
const ffmpeg = require("fluent-ffmpeg");
const mic = require("mic");
const { Readable } = require("stream");
const ffmpegPath = require("@ffmpeg-installer/ffmpeg").path;
require('dotenv').config();

// CHAT
const { Configuration, OpenAIApi } = require("openai");

// OPEN AI
const configuration = new Configuration({
    organization: process.env.OPENAI_ORG,
    apiKey: process.env.OPENAI_API_KEY,
});
const openai = new OpenAIApi(configuration);

// SETUP
ffmpeg.setFfmpegPath(ffmpegPath);

// VARS
let isRecording = false;
const audioFilename = 'recorded_audio.wav';
const micInstance = mic({
    rate: '16000',
    channels: '1',
    fileType: 'wav',
});

// DETECT SPEECH
const file = fs.createWriteStream('determine_speech.wav', { encoding: 'binary' });
const recording = recorder.record();
recording.stream().pipe(file);


recording.stream().on('data', async (data) => {
    let volume = parseInt(calculateVolume(data));
    if (volume > 50 && !isRecording) {
        console.log('You are talking.');
        await recordAudio(audioFilename);
    } else {
        setTimeout(async () => {
            console.log('You are quiet.');
            micInstance.stop();
            console.log('Finished recording');
            const transcription = await transcribeAudio(audioFilename);
            console.log('Transcription:', transcription);
            setTimeout(async () => {
                await askAI(transcription);
            }, 5000);
        }, 5000);
    }
});

function calculateVolume(data) {
    let sum = 0;

    for (let i = 0; i < data.length; i += 2) {
        const sample = data.readInt16LE(i);
        sum += sample * sample;
    }

    const rms = Math.sqrt(sum / (data.length / 2));

    return rms;
}

// TRANSCRIBE
function recordAudio(filename) {
    const micInputStream = micInstance.getAudioStream();
    const output = fs.createWriteStream(filename);
    const writable = new Readable().wrap(micInputStream);

    console.log('Listening...');

    writable.pipe(output);

    micInstance.start();

    micInputStream.on('error', (err) => {
        console.error(err);
    });
}

// Transcribe audio
async function transcribeAudio(filename) {
    const transcript = await openai.createTranscription(
        fs.createReadStream(filename),
        "whisper-1",
    );
    return transcript.data.text;
}

// CHAT
async function askAI(text) {
    let completion = await openai.createChatCompletion({
        model: "gpt-4",
        temperature: 0.2,
        stream: false,
        messages: [
            { role: "user", content: text },
            { role: "system", content: "Act like you are a rude person." }
        ],
    });

    completion = JSON.stringify(completion.data, null, 2);
    console.log(completion);
}


  


• How to convert char buffer (with pcm audio data) to short buffer

  18 décembre 2012, par testCoder

  I have char pAudioBuffer buffer which i got from function ffmpeg :

  int len = avcodec_decode_audio3(av_codec_context,
  
              (int16_t *) pAudioBuffer, &out_size, &packet);

  I know that audio format is two bytes per sample, i need to convert every two bytes to short value, i have tried to use code snippet below, but i often got zero instead of short value :

  int shortBufIndex = 0;
  
  for (int i = 0; i < (out_size); i += 2) {
  
      char b1 = pAudioBuffer[i];
  
      char b2 = pAudioBuffer[i + 1];
  
      short sample = atoi(&b1) + atoi(&b2);
  
      shortBuffer[shortBufIndex] = sample;
  
      shortBufIndex++;
  
      LOGI("BUFFER_ITEM='%d'", sample);
  
  }

  What i'm doing wrong, how to convert every two bytes in char buffer to short and and back.

  UPDATE :

  system's byte order is LITTLE_ENDIAN i have test it like this : Endianness of Android NDK

  How can i convert every two bytes in buffer to sample of short type and back. Please can you provide any code sample.

  UPDATE

  I have tried to access to short as pairs, here is my fixed code, but it not work, i don't hear any sound :

      int shortBufIndex = 0;
  
      for (int i = 0; i < (out_size); i += 2) {
  
          char * byte = (char *) pAudioBuffer[i];
  
          short * sample = byte;
  
          shortBuffer[shortBufIndex] = sample;
  
  }

  What i'm doing wrong ?
  I need conversion like this : byte array to short array and back again in java but in c.