Recherche avancée

Sur d’autres sites (8252)

• lavc/libopenh264enc : add bit rate control select support

  29 avril 2020, par Linjie Fu

  lavc/libopenh264enc : add bit rate control select support
  RC_BITRATE_MODE :
  
      set BITS_EXCEEDED to iCurrentBitsLevel and allows QP adjust
  
      in RcCalculatePictureQp().
  RC_BUFFERBASED_MODE :
  
      use buffer status to adjust the video quality.
  RC_TIMESTAMP_MODE :
  
      bit rate control based on timestamp, introduced in release 1.4.
  Default to use RC_QUALITY_MODE.
  Signed-off-by : Linjie Fu <linjie.fu@intel.com>
  
  Signed-off-by : Martin Storsjö <martin@martin.st>
  

  • [DH] libavcodec/libopenh264enc.c

• What determine number of cpu core usage of filter select=gt(scene,0.1) ?

  15 décembre 2022, par kocoten1992

  I've notice that when using filter gt(scene,0.1), for example :

  


  ffmpeg -i big_buck_bunny.mkv -filter:v "select='gt(scene,0.1)',showinfo" -f null -

  


  Depends on the video, number of cpu cores usage varies extremely (sometimes it 3 cores usage - other time 12 cores usage in different video).

  


  Would like to ask what determine that logic ?

  


  I try to read ffmpeg source code but not familiar with it, a general explanation would be enough, but much appreciate if you point out the line/directory determine that logic in https://github.com/FFmpeg/FFmpeg.

  


  (Also not asking how to reduce cpu usage, interested in the logic determine that).

  


• Detect volume via mic, start recording, end on silence, transcribe and sent to endpoint

  15 juin 2023, par alphadmon

  I have been attempting to get this to work in many ways but I can't seem to get it right. Most of the time I get a part of it to work and then when I try to make other parts work, I generally break other things.

  


  I am intercepting the volume coming from the mic and if it is louder than 50, I start a recording. I then keep recording until there is a silence, if the silence is equal to 5 seconds I then stop the recording.

  


  I then send the recording to be transcribed by whisper using OpenAI API.

  


  Once that is returned, I then want to send it to the open ai chat end point and get the response.

  


  After that, I would like to start listening again.

  


  Here is what I have that is sort of working so far, but the recording is an empty file always :

  


  // DETECT SPEECH
const recorder = require('node-record-lpcm16');

// TRANSCRIBE
const fs = require("fs");
const ffmpeg = require("fluent-ffmpeg");
const mic = require("mic");
const { Readable } = require("stream");
const ffmpegPath = require("@ffmpeg-installer/ffmpeg").path;
require('dotenv').config();

// CHAT
const { Configuration, OpenAIApi } = require("openai");

// OPEN AI
const configuration = new Configuration({
    organization: process.env.OPENAI_ORG,
    apiKey: process.env.OPENAI_API_KEY,
});
const openai = new OpenAIApi(configuration);

// SETUP
ffmpeg.setFfmpegPath(ffmpegPath);

// VARS
let isRecording = false;
const audioFilename = 'recorded_audio.wav';
const micInstance = mic({
    rate: '16000',
    channels: '1',
    fileType: 'wav',
});

// DETECT SPEECH
const file = fs.createWriteStream('determine_speech.wav', { encoding: 'binary' });
const recording = recorder.record();
recording.stream().pipe(file);


recording.stream().on('data', async (data) => {
    let volume = parseInt(calculateVolume(data));
    if (volume > 50 && !isRecording) {
        console.log('You are talking.');
        await recordAudio(audioFilename);
    } else {
        setTimeout(async () => {
            console.log('You are quiet.');
            micInstance.stop();
            console.log('Finished recording');
            const transcription = await transcribeAudio(audioFilename);
            console.log('Transcription:', transcription);
            setTimeout(async () => {
                await askAI(transcription);
            }, 5000);
        }, 5000);
    }
});

function calculateVolume(data) {
    let sum = 0;

    for (let i = 0; i < data.length; i += 2) {
        const sample = data.readInt16LE(i);
        sum += sample * sample;
    }

    const rms = Math.sqrt(sum / (data.length / 2));

    return rms;
}

// TRANSCRIBE
function recordAudio(filename) {
    const micInputStream = micInstance.getAudioStream();
    const output = fs.createWriteStream(filename);
    const writable = new Readable().wrap(micInputStream);

    console.log('Listening...');

    writable.pipe(output);

    micInstance.start();

    micInputStream.on('error', (err) => {
        console.error(err);
    });
}

// Transcribe audio
async function transcribeAudio(filename) {
    const transcript = await openai.createTranscription(
        fs.createReadStream(filename),
        "whisper-1",
    );
    return transcript.data.text;
}

// CHAT
async function askAI(text) {
    let completion = await openai.createChatCompletion({
        model: "gpt-4",
        temperature: 0.2,
        stream: false,
        messages: [
            { role: "user", content: text },
            { role: "system", content: "Act like you are a rude person." }
        ],
    });

    completion = JSON.stringify(completion.data, null, 2);
    console.log(completion);
}