Recherche avancée

Sur d’autres sites (5738)

• Screeching white sound coming while playing audio as a raw stream

  27 avril 2020, par Sri Nithya Sharabheshwarananda

  I. Background

  



  

  1. I am trying to make an application which helps to match subtitles to the audio waveform very accurately at the waveform level, at the word level or even at the character level.

  


  2. The audio is expected to be Sanskrit chants (Yoga, rituals etc.) which are extremely long compound words [ example - aṅganyā-sokta-mātaro-bījam is traditionally one word broken only to assist reading ]

  


  3. The input transcripts / subtitles might be roughly in sync at the sentence/verse level but surely would not be in sync at the word level.

  


  4. The application should be able to figure out points of silence in the audio waveform, so that it can guess the start and end points of each word (or even letter/consonant/vowel in a word), such that the audio-chanting and visual-subtitle at the word level (or even at letter/consonant/vowel level) perfectly match, and the corresponding UI just highlights or animates the exact word (or even letter) in the subtitle line which is being chanted at that moment, and also show that word (or even the letter/consonant/vowel) in bigger font. This app's purpose is to assist learning Sanskrit chanting.

  


  5. It is not expected to be a 100% automated process, nor 100% manual but a mix where the application should assist the human as much as possible.

  


  



  II. Following is the first code I wrote for this purpose, wherein

  



  

  1. First I open a mp3 (or any audio format) file,

  


  2. Seek to some arbitrary point in the timeline of the audio file // as of now playing from zero offset

  


  3. Get the audio data in raw format for 2 purposes - (1) playing it and (2) drawing the waveform.

  


  4. Playing the raw audio data using standard java audio libraries

  


  



  III. The problem I am facing is, between every cycle there is screeching sound.

  



  

  • Probably I need to close the line between cycles ? Sounds simple, I can try.

  


  • But I am also wondering if this overall approach itself is correct ? Any tip, guide, suggestion, link would be really helpful.

  


  • Also I just hard coded the sample-rate etc ( 44100Hz etc. ), are these good to set as default presets or it should depend on the input format ?

  


  



  IV. Here is the code

  



  import com.github.kokorin.jaffree.StreamType;
import com.github.kokorin.jaffree.ffmpeg.FFmpeg;
import com.github.kokorin.jaffree.ffmpeg.FFmpegProgress;
import com.github.kokorin.jaffree.ffmpeg.FFmpegResult;
import com.github.kokorin.jaffree.ffmpeg.NullOutput;
import com.github.kokorin.jaffree.ffmpeg.PipeOutput;
import com.github.kokorin.jaffree.ffmpeg.ProgressListener;
import com.github.kokorin.jaffree.ffprobe.Stream;
import com.github.kokorin.jaffree.ffmpeg.UrlInput;
import com.github.kokorin.jaffree.ffprobe.FFprobe;
import com.github.kokorin.jaffree.ffprobe.FFprobeResult;
import java.io.IOException;
import java.io.OutputStream;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.SourceDataLine;


public class FFMpegToRaw {
    Path BIN = Paths.get("f:\\utilities\\ffmpeg-20190413-0ad0533-win64-static\\bin");
    String VIDEO_MP4 = "f:\\org\\TEMPLE\\DeviMahatmyamRecitationAudio\\03_01_Devi Kavacham.mp3";
    FFprobe ffprobe;
    FFmpeg ffmpeg;

    public void basicCheck() throws Exception {
        if (BIN != null) {
            ffprobe = FFprobe.atPath(BIN);
        } else {
            ffprobe = FFprobe.atPath();
        }
        FFprobeResult result = ffprobe
                .setShowStreams(true)
                .setInput(VIDEO_MP4)
                .execute();

        for (Stream stream : result.getStreams()) {
            System.out.println("Stream " + stream.getIndex()
                    + " type " + stream.getCodecType()
                    + " duration " + stream.getDuration(TimeUnit.SECONDS));
        }    
        if (BIN != null) {
            ffmpeg = FFmpeg.atPath(BIN);
        } else {
            ffmpeg = FFmpeg.atPath();
        }

        //Sometimes ffprobe can't show exact duration, use ffmpeg trancoding to NULL output to get it
        final AtomicLong durationMillis = new AtomicLong();
        FFmpegResult fFmpegResult = ffmpeg
                .addInput(
                        UrlInput.fromUrl(VIDEO_MP4)
                )
                .addOutput(new NullOutput())
                .setProgressListener(new ProgressListener() {
                    @Override
                    public void onProgress(FFmpegProgress progress) {
                        durationMillis.set(progress.getTimeMillis());
                    }
                })
                .execute();
        System.out.println("audio size - "+fFmpegResult.getAudioSize());
        System.out.println("Exact duration: " + durationMillis.get() + " milliseconds");
    }

    public void toRawAndPlay() throws Exception {
        ProgressListener listener = new ProgressListener() {
            @Override
            public void onProgress(FFmpegProgress progress) {
                System.out.println(progress.getFrame());
            }
        };

        // code derived from : https://stackoverflow.com/questions/32873596/play-raw-pcm-audio-received-in-udp-packets

        int sampleRate = 44100;//24000;//Hz
        int sampleSize = 16;//Bits
        int channels   = 1;
        boolean signed = true;
        boolean bigEnd = false;
        String format  = "s16be"; //"f32le"

        //https://trac.ffmpeg.org/wiki/audio types
        final AudioFormat af = new AudioFormat(sampleRate, sampleSize, channels, signed, bigEnd);
        final DataLine.Info info = new DataLine.Info(SourceDataLine.class, af);
        final SourceDataLine line = (SourceDataLine) AudioSystem.getLine(info);

        line.open(af, 4096); // format , buffer size
        line.start();

        OutputStream destination = new OutputStream() {
            @Override public void write(int b) throws IOException {
                throw new UnsupportedOperationException("Nobody uses thi.");
            }
            @Override public void write(byte[] b, int off, int len) throws IOException {
                String o = new String(b);
                boolean showString = false;
                System.out.println("New output ("+ len
                        + ", off="+off + ") -> "+(showString?o:"")); 
                // output wave form repeatedly

                if(len%2!=0) {
                    len -= 1;
                    System.out.println("");
                }
                line.write(b, off, len);
                System.out.println("done round");
            }
        };

        // src : http://blog.wudilabs.org/entry/c3d357ed/?lang=en-US
        FFmpegResult result = FFmpeg.atPath(BIN).
            addInput(UrlInput.fromPath(Paths.get(VIDEO_MP4))).
            addOutput(PipeOutput.pumpTo(destination).
                disableStream(StreamType.VIDEO). //.addArgument("-vn")
                setFrameRate(sampleRate).            //.addArguments("-ar", sampleRate)
                addArguments("-ac", "1").
                setFormat(format)              //.addArguments("-f", format)
            ).
            setProgressListener(listener).
            execute();

        // shut down audio
        line.drain();
        line.stop();
        line.close();

        System.out.println("result = "+result.toString());
    }

    public static void main(String[] args) throws Exception {
        FFMpegToRaw raw = new FFMpegToRaw();
        raw.basicCheck();
        raw.toRawAndPlay();
    }
}



  



  Thank You

  


• FFmpeg cannot extract correct first frame - always extract the second frame [closed]

  21 octobre 2024, par Cai Yu

  I am on macos Ventura 13.2.1, on Macbook Pro M1Max.

  


  The video is number counting animation video. From 2 to 125. The number on first frame is 2. The number on last frame is 125. The number plus one on each frame. You can download the video here.

  


  NOTE that :
First, you hit space bar to play it in Finder, you can see the first frame of the video is number 2.

  


  Second, in professional Video editing software - davinci resolve, on the timeline, you can see the first frame is number 2.

  


  Now I try to use FFmpeg to extract the first frame of the video.
First Try :

  


  ffmpeg -i /Users/chris/Downloads/2to125.mov -vframes 1 /Users/chris/Downloads/1.png


  


  The output png image is number 3. So it's actually the second frame.

  


  Second try :

  


  ffmpeg -i /Users/chris/Downloads/2to125.mov -frames:v 1 /Users/chris/Downloads/1.png


  


  The output png image is still the second frame - the number 3.

  


  Third try :

  


  ffmpeg -i /Users/chris/Downloads/2to125.mov -vf "select=eq(n\,0)" -q:v 3 /Users/chris/Downloads/1.png


  


  The output image is sitll the wrong second frame.

  


  NOTE that :

  


  first : all these three method are from internet. And a lot of people voted correct answer for this question.

  


  second : all their output on my system is the incorrect second frame, not the first frame.

  


  I cannot find what's wrong. Hope friend here help me.

  


• Converting an audio file from .oga format to .mp3 using ffmpeg package on nodeJs produces a max of 3 seconds output file no matter the input duration

  15 janvier 2024, par JnrLouis

  I am downloading an audio file in .oga format and saving it. Then I am trying to convert the file to .mp3 format, but the issue is the output file is always truncated and a maximum of 3 seconds. I have gone through the fluent-ffmpeg library and I can't seem to find what I'm doing wrong.

  


  I have ffmpeg installed and I'm using the fluent-ffmpeg library. The downloaded .oga file doesn't seem to have any issues, the issue is after it gets converted to .mp3.

  


  I also tried converting to .wav, and I faced the same issue.

  


  Below is my current code :

  


  const fs = require("fs");
const ffmpegPath = require('@ffmpeg-installer/ffmpeg').path;
const ffmpeg = require('fluent-ffmpeg');
ffmpeg.setFfmpegPath(ffmpegPath);
const axios = require("axios");

const inputPath = __dirname + '/audio/input.oga';
const outputPath = __dirname + '/audio/output.mp3';

const saveVoiceMessage = async (url) => {
    try {
        const response = await axios({
            method: 'GET',
            url: url,
            responseType: 'stream'
        });
        const inStream = fs.createWriteStream(inputPath);
        await response.data.pipe(inStream);
    } catch (error) {
        console.error(error);
    }
}

const convertToMp3 = async () => {
    try {
        const outStream = fs.createWriteStream(outputPath);
        const inStream = fs.createReadStream(inputPath);
        // I also tried using the inputPath directly, still didn't work
        ffmpeg(inStream)
            .toFormat("mp3")
            .on('error', error => console.log(`Encoding Error: ${error.message}`))
            .on('exit', () => console.log('Audio recorder exited'))
            .on('close', () => console.log('Audio recorder closed'))
            .on('end', () => console.log('Audio Transcoding succeeded !'))
            .pipe(outStream, { end: true })
    } catch (error) {
        console.error(error);
    }
}

const saveAndConvertToMp3 = async (url) => {
    try {
        await saveVoiceMessage(url);
        await convertToMp3();
        }

    } catch (error) {
        console.error(error);
    }
}