Recherche avancée

Sur d’autres sites (7758)

• Are there any libraries on adding effect to audio, like phone, inner monologue, or sounds like a man/woman ? [closed]

  6 mars, par Mathew

  I'm trying to apply different audio effects, such as making audio sound like a phone call. Below is my current approach. As you can see, I'm using multiple filters and simple algorithms to achieve this effect, but the output quality isn't ideal.

  


  Since I need to implement many sound effects/filters, are there any ready-to-use libraries that could help ?

  


  I've looked into FFmpeg filters and noticed mentions of LADSPA/LV2 plugins. Are these viable solutions ? Any other suggestions would be greatly appreciated.

  


  public static void applySceneEffect(String inputPath, String outputPath, int sceneType) {
    LOGGER.info("apply scene effect {} to {}", sceneType, inputPath);

    try (FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(inputPath);
         FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(outputPath, grabber.getAudioChannels())) {

        grabber.setOption("vn", ""); 
        grabber.start();

       
        recorder.setAudioCodec(avcodec.AV_CODEC_ID_PCM_S16LE); 
        recorder.setSampleRate(grabber.getSampleRate());
        recorder.setAudioChannels(grabber.getAudioChannels());
        recorder.setAudioBitrate(grabber.getAudioBitrate());
        recorder.setFormat("wav"); 


        String audioFilter = String.join(",",
                "aresample=8000",      
                "highpass=f=300, lowpass=f=3400",       
                "acompressor=threshold=-15dB:ratio=4:attack=10:release=100", 
                "volume=1.5",          
                "aecho=0.9:0.4:10:0.6"
        );

        FFmpegFrameFilter f1 = new FFmpegFrameFilter(audioFilter, grabber.getAudioChannels());
        f1.setSampleRate(grabber.getSampleRate());
        f1.start();

        recorder.start();

        Random random = new Random();
        double noiseLevel = 0.02; 

   
        while (true) {
            var frame = grabber.grabFrame(true, false, true, true);
            if (frame == null) {
                break;
            }

            ShortBuffer audioBuffer = (ShortBuffer) frame.samples[0];
            short[] audioData = new short[audioBuffer.remaining()];
            audioBuffer.get(audioData);

            applyElectricNoise(audioData, grabber.getSampleRate());

            audioData = applyDistortion(audioData, 1.5, 30000);

            audioBuffer.rewind();
            audioBuffer.put(audioData);
            audioBuffer.flip();


            f1.push(frame); 
            Frame filteredFrame;
            while ((filteredFrame = f1.pull()) != null) {
                recorder.record(filteredFrame); 
            }
        }

        recorder.stop();
        recorder.release();
        grabber.stop();
        grabber.release();
    } catch (FrameGrabber.Exception | FrameRecorder.Exception | FFmpegFrameFilter.Exception e) {
        throw new RuntimeException(e);
    }
}


private static final double NOISE_LEVEL = 0.005; 
private static final int NOISE_FREQUENCY = 60;  

public static void applyElectricNoise(short[] audioData, int sampleRate) {
    Random random = new Random();

    
    for (int i = 0; i < audioData.length; i++) {
        double noise = Math.sin(2 * Math.PI * NOISE_FREQUENCY * i / sampleRate);

        double electricNoise = random.nextGaussian() * NOISE_LEVEL * Short.MAX_VALUE + noise;

        audioData[i] = (short) Math.max(Math.min(audioData[i] + electricNoise, Short.MAX_VALUE), Short.MIN_VALUE); 
    }
}

public static short[] applyTremolo(short[] audioData, int sampleRate, double frequency, double depth) {
    double phase = 0.0;
    double phaseIncrement = 2 * Math.PI * frequency / sampleRate;

    for (int i = 0; i < audioData.length; i++) {
        double modulator = 1.0 - depth + depth * Math.sin(phase); 
        audioData[i] = (short) (audioData[i] * modulator);

        phase += phaseIncrement;
        if (phase > 2 * Math.PI) {
            phase -= 2 * Math.PI;
        }
    }
    return audioData;
}

public static short[] applyDistortion(short[] audioData, double gain, double threshold) {
    for (int i = 0; i < audioData.length; i++) {
        double sample = audioData[i] * gain;

        if (sample > threshold) {
            sample = threshold;
        } else if (sample < -threshold) {
            sample = -threshold;
        }

        audioData[i] = (short) sample;
    }
    return audioData;
}


  


• Unable to set "stream-number" pad property of mpegtsmux

  8 novembre 2024, par TishSerg

  Accroding to gst-inspect-1.0 mpegtsmux, mpegtsmux's sink pads have writable stream-number property :

  


  ...
Pad Templates:
  SINK template: 'sink_%d'
    Availability: On request
    Capabilities:
      ...
    Type: GstBaseTsMuxPad
    Pad Properties:
      ...

      stream-number       : stream number
                            flags: readable, writable
                            Integer. Range: 0 - 31 Default: 0


  


  But when I try to set it, GStreamer says there's no such property. The following listing shows I can run multi-stream pipeline without setting that property, but when I add that property it doesn't work.

  


  PS C:\gstreamer\1.0\msvc_x86_64\bin> ./gst-launch-1.0 mpegtsmux name=mux ! udpsink host=192.168.144.255 port=5600 sync=no `
>> videotestsrc is-live=true pattern=ball ! "video/x-raw, width=1920, height=1080, profile=main" ! x264enc ! mux.sink_300 `
>> videotestsrc is-live=true ! "video/x-raw, width=720, height=576" ! x264enc ! mux.sink_301
Use Windows high-resolution clock, precision: 1 ms
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Redistribute latency...
Redistribute latency...
handling interrupt.9.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:03.773243400
Setting pipeline to NULL ...
Freeing pipeline ...
PS C:\gstreamer\1.0\msvc_x86_64\bin> ./gst-launch-1.0 mpegtsmux name=mux sink_300::stream-number=1 ! udpsink host=192.168.144.255 port=5600 sync=no `
>> videotestsrc is-live=true pattern=ball ! "video/x-raw, width=1920, height=1080, profile=main" ! x264enc ! mux.sink_300 `
>> videotestsrc is-live=true ! "video/x-raw, width=720, height=576" ! x264enc ! mux.sink_301
WARNING: erroneous pipeline: no property "sink_300::stream-number" in element "mpegtsmux"
PS C:\gstreamer\1.0\msvc_x86_64\bin> .\gst-launch-1.0.exe --version
gst-launch-1.0 version 1.24.8
GStreamer 1.24.8
Unknown package origin
PS C:\gstreamer\1.0\msvc_x86_64\bin> .\gst-launch-1.0.exe --version
gst-launch-1.0 version 1.24.9
GStreamer 1.24.9
Unknown package origin
PS C:\gstreamer\1.0\msvc_x86_64\bin> ./gst-launch-1.0 mpegtsmux name=mux sink_300::stream-number=1 ! udpsink host=192.168.144.255 port=5600 sync=no `
>> videotestsrc is-live=true pattern=ball ! "video/x-raw, width=1920, height=1080, profile=main" ! x264enc ! mux.sink_300 `
>> videotestsrc is-live=true ! "video/x-raw, width=720, height=576" ! x264enc ! mux.sink_301
WARNING: erroneous pipeline: no property "sink_300::stream-number" in element "mpegtsmux"


  


  I even updated GStreamer, but still no luck. I tried that because I found news saying there were updates regarding that property :

  


    397 ### MPEG-TS improvements
  398 
  399 -   mpegtsdemux gained support for
  400     -   segment seeking for seamless non-flushing looping, and
  401     -   synchronous KLV
  402 -   mpegtsmux now
  403     -   allows attaching PCR to non-PES streams
  404     -   allows setting of the PES stream number for AAC audio and AVC video streams via a new “stream-number” property on the
  405         muxer sink pads. Currently, the PES stream number is hard-coded to zero for these stream types.


  


  The syntax seems correct (pad_name::pad_prop_name on the element). I ran out of ideas about what I'm doing wrong with that property.

  


  Broader context :

  


  I want to set that property because I want an exact sequence of streams I mixing.

  


  When I feed mpegtsmux with two video streams and one audio stream (from capture devices) without specifying the stream numbers I get them muxed in a random sequence (checking it using ffprobe). Sometimes they are in the desired sequence, but sometimes they aren't. The worst case is when the audio stream is the first stream in the file, so video players get mad when trying to play such .ts file. I have to remux such files using -map key of ffmpeg. If I could set exact stream indices in mpegtsmux (not to be confused with stream PID) I could avoid analyzing actual stream layout and remuxing.

  


  Example of the real layout of the streams (ffprobe output) :

  


  Input #0, mpegts, from '████████████████████████████████████████':
  Duration: 00:20:09.64, start: 3870.816656, bitrate: 6390 kb/s
  Program 1
  Stream #0:2[0x41]: Video: h264 (Baseline) (HDMV / 0x564D4448), yuvj420p(pc, bt709, progressive), 1920x1080, 30 fps, 30 tbr, 90k tbn
  Stream #0:1[0x4b]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, mono, fltp, 130 kb/s
  Program 2
  Stream #0:0[0x42]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(progressive), 720x576, 25 fps, 25 tbr, 90k tbn


  


  You can see 3 streams :

  


  

  • FullHD video with PID 0x41 (defined by me as mpegtsmux0.sink_65) has index 2 while I want it to be 0

  


  • PAL video with PID 0x42 (defined by me as mpegtsmux0.sink_66) has index 0 while I want it to be 1

  


  • Audio with PID 0x4b (defined by me as mpegtsmux0.sink_75) has index 1 while I want it to be 2

  


  


• GStreamer : How to set "stream-number" pad property of mpegtsmux element ? [closed]

  8 novembre 2024, par TishSerg

  Accroding to gst-inspect-1.0 mpegtsmux, mpegtsmux's sink pads have writable stream-number property :

  


  ...
Pad Templates:
  SINK template: 'sink_%d'
    Availability: On request
    Capabilities:
      ...
    Type: GstBaseTsMuxPad
    Pad Properties:
      ...

      stream-number       : stream number
                            flags: readable, writable
                            Integer. Range: 0 - 31 Default: 0


  


  But when I try to set it, GStreamer says there's no such property. The following listing shows I can run multi-stream pipeline without setting that property, but when I add that property it doesn't work.

  


  PS C:\gstreamer\1.0\msvc_x86_64\bin> ./gst-launch-1.0 mpegtsmux name=mux ! udpsink host=192.168.144.255 port=5600 sync=no `
>> videotestsrc is-live=true pattern=ball ! "video/x-raw, width=1920, height=1080, profile=main" ! x264enc ! mux.sink_300 `
>> videotestsrc is-live=true ! "video/x-raw, width=720, height=576" ! x264enc ! mux.sink_301
Use Windows high-resolution clock, precision: 1 ms
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Redistribute latency...
Redistribute latency...
Redistribute latency...
handling interrupt.9.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:03.773243400
Setting pipeline to NULL ...
Freeing pipeline ...
PS C:\gstreamer\1.0\msvc_x86_64\bin> ./gst-launch-1.0 mpegtsmux name=mux sink_300::stream-number=1 ! udpsink host=192.168.144.255 port=5600 sync=no `
>> videotestsrc is-live=true pattern=ball ! "video/x-raw, width=1920, height=1080, profile=main" ! x264enc ! mux.sink_300 `
>> videotestsrc is-live=true ! "video/x-raw, width=720, height=576" ! x264enc ! mux.sink_301
WARNING: erroneous pipeline: no property "sink_300::stream-number" in element "mpegtsmux"
PS C:\gstreamer\1.0\msvc_x86_64\bin> .\gst-launch-1.0.exe --version
gst-launch-1.0 version 1.24.8
GStreamer 1.24.8
Unknown package origin
PS C:\gstreamer\1.0\msvc_x86_64\bin> .\gst-launch-1.0.exe --version
gst-launch-1.0 version 1.24.9
GStreamer 1.24.9
Unknown package origin
PS C:\gstreamer\1.0\msvc_x86_64\bin> ./gst-launch-1.0 mpegtsmux name=mux sink_300::stream-number=1 ! udpsink host=192.168.144.255 port=5600 sync=no `
>> videotestsrc is-live=true pattern=ball ! "video/x-raw, width=1920, height=1080, profile=main" ! x264enc ! mux.sink_300 `
>> videotestsrc is-live=true ! "video/x-raw, width=720, height=576" ! x264enc ! mux.sink_301
WARNING: erroneous pipeline: no property "sink_300::stream-number" in element "mpegtsmux"


  


  I even updated GStreamer, but still no luck. I tried that because I found news saying there were updates regarding that property :

  


    397 ### MPEG-TS improvements
  398 
  399 -   mpegtsdemux gained support for
  400     -   segment seeking for seamless non-flushing looping, and
  401     -   synchronous KLV
  402 -   mpegtsmux now
  403     -   allows attaching PCR to non-PES streams
  404     -   allows setting of the PES stream number for AAC audio and AVC video streams via a new “stream-number” property on the
  405         muxer sink pads. Currently, the PES stream number is hard-coded to zero for these stream types.


  


  The syntax seems correct (pad_name::pad_prop_name on the element). I ran out of ideas about what I'm doing wrong with that property.

  


  Broader context :

  


  I want to set that property because I want an exact sequence of streams I mixing.

  


  When I feed mpegtsmux with two video streams and one audio stream (from capture devices) without specifying the stream numbers I get them muxed in a random sequence (checking it using ffprobe). Sometimes they are in the desired sequence, but sometimes they aren't. The worst case is when the audio stream is the first stream in the file, so video players get mad when trying to play such .ts file. I have to remux such files using -map key of ffmpeg. If I could set exact stream indices in mpegtsmux (not to be confused with stream PID) I could avoid analyzing actual stream layout and remuxing.

  


  Example of the real layout of the streams (ffprobe output) :

  


  Input #0, mpegts, from '████████████████████████████████████████':
  Duration: 00:20:09.64, start: 3870.816656, bitrate: 6390 kb/s
  Program 1
  Stream #0:2[0x41]: Video: h264 (Baseline) (HDMV / 0x564D4448), yuvj420p(pc, bt709, progressive), 1920x1080, 30 fps, 30 tbr, 90k tbn
  Stream #0:1[0x4b]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, mono, fltp, 130 kb/s
  Program 2
  Stream #0:0[0x42]: Video: h264 (High) (HDMV / 0x564D4448), yuv420p(progressive), 720x576, 25 fps, 25 tbr, 90k tbn


  


  You can see 3 streams :

  


  

  • FullHD video with PID 0x41 (defined by me as mpegtsmux0.sink_65) has index 2 while I want it to be 0

  


  • PAL video with PID 0x42 (defined by me as mpegtsmux0.sink_66) has index 0 while I want it to be 1

  


  • Audio with PID 0x4b (defined by me as mpegtsmux0.sink_75) has index 1 while I want it to be 2