Newest 'ffmpeg' Questions - Stack Overflow

http://stackoverflow.com/questions/tagged/ffmpeg

Les articles publiés sur le site

  • Send image and audio data to FFmpeg via named pipes

    5 mai, par Nicke Manarin

    I'm able to send frames one by one to FFmpeg via a name pipe to create a video out of them, but if I try sending audio to a second named pipe, FFmpeg only accepts 1 frame in the frame pipe and starts reading from the audio pipe soon after it.

    ffmpeg.exe -loglevel debug -hwaccel auto 
-f:v rawvideo -r 25 -pix_fmt bgra -video_size 782x601 -i \\.\pipe\video_to_ffmpeg 
-f:a s16le -ac 2 -ar 48000 -i \\.\pipe\audio_to_ffmpeg 
-c:v libx264 -preset fast -pix_fmt yuv420p 
-vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" -crf 23 -f:v mp4 -vsync vfr 
-c:a aac -b:a 128k -ar 48000 -ac 2 
-y "C:\Users\user\Desktop\video.mp4"

    I start both pipes like so:

    _imagePipeServer = new NamedPipeServerStream(ImagePipeName, PipeDirection.Out, 1, PipeTransmissionMode.Byte, PipeOptions.Asynchronous);
_imagePipeStreamWriter = new StreamWriter(_imagePipeServer);
_imagePipeServer.BeginWaitForConnection(null, null);

_audioPipeServer = new NamedPipeServerStream(AudioPipeName, PipeDirection.Out, 1, PipeTransmissionMode.Byte, PipeOptions.Asynchronous);
_audioPipeStreamWriter = new StreamWriter(_audioPipeServer);
_audioPipeServer.BeginWaitForConnection(null, null);

    And send the data to the pipes using these methods:

    public void EncodeFrame(byte[] data)
{
    if (_imagePipeServer?.IsConnected != true)
        throw new FFmpegException("Pipe not connected", Arguments, Output);

    _imagePipeStreamWriter?.BaseStream.Write(data, 0, data.Length);
}

    public void EncodeAudio(ISampleProvider provider, long length)
{
    if (_audioPipeServer?.IsConnected != true)
        throw new FFmpegException("Pipe not connected", Arguments, Output);

    var buffer = new byte[provider.WaveFormat.AverageBytesPerSecond * length / TimeSpan.TicksPerSecond];
    var bytesRead = provider.ToWaveProvider().Read(buffer, 0, buffer.Length);

    if (bytesRead < 1)
        return;

    _audioPipeStreamWriter?.BaseStream.Write(buffer, 0, bytesRead);
    _audioPipeStreamWriter?.BaseStream.Flush();
}

    Not sending the audio (and thus not creating the audio pipe) works, with FFmpeg taking one frame at time and creating the video normally.

    But if I try sending the audio via a secondary pipe, I can only send one frame. This is the output when that happens (Btw, FFmpeg v7.1):

    Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-hwaccel' ... matched as option 'hwaccel' (use HW accelerated decoding) with argument 'auto'.
Reading option '-f:v' ... matched as option 'f' (force container format (auto-detected otherwise)) with argument 'rawvideo'.
Reading option '-r' ... matched as option 'r' (override input framerate/convert to given output framerate (Hz value, fraction or abbreviation)) with argument '25'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'bgra'.
Reading option '-video_size' ... matched as AVOption 'video_size' with argument '782x601'.
Reading option '-i' ... matched as input url with argument '\\.\pipe\video_to_ffmpeg'.
Reading option '-f:a' ... matched as option 'f' (force container format (auto-detected otherwise)) with argument 's16le'.
Reading option '-ac' ... matched as option 'ac' (set number of audio channels) with argument '2'.
Reading option '-ar' ... matched as option 'ar' (set audio sampling rate (in Hz)) with argument '48000'.
Reading option '-i' ... matched as input url with argument '\\.\pipe\audio_to_ffmpeg'.
Reading option '-c:v' ... matched as option 'c' (select encoder/decoder ('copy' to copy stream without reencoding)) with argument 'libx264'.
Reading option '-preset' ... matched as AVOption 'preset' with argument 'fast'.
Reading option '-pix_fmt' ... matched as option 'pix_fmt' (set pixel format) with argument 'yuv420p'.
Reading option '-vf' ... matched as option 'vf' (alias for -filter:v (apply filters to video streams)) with argument 'scale=trunc(iw/2)*2:trunc(ih/2)*2'.
Reading option '-crf' ... matched as AVOption 'crf' with argument '23'.
Reading option '-f:v' ... matched as option 'f' (force container format (auto-detected otherwise)) with argument 'mp4'.
Reading option '-fps_mode' ... matched as option 'fps_mode' (set framerate mode for matching video streams; overrides vsync) with argument 'vfr'.
Reading option '-c:a' ... matched as option 'c' (select encoder/decoder ('copy' to copy stream without reencoding)) with argument 'aac'.
Reading option '-b:a' ... matched as option 'b' (video bitrate (please use -b:v)) with argument '128k'.
Reading option '-ar' ... matched as option 'ar' (set audio sampling rate (in Hz)) with argument '48000'.
Reading option '-ac' ... matched as option 'ac' (set number of audio channels) with argument '2'.
Reading option '-y' ... matched as option 'y' (overwrite output files) with argument '1'.
Reading option 'C:\Users\user\Desktop\video.mp4' ... matched as output url.
Finished splitting the commandline.

Parsing a group of options: global.
Applying option loglevel (set logging level) with argument debug.
Applying option y (overwrite output files) with argument 1.
Successfully parsed a group of options.

Parsing a group of options: input url \\.\pipe\video_to_ffmpeg.
Applying option hwaccel (use HW accelerated decoding) with argument auto.
Applying option f:v (force container format (auto-detected otherwise)) with argument rawvideo.
Applying option r (override input framerate/convert to given output framerate (Hz value, fraction or abbreviation)) with argument 25.
Applying option pix_fmt (set pixel format) with argument bgra.
Successfully parsed a group of options.

Opening an input file: \\.\pipe\video_to_ffmpeg.
[rawvideo @ 000001c302ee08c0] Opening '\\.\pipe\video_to_ffmpeg' for reading
[file @ 000001c302ee1000] Setting default whitelist 'file,crypto,data'
[rawvideo @ 000001c302ee08c0] Before avformat_find_stream_info() pos: 0 bytes read:65536 seeks:0 nb_streams:1
[rawvideo @ 000001c302ee08c0] All info found
[rawvideo @ 000001c302ee08c0] After avformat_find_stream_info() pos: 1879928 bytes read:1879928 seeks:0 frames:1
Input #0, rawvideo, from '\\.\pipe\video_to_ffmpeg':
  Duration: N/A, start: 0.000000, bitrate: 375985 kb/s
  Stream #0:0, 1, 1/25: Video: rawvideo, 1 reference frame (BGRA / 0x41524742), bgra, 782x601, 0/1, 375985 kb/s, 25 tbr, 25 tbn
Successfully opened the file.

Parsing a group of options: input url \\.\pipe\audio_to_ffmpeg.
Applying option f:a (force container format (auto-detected otherwise)) with argument s16le.
Applying option ac (set number of audio channels) with argument 2.
Applying option ar (set audio sampling rate (in Hz)) with argument 48000.
Successfully parsed a group of options.

Opening an input file: \\.\pipe\audio_to_ffmpeg.
[s16le @ 000001c302ef5380] Opening '\\.\pipe\audio_to_ffmpeg' for reading
[file @ 000001c302ef58c0] Setting default whitelist 'file,crypto,data'

    The difference if I try sending 1 frame then some bytes (arbitrary length based on fps) of audio is that I get this extra comment at the end:

    [s16le @ 0000025948c96d00] Before avformat_find_stream_info() pos: 0 bytes read:15360 seeks:0 nb_streams:1

    Extra calls to EncodeFrame() hang forever at the BaseStream.Write(frameBytes, 0, frameBytes.Length) call, suggesting that FFmpeg is no longer reading the data.

    Something is causing FFmpeg to close or stop reading the first pipe and only accept data from the second one.

    Perhaps the command is missing something?

    🏆 Working solution

    I started using two BlockingCollection objects, with the consumers running in separate tasks.

    Start the process, setting up the pipes:

    private Process? _process;
private NamedPipeServerStream? _imagePipeServer;
private NamedPipeServerStream? _audioPipeServer;
private StreamWriter? _imagePipeStreamWriter;
private StreamWriter? _audioPipeStreamWriter;
private readonly BlockingCollection _videoCollection = new();
private readonly BlockingCollection _audioCollection = new();

private const string ImagePipeName = "video_to_ffmpeg";
private const string AudioPipeName = "audio_to_ffmpeg";
private const string PipeStructure = @"\\.\pipe\"; //This part is only sent to FFmpeg, not to the .NET pipe creation.

public void StartEncoding(string arguments)
{
    _process = new Process
    {
        StartInfo = new ProcessStartInfo
        {
            FileName = "path to ffmpeg",
            Arguments = arguments.Replace("{image}", PipeStructure + ImagePipeName).Replace("{audio}", PipeStructure + AudioPipeName),
            RedirectStandardInput = false,
            RedirectStandardOutput = true,
            RedirectStandardError = true,
            UseShellExecute = false,
            CreateNoWindow = true
        }
    };

    StartFramePipeConnection();
    StartAudioPipeConnection();

    _process. Start();
    _process.BeginErrorReadLine();
    _process.BeginOutputReadLine();
}

private void StartFramePipeConnection()
{
    if (_imagePipeServer != null)
    {
        if (_imagePipeServer.IsConnected)
            _imagePipeServer.Disconnect();

        _imagePipeServer.Dispose();
    }

    _imagePipeServer = new NamedPipeServerStream(ImagePipeName, PipeDirection.Out, 1, PipeTransmissionMode.Byte, PipeOptions.Asynchronous);
    _imagePipeStreamWriter = new StreamWriter(_imagePipeServer);
    _imagePipeServer.BeginWaitForConnection(VideoPipe_Connected, null);
}

private void StartAudioPipeConnection()
{
    if (_audioPipeServer != null)
    {
        if (_audioPipeServer.IsConnected)
            _audioPipeServer.Disconnect();

        _audioPipeServer.Dispose();
    }

    _audioPipeServer = new NamedPipeServerStream(AudioPipeName, PipeDirection.Out, 1, PipeTransmissionMode.Byte, PipeOptions.Asynchronous);
    _audioPipeStreamWriter = new StreamWriter(_audioPipeServer);
    _audioPipeServer.BeginWaitForConnection(AudioPipe_Connected, null);
}

    Start sending the data as soon as the pipe gets connected. Once the BlockingCollection gets its signal that no more data is going to be sent, it will leave the foreach block and it will wait for the pipe to drain its data.

    private void VideoPipe_Connected(IAsyncResult ar)
{
    Task.Run(() =>
    {
        try
        {
            foreach (var frameBytes in _videoCollection.GetConsumingEnumerable())
            {                    
                _imagePipeStreamWriter?.BaseStream.Write(frameBytes, 0, frameBytes.Length);
            }

            _imagePipeServer?.WaitForPipeDrain();
            _imagePipeStreamWriter?.Close();
        }
        catch (Exception e)
        {
            //Logging
            throw;
        }
    });
}

private void AudioPipe_Connected(IAsyncResult ar)
{
    Task.Run(() =>
    {
        try
        {
            foreach (var audioChunk in _audioCollection.GetConsumingEnumerable())
            {
                _audioPipeStreamWriter?.BaseStream.Write(audioChunk, 0, audioChunk.Length);
            }

            _audioPipeServer?.WaitForPipeDrain();
            _audioPipeStreamWriter?.Close();
        }
        catch (Exception e)
        {
            //Logging
            throw;
        }
    });
}

    You can send the image and audio data as soon as the BlockingCollections are initiated, no need to wait for the pipes to connect.

    public void EncodeImage(byte[] data)
{
    _videoCollection.Add(data);
}

public void EncodeAudio(ISampleProvider provider, long length)
{
    var sampleCount = (int)(provider.WaveFormat.SampleRate * ((double)length / TimeSpan.TicksPerSecond) * provider.WaveFormat.Channels);
    var floatBuffer = new float[sampleCount];

    var samplesRead = provider.Read(floatBuffer, 0, sampleCount);

    if (samplesRead < 1)
        return 0;

    var byteBuffer = new byte[samplesRead * 4]; //4 bytes per float, f32le.
    Buffer.BlockCopy(floatBuffer, 0, byteBuffer, 0, byteBuffer.Length);

    
    _audioCollection.Add(byteBuffer);
}

    Once you finished producing data, make sure to signal the BlockingCollections:

    public void FinishEncoding()
{
    //Signal the end of video/audio producer.
    _videoCollection.CompleteAdding();
    _audioCollection.CompleteAdding();

    //Waits for 20 seconds for encoding to finish.
    _process?.WaitForExit(20_000);
}

    The FFmpeg arguments were changed slightly:

    -loglevel trace -hwaccel auto 
-f:v rawvideo -probesize 32 -r 25 -pix_fmt bgra -video_size 1109x627 -i {image} 
-f:a f32le -ac 2 -ar 48000 -probesize 32 -i {audio} 
-c:v libx264 -preset fast -pix_fmt yuv420p 
-vf "scale=trunc(iw/2)*2:trunc(ih/2)*2" -crf 23 -f:v mp4 -fps_mode vfr 
-c:a aac -b:a 128k -ar 48000 -ac 2 
-y "C:\Users\user\Desktop\Video.mp4"

  • Streaming different MP3 files using Ezstream and Icecast

    3 mai, par hh083

    I am trying to stream two MP3 files to Icecast using Ezstream and the stream should run in web browsers. The files I am testing with were downloaded as webm and converted to MP3 using ffmpeg. They have the same channels count, same bitrate and same sample rate but different duration.

    My setup: the Ezstream xml configuration file is set to stream MP3 and a program playlist is used to identify what is the next file to be streamed, and no encoders or decoders are used. When I start streaming I save the process ID of the Ezstream process (using the -p argument), and then I use the command kill -10 $(cat currentpid) with currentpid as the file containing the process ID so Ezstream executes the playlist program to get the next file name and skips the current file to play the next one. Basically I am just switching between 1.mp3 and 2.mp3.

    The problem is that on Chrome web browser, when I switch between the two files the player (default HTML5 player) will suddenly stop (sometimes I can switch multiple times before it happens and sometimes it happens quickly) and the error PIPELINE_ERROR_DECODE is what I find when I access player.error in JavaScript. Although Firefox handles the change and continues the stream normally, I am convinced that Firefox here is the exception, that it is not a bug in Chrome (in my case), and that there is something wrong with my setup that needs to be fixed to support other browsers.

    Doing the same using mpv player, I get the following errors but the audio keeps streaming normally (sometimes it takes multiple switches before it happens just like in Chrome):

    [ffmpeg/audio] mp3float: big_values too big
[ffmpeg/audio] mp3float: Error while decoding MPEG audio frame.
Error decoding audio.

    I tried using MP3 encoder and decoder I copied from the Ezstream example files (lame and madplay) but the problem still existed.

    I am not sure if the problem is basic and I cannot see it or it is more complicated. Also I do not have a problem if I need to use other format than MP3 to fix that issue, as long as that format is supported by Ezstream and Icecast.

    Thanks.

  • ffmpeg not honoring sample rate in opus output

    3 mai, par Adam

    I am capturing a live audio stream to Opus, and no matter what I choose for the audio sample rate, I get 48khz output.

    This is my command line

    ./ffmpeg -f alsa -ar 16000 -i sysdefault:CARD=CODEC -f
alsa -ar 16000 -i sysdefault:CARD=CODEC_1 -filter_complex
join=inputs=2:channel_layout=stereo:map=0.1-FR\|1.0-FL,asetpts=expr=N/SR/TB
-ar 16000 -ab 64k -c:a opus -vbr off -compression_level 5 output.ogg

    And this is what ffmpeg responds with:

    Output #0, ogg, to 'output.ogg': Metadata: encoder : Lavf57.48.100 Stream #0:0: Audio: opus (libopus), 16000 Hz, stereo, s16, delay 104, padding 0, 64 kb/s (default) Metadata: encoder : Lavc57.54.100 libopus

    However, it appears that ffmpeg has lied, because when analysing the file again, I get:

    Input #0, ogg, from 'output.ogg': Duration: 00:00:03.21, start: 0.000000, bitrate: 89 kb/s Stream #0:0: Audio: opus, 48000 Hz, stereo, s16, delay 156, padding 0 Metadata: ENCODER : Lavc57.54.100 libopus

    I have tried so many permutations of sample rate, simplifying down to a single audio input etc etc - always with the same result.

    Any ideas?

  • opencv read error :[h264 @ 0x8f915e0] error while decoding MB 53 20, bytestream -7

    2 mai, par Alex Luya

    My configuration:

      ubuntu 16.04
  opencv 3.3.1
  gcc version 5.4.0 20160609
  ffmpeg version 3.4.2-1~16.04.york0

    and I built opencv with:

    cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D PYTHON_EXECUTABLE=$(which python) -D OPENCV_EXTRA_MODULES_PATH=/home/xxx/opencv_contrib/modules -D WITH_QT=ON -D WITH_OPENGL=ON -D WITH_IPP=ON -D WITH_OPENNI2=ON -D WITH_V4L=ON -D WITH_FFMPEG=ON -D WITH_GSTREAMER=OFF -D WITH_OPENMP=ON -D WITH_VTK=ON -D BUILD_opencv_java=OFF -D BUILD_opencv_python3=OFF -D WITH_CUDA=ON -D ENABLE_FAST_MATH=1 -D WITH_NVCUVID=ON -D CUDA_FAST_MATH=ON -D BUILD_opencv_cnn_3dobj=OFF -D FORCE_VTK=ON  -D WITH_CUBLAS=ON -D CUDA_NVCC_FLAGS="-D_FORCE_INLINES" -D WITH_GDAL=ON -D WITH_XINE=ON -D BUILD_EXAMPLES=OFF -D BUILD_DOCS=ON -D BUILD_PERF_TESTS=OFF -D BUILD_TESTS=OFF  -D BUILD_opencv_dnn=OFF -D BUILD_PROTOBUF=OFF -D opencv_dnn_BUILD_TORCH_IMPORTER=OFF -D opencv_dnn_PERF_CAFFE=OFF -D opencv_dnn_PERF_CLCAFFE=OFF -DBUILD_opencv_dnn_modern=OFF -D CUDA_ARCH_BIN=6.1 ..

    and use these python code to read and show:

    import cv2
from com.xxx.cv.core.Image import Image

capture=cv2.VideoCapture("rtsp://192.168.10.184:554/mpeg4?username=xxx&password=yyy")
while True:
    grabbed,content=capture.read()
    if grabbed:
        Image(content).show()
        doSomething()
    else:
        print "nothing grabbed.."

    Everytime, after reading about 50 frames,it will give an error like:

    [h264 @ 0x8f915e0] error while decoding MB 53 20, bytestream -7

    then nothing can be grabbed further,and the strange thing is:

    1,comment doSomething() or
2,keep doSomething() and recording the stream from same IPCamera,then run
  code against recorded video

    both cases,code works fine,can anyone tell how to solve this problem?Thank in advance!

  • ffmpeg.wasm in Angular 19

    2 mai, par Yashar Tabrizi

    I am developing an Angular app that records videos. Since the videos that come out usually have variable and "wrong" framerates, I want to re-encode them using FFmpeg, particularly ffmpeg.wasm.

    I have installed the packages @ffmpeg/ffmpeg, @ffmpeg/core and @ffmpeg/util and I have written the following worker ffmpeg.worker.ts to do the initialization and to execute the FFmpeg processing:

    /// 
import { FFmpeg } from '@ffmpeg/ffmpeg';
import { toBlobURL } from '@ffmpeg/util';


const baseURL = 'https://unpkg.com/@ffmpeg/core@0.12.10/dist/esm';

const ffmpeg = new FFmpeg();

let isLoaded = false;

(async () => {
  await ffmpeg.load({
    coreURL: await toBlobURL(`${baseURL}/ffmpeg-core.js`, "text/javascript"),
    wasmURL: await toBlobURL(`${baseURL}/ffmpeg-core.wasm`, "application/wasm"),
  });
  isLoaded = true;
  self.postMessage({ type: 'ready' });
})();

self.onmessage = async (e: MessageEvent) => {
  if (!isLoaded) {
    self.postMessage({ type: 'error', error: 'FFmpeg not loaded yet!' });
    return;
  }

  if (e.data.byteLength === 0) return;

  try {
    await ffmpeg.writeFile('input', new Uint8Array(e.data));

    await ffmpeg.exec([
      '-i', 'input',
      '-r', '30',
      '-c:v', 'libx264',
      '-preset', 'ultrafast',
      '-pix_fmt', 'yuv420p',
      '-movflags', 'faststart',
      'out.mp4',
    ]);

    const data = await ffmpeg.readFile('out.mp4');
    if (data instanceof Uint8Array) {
      self.postMessage({ type: 'done', file: data.buffer }, [data.buffer]);
    } else {
      self.postMessage({ type: 'error', error: 'Unexpected output from ffmpeg.readFile,' });
    }

  } catch (err) {
    self.postMessage({ type: 'error', error: (err as Error).message });
  } finally {
    await ffmpeg.deleteFile(('input'));
    await ffmpeg.deleteFile(('out.mp4'));
  }
}

    I have a service called cameraService where I do the recording and where I want to do the re-encoding after the recording has stopped, so I have this method that initializes the FFmpeg process:

      private encoder: Worker | null = null;

  private initEncoder() {
    if (this.encoder) return;
    this.encoder = new Worker(
      new URL('../workers/ffmpeg.worker', import.meta.url), // Location of my worker
      { type: 'module' }
    );

    this.encoder.onmessage = (e: MessageEvent) => {
      switch (e.data.type) {
        case 'ready':
          console.log('FFmpeg worker ready.');
          break;

        case 'done':
          this.reEncodedVideo = new Blob([e.data.file], { type: 'video/mp4' });
          this.videoUrlSubject.next(URL.createObjectURL(this.reEncodedVideo));
          console.log('FFmpeg encoding completed.');
          break;
        case 'error':
          console.error('FFmpeg encoding error:', e.data.error);
          break;
      }
    };
  }

    However, the loading of FFmpeg won't work, no matter what I do. Hosting the ffmpeg-core.js and ffmpeg-core.wasm files doesn't help either. I keep getting this message whenever ffmpeg.load() is called:

    The file does not exist at ".../.angular/cache/19.2.0/mover/vite/deps/worker.js?worker_file&type=module" which is in the optimize deps directory. The dependency might be incompatible with the dep optimizer. Try adding it to 'optimizeDeps.exclude'.

    I know this has something to do with Web Workers and their integration with Vite but has anybody been able to implement ffmpeg.wasm in Angular 19 or is there even any way to achieve this? If not FFmpeg, are there alternatives to perform re-encoding after recording a video in Angular 19?

Navigation

Sites Web