Recherche avancée

Sur d’autres sites (11264)

• Failed to decode HLS by FFMpeg command. Invalid NAL unit 0

  9 mars 2024, par Fyodor Khruschov

  On front-end I create stream with chrome.tabCapture.capture or navigator.mediaDevices.getDisplayMedia methods. Then send chunks generated by MediaRecorder to server. On the server I have FFMpeg command which decodes chunks into .mp4 file. This is the command :

  


  ffmpeg -y -i - -preset veryfast -tune zerolatency -filter_complex [0:v]split=3[v1][v2][v3];[v1]scale=w=-2:h=1080,fps=30[v1out];[v2]scale=w=-2:h=720,fps=30[v2out];[v3]scale=w=-2:h=480,fps=30[v3out] -map [v1out] -maxrate:0 6M -bufsize:0 12M -keyint_min 100 -g 100 -map [v2out] -maxrate:1 3M -bufsize:1 6M -keyint_min 100 -g 100 -map [v3out] -maxrate:2 1.5M -bufsize:2 3M -keyint_min 100 -g 100 -c:v libx264 -map a:0 -c:a:0 aac -b:a:0 128k -ac 2 -map a:0 -c:a:1 aac -b:a:1 96k -map a:0 -c:a:2 aac -b:a:2 96k -f hls -hls_time 2 -hls_playlist_type vod -hls_flags independent_segments+temp_file -hls_segment_type fmp4 -hls_segment_filename ./output/ready/output_%v_%03d.m4s -var_stream_map v:0,a:0 v:1,a:1 v:2,a:2 -master_pl_name master.m3u8 ./output/ready/stream_%v.m3u8 -map 0:v:0 -map 0:a:0 -c:v copy -c:a aac ./output/download/video.mp4 -map 0:a:0 -ar 16000 -ac 1 -c:a pcm_s16le ./output/captions/audio.wav -loglevel info


  


  During the process of decoding I have these errors in logs :

  


  [extract_extradata @ 0x60000264b250] Invalid NAL unit 0, skipping.
[h264 @ 0x13ff04e60] Invalid NAL unit 0, skipping.
[h264 @ 0x13ff04e60] co located POCs unavailable
[h264 @ 0x13ff04e60] negative number of zero coeffs at 17 0
[h264 @ 0x13ff04e60] error while decoding MB 17 0
[h264 @ 0x13ff04e60] concealing 3388 DC, 3388 AC, 3388 MV errors in B frame
[h264 @ 0x13ff04e60] missing picture in access unit with size 24158
[h264 @ 0x13ff04e60] Invalid NAL unit 0, skipping.
[h264 @ 0x13ff04e60] data partitioning is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[h264 @ 0x13ff04e60] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
[h264 @ 0x13ff04e60] no frame!
[h264 @ 0x13ff04e60] Unknown SAR index: 18.
[h264 @ 0x13ff04e60] Invalid NAL unit 0, skipping.
[h264 @ 0x13ff04e60] Unknown SAR index: 18.
[h264 @ 0x13ff04e60] number of reference frames (2+4) exceeds max (5; probably corrupt input), discarding one
[h264 @ 0x13ff04e60] number of reference frames (3+3) exceeds max (5; probably corrupt input), discarding one
[h264 @ 0x13ff04e60] number of reference frames (4+2) exceeds max (5; probably corrupt input), discarding one
[h264 @ 0x13ff04e60] FMO is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[h264 @ 0x13ff04e60] sps_id 4 out of range


  


  This issue is very inconsistent and happen in rare cases (I can't understand the logic). Most of the time chunks decoded successfully, but sometimes not.

  


  How to understand where the issue is coming from ? Is it possible for FFMpeg to skip wrong data and generate mp4 file anyway even with glitches, but don't crush ?

  


• How to save audio chunks from client to ffmpeg readable file ?

  22 septembre 2023, par LuckOverflow

  I am live recording audio data from a TS React front-end and need to send it to the server, where it can be saved to a file so that ffmpeg can mix it. The front-end saves the mic data to a blob with type "mimeType : "audio/webm ; codecs=opus" when printed in the browser terminal. I send the exact object that I printed to the server, where logging it indicates it is a, or was passed as a, "Buffer" object.

  


  I have tried saving that Buffer as a webm file, but when I pass that file as an input to ffmpeg ffprobe, I get the error "Format matroska,webm detected only with a low score of 1..." and "EBML header parsing failed.." "Invalid data found when processing input." I have tried several other formats to no success.

  


  I need a way to transform this Buffer object to an audio file that can be mixed by ffmpeg. When I am finished, I also need to be able to do the reverse operation to send it in the same format to another client for playback, which is currently working.

  


  Code that records and sends the audio (TS React) :

  


  React Record

  


  const startRecording = async function () {
    inputStream = await navigator.mediaDevices.getUserMedia({ audio: true });
   
    mediaRecorder.current = new MediaRecorder(inputStream, { mimeType: "audio/webm; codecs=opus" });

    mediaRecorder.current.ondataavailable = e => {
      console.log(e.data)
      if (e.data.size > 0) {
        socket.emit("recording", e.data);
        console.log("Audio data recorded. Transmitting to server via socketio...");
      }
    };

    mediaRecorder.current.start(1000);
  };



  


  Code that receives and tries to save the Buffer to a file (JS Node.js) :

  


  Server Receive

  


  socket.on("recording", (chunk) => {
    console_log("Audio chunk recieved. Transmitting to frontend...");
    socket.broadcast.emit('listening', chunk);

    fs.writeFileSync('out.webm', chunk.toString());
    if (counter > 3) {
      console.log("Trying ffmpeg...");

      ffmpegInstance
        .input('out.webm')
        .complexFilter([
          {
            filter: 'amix'
          }])
        .save('./Music/FFMPEGSTREAM.mp3');
    }

    counter++;
  });


  


  fluent-ffmpeg interface package is includued in the server code, but I have been using ffmpeg in the terminal (Pop OS) to debug. The goal is to save the file to a ram disk and use fluent ffmpeg to mix before sending to a different client for playback. Currently I am just trying to save it to disk and get ffmpeg command line to work on it.

  


  Update :
Problem was that the chunk I was analyzing didn't have the header info. MediaRecorder encodes, then slices it up, not slices it up into your specified time slot and encodes. I have not found a good solution to this. Saving the file, without toString I believe, results in a playable webm when the header is properly included.

  


• Recording audio with MediaRecorder on iPhone with Safari and Chrome only 1 second long ? Mimetype and FFMPEG problem ?

  9 mai 2023, par Avatar

  I am using MediaRecorder to record the Microphone audio on a website.

  


  Javascript :

  


  var blob;
var blob_url;
var stream;
var recorder;
var chunks;

var media = {
    tag: 'audio',
    type: 'audio/ogg',
    ext: '.ogg',
    gUM: {audio: true}
};

navigator.mediaDevices.getUserMedia(media.gUM).then(_stream => 
{
    stream = _stream;

    recorder = new MediaRecorder(stream);

    recorder.ondataavailable = e => 
    {
        // push data to chunks
        chunks.push(e.data);

        // recording has been stopped
        if(recorder.state == 'inactive') 
        {
            // audio data available
            blob = new Blob(chunks, {type: media.type });
            blob_url = URL.createObjectURL(blob);
            
            // send data to server
            uploadfile_audio();
        }
    };

    if(typeof(recorder)=='undefined')
    {
        alert('No microphone access');
        return;
    }

    chunks = [];
    recorder.start();
}


// when stop button is clicked
recorder.stop();
stream.getTracks().forEach( track => track.stop() );


  


  The audio stream (ogg format) is sent to the server.

  


  Since iPad/iPhone do not play ogg files, the recording file is converted to "mp3" using FFMPEG.

  


  This file is stored on the server.

  

  


  This works on Windows and MAC (Chrome and Safari), also on iPad (Safari) but not properly on iPhone (Chrome/Safari). Version : iPhone iOS 15.1.

  


  On iPhone the recording file is only 0:01 min in length. Size is always 17277 Bytes.

  


  What could be the issue ? (I cannot debug because I don't have a Mac.)

  


  Does the stream get interrupted ? Is the recording stopped after 1 second ?

  


  Update 1 :

  


  I have checked the incoming filesize of the browser-recorded file serverside. It seems to be coming in properly, because there are different sizes such as 184 kB.

  


  My guess is now that FFMPEG cannot handle the incoming file correctly. Which might have the wrong mimetype set in Javascript with type: 'audio/ogg',. Is another format needed ?

  


  The conversion code serverside :

  


  PHP

  


  $mp3file = shell_exec("ffmpeg -i ".$file_locationtmp." -vn -ar 44100 -ac 2 -b:a 128k ".$file_locationtmp.".mp3");


  


  I would need to find out the audio recording format used by iPhone but I couldn't.

  


  I tried to find the supporting mimetypes using https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder/isTypeSupported - however, it shows that NO mimetypes are supported on iPhone (neither in Chrome nor Safari).

  


  Update 2 :

  


  I used ffprobe to get the recording file information. It says Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 2234 kb/s (default)

  


  Update 3 :

  


  It seems to be a problem with FFMPEG. See my new question How to convert AAC/MP4A to MP3 using FFMPEG in full length ? Audio file gets cut off after 1 second