Recherche avancée

Sur d’autres sites (6110)

• FFMPEG : Controlling overlay through ZeroMQ in ffmpeg

  9 juillet 2020, par Michel

  For a project we need to manually control an overlay (static image) on top of a video stream. We concluded that we will need ZeroMQ to control this, and we've managed to control an audio filter successfully through ZeroMQ.

  


  To achieve this we've considered two options :

  


  

  1. Use a color filter to control the foreground (image) transparency

  


  2. "Move" the overlay image in front of the video stream, and remove it out of the video canvas

  


  


  We've managed to get both working, but not in combination with ZeroMQ, unfortunately.

  


  Example :

  


  ffmpeg -loglevel verbose    \
              -i $STREAM     \
              -i overlay.png  \
          -bsf:a aac_adtstoasc \
        -threads 12             \
             -vf "zmq=bind_address=tcp\\\://127.0.0.1\\\:1235, [0:v][1:v]overlay=25:25" \
            -c:a aac              \
             -af "azmq=bind_address=tcp\\\://127.0.0.1\\\:1235,volume=1" \
            -c:v h264 output


  


  The above example produces the following error :

  


  Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument


  


  Advice on how this can be achieved is highly appreciated !

  


• How to save audio chunks from client to ffmpeg readable file ?

  22 septembre 2023, par LuckOverflow

  I am live recording audio data from a TS React front-end and need to send it to the server, where it can be saved to a file so that ffmpeg can mix it. The front-end saves the mic data to a blob with type "mimeType : "audio/webm ; codecs=opus" when printed in the browser terminal. I send the exact object that I printed to the server, where logging it indicates it is a, or was passed as a, "Buffer" object.

  


  I have tried saving that Buffer as a webm file, but when I pass that file as an input to ffmpeg ffprobe, I get the error "Format matroska,webm detected only with a low score of 1..." and "EBML header parsing failed.." "Invalid data found when processing input." I have tried several other formats to no success.

  


  I need a way to transform this Buffer object to an audio file that can be mixed by ffmpeg. When I am finished, I also need to be able to do the reverse operation to send it in the same format to another client for playback, which is currently working.

  


  Code that records and sends the audio (TS React) :

  


  React Record

  


  const startRecording = async function () {
    inputStream = await navigator.mediaDevices.getUserMedia({ audio: true });
   
    mediaRecorder.current = new MediaRecorder(inputStream, { mimeType: "audio/webm; codecs=opus" });

    mediaRecorder.current.ondataavailable = e => {
      console.log(e.data)
      if (e.data.size > 0) {
        socket.emit("recording", e.data);
        console.log("Audio data recorded. Transmitting to server via socketio...");
      }
    };

    mediaRecorder.current.start(1000);
  };



  


  Code that receives and tries to save the Buffer to a file (JS Node.js) :

  


  Server Receive

  


  socket.on("recording", (chunk) => {
    console_log("Audio chunk recieved. Transmitting to frontend...");
    socket.broadcast.emit('listening', chunk);

    fs.writeFileSync('out.webm', chunk.toString());
    if (counter > 3) {
      console.log("Trying ffmpeg...");

      ffmpegInstance
        .input('out.webm')
        .complexFilter([
          {
            filter: 'amix'
          }])
        .save('./Music/FFMPEGSTREAM.mp3');
    }

    counter++;
  });


  


  fluent-ffmpeg interface package is includued in the server code, but I have been using ffmpeg in the terminal (Pop OS) to debug. The goal is to save the file to a ram disk and use fluent ffmpeg to mix before sending to a different client for playback. Currently I am just trying to save it to disk and get ffmpeg command line to work on it.

  


  Update :
Problem was that the chunk I was analyzing didn't have the header info. MediaRecorder encodes, then slices it up, not slices it up into your specified time slot and encodes. I have not found a good solution to this. Saving the file, without toString I believe, results in a playable webm when the header is properly included.

  


• Audio PTS is not equal but play time is equal

  20 juillet 2023, par KaGa Wu

  I have a video which has an dubbed Portuguese audio stream. The stream does not have the video head, like BGM of companies playing logos. So, the audio stream to delay some time play, it's first data package PTS is 9s. Then I transcode the audio stream to AAC, it's first package PTS is 2s. In my thinking the audio stream put the first data package in more front, it will play in more front originally, but it plays delay time is right !

  


  Here is the first package PTS, origin and transcoded.

  


  origin

  


  https://files.videohelp.com/u/302682/025_sisu_hevc_1920x800_30s_yuv420p_aac_copy.ts

  


  0x00458B74 Transport Packet { PID = 0x101, Payload = Yes (182), Counter = 15, Start indicator }
Adaptation Field ():
adaptation_field_length = 1
discontinuity_indicator = 0
random_access_indicator = 1
elementary_stream_priority_indicator = 0
PCR_flag = 0
OPCR_flag = 0
splicing_point_flag = 0
transport_private_data_flag = 0
adaptation_field_extension_flag = 0

0x00458B7A PES Packet { stream_id = 0xC0  (audio stream)}
packet_length = 2898
PES_scrambling_control = 0
PES_priority = 0
data_alignment_indicator = 0
copyright = 0
original_or_copy = 0
PTS_DTS_flags = 2
ESCR_flag = 0
ES_rate_flag = 0
DSM_trick_mode_flag = 0
additional_copy_info_flag = 0
PES_CRC_flag = 0
PES_extension_flag = 0
PES_header_data_length = 5
 PTS =  0:  0:  9: 570 (861 300)

0x00458B88 AAC Frame
id = 0
layer = 0
protection_absent = 1
profile = 1
sf_index = 3
private_bit = 0
channel_configuration = 6
original = 0
home = 0
copyright_identification_bit = 0
copyright_identification_start = 0
aac_frame_length = 285
adts_buffer_fullness = 2047
no_raw_data_blocks_in_frame = 0
SamplingRate = 48000
Channels = 6
Duration = 0.021333


  


  transcoded

  


  https://files.videohelp.com/u/302682/025_susi_hevc_1920x800_30s_yuv420p_aac_trans.ts

  


  0x000010E4 Transport Packet { PID = 0x101, Payload = Yes (182), Counter = 0, Start indicator }
Adaptation Field ():
adaptation_field_length = 1
discontinuity_indicator = 0
random_access_indicator = 1
elementary_stream_priority_indicator = 0
PCR_flag = 0
OPCR_flag = 0
splicing_point_flag = 0
transport_private_data_flag = 0
adaptation_field_extension_flag = 0

0x000010EA PES Packet { stream_id = 0xC0  (audio stream)}
packet_length = 467
PES_scrambling_control = 0
PES_priority = 0
data_alignment_indicator = 0
copyright = 0
original_or_copy = 0
PTS_DTS_flags = 2
ESCR_flag = 0
ES_rate_flag = 0
DSM_trick_mode_flag = 0
additional_copy_info_flag = 0
PES_CRC_flag = 0
PES_extension_flag = 0
PES_header_data_length = 5
 PTS =  0:  0:  1: 400 (126 000)

0x000010F8 AAC Frame
id = 0
layer = 0
protection_absent = 1
profile = 1
sf_index = 3
private_bit = 0
channel_configuration = 6
original = 0
home = 0
copyright_identification_bit = 0
copyright_identification_start = 0
aac_frame_length = 43
adts_buffer_fullness = 2047
no_raw_data_blocks_in_frame = 0
SamplingRate = 48000
Channels = 6
Duration = 0.021333


  


  I decode the origin video and transcoded video，and print screen of them.