Recherche avancée

Sur d’autres sites (9911)

• Run ffmpeg audio under Windows with Flutter

  11 janvier, par Chris

  I'd like to stream audio comming from my microphone in my flutter app in the windows desktop version.

  


  Since there is no library that seems to do such thing while supporting windows desktop app, I have tried using Process like this :

  


  // Start the FFmpeg process to capture audio
ffmpegProcess = await Process.start(
  'ffmpeg',
  [
    '-f', 'dshow', // Specify DirectShow input for Windows
    '-i', 'audio="$selectedMic"', // Input audio device (selected mic)
    '-f', 'wav', // Set audio format
    '-ar', '44100', // Set audio sample rate
    '-ac', '1', // Mono channel
    '-b:a', '128k', // Set audio bitrate
    'pipe:1', // Output to stdout
  ],
);

// Listen for data on stdout (audio stream)
ffmpegProcess.stdout.listen((data) async {
  // Send audio data to the server as it comes in
  await sendAudioToServer(data);
});


  


  I've tested the command directly in my terminal (not in the flutter app) and it works fine.

  


  When I run this code in my Flutter app, my task manager also shows a "ffmpeg" process, but somehow there is no stream output in flutter, even after ensuring that the selectedMic variable is correct or even when its hardcoded.

  


  Since this command runs without issue in my terminal and even in python, I am wondering why it does not work in Flutter.

  


  Starting my vscode as administrator also don't solve the issue (I wanted to check if it's a permission issue).

  


  Also relevant : When I run the "ffmpeg -version" command, I get an output for the version, meaning that this is not an installation problem (ffmpeg bin folder is in my PATH environment variable). The problem seems to come from recording from the microphone in flutter, but I don't get why.

  


  ffmpegProcess = await Process.start('ffmpeg', ['-version']);  // this works


  


  I'd love to get some suggestions where the problem could come from or any kind of alternative solutions.

  


• Custom IO writes only header, rest of frames seems omitted

  18 septembre 2023, par Daniel

  I'm using libavformat to read packets from rtsp and remux it to mp4 (fragmented).

  


  Video frames are intact, meaning I don't want to transcode/modify/change anything.
Video frames shall be remuxed into mp4 in their original form. (i.e. : NALUs shall remain the same).

  


  I have updated libavformat to latest (currently 4.4).

  


  Here is my snippet :

  


  //open input, probesize is set to 32, we don't need to decode anything
avformat_open_input

//open output with custom io
avformat_alloc_output_context2(&ofctx,...);
ofctx->pb = avio_alloc_context(buffer, bufsize, 1/*write flag*/, 0, 0, &writeOutput, 0);
ofctx->flags |= AVFMT_FLAG_NOBUFFER | AVFMT_FLAG_FLUSH_PACKETS | AVFMT_FLAG_CUSTOM_IO;

avformat_write_header(...);

//loop
av_read_frame()
LOGPACKET_DETAILS //<- this works, packets are coming
av_write_frame() //<- this doesn't work, my write callback is not called. Also tried with av_write_interleaved_frame, not seem to work.

int writeOutput(void *opaque, uint8_t *buffer, int buffer_size) {
  printf("writeOutput: writing %d bytes: ", buffer_size);
}


  


  avformat_write_header works, it prints the header correctly.

  


  I'm looking for the reason on why my custom IO is not called after a frame has been read.

  


  There must be some more flags should be set to ask avformat to don't care about decoding, just write out whatever comes in.

  


  More information :
Input stream is a VBR encoded H264. It seems av_write_frame calls my write function only in case an SPS, PPS or IDR frame. Non-IDR frames are not passed at all.

  


  Update

  


  I found out if I request IDR frame at every second (I can ask it from the encoder), writeOutput is called at every second.

  


  I created a test : after a client joins, I requested the encoder to create IDRs @1Hz for 10 times. Libav calls writeOutput at 1Hz for 10 seconds, but then encoder sets itself back to create IDR only at every 10 seconds. And then libav calls writeOutput only at every 10s, which makes my decoder fail. In case 1Hz IDRs, decoder is fine.

  


• Stream OpenGL framebuffer over HTTP (via FFmpeg)

  16 juin 2022, par mOfl

  I have an OpenGL application of which rendered images need to be streamed over internet to mobile clients. Previously, it sufficed to simply record the rendering into a video file, which is already working, and now this should be extended to subsequent streaming.

  



  What is working right now :

  



  

  • Render a scene to an OpenGL framebuffer object

  


  • Capture the FBO content using NvIFR

  


  • Encode it to H.264 using NvENC (no CPU round trip required)

  


  • Download the encoded frame to host memory as a byte array

  


  • Append this frame to a video file

  


  



  None of this steps involves FFmpeg or any other library so far. I now want to replace the last step with "Stream the current frame's byte array over internet" and I assume that using FFmpeg and FFserver would be a reasonable choice for this. Am I correct ? If not, what would be the proper way ?

  



  If so, how do I approach this within my C++ code ? As pointed out, the frame is already encoded. Also, there is no sound or other stuff, simply a H.264 encoded frame as byte array that is updated irregularly and should be converted into a steady video stream. I assume that this would be FFmpeg's job and that the subsequent streaming via FFserver would be simple from there. What I don't know is how to feed my data to FFmpeg in the first place, as all FFmpeg tutorials I found (in a non-exhaustive search) work on a file or webcam/capture device as data source, not volatile data in main memory.

  



  The file mentioned above that I am already able to create is a C++ file stream to which I append each single frame, meaning that different framerates of video and rendering are not treated correctly. This also needs to be taken care of at some point.

  



  Can somebody point me in the right direction ? Can I forward data from my application to FFmpeg to build a proper video feed without writing to the hard disk ? Tutorials are greatly appreciated. By the way FFmpeg/FFserver is not mandatory. If you have a better idea for streaming of OpenGL framebuffer contents, I'm eager to know.