Recherche avancée

Sur d’autres sites (7271)

• Stream microphone from client browser to remote server and pass audio in real time to ffmpeg to combine with a second video source

  4 mai 2021, par fakeguybrushthreepwood

  As a beginner at working with these kinds of real-time streaming services, I've spent hours trying to work out how this is possible, but can't seem to work out I'd precisely go about it.

  


  I'm prototyping a personal basic web app that does the following :

  


  


  1. In a web browser, the web application has a button that says 'Stream Microphone' - when pressed it streams the audio from the user's microphone (the user obviously has to consent to give permission to send their microphone audio) through to the server which I was presuming would be running node.js (no specific reason at this point, just thought this is how I'd go about doing it).

     


  


  2. The server receives the audio close enough to real-time somehow (not sure how I'd do this).

     


  


  3. I can then run ffmpeg on the command line and take the real-time audio coming in real-time and add it as the sound to a video file (let's just say I'm going to play testmovie.mp4) that I want to play.

     


  


  


  I've looked at various solutions - such as maybe using WebRTC, RTP/RTSP, Piping audio into ffmpeg, Gstreamer, Kurento, Flashphoner and/or Wowza - but somehow they look overly complicated and usually seem to focus on video along with audio. I just need to work with audio.

  


• Extended client ownership of MediaCodec encoder output buffers for RTMP streaming

  13 février 2014, par dbro

  Background :

  I've connected Android's MediaCodec to FFmpeg for muxing a variety of formats not supported by MediaMuxer, including rtmp:// output via a .flv container. Such streaming muxers require longer, unpredictable ownership of MediaCodec's output buffers, as they may perform networking I/O on any packet processing step. For my video stream, I'm using MediaCodec configured for Surface input. To decouple muxing from encoding, I queue MediaCodec's ByteBuffer output buffers to my muxer via a Handler.

  All works splendidly if I mux the .flv output to file, rather than rtmp endpoint.

  Problem :

  When muxing to rtmp://... endpoint I notice my streaming application begins to block on calls to eglSwapBuffers(mEGLDisplay, mEncodingEGLSurface) at dequeueOutputBuffer() once I'm retaining even a few MediaCodec output buffers in my muxing queue as MediaCodec seems to be locked to only 4 output buffers.

  Any tricks to avoid copying all encoder output returned by MediaCodec#dequeueOutputBuffers and immediately calling releaseOutputBuffer(...) ?

  The full source of my project is available on Github. Specifically, see :

  • AndroidEncoder.java : Abstract Encoder class with shared behavior between Audio and Video encoders : mainly drainEncoder(). Writes data to a Muxer instance.
  • FFmpegMuxer.java : Implements Muxer
  • CameraEncoder.java. Sends camera frames to an AndroidEncoder subclass configured for Video encoding.

  Systrace

  

  Here's some systrace output streaming 720p @ 2Mbps video to Zencoder.

  Solved

  Copying then releasing the MediaCodec encoder output ByteBuffers as soon as they're available solves the issue without significantly affecting performance. I recycle the ByteBuffer copies in an ArrayDeque<bytebuffer></bytebuffer> for each muxer track, which limits the number of allocations.

• flv reencode to mp4 for iphone/ipod via ffmpeg and x264 (quality issue)

  3 octobre 2011, par zeroasterisk

  There are a lot of questions on this topic, and I've read most of them and most of the google search results I could come up with.

  When I use FFMPEG to convert a FLV to a iphone3 compatble MP4 file, it just doesn't preserver enough of the quality. Yes, I've worked the hell out of -sameq and -b and -bt settings, text just isn't readable.

  Next I tried to split the video out and process it directly, using these instructions :
  https://sites.google.com/site/linuxencoding/x264-encoding-guide

  The problem is myplayer (via ffmpeg) was not able to determine the duration of the FLV (even though the metadata was set).

  (I assume) Because of that unknown duration, when I create the MP4 file, the resulting x264 file plays through super-fast while the audio plays at the normal rate.

  user@server:/tmp# mplayer -nosound -benchmark -sws 9 -vf dsize=640:480:0,scale=0:0,expand=640:480 -vo yuv4mpeg:file=>(x264 --demuxer y4m --crf 0 --preset slow --threads auto --output output.264 - 2>x264.log) 'input.flv'
  
  MPlayer 1.0rc4-4.4.5 (C) 2000-2010 MPlayer Team
  
  mplayer: could not connect to socket
  
  mplayer: No such file or directory
  
  Failed to open LIRC support. You will not be able to use your remote control.
  
  
  
  Playing input.flv.
  
  libavformat file format detected.
  
  [flv @ 0x1202460]Estimating duration from bitrate, this may be inaccurate
  
  [lavf] stream 0: video (vp6f), -vid 0
  
  [lavf] stream 1: audio (nellymoser), -aid 0
  
  VIDEO:  [VP6F]  1680x992  0bpp  1000.000 fps   33.4 kbps ( 4.1 kbyte/s)
  
  Clip info:
  
   audiocodecid: 6
  
   audiodatarate: 86
  
   audiosamplerate: 44100
  
   audiosamplesize: 16
  
   audiosize: 6097005
  
   canSeekToEnd: true
  
   datasize: 8609138
  
   duration: 567
  
   framerate: 2
  
   hasAudio: true
  
   hasCuePoints: false
  
   hasKeyframes: true
  
   hasMetadata: true
  
   hasVideo: true
  
   height: 992
  
   lasttimestamp: 567
  
   metadatacreator: flvtool++ (Facebook, Motion project, dweatherford)
  
   stereo: false
  
   totalframes: 1043
  
   videocodecid: 4
  
   videodatarate: 33
  
   videosize: 2316256
  
   width: 1680
  
  Using (default) progressive frame mode.Opening video filter: [expand w=640 h=480]
  
  Expand: 640 x 480, -1 ; -1, osd: 0, aspect: 0.000000, round: 1
  
  Opening video filter: [scale w=0 h=0]
  
  Opening video filter: [dsize=640:480:0]
  
  ==========================================================================
  
  Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
  
  Selected video codec: [ffvp6f] vfm: ffmpeg (FFmpeg VP6 Flash)
  
  ==========================================================================
  
  Audio: no sound
  
  Starting playback...
  
  Movie-Aspect is undefined - no prescaling applied.
  
  [swscaler @ 0x7f0c738b9620]Lanczos scaler, from yuv420p to yuv420p using MMX2
  
  VO: [yuv4mpeg] 640x480 => 641x480 Planar YV12

  I have also tried specifying FPS, but no change in results

  user@server:/tmp# mplayer -nosound -fps 25-benchmark -sws 9 -vf dsize=640:480:0,scale=0:0,expand=640:480 -vo yuv4mpeg:file=>(x264 --demuxer y4m --fps 25 --crf 0 --preset slow --threads auto --output output.264 - 2>x264.log) 'input.flv'

  Can someone tell me how to either :

  1. fix my split A/V processing/timing/duration issues ?
  2. improve the
     quality of the FFMPEG conversion of FLV to iphone3 compatible
     format ?