Recherche avancée
Médias (1)
-
Sintel MP4 Surround 5.1 Full
13 mai 2011, par
Mis à jour : Février 2012
Langue : English
Type : Video
Autres articles (86)
-
Websites made with MediaSPIP
2 mai 2011, parThis page lists some websites based on MediaSPIP.
-
Creating farms of unique websites
13 avril 2011, parMediaSPIP platforms can be installed as a farm, with a single "core" hosted on a dedicated server and used by multiple websites.
This allows (among other things) : implementation costs to be shared between several different projects / individuals rapid deployment of multiple unique sites creation of groups of like-minded sites, making it possible to browse media in a more controlled and selective environment than the major "open" (...) -
Le profil des utilisateurs
12 avril 2011, parChaque utilisateur dispose d’une page de profil lui permettant de modifier ses informations personnelle. Dans le menu de haut de page par défaut, un élément de menu est automatiquement créé à l’initialisation de MediaSPIP, visible uniquement si le visiteur est identifié sur le site.
L’utilisateur a accès à la modification de profil depuis sa page auteur, un lien dans la navigation "Modifier votre profil" est (...)
Sur d’autres sites (11017)
-
Decode audio using ffmpeg (packet-by-packet) in Java
27 mai 2022, par quad478In my application, I receive an audio stream from an IP-camera via RTP using NETTY.
The stream from IP-camera comes in the "G.711 mulaw" format, I would like to recode it to the "AAC" format.
I can't use files in this task as it's a live stream, and each packet needs to be decoded and delivered to the client (browser) immediately.
For this task, I wanted to use the ffmpeg child process :
when connecting to the camera, create a ffmpeg process and send to stdin (ffmpeg) each received packet, then I would like to receive the decoded packet from stdout.
Here is the command I run ffmeg with :


"ffmpeg.exe -f mulaw -re -i - -f adts -"


I'm not sure if "-re" should be used here, but without this option, ffmpeg outputs the decode result only after stdin is closed and the process exits.
The problem is that I don't get anything on stdout after sending the packet to ffmpeg.


Decoder code :


package ru.ngslab.insentry.web.video.protocols.rtp;

import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.MessageToMessageDecoder;

import java.io.Closeable;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

public class RtpFfmpegDecoder extends MessageToMessageDecoder<rtppacket> implements Closeable {

 private final Process ffmegProcess;
 private final OutputStream ffmpegOutput;
 private final InputStream ffmegInput;
 private final ExecutorService ffmpegInputReaderService = Executors.newSingleThreadExecutor();

 public RtpFfmpegDecoder() {

 //Start Ffmpeg process
 ProcessBuilder ffmpegBuilder = new ProcessBuilder("ffmpeg.exe", "-f", "mulaw",
 "-re", "-i", "-", "-f", "adts", "-").redirectError(ProcessBuilder.Redirect.INHERIT);
 try {
 ffmegProcess = ffmpegBuilder.start();
 ffmpegOutput = ffmegProcess.getOutputStream();
 ffmegInput = ffmegProcess.getInputStream();
 } catch (IOException e) {
 throw new IllegalStateException(e);
 }
 }

 @Override
 protected void decode(ChannelHandlerContext channelHandlerContext, RtpPacket rtpPacket, List list) throws Exception {

 //start read ffmpeg output in another thread
 Future future = ffmpegInputReaderService.submit(this::startReadFFmpegOutput);

 //write rtp- packet bytes to ffmpeg-input
 ByteBuf data = rtpPacket.getData();
 byte[] rtpData = new byte[data.readableBytes()];
 data.getBytes(data.readerIndex(), rtpData);
 ffmpegOutput.write(rtpData);
 ffmpegOutput.flush();

 //waiting here for the decoding result from ffmpeg
 //blocks here
 byte[] result = future.get();
 //then process result...
 }

 private byte[] startReadFFmpegOutput() {
 try {
 //I don't know how many bytes to expect here, for test purposes I use 1024
 var bytes = new byte[1024];
 ffmegInput.read(bytes);
 return bytes;
 } catch (IOException e) {
 throw new IllegalStateException(e);
 }
 }

 @Override
 public void close() throws IOException {
 //Close streams code...
 }
}
</rtppacket>

This doesn't work because ffmpeg doesn't send anything after receiving the packet.
No errors in log, no output data.
Just wait for result here :


byte[] result = future.get();


Normally, ffmpeg only outputs after stdin is closed and the process stops.
It may be necessary to run ffmpeg with some special
parameters so that it outputs each received packet at once ?


I would be very grateful for any help


-
Build ffmpeg on a build machine
18 juillet 2019, par RDIBuild ffmpeg on build PC using libx264 and shared libraries (not static).
I am building on a Red Hat 6.6 Server and final target machine is CentOS 6.6.
I am trying, as said, to build ffmpeg with encoding enabled (with libx264) and shared libraries ; of course I do not want to install the libraries on the build PC, they should be only extracted and then delivered together with the final RPM.
After the "./configure" I get all RPMs (related to ffmpeg) but when trying to installing ffmpeg-libs on the build pc it fails because the libx264.so.157 is not found, even if as test I installed it (configure/make/make install) and present at /usr/local/lib.Where am I wrong ?
Thanks
This is my SPEC file at the moment :
ldconfig /usr/local/lib
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
# configure
./configure \
--enable-gpl --disable-static --enable-shared --extra-cflags="-I/usr/local/include" --extra-ldflags="-L/usr/local/lib" --extra-libs=-ldl --disable-autodetect --disable-doc --disable-postproc --disable-ffplay --disable-everything --enable-encoder=aac --enable-encoder=png --enable-encoder=mjpeg --enable-encoder=libx264 --enable-decoder=aac --enable-decoder=h264 --enable-decoder=mpeg4 --enable-decoder=rawvideo --enable-decoder=png --enable-muxer=mp4 --enable-muxer=stream_segment --enable-muxer=image2 --enable-demuxer=aac --enable-demuxer=h264 --enable-demuxer=mov --enable-demuxer=rtp --enable-parser=aac --enable-parser=h264 --enable-parser=mpeg4video --enable-bsf=aac_adtstoasc --enable-protocol=file --enable-protocol=http --enable-protocol=tcp --enable-protocol=rtp --enable-protocol=udp --enable-indev=xcbgrab --disable-alsa --enable-libxcb --enable-libxcb-xfixes --enable-libxcb-shape --enable-zlib --prefix=%{_prefix} --bindir=%{_bindir} --datadir=%{_datadir}/%{name} --shlibdir=%{_libdir} --enable-alsa --enable-avfilter --enable-avresample --enable-libx264 --enable-filter=scale \ -
MediaCodec AV Sync when decoding
12 juin 2020, par ClassAAll of the questions regarding syncing audio and video, when decoding using
MediaCodec, suggests that we should use an "AV Sync" mechanism to sync the video and audio using their timestamps.


Here is what I do to achieve this :



I have 2 threads, one for decoding video and one for audio. To sync the video and audio I'm using
Extractor.getSampleTime()to determine if I should release the audio or video buffers, please see below :


//This is called after configuring MediaCodec(both audio and video)
private void startPlaybackThreads(){
 //Audio playback thread
 mAudioWorkerThread = new Thread("AudioThread") {
 @Override
 public void run() {
 if (!Thread.interrupted()) {
 try {
 //Check info below
 if (shouldPushAudio()) {
 workLoopAudio();
 }
 } catch (Exception e) {
 e.printStackTrace();
 }
 }
 }
 };
 mAudioWorkerThread.start();

 //Video playback thread
 mVideoWorkerThread = new Thread("VideoThread") {
 @Override
 public void run() {
 if (!Thread.interrupted()) {
 try {
 //Check info below
 if (shouldPushVideo()) {
 workLoopVideo();
 }
 } catch (Exception e) {
 e.printStackTrace();
 }
 }
 }
 };
 mVideoWorkerThread.start();
}

//Check if more buffers should be sent to the audio decoder
private boolean shouldPushAudio(){
 int audioTime =(int) mAudioExtractor.getSampleTime();
 int videoTime = (int) mExtractor.getSampleTime();
 return audioTime <= videoTime;
}
//Check if more buffers should be sent to the video decoder
private boolean shouldPushVideo(){
 int audioTime =(int) mAudioExtractor.getSampleTime();
 int videoTime = (int) mExtractor.getSampleTime();
 return audioTime > videoTime;
}



Inside
workLoopAudio()andworkLoopVideo()is all myMediaCodeclogic (I decided not to post it because it's not relevant).


So what I do is, I get the sample time of the video and the audio tracks, I then check which one is bigger(further ahead). If the video is "ahead" then I pass more buffers to my audio decoder and visa versa.



This seems to be working fine - The video and audio are playing in sync.





My question :


I would like to know if my approach is correct(is this how we should be doing it, or is there another/better way) ? I could not find any working examples of this(written in java/kotlin), thus the question.




EDIT 1 :



I've found that the audio trails behind the video (very slightly) when I decode/play a video that was encoded using
FFmpeg. If I use a video that was not encoded usingFFmpegthen the video and audio syncs perfectly.


The
FFmpegcommand is nothing out of the ordinary :


-i inputPath -crf 18 -c:v libx264 -preset ultrafast OutputPath



I will be providing additional information below :



I initialize/create
AudioTracklike this :


//Audio
mAudioExtractor = new MediaExtractor();
mAudioExtractor.setDataSource(mSource);
int audioTrackIndex = selectAudioTrack(mAudioExtractor);
if (audioTrackIndex < 0){
 throw new IOException("Can't find Audio info!");
}
mAudioExtractor.selectTrack(audioTrackIndex);
mAudioFormat = mAudioExtractor.getTrackFormat(audioTrackIndex);
mAudioMime = mAudioFormat.getString(MediaFormat.KEY_MIME);

mAudioChannels = mAudioFormat.getInteger(MediaFormat.KEY_CHANNEL_COUNT);
mAudioSampleRate = mAudioFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE);

final int min_buf_size = AudioTrack.getMinBufferSize(mAudioSampleRate, (mAudioChannels == 1 ? AudioFormat.CHANNEL_OUT_MONO : AudioFormat.CHANNEL_OUT_STEREO), AudioFormat.ENCODING_PCM_16BIT);
final int max_input_size = mAudioFormat.getInteger(MediaFormat.KEY_MAX_INPUT_SIZE);
mAudioInputBufSize = min_buf_size > 0 ? min_buf_size * 4 : max_input_size;
if (mAudioInputBufSize > max_input_size) mAudioInputBufSize = max_input_size;
final int frameSizeInBytes = mAudioChannels * 2;
mAudioInputBufSize = (mAudioInputBufSize / frameSizeInBytes) * frameSizeInBytes;

mAudioTrack = new AudioTrack(AudioManager.STREAM_MUSIC,
 mAudioSampleRate,
 (mAudioChannels == 1 ? AudioFormat.CHANNEL_OUT_MONO : AudioFormat.CHANNEL_OUT_STEREO),
 AudioFormat.ENCODING_PCM_16BIT,
 AudioTrack.getMinBufferSize(mAudioSampleRate, mAudioChannels == 1 ? AudioFormat.CHANNEL_OUT_MONO : AudioFormat.CHANNEL_OUT_STEREO, AudioFormat.ENCODING_PCM_16BIT),
 AudioTrack.MODE_STREAM);

try {
 mAudioTrack.play();
} catch (final Exception e) {
 Log.e(TAG, "failed to start audio track playing", e);
 mAudioTrack.release();
 mAudioTrack = null;
}



And I write to the
AudioTracklike this :


//Called from within workLoopAudio, when releasing audio buffers
if (bufferAudioIndex >= 0) {
 if (mAudioBufferInfo.size > 0) {
 internalWriteAudio(mAudioOutputBuffers[bufferAudioIndex], mAudioBufferInfo.size);
 }
 mAudioDecoder.releaseOutputBuffer(bufferAudioIndex, false);
}

private boolean internalWriteAudio(final ByteBuffer buffer, final int size) {
 if (mAudioOutTempBuf.length < size) {
 mAudioOutTempBuf = new byte[size];
 }
 buffer.position(0);
 buffer.get(mAudioOutTempBuf, 0, size);
 buffer.clear();
 if (mAudioTrack != null)
 mAudioTrack.write(mAudioOutTempBuf, 0, size);
 return true;
}



"NEW" Question :



The audio trails about 200ms behind the video if I use a video that was encoded using
FFmpeg, is there a reason why this could be happening ?

Syndiquer tout le site
