Recherche avancée
Médias (1)
-
Rennes Emotion Map 2010-11
19 octobre 2011, par
Mis à jour : Juillet 2013
Langue : français
Type : Texte
Autres articles (37)
-
Contribute to a better visual interface
13 avril 2011MediaSPIP is based on a system of themes and templates. Templates define the placement of information on the page, and can be adapted to a wide range of uses. Themes define the overall graphic appearance of the site.
Anyone can submit a new graphic theme or template and make it available to the MediaSPIP community. -
Qu’est ce qu’un éditorial
21 juin 2013, parEcrivez votre de point de vue dans un article. Celui-ci sera rangé dans une rubrique prévue à cet effet.
Un éditorial est un article de type texte uniquement. Il a pour objectif de ranger les points de vue dans une rubrique dédiée. Un seul éditorial est placé à la une en page d’accueil. Pour consulter les précédents, consultez la rubrique dédiée.
Vous pouvez personnaliser le formulaire de création d’un éditorial.
Formulaire de création d’un éditorial Dans le cas d’un document de type éditorial, les (...) -
Dépôt de média et thèmes par FTP
31 mai 2013, parL’outil MédiaSPIP traite aussi les média transférés par la voie FTP. Si vous préférez déposer par cette voie, récupérez les identifiants d’accès vers votre site MédiaSPIP et utilisez votre client FTP favori.
Vous trouverez dès le départ les dossiers suivants dans votre espace FTP : config/ : dossier de configuration du site IMG/ : dossier des média déjà traités et en ligne sur le site local/ : répertoire cache du site web themes/ : les thèmes ou les feuilles de style personnalisées tmp/ : dossier de travail (...)
Sur d’autres sites (3462)
-
Why does changing the sample rate of my AudioEngine cause video/audio to be out of sync ?
5 mars 2024, par ZeunO8I am playing back a video file with audio. The video files' audio sample rate is 44100, if I set my AudioEngines' sample rate to 44100 the video & audio stay in sync when playing back, however if I change my AudioEngines' sample rate to something higher (like 48000), the video and audio are out of sync (with the video 1+seconds in-front of the audio)


I am opening Video and Audio codec contexts like so :


void Video::openVideo()
{
 const AVCodec *videoCodec = 0;
 AVCodecParameters *videoCodecParameters = 0;

 for (int i = 0; i < formatContext->nb_streams; i++)
 {
 if (formatContext->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO)
 {
 videoCodecParameters = formatContext->streams[i]->codecpar;
 videoCodec = avcodec_find_decoder(videoCodecParameters->codec_id);
 videoStreamIndex = i;
 break;
 }
 }

 if (videoStreamIndex == -1)
 {
 Logger(LogType::ERROR, "%s\n", "Video::openVideo: videoStreamIndex is -1, indicating no video stream available");
 return;
 }

 videoStream = formatContext->streams[videoStreamIndex];
 if (!videoStream)
 {
 Logger(LogType::ERROR, "%s\n", "Could not retrieve video stream.");
 return;
 }

 AVRational frameRate = av_guess_frame_rate(formatContext, videoStream, NULL);

 framesPerSecond = frameRate.num / static_cast<double>(frameRate.den);

 int64_t duration = formatContext->duration;
 totalVideoTimeSeconds = (double)duration / AV_TIME_BASE;

 videoFrameCount = totalVideoTimeSeconds * framesPerSecond;

 uInteger64 frameWidth = videoCodecParameters->width,
 frameHeight = videoCodecParameters->height;

 textures.push_back({ReleaseType::Delete, new Texture(frameWidth, frameHeight, ETextureFormat::RGB8, ETextureType::UnsignedByte), 1});
 texturePointer = textures._data[0].pointer;

 videoCodecContext = avcodec_alloc_context3(videoCodec);
 avcodec_parameters_to_context(videoCodecContext, videoCodecParameters);

 if (avcodec_open2(videoCodecContext, videoCodec, NULL) < 0)
 {
 Logger(LogType::ERROR, "%s\n", "Video::openVideo(avcodec_open2): error");
 return;
 }

 videoSwsContext = sws_getContext(videoCodecContext->width, videoCodecContext->height, videoCodecContext->pix_fmt,
 videoCodecContext->width, videoCodecContext->height, AV_PIX_FMT_RGB24,
 SWS_BILINEAR, 0, 0, 0);

 rgbBufferSize = av_image_get_buffer_size(AV_PIX_FMT_RGB24, videoCodecContext->width, videoCodecContext->height, 1);
 rgbBuffer = (uInteger8 *)av_malloc(rgbBufferSize * sizeof(uInteger8));

 videoFrame = av_frame_alloc();
 rgbFrame = av_frame_alloc();

 av_image_fill_arrays(rgbFrame->data, rgbFrame->linesize, rgbBuffer, AV_PIX_FMT_RGB24, videoCodecContext->width, videoCodecContext->height, 1);
};
/*
 */
void Video::openAudio()
{
 const AVCodec *audioCodec = 0;
 AVCodecParameters *audioCodecParameters = 0;

 for (int i = 0; i < formatContext->nb_streams; i++)
 {
 if (formatContext->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO)
 {
 audioCodecParameters = formatContext->streams[i]->codecpar;
 audioCodec = avcodec_find_decoder(audioCodecParameters->codec_id);
 audioStreamIndex = i;
 break;
 }
 }

 if (audioStreamIndex == -1)
 {
 Logger(LogType::ERROR, "%s\n", "Video::openAudio: audioStreamIndex is -1, indicating no audio stream available");
 return;
 }
 audioStream = formatContext->streams[audioStreamIndex];

 audioCodecContext = avcodec_alloc_context3(audioCodec);
 avcodec_parameters_to_context(audioCodecContext, audioCodecParameters);

 if (avcodec_open2(audioCodecContext, audioCodec, NULL) < 0)
 {
 // Handle error
 return;
 }

 auto AV_CH = MAToAV_ChannelLayout();
 auto AV_SAMPLE_FMT = MAToAV_SampleFormat();
 DEFINE_AUDIO_ENGINE

 audioSwrContext = swr_alloc_set_opts(NULL,
 AV_CH,
 AV_SAMPLE_FMT,
 audioEngine.sampleRate,
 audioCodecContext->channel_layout,
 audioCodecContext->sample_fmt,
 audioCodecContext->sample_rate,
 0, NULL);

 if (!audioSwrContext || swr_init(audioSwrContext) < 0)
 {
 // Handle error
 return;
 }

 audioFrame = av_frame_alloc();
 audioSwrFrame = av_frame_alloc();
 audioSwrFrame->format = AV_SAMPLE_FMT;
 audioSwrFrame->channel_layout = AV_CH;
 audioSwrFrame->sample_rate = audioEngine.sampleRate;
 audioSwrFrame->nb_samples = audioEngine.frameSize;
};
</double>

Note I set up a SwrContext for resampling the audio from its in sample rate to its out sample rate.


I process demuxed audio packets in my audio callback like so :


void Video::outputFrames(Floating32 *frames, const Integer32 &channelCount, const unsigned long &frameCount, const AudioTime_t &time)
{
 if (firstAudioFrame)
 {
 firstAudioFrame_us = (uv_hrtime() - startTimeNs) / 1'000.0L;
 Logger(LogType::INFO, "First Audio Frame: %Lfs\n", firstAudioFrame_us / 1'000'000.0L);
 Logger(LogType::INFO, "AudioStream->StartTime: %li\n", audioStream->start_time);
 firstAudioFrame = false;
 }
 Boolean audioFrameDecoded = false;
 uInteger32 outputSamples = 0;
 uInteger32 frameIndex = 0;
 {
 std::lock_guard lock(audioFrameMutex);
 while (outputSamples < frameCount * channelCount && !audioFrameQueue.empty())
 {
 frames[frameIndex++] = audioFrameQueue.front();
 outputSamples++;
 audioFrameQueue.pop();
 }
 }
 Floating128 packetTimestamp = 0;
 Boolean currentAudioTimeSet = false;
 while (outputSamples < frameCount * channelCount)
 {
 if (!audioQueue.empty())
 {
 auto packet = audioQueue.pop();
 if (avcodec_send_packet(audioCodecContext, packet) == 0)
 {
 while (avcodec_receive_frame(audioCodecContext, audioFrame) == 0)
 {
 swr_convert_frame(audioSwrContext, audioSwrFrame, audioFrame);
 
 int64_t pts = av_rescale_q(audioFrame->pts, audioStream->time_base, {1, AV_TIME_BASE});

 if (justSeekedAudio &&
 !Common::isWithinDistance<floating128>((Floating128)pts / 1'000'000.0L, seekingToTimeSeconds, 0.05))
 {
 break;
 }
 else if (justSeekedAudio)
 {
 justSeekedAudio = false;
 }

 if (!currentAudioTimeSet)
 {
 currentAudioTimeSeconds = (Floating128)audioPts / 1'000'000.0L;
 audioPts = pts;
 currentAudioTimeSet = true;
 }

 Integer32 numSamples = audioSwrFrame->nb_samples * channelCount;
 Integer32 oNumSamples = numSamples;
 if (outputSamples + numSamples > frameCount * channelCount)
 {
 numSamples = frameCount * channelCount - outputSamples;
 }
 Integer32 remainingSamples = oNumSamples - numSamples;
 {
 std::lock_guard lock(audioFrameMutex);
 for (uInteger32 sampleIndex = numSamples; sampleIndex < oNumSamples; sampleIndex++)
 {
 audioFrameQueue.push(((Floating32 *)audioSwrFrame->data[0])[sampleIndex]);
 }
 }
 memcpy(frames + outputSamples, audioSwrFrame->data[0], numSamples * sizeof(Floating32));
 outputSamples += numSamples;

 if (outputSamples >= frameCount * channelCount)
 {
 audioFrameDecoded = true;
 break;
 }
 }
 }
 av_packet_unref(packet);
 av_packet_free(&packet);
 }
 else
 {
 std::fill(frames + outputSamples, frames + frameCount * channelCount, 0.0f);
 break;
 }
 }

 if (currentAudioTimeSeconds >= totalVideoTimeSeconds)
 {
 DEFINE_AUDIO_ENGINE
 audioEngine.pipeline.stages.remove(stagePointer);
 }
};
</floating128>

I am handling rendering of frames in an onRender callback (Video IEntity) like so :


Boolean Video::renderCallback()
{
 if (firstVideoFrame)
 {
 Logger(LogType::INFO, "First Video Frame: %Lfs\n", (uv_hrtime() - startTimeNs) / 1'000'000'000.0L);
 Logger(LogType::INFO, "VideoStream->StartTime: %li\n", videoStream->start_time);
 firstVideoFrame = false;
 }
 int64_t closestPts = -1;
 {
 std::lock_guard lock(videoFramesMutex);

 std::sort(videoFrames.begin(), videoFrames.end(), [&](const auto &a, const auto &b)
 { return a->pts < b->pts; });

 auto smallestDiff = std::numeric_limits<floating128>::max();
 AVFrame *closestFrame = 0;
 Integer32 closestVideoFrameIndex = -1;
 Integer32 _videoFrameIndex = 0;
 auto videoFramesSize = videoFrames.size();
 for (;_videoFrameIndex < videoFramesSize; _videoFrameIndex++)
 {
 auto &videoFrame = videoFrames[_videoFrameIndex];
 int64_t videoPts = av_rescale_q(videoFrame->pts, videoStream->time_base, AV_TIME_BASE_Q);
 Floating128 diff = (audioPts - firstAudioFrame_us) - videoPts ;
 if (diff >= 0 && diff < smallestDiff)
 {
 smallestDiff = diff;
 closestFrame = videoFrame;
 closestPts = videoPts;
 closestVideoFrameIndex = _videoFrameIndex;
 }
 }
 auto nextAVFrame = closestFrame;
 videoFrameIndex = closestVideoFrameIndex;

 if (nextAVFrame == currentAVFrame || !nextAVFrame)
 {
 return false;
 }

 currentAVFrame = nextAVFrame;

 sws_scale(videoSwsContext, (uint8_t const *const *)currentAVFrame->data, currentAVFrame->linesize, 0, videoCodecContext->height,
 rgbFrame->data, rgbFrame->linesize);
 }

 // flip frame vertically
 int bytesPerPixel = 3;

 int rowBytes = videoCodecContext->width * bytesPerPixel;

 uint8_t *tempRow = new uint8_t[rowBytes];

 for (int i = 0; i < videoCodecContext->height / 2; ++i)
 {
 uint8_t *rowTop = rgbBuffer + i * rowBytes;
 uint8_t *rowBottom = rgbBuffer + (videoCodecContext->height - i - 1) * rowBytes;

 memcpy(tempRow, rowTop, rowBytes);
 memcpy(rowTop, rowBottom, rowBytes);
 memcpy(rowBottom, tempRow, rowBytes);
 }

 delete[] tempRow;

 texturePointer->update(rgbBuffer);

 if (videoFrameIndex >= framesPerSecond * 2)
 {
 std::lock_guard lock(videoFramesMutex);
 videoFrames.erase(videoFrames.begin(), videoFrames.begin() + framesPerSecond * 2);
 videoFrameIndex -= framesPerSecond * 2;
 }

 {
 std::lock_guard lock(demuxThreadMutex);
 std::lock_guard lock2(videoFramesThreadMutex);
 currentVideoTimeSeconds = closestPts / 1'000'000.0L;
 }
 demuxThreadCond.notify_one();
 videoFramesThreadCond.notify_one();

 return false;
};
</floating128>

What could be causing the desync of audio/video when changing the output sample rate ?


The audio playback sounds fine (no speedups or crackling) when changing the output sample rate.


-
avformat/pcm : decrease target audio frame per sec to 10
3 mars 2024, par Marton Balintavformat/pcm : decrease target audio frame per sec to 10
This makes the wav and pcm demuxer demux bigger packets, which is more
efficient.As a side effect of the bigger packets, audio durations can become less exact
for command lines such as "ffmpeg -i $INPUT -c:a copy -t 1.0 $OUTPUT".Signed-off-by : Marton Balint <cus@passwd.hu>
- [DH] libavformat/pcm.c
- [DH] tests/ref/acodec/pcm-s16be
- [DH] tests/ref/acodec/pcm-s16be_planar
- [DH] tests/ref/acodec/pcm-s16le_planar
- [DH] tests/ref/acodec/pcm-s24be
- [DH] tests/ref/acodec/pcm-s24le_planar
- [DH] tests/ref/acodec/pcm-s32be
- [DH] tests/ref/acodec/pcm-s32le_planar
- [DH] tests/ref/acodec/pcm-s8_planar
- [DH] tests/ref/acodec/pcm-u16be
- [DH] tests/ref/acodec/pcm-u16le
- [DH] tests/ref/acodec/pcm-u24be
- [DH] tests/ref/acodec/pcm-u24le
- [DH] tests/ref/acodec/pcm-u32be
- [DH] tests/ref/acodec/pcm-u32le
- [DH] tests/ref/acodec/s302m
- [DH] tests/ref/fate/dcinema-encode
- [DH] tests/ref/fate/filter-acrossfade
- [DH] tests/ref/fate/filter-adelay
- [DH] tests/ref/fate/filter-aecho
- [DH] tests/ref/fate/filter-aemphasis-50fm
- [DH] tests/ref/fate/filter-aemphasis-75kf
- [DH] tests/ref/fate/filter-afade-esin
- [DH] tests/ref/fate/filter-afade-exp
- [DH] tests/ref/fate/filter-afade-hsin
- [DH] tests/ref/fate/filter-afade-iqsin
- [DH] tests/ref/fate/filter-afade-log
- [DH] tests/ref/fate/filter-afade-qsin
- [DH] tests/ref/fate/filter-agate
- [DH] tests/ref/fate/filter-alimiter
- [DH] tests/ref/fate/filter-amerge
- [DH] tests/ref/fate/filter-anequalizer
- [DH] tests/ref/fate/filter-apad
- [DH] tests/ref/fate/filter-asegment-samples-absolute
- [DH] tests/ref/fate/filter-asegment-samples-relative
- [DH] tests/ref/fate/filter-asegment-timestamps-absolute
- [DH] tests/ref/fate/filter-asegment-timestamps-relative
- [DH] tests/ref/fate/filter-asetrate
- [DH] tests/ref/fate/filter-atrim-mixed
- [DH] tests/ref/fate/filter-atrim-time
- [DH] tests/ref/fate/filter-chorus
- [DH] tests/ref/fate/filter-compand
- [DH] tests/ref/fate/filter-crystalizer
- [DH] tests/ref/fate/filter-dcshift
- [DH] tests/ref/fate/filter-earwax
- [DH] tests/ref/fate/filter-extrastereo
- [DH] tests/ref/fate/filter-pan-downmix1
- [DH] tests/ref/fate/filter-pan-downmix2
- [DH] tests/ref/fate/filter-pan-mono1
- [DH] tests/ref/fate/filter-pan-mono2
- [DH] tests/ref/fate/filter-pan-stereo1
- [DH] tests/ref/fate/filter-pan-stereo2
- [DH] tests/ref/fate/filter-pan-stereo3
- [DH] tests/ref/fate/filter-pan-stereo4
- [DH] tests/ref/fate/filter-pan-upmix1
- [DH] tests/ref/fate/filter-pan-upmix2
- [DH] tests/ref/fate/filter-stereotools
- [DH] tests/ref/fate/flcl1905
- [DH] tests/ref/fate/g722-encode
- [DH] tests/ref/fate/g726-encode-2bit
- [DH] tests/ref/fate/g726-encode-3bit
- [DH] tests/ref/fate/g726-encode-4bit
- [DH] tests/ref/fate/g726-encode-5bit
- [DH] tests/ref/fate/matroska-move-cues-to-front
- [DH] tests/ref/fate/mov-channel-description
- [DH] tests/ref/fate/mov-mp4-pcm
- [DH] tests/ref/fate/mov-mp4-pcm-float
- [DH] tests/ref/lavf/ast
- [DH] tests/ref/lavf/mov
- [DH] tests/ref/lavf/mov_rtphint
- [DH] tests/ref/lavf/s16.voc
- [DH] tests/ref/lavf/smjpeg
- [DH] tests/ref/lavf/voc
- [DH] tests/ref/seek/acodec-adpcm-ima_wav
- [DH] tests/ref/seek/acodec-adpcm-ima_wav-trellis
- [DH] tests/ref/seek/acodec-adpcm-ms
- [DH] tests/ref/seek/acodec-adpcm-ms-trellis
- [DH] tests/ref/seek/acodec-adpcm-yamaha
- [DH] tests/ref/seek/acodec-adpcm-yamaha-trellis
- [DH] tests/ref/seek/acodec-pcm-alaw
- [DH] tests/ref/seek/acodec-pcm-f32be
- [DH] tests/ref/seek/acodec-pcm-f32le
- [DH] tests/ref/seek/acodec-pcm-f64be
- [DH] tests/ref/seek/acodec-pcm-f64le
- [DH] tests/ref/seek/acodec-pcm-mulaw
- [DH] tests/ref/seek/acodec-pcm-s16le
- [DH] tests/ref/seek/acodec-pcm-s24le
- [DH] tests/ref/seek/acodec-pcm-s32le
- [DH] tests/ref/seek/acodec-pcm-u8
- [DH] tests/ref/seek/lavf-al
- [DH] tests/ref/seek/lavf-au
- [DH] tests/ref/seek/lavf-mov
- [DH] tests/ref/seek/lavf-ul
- [DH] tests/ref/seek/lavf-voc
- [DH] tests/ref/seek/lavf-wav
-
avformat/wavdec : use ff_pcm_default_packet_size for the default packet size
7 mars 2024, par Marton Balintavformat/wavdec : use ff_pcm_default_packet_size for the default packet size
Signed-off-by : Marton Balint <cus@passwd.hu>
- [DH] libavformat/wavdec.c
- [DH] tests/ref/fate/dcinema-encode
- [DH] tests/ref/fate/filter-chorus
- [DH] tests/ref/fate/flcl1905
- [DH] tests/ref/fate/g722-encode
- [DH] tests/ref/fate/g726-encode-2bit
- [DH] tests/ref/fate/g726-encode-3bit
- [DH] tests/ref/fate/g726-encode-4bit
- [DH] tests/ref/fate/g726-encode-5bit
- [DH] tests/ref/fate/matroska-move-cues-to-front
- [DH] tests/ref/seek/acodec-adpcm-ima_wav
- [DH] tests/ref/seek/acodec-adpcm-ima_wav-trellis
- [DH] tests/ref/seek/acodec-adpcm-ms
- [DH] tests/ref/seek/acodec-adpcm-ms-trellis
- [DH] tests/ref/seek/acodec-adpcm-yamaha
- [DH] tests/ref/seek/acodec-adpcm-yamaha-trellis
Syndiquer tout le site
