Recherche avancée
Médias (16)
-
#7 Ambience
16 octobre 2011, par
Mis à jour : Juin 2015
Langue : English
Type : Audio
-
#6 Teaser Music
16 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
#5 End Title
16 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
#3 The Safest Place
16 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
#4 Emo Creates
15 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
#2 Typewriter Dance
15 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
Autres articles (39)
-
Submit enhancements and plugins
13 avril 2011If you have developed a new extension to add one or more useful features to MediaSPIP, let us know and its integration into the core MedisSPIP functionality will be considered.
You can use the development discussion list to request for help with creating a plugin. As MediaSPIP is based on SPIP - or you can use the SPIP discussion list SPIP-Zone. -
Librairies et binaires spécifiques au traitement vidéo et sonore
31 janvier 2010, parLes logiciels et librairies suivantes sont utilisées par SPIPmotion d’une manière ou d’une autre.
Binaires obligatoires FFMpeg : encodeur principal, permet de transcoder presque tous les types de fichiers vidéo et sonores dans les formats lisibles sur Internet. CF ce tutoriel pour son installation ; Oggz-tools : outils d’inspection de fichiers ogg ; Mediainfo : récupération d’informations depuis la plupart des formats vidéos et sonores ;
Binaires complémentaires et facultatifs flvtool2 : (...) -
Support audio et vidéo HTML5
10 avril 2011MediaSPIP utilise les balises HTML5 video et audio pour la lecture de documents multimedia en profitant des dernières innovations du W3C supportées par les navigateurs modernes.
Pour les navigateurs plus anciens, le lecteur flash Flowplayer est utilisé.
Le lecteur HTML5 utilisé a été spécifiquement créé pour MediaSPIP : il est complètement modifiable graphiquement pour correspondre à un thème choisi.
Ces technologies permettent de distribuer vidéo et son à la fois sur des ordinateurs conventionnels (...)
Sur d’autres sites (4598)
-
Vulkan image data to AVFrames and to video
12 avril 2024, par W4zab1I am trying to encode Vulkan image data into video with MPEG4 format. For some reason the output videofile is corrupted. FFProbe shows discontinuity in timestamps, and the frames are corrupted.
First I prepare my video encoder

Then I get FrameEnded events from my engine where I can get the image data from the vulkan swapchain.

I then convert the image data from vulkan to AVFrames (RGBA to YUV420P), then I pass the frames into queue.

This queue is then handled in another thread, where the frames are processed, and written into video.

I am bit of a noob with ffmpeg, so there can be some code that does not make sense.

This seems right straight forward logic, but there is probably some problems with codec params, way I am converting the imagedata to AVFrame, or something of that sort.

The videofile still gets created, and has some data in it (it is > 0 bytes, and longer the recording, bigger the filesize).

There is no errors from ffmpeg with log_level set to DEBUG.

struct FrameData {
 AVFrame* frame;
 int frame_index;
};

class EventListenerVideoCapture : public VEEventListenerGLFW {
private:
 AVFormatContext* format_ctx = nullptr;
 AVCodec* video_codec = nullptr;
 AVCodecContext* codec_context = nullptr;
 AVStream* video_stream = nullptr;
 AVDictionary* muxer_opts = nullptr;
 int frame_index = 0;

 std::queue frame_queue;
 std::mutex queue_mtx;
 std::condition_variable queue_cv;
 std::atomic<bool> stop_processing{ false };
 std::thread video_processing_thread;
 int prepare_video_encoder()
 {
 av_log_set_level(AV_LOG_DEBUG);
 // Add video stream to format context
 avformat_alloc_output_context2(&format_ctx, nullptr, nullptr, "video.mpg");
 video_stream = avformat_new_stream(format_ctx, NULL);
 video_codec = (AVCodec*)avcodec_find_encoder(AV_CODEC_ID_MPEG4);
 codec_context = avcodec_alloc_context3(video_codec);
 if (!format_ctx) { std::cerr << "Error: Failed to allocate format context" << std::endl; system("pause"); }
 if (!video_stream) { std::cerr << "Error: Failed to create new stream" << std::endl; system("pause"); }
 if (!video_codec) { std::cerr << "Error: Failed to find video codec" << std::endl; system("pause"); }
 if (!codec_context) { std::cerr << "Error: Failed to allocate codec context" << std::endl; system("pause"); }

 if (avio_open(&format_ctx->pb, "video.mpg", AVIO_FLAG_WRITE) < 0) { std::cerr << "Error: Failed to open file for writing!" << std::endl; return -1; }

 av_opt_set(codec_context->priv_data, "preset", "fast", 0);

 codec_context->codec_id = AV_CODEC_ID_MPEG4;
 codec_context->codec_type = AVMEDIA_TYPE_VIDEO;
 codec_context->pix_fmt = AV_PIX_FMT_YUV420P;
 codec_context->width = getWindowPointer()->getExtent().width;
 codec_context->height = getWindowPointer()->getExtent().height;
 codec_context->bit_rate = 1000 * 1000; // Bitrate
 codec_context->time_base = { 1, 30 }; // 30 FPS
 codec_context->gop_size = 10;

 av_dict_set(&muxer_opts, "movflags", "faststart", 0);

 //Unecessary? Since the params are copied anyways
 video_stream->time_base = codec_context->time_base;

 //Try to open codec after changes
 //copy codec_context params to videostream
 //and write headers to format_context
 if (avcodec_open2(codec_context, video_codec, NULL) < 0) { std::cerr << "Error: Could not open codec!" << std::endl; return -1; }
 if (avcodec_parameters_from_context(video_stream->codecpar, codec_context) < 0) { std::cerr << "Error: Could not copy params from context to stream!" << std::endl; return -1; };
 if (avformat_write_header(format_ctx, &muxer_opts) < 0) { std::cerr << "Error: Failed to write output file headers!" << std::endl; return -1; }
 return 0;
 }

 void processFrames() {
 while (!stop_processing) {
 FrameData* frameData = nullptr;
 {
 std::unique_lock lock(queue_mtx);
 queue_cv.wait(lock, [&]() { return !frame_queue.empty() || stop_processing; });

 if (stop_processing && frame_queue.empty())
 break;

 frameData = frame_queue.front();
 frame_queue.pop();
 }

 if (frameData) {
 encodeAndWriteFrame(frameData);
 AVFrame* frame = frameData->frame;
 av_frame_free(&frame); // Free the processed frame
 delete frameData;
 }
 }
 }

 void encodeAndWriteFrame(FrameData* frameData) {

 // Validation
 if (!frameData->frame) { std::cerr << "Error: Frame was null! " << std::endl; return; }
 if (frameData->frame->format != codec_context->pix_fmt) { std::cerr << "Error: Frame format mismatch!" << std::endl; return; }
 if ( av_frame_get_buffer(frameData->frame, 0) < 0) { std::cerr << "Error allocating frame buffer: " << std::endl; return; }
 if (!codec_context) return;

 AVPacket* pkt = av_packet_alloc();
 if (!pkt) { std::cerr << "Error: Failed to allocate AVPacket" << std::endl; system("pause"); }

 int ret = avcodec_send_frame(codec_context, frameData->frame);
 if (ret < 0) { 
 std::cerr << "Error receiving packet from codec: " << ret << std::endl;
 delete frameData;
 av_packet_free(&pkt); return; 
 }

 while (ret >= 0) {
 ret = avcodec_receive_packet(codec_context, pkt);

 //Error checks
 if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) { break; }
 else if (ret < 0) { std::cerr << "Error receiving packet from codec: " << ret << std::endl; av_packet_free(&pkt); return; }
 if (!video_stream) { std::cerr << "Error: video stream is null!" << std::endl; av_packet_free(&pkt); return; }
 
 int64_t frame_duration = codec_context->time_base.den / codec_context->time_base.num;
 pkt->stream_index = video_stream->index;
 pkt->duration = frame_duration;
 pkt->pts = frameData->frame_index * frame_duration;

 int write_ret = av_interleaved_write_frame(format_ctx, pkt);
 if (write_ret < 0) { std::cerr << "Error: failed to write a frame! " << write_ret << std::endl;}

 av_packet_unref(pkt);
 }

 av_packet_free(&pkt);

 }

protected:
 virtual void onFrameEnded(veEvent event) override {
 // Get the image data from vulkan
 VkExtent2D extent = getWindowPointer()->getExtent();
 uint32_t imageSize = extent.width * extent.height * 4;
 VkImage image = getEnginePointer()->getRenderer()->getSwapChainImage();

 uint8_t *dataImage = new uint8_t[imageSize];
 
 vh::vhBufCopySwapChainImageToHost(getEnginePointer()->getRenderer()->getDevice(),
 getEnginePointer()->getRenderer()->getVmaAllocator(),
 getEnginePointer()->getRenderer()->getGraphicsQueue(),
 getEnginePointer()->getRenderer()->getCommandPool(),
 image, VK_FORMAT_R8G8B8A8_UNORM,
 VK_IMAGE_ASPECT_COLOR_BIT, VK_IMAGE_LAYOUT_PRESENT_SRC_KHR,
 dataImage, extent.width, extent.height, imageSize);
 
 // Create AVFrame for the converted image data
 AVFrame* frame = av_frame_alloc();
 if (!frame) { std::cout << "Could not allocate memory for frame!" << std::endl; return; }

 frame->format = AV_PIX_FMT_YUV420P;
 frame->width = extent.width;
 frame->height = extent.height;
 if (av_frame_get_buffer(frame, 0) < 0) { std::cerr << "Failed to allocate frame buffer! " << std::endl; return;} ;

 // Prepare context for converting from RGBA to YUV420P
 SwsContext* sws_ctx = sws_getContext(
 extent.width, extent.height, AV_PIX_FMT_RGBA,
 extent.width, extent.height, AV_PIX_FMT_YUV420P,
 SWS_BILINEAR, nullptr, nullptr, nullptr);

 // Convert the vulkan image data to AVFrame
 uint8_t* src_data[1] = { dataImage };
 int src_linesize[1] = { extent.width * 4 };
 int scale_ret = sws_scale(sws_ctx, src_data, src_linesize, 0, extent.height,
 frame->data, frame->linesize);

 if (scale_ret <= 0) { std::cerr << "Failed to scale the image to frame" << std::endl; return; }

 sws_freeContext(sws_ctx);
 delete[] dataImage;

 // Add frame to the queue
 {
 std::lock_guard lock(queue_mtx);

 FrameData* frameData = new FrameData;
 frameData->frame = frame;
 frameData->frame_index = frame_index;
 frame_queue.push(frameData);

 frame_index++;
 }

 // Notify processing thread
 queue_cv.notify_one();
 }

public:
 EventListenerVideoCapture(std::string name) : VEEventListenerGLFW(name) {
 //Prepare the video encoder
 int ret = prepare_video_encoder();
 if (ret < 0)
 {
 std::cerr << "Failed to prepare video encoder! " << std::endl;
 exit(-1);
 }
 else
 {
 // Start video processing thread
 video_processing_thread = std::thread(&EventListenerVideoCapture::processFrames, this);
 }
 }

 ~EventListenerVideoCapture() {
 // Stop video processing thread
 stop_processing = true;
 queue_cv.notify_one(); // Notify processing thread to stop

 if (video_processing_thread.joinable()) {
 video_processing_thread.join();
 }

 // Flush codec and close output file
 avcodec_send_frame(codec_context, nullptr);
 av_write_trailer(format_ctx);

 av_dict_free(&muxer_opts);
 avio_closep(&format_ctx->pb);
 avcodec_free_context(&codec_context);
 avformat_free_context(format_ctx);
 }
};

</bool>

I have tried changing the codec params, debugging and printing the videoframe data with no success.


-
avformat/wavdec : use ff_pcm_default_packet_size for the default packet size
7 mars 2024, par Marton Balintavformat/wavdec : use ff_pcm_default_packet_size for the default packet size
Signed-off-by : Marton Balint <cus@passwd.hu>
- [DH] libavformat/wavdec.c
- [DH] tests/ref/fate/dcinema-encode
- [DH] tests/ref/fate/filter-chorus
- [DH] tests/ref/fate/flcl1905
- [DH] tests/ref/fate/g722-encode
- [DH] tests/ref/fate/g726-encode-2bit
- [DH] tests/ref/fate/g726-encode-3bit
- [DH] tests/ref/fate/g726-encode-4bit
- [DH] tests/ref/fate/g726-encode-5bit
- [DH] tests/ref/fate/matroska-move-cues-to-front
- [DH] tests/ref/seek/acodec-adpcm-ima_wav
- [DH] tests/ref/seek/acodec-adpcm-ima_wav-trellis
- [DH] tests/ref/seek/acodec-adpcm-ms
- [DH] tests/ref/seek/acodec-adpcm-ms-trellis
- [DH] tests/ref/seek/acodec-adpcm-yamaha
- [DH] tests/ref/seek/acodec-adpcm-yamaha-trellis
-
avformat/pcm : decrease target audio frame per sec to 10
3 mars 2024, par Marton Balintavformat/pcm : decrease target audio frame per sec to 10
This makes the wav and pcm demuxer demux bigger packets, which is more
efficient.As a side effect of the bigger packets, audio durations can become less exact
for command lines such as "ffmpeg -i $INPUT -c:a copy -t 1.0 $OUTPUT".Signed-off-by : Marton Balint <cus@passwd.hu>
- [DH] libavformat/pcm.c
- [DH] tests/ref/acodec/pcm-s16be
- [DH] tests/ref/acodec/pcm-s16be_planar
- [DH] tests/ref/acodec/pcm-s16le_planar
- [DH] tests/ref/acodec/pcm-s24be
- [DH] tests/ref/acodec/pcm-s24le_planar
- [DH] tests/ref/acodec/pcm-s32be
- [DH] tests/ref/acodec/pcm-s32le_planar
- [DH] tests/ref/acodec/pcm-s8_planar
- [DH] tests/ref/acodec/pcm-u16be
- [DH] tests/ref/acodec/pcm-u16le
- [DH] tests/ref/acodec/pcm-u24be
- [DH] tests/ref/acodec/pcm-u24le
- [DH] tests/ref/acodec/pcm-u32be
- [DH] tests/ref/acodec/pcm-u32le
- [DH] tests/ref/acodec/s302m
- [DH] tests/ref/fate/dcinema-encode
- [DH] tests/ref/fate/filter-acrossfade
- [DH] tests/ref/fate/filter-adelay
- [DH] tests/ref/fate/filter-aecho
- [DH] tests/ref/fate/filter-aemphasis-50fm
- [DH] tests/ref/fate/filter-aemphasis-75kf
- [DH] tests/ref/fate/filter-afade-esin
- [DH] tests/ref/fate/filter-afade-exp
- [DH] tests/ref/fate/filter-afade-hsin
- [DH] tests/ref/fate/filter-afade-iqsin
- [DH] tests/ref/fate/filter-afade-log
- [DH] tests/ref/fate/filter-afade-qsin
- [DH] tests/ref/fate/filter-agate
- [DH] tests/ref/fate/filter-alimiter
- [DH] tests/ref/fate/filter-amerge
- [DH] tests/ref/fate/filter-anequalizer
- [DH] tests/ref/fate/filter-apad
- [DH] tests/ref/fate/filter-asegment-samples-absolute
- [DH] tests/ref/fate/filter-asegment-samples-relative
- [DH] tests/ref/fate/filter-asegment-timestamps-absolute
- [DH] tests/ref/fate/filter-asegment-timestamps-relative
- [DH] tests/ref/fate/filter-asetrate
- [DH] tests/ref/fate/filter-atrim-mixed
- [DH] tests/ref/fate/filter-atrim-time
- [DH] tests/ref/fate/filter-chorus
- [DH] tests/ref/fate/filter-compand
- [DH] tests/ref/fate/filter-crystalizer
- [DH] tests/ref/fate/filter-dcshift
- [DH] tests/ref/fate/filter-earwax
- [DH] tests/ref/fate/filter-extrastereo
- [DH] tests/ref/fate/filter-pan-downmix1
- [DH] tests/ref/fate/filter-pan-downmix2
- [DH] tests/ref/fate/filter-pan-mono1
- [DH] tests/ref/fate/filter-pan-mono2
- [DH] tests/ref/fate/filter-pan-stereo1
- [DH] tests/ref/fate/filter-pan-stereo2
- [DH] tests/ref/fate/filter-pan-stereo3
- [DH] tests/ref/fate/filter-pan-stereo4
- [DH] tests/ref/fate/filter-pan-upmix1
- [DH] tests/ref/fate/filter-pan-upmix2
- [DH] tests/ref/fate/filter-stereotools
- [DH] tests/ref/fate/flcl1905
- [DH] tests/ref/fate/g722-encode
- [DH] tests/ref/fate/g726-encode-2bit
- [DH] tests/ref/fate/g726-encode-3bit
- [DH] tests/ref/fate/g726-encode-4bit
- [DH] tests/ref/fate/g726-encode-5bit
- [DH] tests/ref/fate/matroska-move-cues-to-front
- [DH] tests/ref/fate/mov-channel-description
- [DH] tests/ref/fate/mov-mp4-pcm
- [DH] tests/ref/fate/mov-mp4-pcm-float
- [DH] tests/ref/lavf/ast
- [DH] tests/ref/lavf/mov
- [DH] tests/ref/lavf/mov_rtphint
- [DH] tests/ref/lavf/s16.voc
- [DH] tests/ref/lavf/smjpeg
- [DH] tests/ref/lavf/voc
- [DH] tests/ref/seek/acodec-adpcm-ima_wav
- [DH] tests/ref/seek/acodec-adpcm-ima_wav-trellis
- [DH] tests/ref/seek/acodec-adpcm-ms
- [DH] tests/ref/seek/acodec-adpcm-ms-trellis
- [DH] tests/ref/seek/acodec-adpcm-yamaha
- [DH] tests/ref/seek/acodec-adpcm-yamaha-trellis
- [DH] tests/ref/seek/acodec-pcm-alaw
- [DH] tests/ref/seek/acodec-pcm-f32be
- [DH] tests/ref/seek/acodec-pcm-f32le
- [DH] tests/ref/seek/acodec-pcm-f64be
- [DH] tests/ref/seek/acodec-pcm-f64le
- [DH] tests/ref/seek/acodec-pcm-mulaw
- [DH] tests/ref/seek/acodec-pcm-s16le
- [DH] tests/ref/seek/acodec-pcm-s24le
- [DH] tests/ref/seek/acodec-pcm-s32le
- [DH] tests/ref/seek/acodec-pcm-u8
- [DH] tests/ref/seek/lavf-al
- [DH] tests/ref/seek/lavf-au
- [DH] tests/ref/seek/lavf-mov
- [DH] tests/ref/seek/lavf-ul
- [DH] tests/ref/seek/lavf-voc
- [DH] tests/ref/seek/lavf-wav
Syndiquer tout le site
