Recherche avancée
Autres articles (69)
-
Support de tous types de médias
10 avril 2011Contrairement à beaucoup de logiciels et autres plate-formes modernes de partage de documents, MediaSPIP a l’ambition de gérer un maximum de formats de documents différents qu’ils soient de type : images (png, gif, jpg, bmp et autres...) ; audio (MP3, Ogg, Wav et autres...) ; vidéo (Avi, MP4, Ogv, mpg, mov, wmv et autres...) ; contenu textuel, code ou autres (open office, microsoft office (tableur, présentation), web (html, css), LaTeX, Google Earth) (...)
-
Ecrire une actualité
21 juin 2013, parPrésentez les changements dans votre MédiaSPIP ou les actualités de vos projets sur votre MédiaSPIP grâce à la rubrique actualités.
Dans le thème par défaut spipeo de MédiaSPIP, les actualités sont affichées en bas de la page principale sous les éditoriaux.
Vous pouvez personnaliser le formulaire de création d’une actualité.
Formulaire de création d’une actualité Dans le cas d’un document de type actualité, les champs proposés par défaut sont : Date de publication ( personnaliser la date de publication ) (...) -
Librairies et binaires spécifiques au traitement vidéo et sonore
31 janvier 2010, parLes logiciels et librairies suivantes sont utilisées par SPIPmotion d’une manière ou d’une autre.
Binaires obligatoires FFMpeg : encodeur principal, permet de transcoder presque tous les types de fichiers vidéo et sonores dans les formats lisibles sur Internet. CF ce tutoriel pour son installation ; Oggz-tools : outils d’inspection de fichiers ogg ; Mediainfo : récupération d’informations depuis la plupart des formats vidéos et sonores ;
Binaires complémentaires et facultatifs flvtool2 : (...)
Sur d’autres sites (7574)
-
Encoding/Decoding H264 using libav in C++ [closed]
20 mai, par gbock93I want to build an application to


- 

- capture frames in YUYV 4:2:2 format
- encode them to H264
- send over network
- decode the received data
- display the video stream












To do so I wrote 2 classes, H264Encoder and H264Decoder.


I post only the .cpp contents, the .h are trivial :


H264Encoder.cpp


#include 

#include <stdexcept>
#include <iostream>

H264Encoder::H264Encoder(unsigned int width_, unsigned int height_, unsigned int fps_):
 m_width(width_),
 m_height(height_),
 m_fps(fps_),
 m_frame_index(0),
 m_context(nullptr),
 m_frame(nullptr),
 m_packet(nullptr),
 m_sws_ctx(nullptr)
{
 // Find the video codec
 AVCodec* codec;
 codec = avcodec_find_encoder(AV_CODEC_ID_H264);
 if (!codec)
 throw std::runtime_error("[Encoder]: Error: Codec not found");

 // Allocate codec
 m_context = avcodec_alloc_context3(codec);
 if (!m_context)
 throw std::runtime_error("[Encoder]: Error: Could not allocate codec context");

 // Configure codec
 av_opt_set(m_context->priv_data, "preset", "ultrafast", 0);
 av_opt_set(m_context->priv_data, "tune", "zerolatency", 0);
 av_opt_set(m_context->priv_data, "crf", "35", 0); // Range: [0; 51], sane range: [18; 26], lower -> higher compression

 m_context->width = (int)width_;
 m_context->height = (int)height_;
 m_context->time_base = {1, (int)fps_};
 m_context->framerate = {(int)fps_, 1};
 m_context->codec_id = AV_CODEC_ID_H264;
 m_context->pix_fmt = AV_PIX_FMT_YUV420P; // H265|4 codec take as input only AV_PIX_FMT_YUV420P
 m_context->bit_rate = 400000;
 m_context->gop_size = 10;
 m_context->max_b_frames = 1;

 // Open codec
 if (avcodec_open2(m_context, codec, nullptr) < 0)
 throw std::runtime_error("[Encoder]: Error: Could not open codec");

 // Allocate frame and its buffer
 m_frame = av_frame_alloc();
 if (!m_frame) 
 throw std::runtime_error("[Encoder]: Error: Could not allocate frame");

 m_frame->format = m_context->pix_fmt;
 m_frame->width = m_context->width;
 m_frame->height = m_context->height;

 if (av_frame_get_buffer(m_frame, 0) < 0)
 throw std::runtime_error("[Encoder]: Error: Cannot allocate frame buffer");
 
 // Allocate packet
 m_packet = av_packet_alloc();
 if (!m_packet) 
 throw std::runtime_error("[Encoder]: Error: Could not allocate packet");

 // Convert from YUYV422 to YUV420P
 m_sws_ctx = sws_getContext(
 width_, height_, AV_PIX_FMT_YUYV422,
 width_, height_, AV_PIX_FMT_YUV420P,
 SWS_BILINEAR, nullptr, nullptr, nullptr
 );
 if (!m_sws_ctx) 
 throw std::runtime_error("[Encoder]: Error: Could not allocate sws context");

 //
 printf("[Encoder]: H264Encoder ready.\n");
}

H264Encoder::~H264Encoder()
{
 sws_freeContext(m_sws_ctx);
 av_packet_free(&m_packet);
 av_frame_free(&m_frame);
 avcodec_free_context(&m_context);

 printf("[Encoder]: H264Encoder destroyed.\n");
}

std::vector H264Encoder::encode(const cv::Mat& img_)
{
 /*
 - YUYV422 is a packed format. It has 3 components (av_pix_fmt_desc_get((AVPixelFormat)AV_PIX_FMT_YUYV422)->nb_components == 3) but
 data is stored in a single plane (av_pix_fmt_count_planes((AVPixelFormat)AV_PIX_FMT_YUYV422) == 1).
 - YUV420P is a planar format. It has 3 components (av_pix_fmt_desc_get((AVPixelFormat)AV_PIX_FMT_YUV420P)->nb_components == 3) and
 each component is stored in a separate plane (av_pix_fmt_count_planes((AVPixelFormat)AV_PIX_FMT_YUV420P) == 3) with its
 own stride.
 */
 std::cout << "[Encoder]" << std::endl;
 std::cout << "[Encoder]: Encoding img " << img_.cols << "x" << img_.rows << " | element size " << img_.elemSize() << std::endl;
 assert(img_.elemSize() == 2);

 uint8_t* input_data[1] = {(uint8_t*)img_.data};
 int input_linesize[1] = {2 * (int)m_width};
 
 if (av_frame_make_writable(m_frame) < 0)
 throw std::runtime_error("[Encoder]: Error: Cannot make frame data writable");

 // Convert from YUV422 image to YUV420 frame. Apply scaling if necessary
 sws_scale(
 m_sws_ctx,
 input_data, input_linesize, 0, m_height,
 m_frame->data, m_frame->linesize
 );
 m_frame->pts = m_frame_index;

 int n_planes = av_pix_fmt_count_planes((AVPixelFormat)m_frame->format);
 std::cout << "[Encoder]: Sending Frame " << m_frame_index << " with dimensions " << m_frame->width << "x" << m_frame->height << "x" << n_planes << std::endl;
 for (int i=0; iframerate.num) + 1;
 break;
 case AVERROR(EAGAIN):
 throw std::runtime_error("[Encoder]: avcodec_send_frame: EAGAIN");
 case AVERROR_EOF:
 throw std::runtime_error("[Encoder]: avcodec_send_frame: EOF");
 case AVERROR(EINVAL):
 throw std::runtime_error("[Encoder]: avcodec_send_frame: EINVAL");
 case AVERROR(ENOMEM):
 throw std::runtime_error("[Encoder]: avcodec_send_frame: ENOMEM");
 default:
 throw std::runtime_error("[Encoder]: avcodec_send_frame: UNKNOWN");
 }

 // Receive packet from codec
 std::vector result;
 while(ret >= 0)
 {
 ret = avcodec_receive_packet(m_context, m_packet);

 switch (ret)
 {
 case 0:
 std::cout << "[Encoder]: Received packet from codec of size " << m_packet->size << " bytes " << std::endl;
 result.insert(result.end(), m_packet->data, m_packet->data + m_packet->size);
 av_packet_unref(m_packet);
 break;

 case AVERROR(EAGAIN):
 std::cout << "[Encoder]: avcodec_receive_packet: EAGAIN" << std::endl;
 break;
 case AVERROR_EOF:
 std::cout << "[Encoder]: avcodec_receive_packet: EOF" << std::endl;
 break;
 case AVERROR(EINVAL):
 throw std::runtime_error("[Encoder]: avcodec_receive_packet: EINVAL");
 default:
 throw std::runtime_error("[Encoder]: avcodec_receive_packet: UNKNOWN");
 }
 }

 std::cout << "[Encoder]: Encoding complete" << std::endl;
 return result;
}
</iostream></stdexcept>

H264Decoder.cpp


#include 

#include <iostream>
#include <stdexcept>

H264Decoder::H264Decoder():
 m_context(nullptr),
 m_frame(nullptr),
 m_packet(nullptr)
{
 // Find the video codec
 AVCodec* codec;
 codec = avcodec_find_decoder(AV_CODEC_ID_H264);
 if (!codec)
 throw std::runtime_error("[Decoder]: Error: Codec not found");

 // Allocate codec
 m_context = avcodec_alloc_context3(codec);
 if (!m_context)
 throw std::runtime_error("[Decoder]: Error: Could not allocate codec context");

 // Open codec
 if (avcodec_open2(m_context, codec, nullptr) < 0)
 throw std::runtime_error("[Decoder]: Error: Could not open codec");

 // Allocate frame
 m_frame = av_frame_alloc();
 if (!m_frame)
 throw std::runtime_error("[Decoder]: Error: Could not allocate frame");

 // Allocate packet
 m_packet = av_packet_alloc();
 if (!m_packet) 
 throw std::runtime_error("[Decoder]: Error: Could not allocate packet");

 //
 printf("[Decoder]: H264Decoder ready.\n");
}

H264Decoder::~H264Decoder()
{
 av_packet_free(&m_packet);
 av_frame_free(&m_frame);
 avcodec_free_context(&m_context);

 printf("[Decoder]: H264Decoder destroyed.\n");
}

bool H264Decoder::decode(uint8_t* data_, size_t size_, cv::Mat& img_)
{
 std::cout << "[Decoder]" << std::endl;
 std::cout << "[Decoder]: decoding " << size_ << " bytes of data" << std::endl;

 // Fill packet
 m_packet->data = data_;
 m_packet->size = size_;

 if (size_ == 0)
 return false;

 // Send packet to codec
 int send_result = avcodec_send_packet(m_context, m_packet);

 switch (send_result)
 {
 case 0:
 std::cout << "[Decoder]: Sent packet to codec" << std::endl;
 break;
 case AVERROR(EAGAIN):
 throw std::runtime_error("[Decoder]: avcodec_send_packet: EAGAIN");
 case AVERROR_EOF:
 throw std::runtime_error("[Decoder]: avcodec_send_packet: EOF");
 case AVERROR(EINVAL):
 throw std::runtime_error("[Decoder]: avcodec_send_packet: EINVAL");
 case AVERROR(ENOMEM):
 throw std::runtime_error("[Decoder]: avcodec_send_packet: ENOMEM");
 default:
 throw std::runtime_error("[Decoder]: avcodec_send_packet: UNKNOWN");
 }

 // Receive frame from codec
 int n_planes;
 uint8_t* output_data[1];
 int output_line_size[1];

 int receive_result = avcodec_receive_frame(m_context, m_frame);

 switch (receive_result)
 {
 case 0:
 n_planes = av_pix_fmt_count_planes((AVPixelFormat)m_frame->format);
 std::cout << "[Decoder]: Received Frame with dimensions " << m_frame->width << "x" << m_frame->height << "x" << n_planes << std::endl;
 for (int i=0; i/
 std::cout << "[Decoder]: Decoding complete" << std::endl;
 return true;
}
</stdexcept></iostream>

To test the two classes I put together a main.cpp to grab a frame, encode/decode and display the decoded frame (no network transmission in place) :


main.cpp


while(...)
{
 // get frame from custom camera class. Format is YUYV 4:2:2
 camera.getFrame(camera_frame);
 // Construct a cv::Mat to represent the grabbed frame
 cv::Mat camera_frame_yuyv = cv::Mat(camera_frame.height, camera_frame.width, CV_8UC2, camera_frame.data.data());
 // Encode image
 std::vector encoded_data = encoder.encode(camera_frame_yuyv);
 if (!encoded_data.empty())
 {
 // Decode image
 cv::Mat decoded_frame;
 if (decoder.decode(encoded_data.data(), encoded_data.size(), decoded_frame))
 {
 // Display image
 cv::imshow("Camera", decoded_frame);
 cv::waitKey(1);
 }
 }
}


Compiling and executing the code I get random results between subsequent executions :


- 

- Sometimes the whole loop runs without problems and I see the decoded image.
- Sometimes the program crashes at the
sws_scale(...)call in the decoder with"Assertion desc failed at src/libswscale/swscale_internal.h:757". - Sometimes the loop runs but I see a black image and the message
Slice parameters 0, 720 are invalidis displayed when executing thesws_scale(...)call in the decoder.








Why is the behaviour so random ? What am I doing wrong with the libav API ?


Some resources I found useful :


- 

- This article on encoding
- This article on decoding






-
How to mention real image instead of dummy image in ffmpeg api-example.c
2 mars 2013, par MohanI am using
video_encode_examplefunction fromapi-example.cof FFmpeg,
which basically creates 25 dummy images and encodes into a one second video.
How ever i am unable to mention real images instead of dummy ones.
If any one know how to do this for xcode objective C, pl submit a reply.
Below is the function/*
* Video encoding example
*/
static void video_encode_example(const char *filename)
{
AVCodec *codec;
AVCodecContext *c= NULL;
int i, out_size, size, x, y, outbuf_size;
FILE *f;
AVFrame *picture;
uint8_t *outbuf, *picture_buf;
printf("Video encoding\n");
/* find the mpeg1 video encoder */
codec = avcodec_find_encoder(CODEC_ID_MPEG1VIDEO);
if (!codec) {
fprintf(stderr, "codec not found\n");
exit(1);
}
c= avcodec_alloc_context();
picture= avcodec_alloc_frame();
/* put sample parameters */
c->bit_rate = 400000;
/* resolution must be a multiple of two */
c->width = 352;
c->height = 288;
/* frames per second */
c->time_base= (AVRational){1,25};
c->gop_size = 10; /* emit one intra frame every ten frames */
c->max_b_frames=1;
c->pix_fmt = PIX_FMT_YUV420P;
/* open it */
if (avcodec_open(c, codec) < 0) {
fprintf(stderr, "could not open codec\n");
exit(1);
}
f = fopen(filename, "wb");
if (!f) {
fprintf(stderr, "could not open %s\n", filename);
exit(1);
}
/* alloc image and output buffer */
outbuf_size = 100000;
outbuf = malloc(outbuf_size);
size = c->width * c->height;
picture_buf = malloc((size * 3) / 2); /* size for YUV 420 */
picture->data[0] = picture_buf;
picture->data[1] = picture->data[0] + size;
picture->data[2] = picture->data[1] + size / 4;
picture->linesize[0] = c->width;
picture->linesize[1] = c->width / 2;
picture->linesize[2] = c->width / 2;
/* encode 1 second of video */
for(i=0;i<25;i++) {
fflush(stdout);
/* prepare a dummy image */
/* Y */
for(y=0;yheight;y++) {
for(x=0;xwidth;x++) {
picture->data[0][y * picture->linesize[0] + x] = x + y + i * 3;
}
}
/* Cb and Cr */
for(y=0;yheight/2;y++) {
for(x=0;xwidth/2;x++) {
picture->data[1][y * picture->linesize[1] + x] = 128 + y + i * 2;
picture->data[2][y * picture->linesize[2] + x] = 64 + x + i * 5;
}
}
/* encode the image */
out_size = avcodec_encode_video(c, outbuf, outbuf_size, picture);
printf("encoding frame %3d (size=%5d)\n", i, out_size);
fwrite(outbuf, 1, out_size, f);
}
/* get the delayed frames */
for(; out_size; i++) {
fflush(stdout);
out_size = avcodec_encode_video(c, outbuf, outbuf_size, NULL);
printf("write frame %3d (size=%5d)\n", i, out_size);
fwrite(outbuf, 1, out_size, f);
}
/* add sequence end code to have a real mpeg file */
outbuf[0] = 0x00;
outbuf[1] = 0x00;
outbuf[2] = 0x01;
outbuf[3] = 0xb7;
fwrite(outbuf, 1, 4, f);
fclose(f);
free(picture_buf);
free(outbuf);
avcodec_close(c);
av_free(c);
av_free(picture);
printf("\n");
} -
Green screen writing FLV file libavformat
5 mai 2013, par YYZI've written a piece of C++ code that can capture webcam video frame, decode them, convert them to YUV420P, encode them and then write them to a file. If I use the mpeg2 codec and write to a .mpg file, everything works perfectly. But, if I use flv, then the output produced is just a green screen. I'm not sure if there are different encoder settings I need to set for encoding flv video ? Here's my code(the relevant parts) :
Encoder settings :
c->codec_id = codec_id;
c->bit_rate = 400000;
// Resolution must be a multiple of two.
c->width = 352;
c->height = 288;
c->time_base.den = STREAM_FRAME_RATE;
c->time_base.num = 1;
//emit one intra frame every twelve frames at most
c->gop_size = 12;
c->pix_fmt = STREAM_PIX_FMT;Write the frames
int ret;
uint8_t *buffer = NULL;
static struct SwsContext *sws_ctx;
//Setup the codec context, and set its width and height to be equal to the input video width and height
AVCodecContext *c = st->codec;
c->width = pCodecCtx->width;
c->height = pCodecCtx->height;
av_init_packet(&packet);
frameYUV = avcodec_alloc_frame();
//Determine how big the buffer will need to be to store the captured frame
numBytes = avpicture_get_size(STREAM_PIX_FMT,pCodecCtx->width,pCodecCtx->height);
//Allocate the needed buffer size
buffer = new uint8_t[numBytes];
sws_ctx = sws_getContext(pCodecCtx->width,pCodecCtx->height,pCodecCtx->pix_fmt,
pCodecCtx->width,pCodecCtx->height,
STREAM_PIX_FMT,SWS_BICUBIC,NULL,NULL,NULL);
//Fill the output frame
avpicture_fill((AVPicture *)frameYUV,buffer,STREAM_PIX_FMT,pCodecCtx->width,pCodecCtx->height);
//Read a video frame in
av_read_frame(pFormatCtx,&packet);
//Decode the contents of packet into pFrame
avcodec_decode_video2(pCodecCtx,pFrame,&frameFinished,&packet);
//Scale pFrame into frameYUV, and convert to PIXFMTYUV420P
sws_scale
(
sws_ctx,
(uint8_t const * const *)pFrame->data,
pFrame->linesize,
0,
pCodecCtx->height,
frameYUV->data,
frameYUV->linesize
);
av_init_packet(&samsPacket);
//Encode frameYUV
avcodec_encode_video2(c, &samsPacket, frameYUV, &gotSamsPacket);
AVPacket pkt = { 0 };
int got_packet;
av_init_packet(&pkt);
// encode the image
ret = avcodec_encode_video2(c, &pkt, frame, &got_packet);
if (ret < 0){
debugLogStreamSave->debug("Error encoding video frame");
exit(1);
}
if (!ret && got_packet && pkt.size){
pkt.stream_index = st->index;
// Write the compressed frame to our output
ret = av_interleaved_write_frame(oc, &pkt);Any help would be appreciated !
Syndiquer tout le site
