Newest 'libx264' Questions - Stack Overflow
Articles published on the website
-
Creating a video from images using ffmpeg libav and libx264?
8 March 2021, by marikanerI am trying to create a video from images using the ffmpeg library. The images have a size of 1920x1080 and are supposed to be encoded with H.264 using a .mkv container. I have come across various problems, thinking I am getting closer to a solution, but this one I am really stuck on. With the settings I use, the first X frames (around 40, depending on what and how many images I use for the video) of my video are not encoded. avcodec_encode_video2 does not return any error (return value is 0) with got_picture_ptr = 0. The result is a video that actually looks as expected, but the first seconds are weirdly jumpy.
So this is how I create the video file:
// m_codecContext is an instance variable of type AVCodecContext * // m_formatCtx is an instance variable of type AVFormatContext * // outputFileName is a valid filename ending with .mkv AVOutputFormat *oformat = av_guess_format(NULL, outputFileName, NULL); if (oformat == NULL) { oformat = av_guess_format("mpeg", NULL, NULL); } // oformat->video_codec is AV_CODEC_ID_H264 AVCodec *codec = avcodec_find_encoder(oformat->video_codec); m_codecContext = avcodec_alloc_context3(codec); m_codecContext->codec_id = oformat->video_codec; m_codecContext->codec_type = AVMEDIA_TYPE_VIDEO; m_codecContext->gop_size = 30; m_codecContext->bit_rate = width * height * 4 m_codecContext->width = width; m_codecContext->height = height; m_codecContext->time_base = (AVRational){1,frameRate}; m_codecContext->max_b_frames = 1; m_codecContext->pix_fmt = AV_PIX_FMT_YUV420P; m_formatCtx = avformat_alloc_context(); m_formatCtx->oformat = oformat; m_formatCtx->video_codec_id = oformat->video_codec; snprintf(m_formatCtx->filename, sizeof(m_formatCtx->filename), "%s", outputFileName); AVStream *videoStream = avformat_new_stream(m_formatCtx, codec); if(!videoStream) { printf("Could not allocate stream\n"); } videoStream->codec = m_codecContext; if(m_formatCtx->oformat->flags & AVFMT_GLOBALHEADER) { m_codecContext->flags |= CODEC_FLAG_GLOBAL_HEADER; } avcodec_open2(m_codecContext, codec, NULL) < 0); avio_open(&m_formatCtx->pb, outputFileName.toStdString().c_str(), AVIO_FLAG_WRITE); avformat_write_header(m_formatCtx, NULL);this is how the frames are added:
void VideoCreator::writeImageToVideo(const QSharedPointer&img, int frameIndex) { AVFrame *frame = avcodec_alloc_frame(); /* alloc image and output buffer */ int size = m_codecContext->width * m_codecContext->height; int numBytes = avpicture_get_size(m_codecContext->pix_fmt, m_codecContext->width, m_codecContext->height); uint8_t *outbuf = (uint8_t *)malloc(numBytes); uint8_t *picture_buf = (uint8_t *)av_malloc(numBytes); int ret = av_image_fill_arrays(frame->data, frame->linesize, picture_buf, m_codecContext->pix_fmt, m_codecContext->width, m_codecContext->height, 1); frame->data[0] = picture_buf; frame->data[1] = frame->data[0] + size; frame->data[2] = frame->data[1] + size/4; frame->linesize[0] = m_codecContext->width; frame->linesize[1] = m_codecContext->width/2; frame->linesize[2] = m_codecContext->width/2; fflush(stdout); for (int y = 0; y < m_codecContext->height; y++) { for (int x = 0; x < m_codecContext->width; x++) { unsigned char b = img->bits()[(y * m_codecContext->width + x) * 4 + 0]; unsigned char g = img->bits()[(y * m_codecContext->width + x) * 4 + 1]; unsigned char r = img->bits()[(y * m_codecContext->width + x) * 4 + 2]; unsigned char Y = (0.257 * r) + (0.504 * g) + (0.098 * b) + 16; frame->data[0][y * frame->linesize[0] + x] = Y; if (y % 2 == 0 && x % 2 == 0) { unsigned char V = (0.439 * r) - (0.368 * g) - (0.071 * b) + 128; unsigned char U = -(0.148 * r) - (0.291 * g) + (0.439 * b) + 128; frame->data[1][y/2 * frame->linesize[1] + x/2] = U; frame->data[2][y/2 * frame->linesize[2] + x/2] = V; } } } int pts = frameIndex;//(1.0 / 30.0) * 90.0 * frameIndex; frame->pts = pts;//av_rescale_q(m_codecContext->coded_frame->pts, m_codecContext->time_base, formatCtx->streams[0]->time_base); //(1.0 / 30.0) * 90.0 * frameIndex; int got_packet_ptr; AVPacket packet; av_init_packet(&packet); packet.data = outbuf; packet.size = numBytes; packet.stream_index = formatCtx->streams[0]->index; packet.flags |= AV_PKT_FLAG_KEY; packet.pts = packet.dts = pts; m_codecContext->coded_frame->pts = pts; ret = avcodec_encode_video2(m_codecContext, &packet, frame, &got_packet_ptr); if (got_packet_ptr != 0) { m_codecContext->coded_frame->pts = pts; // Set the time stamp if (m_codecContext->coded_frame->pts != (0x8000000000000000LL)) { pts = av_rescale_q(m_codecContext->coded_frame->pts, m_codecContext->time_base, formatCtx->streams[0]->time_base); } packet.pts = pts; if(m_codecContext->coded_frame->key_frame) { packet.flags |= AV_PKT_FLAG_KEY; } std::cout << "pts: " << packet.pts << ", dts: " << packet.dts << std::endl; av_interleaved_write_frame(formatCtx, &packet); av_free_packet(&packet); } free(picture_buf); free(outbuf); av_free(frame); printf("\n"); } and this is the cleanup:
int numBytes = avpicture_get_size(m_codecContext->pix_fmt, m_codecContext->width, m_codecContext->height); int got_packet_ptr = 1; int ret; // for(; got_packet_ptr != 0; i++) while (got_packet_ptr) { uint8_t *outbuf = (uint8_t *)malloc(numBytes); AVPacket packet; av_init_packet(&packet); packet.data = outbuf; packet.size = numBytes; ret = avcodec_encode_video2(m_codecContext, &packet, NULL, &got_packet_ptr); if (got_packet_ptr) { av_interleaved_write_frame(m_formatCtx, &packet); } av_free_packet(&packet); free(outbuf); } av_write_trailer(formatCtx); avcodec_close(m_codecContext); av_free(m_codecContext); printf("\n");I assume it might be tied to the PTS and DTS values, but I have tried EVERYTHING. The frame index seems to make the most sense. The images are correct, I can save them to files without any problems. I am running out of ideas. I would be incredibly thankful if there was someone out there who knew better than me...
Cheers, marikaner
UPDATE:
If this is of any help this is the output at the end of the video encoding:
[libx264 @ 0x7fffc00028a0] frame I:19 Avg QP:14.24 size:312420 [libx264 @ 0x7fffc00028a0] frame P:280 Avg QP:19.16 size:148867 [libx264 @ 0x7fffc00028a0] frame B:181 Avg QP:21.31 size: 40540 [libx264 @ 0x7fffc00028a0] consecutive B-frames: 24.6% 75.4% [libx264 @ 0x7fffc00028a0] mb I I16..4: 30.9% 45.5% 23.7% [libx264 @ 0x7fffc00028a0] mb P I16..4: 4.7% 9.1% 4.5% P16..4: 23.5% 16.6% 12.6% 0.0% 0.0% skip:28.9% [libx264 @ 0x7fffc00028a0] mb B I16..4: 0.6% 0.5% 0.3% B16..8: 26.7% 11.0% 5.5% direct: 3.9% skip:51.5% L0:39.4% L1:45.0% BI:15.6% [libx264 @ 0x7fffc00028a0] final ratefactor: 19.21 [libx264 @ 0x7fffc00028a0] 8x8 transform intra:48.2% inter:47.3% [libx264 @ 0x7fffc00028a0] coded y,uvDC,uvAC intra: 54.9% 53.1% 30.4% inter: 25.4% 13.5% 4.2% [libx264 @ 0x7fffc00028a0] i16 v,h,dc,p: 41% 29% 11% 19% [libx264 @ 0x7fffc00028a0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 26% 31% 3% 4% 3% 7% 3% 6% [libx264 @ 0x7fffc00028a0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 30% 26% 14% 4% 5% 4% 7% 4% 7% [libx264 @ 0x7fffc00028a0] i8c dc,h,v,p: 58% 26% 13% 3% [libx264 @ 0x7fffc00028a0] Weighted P-Frames: Y:17.1% UV:3.6% [libx264 @ 0x7fffc00028a0] ref P L0: 63.1% 21.4% 11.4% 4.1% 0.1% [libx264 @ 0x7fffc00028a0] ref B L0: 85.7% 14.3% [libx264 @ 0x7fffc00028a0] kb/s:27478.30 -
Convert .mp4 to .ts using H264 at constant framerate 25fps [closed]
4 March 2021, by losing-my-marblesI've been trying to do this conversion for a couple of days reading through forums
ffmpeg -i in1.mp4 -vcodec libx264 -s 720x576 -vf "fps=25"-vsync cfr -b:v 2300k -acodec aac -b:a 128k out1.tsI've tried the below flags found in other forums.
... -r 25 -vf fps=25 -filter:v fps=25 -filter:v fps=fps=25I then read the file using mediainfo and keep getting the same result;
Frame rate mode : VFR Frame rate mode : Variable Frame rate : 46.875 Frame rate : 46.875 FPS (1024 SPF) -
iOS Build PJSIP with FFmpeg+libx264
22 February 2021, by MeonardoI have built the FFmpeg with libx264 into static libs, here is my directory tree.
./ffmpeg ├── include │ ├── libavcodec │ │ ├── ac3_parser.h │ │ ├── adts_parser.h │ │ ├── avcodec.h │ │ ├── avdct.h │ │ ├── avfft.h │ │ ├── bsf.h │ │ ├── codec.h │ │ ├── codec_desc.h │ │ ├── codec_id.h │ │ ├── codec_par.h │ │ ├── d3d11va.h │ │ ├── dirac.h │ │ ├── dv_profile.h │ │ ├── dxva2.h │ │ ├── jni.h │ │ ├── mediacodec.h │ │ ├── packet.h │ │ ├── qsv.h │ │ ├── vaapi.h │ │ ├── vdpau.h │ │ ├── version.h │ │ ├── videotoolbox.h │ │ ├── vorbis_parser.h │ │ └── xvmc.h │ ├── libavdevice │ │ ├── avdevice.h │ │ └── version.h │ ├── libavfilter │ │ ├── avfilter.h │ │ ├── buffersink.h │ │ ├── buffersrc.h │ │ └── version.h │ ├── libavformat │ │ ├── avformat.h │ │ ├── avio.h │ │ └── version.h │ ├── libavutil │ │ ├── adler32.h │ │ ├── aes.h │ │ ├── aes_ctr.h │ │ ├── attributes.h │ │ ├── audio_fifo.h │ │ ├── avassert.h │ │ ├── avconfig.h │ │ ├── avstring.h │ │ ├── avutil.h │ │ ├── base64.h │ │ ├── blowfish.h │ │ ├── bprint.h │ │ ├── bswap.h │ │ ├── buffer.h │ │ ├── camellia.h │ │ ├── cast5.h │ │ ├── channel_layout.h │ │ ├── common.h │ │ ├── cpu.h │ │ ├── crc.h │ │ ├── des.h │ │ ├── dict.h │ │ ├── display.h │ │ ├── dovi_meta.h │ │ ├── downmix_info.h │ │ ├── encryption_info.h │ │ ├── error.h │ │ ├── eval.h │ │ ├── ffversion.h │ │ ├── fifo.h │ │ ├── file.h │ │ ├── frame.h │ │ ├── hash.h │ │ ├── hdr_dynamic_metadata.h │ │ ├── hmac.h │ │ ├── hwcontext.h │ │ ├── hwcontext_cuda.h │ │ ├── hwcontext_d3d11va.h │ │ ├── hwcontext_drm.h │ │ ├── hwcontext_dxva2.h │ │ ├── hwcontext_mediacodec.h │ │ ├── hwcontext_opencl.h │ │ ├── hwcontext_qsv.h │ │ ├── hwcontext_vaapi.h │ │ ├── hwcontext_vdpau.h │ │ ├── hwcontext_videotoolbox.h │ │ ├── hwcontext_vulkan.h │ │ ├── imgutils.h │ │ ├── intfloat.h │ │ ├── intreadwrite.h │ │ ├── lfg.h │ │ ├── log.h │ │ ├── lzo.h │ │ ├── macros.h │ │ ├── mastering_display_metadata.h │ │ ├── mathematics.h │ │ ├── md5.h │ │ ├── mem.h │ │ ├── motion_vector.h │ │ ├── murmur3.h │ │ ├── opt.h │ │ ├── parseutils.h │ │ ├── pixdesc.h │ │ ├── pixelutils.h │ │ ├── pixfmt.h │ │ ├── random_seed.h │ │ ├── rational.h │ │ ├── rc4.h │ │ ├── replaygain.h │ │ ├── ripemd.h │ │ ├── samplefmt.h │ │ ├── sha.h │ │ ├── sha512.h │ │ ├── spherical.h │ │ ├── stereo3d.h │ │ ├── tea.h │ │ ├── threadmessage.h │ │ ├── time.h │ │ ├── timecode.h │ │ ├── timestamp.h │ │ ├── tree.h │ │ ├── twofish.h │ │ ├── tx.h │ │ ├── version.h │ │ ├── video_enc_params.h │ │ └── xtea.h │ ├── libpostproc │ │ ├── postprocess.h │ │ └── version.h │ ├── libswresample │ │ ├── swresample.h │ │ └── version.h │ ├── libswscale │ │ ├── swscale.h │ │ └── version.h │ └── libx264 │ ├── x264.h │ └── x264_config.h └── lib ├── libavcodec.a ├── libavdevice.a ├── libavfilter.a ├── libavformat.a ├── libavutil.a ├── libpostproc.a ├── libswresample.a ├── libswscale.a └── libx264.aThe final lib was created successfully, no error message in log file, but when I import the lib to my Xcode Project, the linker give me tons of errors(missing symbols like:
Undefined symbol: _pjsua_vid_codec_set_priority). I found thepjsua_vid.ois very small(200Bytes) compare to my pjsip+openh264 build file(was 47KB), perhaps this is the reason cause the link error.Here is my build log: log_file
the build script I am using:
#!/bin/sh # see http://stackoverflow.com/a/3915420/318790 function realpath { echo $(cd $(dirname "$1"); pwd)/$(basename "$1"); } __FILE__=`realpath "$0"` __DIR__=`dirname "${__FILE__}"` # download function download() { "${__DIR__}/download.sh" "$1" "$2" #--no-cache } BASE_DIR="$1" PJSIP_URL="https://github.com/pjsip/pjproject/archive/2.10.zip" #http://www.pjsip.org/release/2.8.0/pjproject-2.8.0.tar.bz2 PJSIP_DIR="$1/src" LIB_PATHS=("pjlib/lib" \ "pjlib-util/lib" \ "pjmedia/lib" \ "pjnath/lib" \ "pjsip/lib" \ "third_party/lib") OPENSSL_PREFIX= FFMPEG_PREFIX= OPENH264_PREFIX= OPUS_PREFIX= while [ "$#" -gt 0 ]; do case $1 in --with-openssl) if [ "$#" -gt 1 ]; then OPENSSL_PREFIX=$(python -c "import os,sys; print os.path.realpath(sys.argv[1])" "$2") shift 2 continue else echo 'ERROR: Must specify a non-empty "--with-openssl PREFIX" argument.' >&2 exit 1 fi ;; --with-openh264) if [ "$#" -gt 1 ]; then OPENH264_PREFIX=$(python -c "import os,sys; print os.path.realpath(sys.argv[1])" "$2") shift 2 continue else echo 'ERROR: Must specify a non-empty "--with-openh264 PREFIX" argument.' >&2 exit 1 fi ;; --with-ffmpeg) if [ "$#" -gt 1 ]; then FFMPEG_PREFIX=$(python -c "import os,sys; print os.path.realpath(sys.argv[1])" "$2") shift 2 continue else echo 'ERROR: Must specify a non-empty "--with-ffmpeg PREFIX" argument.' >&2 exit 1 fi ;; --with-opus) if [ "$#" -gt 1 ]; then OPUS_PREFIX=$(python -c "import os,sys; print os.path.realpath(sys.argv[1])" "$2") shift 2 continue else echo 'ERROR: Must specify a non-empty "--with-opus PREFIX" argument.' >&2 exit 1 fi ;; esac shift done function config_site() { SOURCE_DIR=$1 PJSIP_CONFIG_PATH="${SOURCE_DIR}/pjlib/include/pj/config_site.h" HAS_VIDEO= echo "Creating config_site.h ..." if [ -f "${PJSIP_CONFIG_PATH}" ]; then rm "${PJSIP_CONFIG_PATH}" fi echo "#define PJ_CONFIG_IPHONE 1" >> "${PJSIP_CONFIG_PATH}" echo "#define PJ_HAS_IPV6 1" >> "${PJSIP_CONFIG_PATH}" # Enable IPV6 if [[ ${OPENH264_PREFIX} ]]; then # echo "#define PJMEDIA_HAS_VID_TOOLBOX_CODEC 1" >> "${PJSIP_CONFIG_PATH}" # echo "#define PJMEDIA_HAS_OPENH264_CODEC 1" >> "${PJSIP_CONFIG_PATH}" echo "#define PJMEDIA_HAS_FFMPEG_VID_CODEC 1" >> "${PJSIP_CONFIG_PATH}" HAS_VIDEO=1 fi if [[ ${HAS_VIDEO} ]]; then echo "#define PJMEDIA_HAS_VIDEO 1" >> "${PJSIP_CONFIG_PATH}" echo "#define PJMEDIA_VIDEO_DEV_HAS_OPENGL 1" >> "${PJSIP_CONFIG_PATH}" echo "#define PJMEDIA_VIDEO_DEV_HAS_OPENGL_ES 1" >> "${PJSIP_CONFIG_PATH}" echo "#define PJMEDIA_VIDEO_DEV_HAS_IOS_OPENGL 1" >> "${PJSIP_CONFIG_PATH}" echo "#includeES3/glext.h>" >> "${PJSIP_CONFIG_PATH}" fi echo "#include config_site_sample.h>" >> "${PJSIP_CONFIG_PATH}" } function clean_libs () { ARCH=${1} for SRC_DIR in ${LIB_PATHS[*]}; do DIR="${PJSIP_DIR}/${SRC_DIR}" if [ -d "${DIR}" ]; then rm -rf "${DIR}"/* fi DIR="${PJSIP_DIR}/${SRC_DIR}-${ARCH}" if [ -d "${DIR}" ]; then rm -rf "${DIR}" fi done } function copy_libs () { ARCH=${1} for SRC_DIR in ${LIB_PATHS[*]}; do SRC_DIR="${PJSIP_DIR}/${SRC_DIR}" DST_DIR="${SRC_DIR}-${ARCH}" if [ -d "${DST_DIR}" ]; then rm -rf "${DST_DIR}" fi cp -R "${SRC_DIR}" "${DST_DIR}" rm -rf "${SRC_DIR}"/* # delete files because this directory will be used for the final lipo output done } function _build() { pushd . > /dev/null cd ${PJSIP_DIR} ARCH=$1 LOG=${BASE_DIR}/${ARCH}.log # configure CONFIGURE="./configure-iphone" if [[ ${OPENSSL_PREFIX} ]]; then CONFIGURE="${CONFIGURE} --with-ssl=${OPENSSL_PREFIX}" fi # if [[ ${OPENH264_PREFIX} ]]; then # CONFIGURE="${CONFIGURE} --with-openh264=${OPENH264_PREFIX}" # fi if [[ ${FFMPEG_PREFIX} ]]; then CONFIGURE="${CONFIGURE} --with-ffmpeg=${FFMPEG_PREFIX}" fi if [[ ${OPUS_PREFIX} ]]; then CONFIGURE="${CONFIGURE} --with-opus=${OPUS_PREFIX}" fi # flags if [[ ! ${CFLAGS} ]]; then export CFLAGS= fi if [[ ! ${LDFLAGS} ]]; then export LDFLAGS= fi if [[ ${OPENSSL_PREFIX} ]]; then export CFLAGS="${CFLAGS} -I${OPENSSL_PREFIX}/include" export LDFLAGS="${LDFLAGS} -L${OPENSSL_PREFIX}/lib" fi # if [[ ${OPENH264_PREFIX} ]]; then # export CFLAGS="${CFLAGS} -I${OPENH264_PREFIX}/include" # export LDFLAGS="${LDFLAGS} -L${OPENH264_PREFIX}/lib" # fi if [[ ${FFMPEG_PREFIX} ]]; then export CFLAGS="${CFLAGS} -I${FFMPEG_PREFIX}/include" export LDFLAGS="${LDFLAGS} -L${FFMPEG_PREFIX}/lib" fi export LDFLAGS="${LDFLAGS} -lstdc++" echo "Building for ${ARCH}..." clean_libs ${ARCH} make distclean > ${LOG} 2>&1 ARCH="-arch ${ARCH}" ${CONFIGURE} >> ${LOG} 2>&1 make dep >> ${LOG} 2>&1 make clean >> ${LOG} make lib >> ${LOG} 2>&1 copy_libs ${ARCH} } # function armv7() { # export DEVPATH="`xcrun -sdk iphoneos --show-sdk-platform-path`/Developer" # export CFLAGS="-miphoneos-version-min=8.0" # export LDFLAGS= # _build "armv7" # } # function armv7s() { # export DEVPATH="`xcrun -sdk iphoneos --show-sdk-platform-path`/Developer" # export CFLAGS="-miphoneos-version-min=8.0" # export LDFLAGS= # _build "armv7s" # } function arm64() { export DEVPATH="`xcrun -sdk iphoneos --show-sdk-platform-path`/Developer" export CFLAGS="-miphoneos-version-min=8.0" export LDFLAGS= _build "arm64" } function i386() { export DEVPATH="`xcrun -sdk iphonesimulator --show-sdk-platform-path`/Developer" export CFLAGS="-O2 -m32 -mios-simulator-version-min=8.0" export LDFLAGS="-O2 -m32 -mios-simulator-version-min=8.0" _build "i386" } function x86_64() { export DEVPATH="`xcrun -sdk iphonesimulator --show-sdk-platform-path`/Developer" export CFLAGS="-O2 -m32 -mios-simulator-version-min=8.0" export LDFLAGS="-O2 -m32 -mios-simulator-version-min=8.0" _build "x86_64" } function lipo() { TMP=`mktemp -t lipo` echo "Lipo libs... (${TMP})" for LIB_DIR in ${LIB_PATHS[*]}; do # loop over libs DST_DIR="${PJSIP_DIR}/${LIB_DIR}" # use the first architecture to find all libraries PATTERN_DIR="${DST_DIR}-$1" for PATTERN_FILE in `ls -l1 "${PATTERN_DIR}"`; do OPTIONS="" # loop over all architectures and collect the current library for ARCH in "$@"; do FILE="${DST_DIR}-${ARCH}/${PATTERN_FILE/-$1-/-${ARCH}-}" if [ -e "${FILE}" ]; then OPTIONS="$OPTIONS -arch ${ARCH} ${FILE}" fi done if [ "$OPTIONS" != "" ]; then OUTPUT_PREFIX=$(dirname "${DST_DIR}") OUTPUT="${OUTPUT_PREFIX}/lib/${PATTERN_FILE/-$1-/-}" OPTIONS="${OPTIONS} -create -output ${OUTPUT}" echo "$OPTIONS" >> "${TMP}" fi done done while read LINE; do xcrun -sdk iphoneos lipo ${LINE} done < "${TMP}" } # download "${PJSIP_URL}" "${PJSIP_DIR}" config_site "${PJSIP_DIR}" arm64 && i386 && x86_64 lipo arm64 i386 x86_64 Thanks for any advice.
-
Generating all P frame intra refresh H264 with x264
3 February 2021, by AndrewI am trying to generate a periodic intra refresh h264 stream with only P frames using ffmpeg and x264 but I always get an I frame at the start.
Is there a way with x264 to create a P frame only stream?
Commands I am using:
ffmpeg -f lavfi -re -i testsrc=duration=5:size=1920x1080:rate=30000/1001 -s 1920x1080 -pix_fmt yuv420p -f rawvideo out.yuv x264 --input-res 1920x1080 --intra-refresh out.yuv --b-pyramid none -b 0 --ref 0 -o out.264Verification:
ffprobe -show_frames out.264 |grep pict_type=IOr just looking at the x264 output e.g.
yuv [info]: 1920x1080p 0:0 @ 25/1 fps (cfr) x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 x264 [info]: profile High, level 4.0, 4:2:0, 8-bit x264 [info]: frame I:1 Avg QP:13.63 size: 12189 x264 [info]: frame P:149 Avg QP:13.59 size: 874 x264 [info]: mb I I16..4: 78.7% 18.6% 2.7% x264 [info]: mb P I16..4: 2.3% 0.1% 0.0% P16..4: 3.2% 0.3% 0.0% 0.0% 0.0% skip:94.1% x264 [info]: 8x8 transform intra:7.6% inter:91.5% x264 [info]: coded y,uvDC,uvAC intra: 1.3% 18.9% 3.6% inter: 0.1% 1.1% 0.1% x264 [info]: i16 v,h,dc,p: 86% 6% 1% 7% x264 [info]: i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 42% 22% 36% 0% 0% 0% 0% 0% 0% x264 [info]: i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 29% 30% 32% 3% 1% 3% 0% 3% 0% x264 [info]: i8c dc,h,v,p: 28% 7% 55% 9% x264 [info]: Weighted P-Frames: Y:0.0% UV:0.0% x264 [info]: kb/s:189.96 encoded 150 frames, 66.76 fps, 189.96 kb/s -
Proper use of `nalu_process` callback in x264
26 January 2021, by gsprI wish to make use of libx264's low-latency encoding mechanism, whereby a user-provided callback is called as soon as a single NAL unit is available instead of having to wait for a whole frame to be encoded before starting processing.
The x264 documentation states the following about that facility:
/* Optional low-level callback for low-latency encoding. Called for each output NAL unit * immediately after the NAL unit is finished encoding. This allows the calling application * to begin processing video data (e.g. by sending packets over a network) before the frame * is done encoding. * * This callback MUST do the following in order to work correctly: * 1) Have available an output buffer of at least size nal->i_payload*3/2 + 5 + 64. * 2) Call x264_nal_encode( h, dst, nal ), where dst is the output buffer. * After these steps, the content of nal is valid and can be used in the same way as if * the NAL unit were output by x264_encoder_encode. * * This does not need to be synchronous with the encoding process: the data pointed to * by nal (both before and after x264_nal_encode) will remain valid until the next * x264_encoder_encode call. The callback must be re-entrant. * * This callback does not work with frame-based threads; threads must be disabled * or sliced-threads enabled. This callback also does not work as one would expect * with HRD -- since the buffering period SEI cannot be calculated until the frame * is finished encoding, it will not be sent via this callback. * * Note also that the NALs are not necessarily returned in order when sliced threads is * enabled. Accordingly, the variable i_first_mb and i_last_mb are available in * x264_nal_t to help the calling application reorder the slices if necessary. * * When this callback is enabled, x264_encoder_encode does not return valid NALs; * the calling application is expected to acquire all output NALs through the callback. * * It is generally sensible to combine this callback with a use of slice-max-mbs or * slice-max-size. * * The opaque pointer is the opaque pointer from the input frame associated with this * NAL unit. This helps distinguish between nalu_process calls from different sources, * e.g. if doing multiple encodes in one process. */ void (*nalu_process)( x264_t *h, x264_nal_t *nal, void *opaque );This seems straight forward enough. However, when I run the following dummy code, I get a segfault on the marked line. I've tried to add some debugging to
x264_nal_encodeitself to understand where it goes wrong, but it seems to be the function call itself that results in a segfault. Am I missing something here? (Let's ignore the fact that the use ofassertprobably makescbnon-reentrant – it's only there to indicate to the reader that my workspace buffer is more than large enough.)#include #include #include #include #include #include #define WS_SIZE 10000000 uint8_t * workspace; void cb(x264_t * h, x264_nal_t * nal, void * opaque) { assert((nal->i_payload*3)/2 + 5 + 64 < WS_SIZE); x264_nal_encode(h, workspace, nal); // Segfault here. // Removed: Process nal. } int main(int argc, char ** argv) { uint8_t * fake_frame = malloc(1280*720*3); memset(fake_frame, 0, 1280*720*3); workspace = malloc(WS_SIZE); x264_param_t param; int status = x264_param_default_preset(¶m, "ultrafast", "zerolatency"); assert(status == 0); param.i_csp = X264_CSP_RGB; param.i_width = 1280; param.i_height = 720; param.i_threads = 1; param.i_lookahead_threads = 1; param.i_frame_total = 0; param.i_fps_num = 30; param.i_fps_den = 1; param.i_slice_max_size = 1024; param.b_annexb = 1; param.nalu_process = cb; status = x264_param_apply_profile(¶m, "high444"); assert(status == 0); x264_t * h = x264_encoder_open(¶m); assert(h); x264_picture_t pic; status = x264_picture_alloc(&pic, param.i_csp, param.i_width, param.i_height); assert(pic.img.i_plane == 1); x264_picture_t pic_out; x264_nal_t * nal; // Not used. We process NALs in cb. int i_nal; for (int i = 0; i < 100; ++i) { pic.i_pts = i; pic.img.plane[0] = fake_frame; status = x264_encoder_encode(h, &nal, &i_nal, &pic, &pic_out); } x264_encoder_close(h); x264_picture_clean(&pic); free(workspace); free(fake_frame); return 0; }Edit: The segfault happens the first time
cbcallsx264_nal_encode. If I switch to a different preset, where more frames are encoded before the first callback happens, then several successful calls tox264_encoder_encodeare made before the first callback, and hence segfault, occurs.
