Newest 'x264' Questions - Stack Overflow

Les articles publiés sur le site

• How to get the bit size of a marcoblock in x264 ?

  29 octobre 2021, par Corvo Attano

  How to get the bit size of a marcoblock in x264 after encode ? Does x264 has any API to get the bit size of each marcoblocks after encoding a slice or a frame? If the answer is no, which function should I modify to handle the bit size after encoding marcoblocks ?

• WebRTC Stream Freezes When Picture Complexity Increases

  12 octobre 2021, par user1259576

  I am developing an application that uses WebRTC to display a live video stream being captured from a V4L2 source. The stream originates from a Linux box that has a DVI-USB capture card, is encoded to H264 by ffmpeg and sent to RTP, received by a Janus WebRTC server which is accessed by the web interface.

  Here is my current ffmpeg command - pretty simple: ffmpeg -f v4l2 -i /dev/video0 -vf "transpose=1,scale=768:1024" -vcodec libx264 -profile:v baseline -pix_fmt yuv420p -f rtp rtp://10.116.80.86:8004

  I can't go into details, but the DVI source generates a portrait 768x1024 image that initially is a simple image where the only movement is a small clock near the center that increments every second. At this stage, everything appears to work great. The image is high-quality and continuous/smooth in the browser.

  Once I interact with the DVI source, a more complex image is generated, with some text/lines in the upper half. Still not very complex - only 2 colors involved and some basic 1px line shapes, and only the little clock is moving. At this point, the video starts to freeze frequently, and only updates once in a while for a few seconds. Bandwidth should not be an issue here, and the bitrate appears to stay high. However, many fewer frames are decoded.

  I have also tried scaling the video down to 480x640 from 768x1024 and with that change the issue does not occur. However, I really need the full resolution and, again, there should not be a bandwidth issue here.

  I have also tried capturing the output of ffmpeg to a file rather than streaming to RTP and in the file everything is good.

  Here is a screenshot of the WebRTC internals (in Edge) for this stream. You can clearly see when the video image changes from the simple clock to including more shapes & text (nothing is changed here other than the image from the DVI source):

  

  In Firefox, the video just freezes whenever frames are not decoded. In Edge, the video goes black after a moment with no frames decoded.

  Any ideas as to what might be causing this?

• Encoding images to h264 and rtp output : SDP file without sprop-parameter-sets does not play

  14 septembre 2021, par oarfish

  tl;dr: I try to encode acquired camera frames to h264, send via RTP and play this back on another device. SDP file generated by ffmpeg for a sample video has info which my own SDP file misses. My SDP file plays in ffplay, but not VLC, while both play ffmpeg's SDP file. I am suspecting missing sprop-parameter-sets in my SDP file.

  Ultimately I want to play this back in VLC.

  ---

  I am writing code that encodes images to h264 and outputs to an RTP server (or client? anyway the part that is listening). I generate an SDP file for this.

  • ffplay plays the stream without problem
  • mplayer shows a green box embedded in a larger black box, but I read somewhere it only supports mpegts over RTP, so not sure
  • VLC does not play the SDP file.

  Now when instead I use some random video and have ffmpeg output an SDP file like so

  ffmpeg     -re     -i some.mp4     -an     -c:v copy -f rtp -sdp_file
  video.sdp     "rtp://127.0.0.1:5004"
  

  I can see that the generated SDP file – which plays in both ffplay and VLC – includes the base64 encoded sprop-parameter-sets field, and removing this causes the stream to not play.

  > cat video.sdp
  v=0
  o=- 0 0 IN IP4 127.0.0.1
  s=No Name
  c=IN IP4 127.0.0.1
  t=0 0
  a=tool:libavformat 58.76.100
  m=video 5004 RTP/AVP 96
  b=AS:1034
  a=rtpmap:96 H264/90000
  a=fmtp:96 packetization-mode=1;
  sprop-parameter-sets=Z2QANKzZQDAA7fiMBagICAoAAAMAAgAAAwDwHjBjLA==,aOvjyyLA;
  profile-level-id=640034
  

  My own SDP file on the other hand, does not contain this information, and VLC hangs for 10s and then stops trying with "no data received".

  > cat test.sdp
  v=0
  o=- 0 0 IN IP4 127.0.0.1
  s=No Name
  c=IN IP4 127.0.0.1
  t=0 0
  a=tool:libavformat 58.76.100
  m=video 44499 RTP/AVP 96
  b=AS:2000
  a=rtpmap:96 H264/90000
  a=fmtp:96 packetization-mode=1
  

  So my theory is that my custom code must somehow add this SPS information to the SDP file. But despite hours of searching, I could not find a structured way to set the extradata field on the AVStream's AVCodecParams. The code I'm using is roughly this (I'm sure there's unrelated errors in there):

  // variables
  std::vector imgbuf;
  AVFormatContext *ofmt_ctx = nullptr;
  AVCodec *out_codec = nullptr;
  AVStream *out_stream = nullptr;
  AVCodecContext *out_codec_ctx = nullptr;
  SwsContext *swsctx = nullptr;
  cv::Mat canvas_;
  unsigned int height_;
  unsigned int width_;
  unsigned int fps_;
  AVFrame *frame_ = nullptr;
  
  AVOutputFormat *format = av_guess_format("rtp", nullptr, nullptr);
  const auto url = std::string("rtp://127.0.0.1:5001");
  avformat_alloc_output_context2(ofmt_ctx, format, format->name, url.c_str());
  
  out_codec = avcodec_find_encoder(AV_CODEC_ID_H264);
  stream = avformat_new_stream(ofmt_ctx, out_codec);
  out_codec_ctx = avcodec_alloc_context3(out_codec);
  
  // then, for each incoming image:
  while (receive_image) {
    static bool first_time = true;
    if (first_time) {
      // discover necessary params such as image dimensions from the first
      // received image
      first_time = false;
      height_ = image.rows;
      width_ = image.cols;
  
      codec_ctx->codec_tag = 0;
      codec_ctx->bit_rate = 2e6;
      // does nothing, unfortunately
      codec_ctx->thread_count = 1;
      codec_ctx->codec_id = AV_CODEC_ID_H264;
      codec_ctx->codec_type = AVMEDIA_TYPE_VIDEO;
      codec_ctx->width = width_;
      codec_ctx->height = height_;
      codec_ctx->gop_size = 6;
      codec_ctx->pix_fmt = AV_PIX_FMT_YUV420P;
      codec_ctx->framerate = fps_;
      codec_ctx->time_base = av_inv_q(fps_);
  
      avcodec_parameters_from_context(stream, out_codec_ctx);
  
      // this stuff is empty: is that the problem?
      stream->codecpar->extradata = codec_ctx->extradata;
      stream->codecpar->extradata_size = codec_ctx->extradata_size;
  
      AVDictionary *codec_options = nullptr;
      av_dict_set(&codec_options, "profile", "high", 0);
      av_dict_set(&codec_options, "preset", "ultrafast", 0);
      av_dict_set(&codec_options, "tune", "zerolatency", 0);
  
      // open video encoder
      avcodec_open2(codec_ctx, codec, &codec_options);
  
      stream->time_base.num = 1;
      stream->time_base.den = fps_;
      avio_open(&(ofmt_ctx->pb), ofmt_ctx->filename, AVIO_FLAG_WRITE);
  
      /* Write a file for VLC */
      char buf[200000];
      AVFormatContext *ac[] = {ofmt_ctx};
      av_sdp_create(ac, 1, buf, 20000);
      printf("sdp:\n%s\n", buf);
      FILE *fsdp = fopen("test.sdp", "w");
      fprintf(fsdp, "%s", buf);
      fclose(fsdp);
  
      swsctx = sws_getContext(width_, height_, AV_PIX_FMT_BGR24, width_, height_,
                              out_codec_ctx->pix_fmt, SWS_BICUBIC, nullptr,
                              nullptr, nullptr);
    }
  
    if (!frame_) {
      frame_ = av_frame_alloc();
  
      std::uint8_t *framebuf = new uint8_t[av_image_get_buffer_size(
          codec_ctx->pix_fmt, width_, height_, 1)];
      av_image_fill_arrays(frame_->data, frame_->linesize, framebuf,
                           codec_ctx->pix_fmt, width, height, 1);
      frame_->width = width_;
      frame_->height = height_;
      frame_->format = static_cast(codec_ctx->pix_fmt);
      success = avformat_write_header(ofmt_ctx, nullptr);
    }
    if (imgbuf.empty()) {
      imgbuf.resize(height_ * width_ * 3 + 16);
      canvas_ = cv::Mat(height_, width_, CV_8UC3, imgbuf.data(), width_ * 3);
    } else {
      image.copyTo(canvas_);
    }
  
    const int stride[] = {static_cast(image.step[0])};
  
    sws_scale(swsctx, &canvas_.data, stride, 0, canvas_.rows, frame_->data,
              frame_->linesize);
    frame_->pts += av_rescale_q(1, out_codec_ctx->time_base, stream->time_base);
  
    AVPacket pkt = {0};
    avcodec_send_frame(out_codec_ctx, frame_);
    avcodec_receive_packet(out_codec_ctx, &pkt);
    av_interleaved_write_frame(ofmt_ctx, &pkt);
  }
  

  Can anyone offer some advice here?

  --

  Update

  When setting

  this->out_codec_ctx->flags |=AV_CODEC_FLAG_GLOBAL_HEADER;
  

  extradata is actually present in the codec context, but I had to move avcodec_parameters_from_context() after avcodec_open2(), as the extradata is empty before opening the codec. I now get sprop-parameter-sets in the SDP file, but VLC still does not play it.

• Error in compiling x264 on ARM 64-bit platform

  2 septembre 2021, par yanzhang.guo

  The same x264 source code I compiled on Debian 10 (Buster) system has no problem, but I can't compile it on an ARM 64-bit platform. Why?

  Maybe there is something wrong with the Makefile, but I don't know where it is.

  1. Configure

     ./configure --disable-asm --enable-shared  --host=aarch64-linux
     
     platform:      AARCH64
     byte order:    little-endian
     system:        LINUX
     cli:           yes
     libx264:       internal
     shared:        yes
     static:        no
     asm:           no
     interlaced:    yes
     avs:           avxsynth
     lavf:          no
     ffms:          no
     mp4:           no
     gpl:           yes
     thread:        posix
     opencl:        yes
     filters:       crop select_every
     lto:           no
     debug:         no
     gprof:         no
     strip:         no
     PIC:           yes
     bit depth:     all
     chroma format: all
     

  2. Edit file config.mak,

     SRCPATH=.
     prefix=/usr/local
     exec_prefix=${prefix}
     bindir=${exec_prefix}/bin
     libdir=${exec_prefix}/lib
     includedir=${prefix}/include
     SYS_ARCH=AARCH64
     SYS=LINUX
     CC=aarch64-linux-gnu-gcc
     CFLAGS=-Wno-maybe-uninitialized -Wshadow -O3 -ffast-math  -Wall -I. -I$(SRCPATH) -std=gnu99 -
     D_GNU_SOURCE -fPIC -fomit-frame-pointer -fno-tree-vectorize
     COMPILER=GNU
     COMPILER_STYLE=GNU
     DEPMM=-MM -g0
     DEPMT=-MT
     LD=aarch64-linux-gnu-gcc -o
     LDFLAGS= -lm -lpthread -ldl
     LIBX264=libx264.a
     AR=aarch64-linux-gnu-gcc-ar rc
     RANLIB=aarch64-linux-gnu-gcc-ranlib
     STRIP=aarch64-linux-gnu-strip
     INSTALL=install
     AS=
     ASFLAGS= -I. -I$(SRCPATH) -DSTACK_ALIGNMENT=16 -DPIC
     RC=
     RCFLAGS=
     EXE=
     HAVE_GETOPT_LONG=1
     DEVNULL=/dev/null
     PROF_GEN_CC=-fprofile-generate
     PROF_GEN_LD=-fprofile-generate
     PROF_USE_CC=-fprofile-use
     PROF_USE_LD=-fprofile-use
     HAVE_OPENCL=yes
     CC_O=-o $@
     default: cli
     install: install-cli
     SOSUFFIX=so
     SONAME=libx264.so.157
     SOFLAGS=-shared -Wl,-soname,$(SONAME)  -Wl,-Bsymbolic
     default: lib-shared
     install: install-lib-shared
     LDFLAGSCLI = -ldl
     CLI_LIBX264 = $(LIBX264)
     

  3. Make, an error occurred, details in error.

  4. Add file config.h after configure,

      #define HAVE_MALLOC_H 1
      #define ARCH_X86_64 1
      #define SYS_LINUX 1
      #define STACK_ALIGNMENT 64
      #define HAVE_POSIXTHREAD 1
      #define HAVE_CPU_COUNT 1
      #define HAVE_THREAD 1
      #define HAVE_LOG2F 1
      #define HAVE_STRTOK_R 1
      #define HAVE_CLOCK_GETTIME 1
      #define HAVE_MMAP 1
      #define HAVE_THP 1
      #define HAVE_AVS 1
      #define USE_AVXSYNTH 1
      #define HAVE_VECTOREXT 1
      #define fseek fseeko
      #define ftell ftello
      #define HAVE_BITDEPTH8 1
      #define HAVE_BITDEPTH10 1
      #define HAVE_GPL 1
      #define HAVE_INTERLACED 1
      #define HAVE_OPENCL (BIT_DEPTH==8)
      #define HAVE_ALTIVEC 0
      #define HAVE_ALTIVEC_H 0
      #define HAVE_MMX 0
      #define HAVE_ARMV6 0
      #define HAVE_ARMV6T2 0
      #define HAVE_NEON 0
      #define HAVE_BEOSTHREAD 0
      #define HAVE_WIN32THREAD 0
      #define HAVE_SWSCALE 0
      #define HAVE_LAVF 0
      #define HAVE_FFMS 0
      #define HAVE_GPAC 0
      #define HAVE_LSMASH 0
      #define HAVE_X86_INLINE_ASM 0
      #define HAVE_AS_FUNC 0
      #define HAVE_INTEL_DISPATCHER 0
      #define HAVE_MSA 0
      #define HAVE_WINRT 0
      #define HAVE_VSX 0
      #define HAVE_ARM_INLINE_ASM 0
     

• High Quality Screen Capture with FFmpeg

  27 août 2021, par plumzl

  I'm trying to figure out a command for high quality screen capture on REHL.

  I have tried various options, but none really worked. The quality is not good enough. I'm quite new to ffmpeg and doesn't quite understand all the different flags. So any help would be appreciated!

  The commands I have tried:

  ffmpeg -y -f x11grab -r 30 -s 1920x1080 -i :0.0+0,0 -vcodec libx264 -strict -2 -b:v 16M output.mov
  

  The first command I tried. The weird thing is it doesn't seem to preserve the bitrate specified -b:v 16M. The output video's bitrate is only 2mb/s. Is that not the correct way to specify bitrate when doing screen capture?

  ffmpeg -f x11grab -s 1920x1080 -r 30 -i :0.0 -qscale 0 -vcodec huffyuv output.mov
  

  The huffyuv codec is supposed to be lossless, but the quality doesn't seem to be any better than the libx264 output.

  ffmpeg -f x11grab -r 30 -s 1920x1080 -i :0.0+0,0 -vcodec copy -pix_fmt yuv420p -qscale 0 -an output.mkv
  ffmpeg -f rawvideo -pix_fmt yuv420p -s:v 1920x1080 -r 30 -i output.mkv -an -c:v libx264 -qscale 0 result.mov 
  

  I'm even trying to capture rawvideo and then convert it to libx264 codec viedeo. But the output is corrupted.

  I'm using the version of ffmpeg that's built on April 23rd this year. It doesn't have the preset like -vpre lossless_ultrafast which I see mentioned in a lot posts.

  Edit:

  After more research, this is the command I end up using. I have to use h264 codec and yuv420p as pixel_fmt since I need to be able to play the quicktimes using quicktime.

  ffmpeg -y -f x11grab -framerate 30 -video_size 1920x1080 -i :0.0+0,0 -c:v libx264 -pix_fmt yuv420p -qp 0 -preset ultrafast output.mov
  

  The quality doesn't seem to be much improved even though -qp is set to 0. But it could also because of the rgb conversion to yuv color space according to this post.

  https://trac.ffmpeg.org/wiki/Encode/H.264