Newest 'libx264' Questions - Stack Overflow

Les articles publiés sur le site

• How libx264 and ffmpeg encode and decode H.264

  6 février 2017, par e_soroush

  I want to know what exactly get lost in H.264 encoding. I am encoding a video using libx264 and decode it with ffmpeg and again I encode the resulting video with libx264, with the same parameters and decode it for the second time. What I anticipate is first and second decoded videos must be the same in size and appearance, however, it's not! For example, I have 13000 bytes in first slice NAL units in the first coded frame and 12000 bytes in the second coded frames and that's odd. Because whatever lossy information existed in a frame should have been eliminated in the first encoding process and second encoding should be a process that its result same NAL units. Am I wrong or somewhere in my implementations I mistook?

• How libx264 and ffmpeg encode and decode H.264

  6 février 2017, par e_soroush

  I want to know what exactly get lost in H.264 encoding. I am encoding a video using libx264 and decode it with ffmpeg and again I encode the resulting video with libx264, with the same parameters and decode it for the second time. What I anticipate is first and second decoded videos must be the same in size and appearance, however, it's not! For example, I have 13000 bytes in first slice NAL units in the first coded frame and 12000 bytes in the second coded frames and that's odd. Because whatever lossy information existed in a frame should have been eliminated in the first encoding process and second encoding should be a process that its result same NAL units. Am I wrong or somewhere in my implementations I mistook?

• Live555 : X264 Stream Live source based on "testOnDemandRTSPServer"

  12 janvier 2017, par user2660369

  I am trying to create a rtsp Server that streams the OpenGL output of my program. I had a look at How to write a Live555 FramedSource to allow me to stream H.264 live, but I need the stream to be unicast. So I had a look at testOnDemandRTSPServer. Using the same Code fails. To my understanding I need to provide memory in which I store my h264 frames so the OnDemandServer can read them on Demand.

  H264VideoStreamServerMediaSubsession.cpp

  H264VideoStreamServerMediaSubsession*
  H264VideoStreamServerMediaSubsession::createNew(UsageEnvironment& env,
                            Boolean reuseFirstSource) {
    return new H264VideoStreamServerMediaSubsession(env, reuseFirstSource);
  }
  
  H264VideoStreamServerMediaSubsession::H264VideoStreamServerMediaSubsession(UsageEnvironment& env, Boolean reuseFirstSource)
    : OnDemandServerMediaSubsession(env, reuseFirstSource), fAuxSDPLine(NULL), fDoneFlag(0), fDummyRTPSink(NULL) {
  }
  
  H264VideoStreamServerMediaSubsession::~H264VideoStreamServerMediaSubsession() {
    delete[] fAuxSDPLine;
  }
  
  static void afterPlayingDummy(void* clientData) {
    H264VideoStreamServerMediaSubsession* subsess = (H264VideoStreamServerMediaSubsession*)clientData;
    subsess->afterPlayingDummy1();
  }
  
  void H264VideoStreamServerMediaSubsession::afterPlayingDummy1() {
    // Unschedule any pending 'checking' task:
    envir().taskScheduler().unscheduleDelayedTask(nextTask());
    // Signal the event loop that we're done:
    setDoneFlag();
  }
  
  static void checkForAuxSDPLine(void* clientData) {
    H264VideoStreamServerMediaSubsession* subsess = (H264VideoStreamServerMediaSubsession*)clientData;
    subsess->checkForAuxSDPLine1();
  }
  
  void H264VideoStreamServerMediaSubsession::checkForAuxSDPLine1() {
    char const* dasl;
  
    if (fAuxSDPLine != NULL) {
      // Signal the event loop that we're done:
      setDoneFlag();
    } else if (fDummyRTPSink != NULL && (dasl = fDummyRTPSink->auxSDPLine()) != NULL) {
      fAuxSDPLine = strDup(dasl);
      fDummyRTPSink = NULL;
  
      // Signal the event loop that we're done:
      setDoneFlag();
    } else {
      // try again after a brief delay:
      int uSecsToDelay = 100000; // 100 ms
      nextTask() = envir().taskScheduler().scheduleDelayedTask(uSecsToDelay,
                    (TaskFunc*)checkForAuxSDPLine, this);
    }
  }
  
  char const* H264VideoStreamServerMediaSubsession::getAuxSDPLine(RTPSink* rtpSink, FramedSource* inputSource) {
    if (fAuxSDPLine != NULL) return fAuxSDPLine; // it's already been set up (for a previous client)
  
    if (fDummyRTPSink == NULL) { // we're not already setting it up for another, concurrent stream
      // Note: For H264 video files, the 'config' information ("profile-level-id" and "sprop-parameter-sets") isn't known
      // until we start reading the file.  This means that "rtpSink"s "auxSDPLine()" will be NULL initially,
      // and we need to start reading data from our file until this changes.
      fDummyRTPSink = rtpSink;
  
      // Start reading the file:
      fDummyRTPSink->startPlaying(*inputSource, afterPlayingDummy, this);
  
      // Check whether the sink's 'auxSDPLine()' is ready:
      checkForAuxSDPLine(this);
    }
  
    envir().taskScheduler().doEventLoop(&fDoneFlag);
  
    return fAuxSDPLine;
  }
  
  FramedSource* H264VideoStreamServerMediaSubsession::createNewStreamSource(unsigned /*clientSessionId*/, unsigned& estBitrate) {
    estBitrate = 500; // kb
    megamol::remotecontrol::View3D_MRC *parent = (megamol::remotecontrol::View3D_MRC*)this->parent;
    return H264VideoStreamFramer::createNew(envir(), parent->h264FramedSource);
  }
  
  RTPSink* H264VideoStreamServerMediaSubsession::createNewRTPSink(Groupsock* rtpGroupsock, unsigned char rtpPayloadTypeIfDynamic, FramedSource* /*inputSource*/) {
    return H264VideoRTPSink::createNew(envir(), rtpGroupsock, rtpPayloadTypeIfDynamic);
  }
  

  FramedSource.cpp

  H264FramedSource* H264FramedSource::createNew(UsageEnvironment& env,
                                            unsigned preferredFrameSize,
                                            unsigned playTimePerFrame)
  {
      return new H264FramedSource(env, preferredFrameSize, playTimePerFrame);
  }
  
  H264FramedSource::H264FramedSource(UsageEnvironment& env,
                                 unsigned preferredFrameSize,
                                 unsigned playTimePerFrame)
      : FramedSource(env),
      fPreferredFrameSize(fMaxSize),
      fPlayTimePerFrame(playTimePerFrame),
      fLastPlayTime(0),
      fCurIndex(0)
  {
  
      x264_param_default_preset(&param, "veryfast", "zerolatency");
      param.i_threads = 1;
      param.i_width = 1024;
      param.i_height = 768;
      param.i_fps_num = 30;
      param.i_fps_den = 1;
      // Intra refres:
      param.i_keyint_max = 60;
      param.b_intra_refresh = 1;
      //Rate control:
      param.rc.i_rc_method = X264_RC_CRF;
      param.rc.f_rf_constant = 25;
      param.rc.f_rf_constant_max = 35;
      param.i_sps_id = 7;
      //For streaming:
      param.b_repeat_headers = 1;
      param.b_annexb = 1;
      x264_param_apply_profile(&param, "baseline");
  
      param.i_log_level = X264_LOG_ERROR;
  
      encoder = x264_encoder_open(&param);
      pic_in.i_type            = X264_TYPE_AUTO;
      pic_in.i_qpplus1         = 0;
      pic_in.img.i_csp         = X264_CSP_I420;
      pic_in.img.i_plane       = 3;
  
  
      x264_picture_alloc(&pic_in, X264_CSP_I420, 1024, 768);
  
      convertCtx = sws_getContext(1024, 768, PIX_FMT_RGBA, 1024, 768, PIX_FMT_YUV420P, SWS_FAST_BILINEAR, NULL, NULL, NULL);
      eventTriggerId = envir().taskScheduler().createEventTrigger(deliverFrame0);
  }
  
  H264FramedSource::~H264FramedSource()
  {
      envir().taskScheduler().deleteEventTrigger(eventTriggerId);
      eventTriggerId = 0;
  }
  
  void H264FramedSource::AddToBuffer(uint8_t* buf, int surfaceSizeInBytes)
  {
      uint8_t* surfaceData = (new uint8_t[surfaceSizeInBytes]);
  
      memcpy(surfaceData, buf, surfaceSizeInBytes);
  
      int srcstride = 1024*4;
      sws_scale(convertCtx, &surfaceData, &srcstride,0, 768, pic_in.img.plane, pic_in.img.i_stride);
      x264_nal_t* nals = NULL;
      int i_nals = 0;
      int frame_size = -1;
  
  
      frame_size = x264_encoder_encode(encoder, &nals, &i_nals, &pic_in, &pic_out);
  
      static bool finished = false;
  
      if (frame_size >= 0)
      {
      static bool alreadydone = false;
      if(!alreadydone)
      {
  
          x264_encoder_headers(encoder, &nals, &i_nals);
          alreadydone = true;
      }
      for(int i = 0; i < i_nals; ++i)
      {
          m_queue.push(nals[i]);
      }
      }
      delete [] surfaceData;
      surfaceData = nullptr;
  
      envir().taskScheduler().triggerEvent(eventTriggerId, this);
  }
  
  void H264FramedSource::doGetNextFrame()
  {
      deliverFrame();
  }
  
  void H264FramedSource::deliverFrame0(void* clientData)
  {
      ((H264FramedSource*)clientData)->deliverFrame();
  }
  
  void H264FramedSource::deliverFrame()
  {
      x264_nal_t nalToDeliver;
  
      if (fPlayTimePerFrame > 0 && fPreferredFrameSize > 0) {
      if (fPresentationTime.tv_sec == 0 && fPresentationTime.tv_usec == 0) {
          // This is the first frame, so use the current time:
          gettimeofday(&fPresentationTime, NULL);
      } else {
          // Increment by the play time of the previous data:
          unsigned uSeconds   = fPresentationTime.tv_usec + fLastPlayTime;
          fPresentationTime.tv_sec += uSeconds/1000000;
          fPresentationTime.tv_usec = uSeconds%1000000;
      }
  
      // Remember the play time of this data:
      fLastPlayTime = (fPlayTimePerFrame*fFrameSize)/fPreferredFrameSize;
      fDurationInMicroseconds = fLastPlayTime;
      } else {
      // We don't know a specific play time duration for this data,
      // so just record the current time as being the 'presentation time':
      gettimeofday(&fPresentationTime, NULL);
      }
  
      if(!m_queue.empty())
      {
      m_queue.wait_and_pop(nalToDeliver);
  
      uint8_t* newFrameDataStart = (uint8_t*)0xD15EA5E;
  
      newFrameDataStart = (uint8_t*)(nalToDeliver.p_payload);
      unsigned newFrameSize = nalToDeliver.i_payload;
  
      // Deliver the data here:
      if (newFrameSize > fMaxSize) {
          fFrameSize = fMaxSize;
          fNumTruncatedBytes = newFrameSize - fMaxSize;
      }
      else {
          fFrameSize = newFrameSize;
      }
  
      memcpy(fTo, nalToDeliver.p_payload, nalToDeliver.i_payload);
  
      FramedSource::afterGetting(this);
      }
  }
  

  Relevant part of the RTSP-Server Therad

    RTSPServer* rtspServer = RTSPServer::createNew(*(parent->env), 8554, NULL);
    if (rtspServer == NULL) {
      *(parent->env) << "Failed to create RTSP server: " << (parent->env)->getResultMsg() << "\n";
      exit(1);
    }
    char const* streamName = "Stream";
    parent->h264FramedSource = H264FramedSource::createNew(*(parent->env), 0, 0);
    H264VideoStreamServerMediaSubsession *h264VideoStreamServerMediaSubsession = H264VideoStreamServerMediaSubsession::createNew(*(parent->env), true);
    h264VideoStreamServerMediaSubsession->parent = parent;
    sms->addSubsession(h264VideoStreamServerMediaSubsession);
    rtspServer->addServerMediaSession(sms);
  
    parent->env->taskScheduler().doEventLoop(); // does not return
  

  Once a connection exists the render loop calls

  h264FramedSource->AddToBuffer(videoData, 1024*768*4);
  

• Why does moviepy complain about bitrate while generating audiofile ?

  11 janvier 2017, par Roman

  I have just try to use moviepy library for the first time. Generation of movies from numpy arrays was really simple, intuitive and worked out of box. This is what I tried:

  from moviepy.editor import VideoClip
  import numpy as np
  
  def make_frame(t):
  
      val = int(255.0*(t/3.0))
  
      ls = []
      for height in range(100):
          row = []
          for width in range(300):
              row.append([val,0,0])
          ls.append(row)
      frame = np.array(ls)
      return frame
  
  animation = VideoClip(make_frame, duration = 3)
  
  animation.write_gif('first_try.gif', fps=24)
  animation.write_videofile('first_try.mp4', fps=24)
  

  Then I wanted to use moviepy to generate sound. In theory it should work in a very similar way. Here is what I tried:

  from moviepy.editor import AudioClip
  import numpy as np
  
  make_frame = lambda t : 2*[ np.sin(404 * 2 * np.pi * t) ]
  clip = AudioClip(make_frame, duration=5)
  
  clip.write_audiofile('sound.mp4')
  

  However, I got an error message:

  [MoviePy] Writing audio in sound.mp4
  |----------| 0/111   0% [elapsed: 00:00 left: ?, ? iters/sec]Traceback (most recent call last):
    File "sound.py", line 9, in 
      clip.write_audiofile('sound.mp4')
    File "", line 2, in write_audiofile
    File "/usr/local/lib/python2.7/dist-packages/moviepy/decorators.py", line 54, in requires_duration
      return f(clip, *a, **k)
    File "/usr/local/lib/python2.7/dist-packages/moviepy/audio/AudioClip.py", line 204, in write_audiofile
      verbose=verbose, ffmpeg_params=ffmpeg_params)
    File "", line 2, in ffmpeg_audiowrite
    File "/usr/local/lib/python2.7/dist-packages/moviepy/decorators.py", line 54, in requires_duration
      return f(clip, *a, **k)
    File "/usr/local/lib/python2.7/dist-packages/moviepy/audio/io/ffmpeg_audiowriter.py", line 162, in ffmpeg_audiowrite
      writer.write_frames(chunk)
    File "/usr/local/lib/python2.7/dist-packages/moviepy/audio/io/ffmpeg_audiowriter.py", line 122, in write_frames
      raise IOError(error)
  IOError: [Errno 32] Broken pipe
  
  MoviePy error: FFMPEG encountered the following error while writing file sound.mp4:
  
  Invalid encoder type 'libx264'
  
  
  The audio export failed, possily because the bitrate you specified was two high or too low for the video codec.
  

  Does anybody know what this error means and how this problem can be resolved?

• How to encode h.264 with libavcodec/x264 ?

  23 décembre 2016, par szatmary

  I am attempting to encode video using libavcodec/libavformat. Audio works great, but when I try to encode video I get the following errors:

  [libx264 @ 0x10182a000]broken ffmpeg default settings detected  
  [libx264 @ 0x10182a000]use an encoding preset (vpre)  
  

  easy to fix using the command line ffmpeg, but I am trying to do this in C. my options are

  AVStream *pVideoOutStream = av_new_stream(pOutFormatCtx, 0);  
  AVCodecContext *pVideoOutCodecCtx  = pVideoOutStream->codec;  
  
  pVideoOutCodecCtx->codec_id        = CODEC_ID_H264;    
  pVideoOutCodecCtx->codec_type      = CODEC_TYPE_VIDEO;  
  pVideoOutCodecCtx->bit_rate        = pVideoInCodecCtx->bit_rate;  
  pVideoOutCodecCtx->width           = pVideoInCodecCtx->width;    
  pVideoOutCodecCtx->height          = pVideoInCodecCtx->height;  
  pVideoOutCodecCtx->pix_fmt         = pVideoInCodecCtx->pix_fmt;    
  pVideoOutCodecCtx->sample_rate     = pVideoInCodecCtx->sample_rate;    
  pVideoOutCodecCtx->gop_size        = 30;  
  

  but avcodec_open() fails.

  What other values do I need to set to make x264 happy?