Recherche avancée

Sur d’autres sites (7263)

• Why codecs x264/x265 ignores pts and dts of input frame ?

  1er mars 2018, par Iceman

  I’m trying to encode images from a webcam with libx265 (libx264 tried earlier) ...
  The webcam can not shoot with stable FPS because of the different amount of light entering the matrix and, as a result, different delays. Therefore, I count the fps and dts of the incoming frame and set these values for the corresponding parameters of the x265_image object, and init the encoder fpsNum with 1000 and fpsDenom with 1 (for millisecond timebase).
  The problem is that the encoder ignores pts and dts of input image and encodes at 1000 fps ! The same trick with timebase produces smooth record with libvpx. Why it does not work with x264/x265 codecs ?

  Here is parameters initialization :

  ...
  
      error = (x265_param_default_preset(param, "fast", "zerolatency") != 0);
  
      if(!error){
  
          param->sourceWidth = width;
  
          param->sourceHeight = height;
  
          param->frameNumThreads = 1;
  
          param->fpsNum = 1000;
  
          param->fpsDenom =  1;
  
          // Intra refres:
  
          param->keyframeMax = 15;
  
          param->intraRefine = 1;
  
          // Rate control:
  
          param->rc.rateControlMode = X265_RC_CQP;
  
          param->rc.rfConstant = 12;
  
          param->rc.rfConstantMax = 48;
  
          // For streaming:
  
          param->bRepeatHeaders = 1;
  
          param->bAnnexB = 1;
  
          encoder = x265_encoder_open(param);
  
          ...
  
      }
  
  ...

  Here is frame adding function :

  bool hevc::Push(unsigned char *data){
  
      if(!error){
  
          std::lock_guard lock(m_framestack);
  
          if( timer > 0){
  
              framestack.back()->dts = clock() - timer;
  
              timer+= framestack.back()->dts;
  
          }
  
          else{timer = clock();}
  
          x265_picture *picture = x265_picture_alloc();
  
          if( picture){
  
              x265_picture_init(param, picture);
  
              picture->height = param->sourceHeight;
  
              picture->stride[0] = param->sourceWidth;
  
              picture->stride[1] = picture->stride[2] = picture->stride[0] / 2;
  
              picture->planes[0] = new char[  luma_size];
  
              picture->planes[1] = new char[chroma_size];
  
              picture->planes[2] = new char[chroma_size];
  
  
  
              colorspaces::BGRtoI420(param->sourceWidth, param->sourceHeight, data, (byte*)picture->planes[0], (byte*)picture->planes[1], (byte*)picture->planes[2]);
  
              picture->pts = picture->dts = 0;
  
              framestack.emplace_back(picture);
  
          }
  
          else{error = true;}
  
      }
  
      return !error;
  
  }

  Global PTS is increasing right after x265_encoder_encode call :
  pts+= pic_in->dts; and sets as a pts of new image from framestack queue when it comes to encoder.

  Can the x265/x264 codecs encode at variable fps ? How to configure it if yes ?

• FFMPEG segment records a minute worth of data but metadata shows 0.1s length video

  1er février 2018, par Soft Strategy

  I’m recording a stream from a Live555 restream of a rtsp camera. I record in chunks of one minute videos. This is the setup :

  RTSP Camera -> Live555 Server -> Recorder

  Without the Live555 restreamer server I always record the rtsp camera stream with ffmpeg with no issues. This is the command :

  ffmpeg -rtsp_transport tcp -y -i rtsp://IPcamera/url/to/video -c copy -f segment -segment_time 60 -reset_timestamps 1 -segment_atclocktime 1 -strftime 1 /video/%Y-%m-%d_%H-%M-%Stest.ts

  I record in a ts container to mitigate framerate inaccuracies.

  If I capture the proxy stream after few ours with no errors, something happens the stream starts to record files with the usual space on disk (around 5-10MB each) but when I check the metadata, the duration is 0.16 seconds. If you watch the video, it’s of course too fast but shows the clock hard coded in the image that runs through the whole minute, so the data is there.

  The command I use with the restream, just an input change.

  ffmpeg -rtsp_transport tcp -y -i rtsp://IPstreamServer/proxyStream-1 -c copy -f segment -segment_time 60 -reset_timestamps 1 -segment_atclocktime 1 -strftime 1 /video/%Y-%m-%d_%H-%M-%Stest.ts

  It’s like ffmpeg is messing up the metadata. With the tcp setting there are no packets lost.

  What could be the problem ?

  Recording on Debian 8.8, ffmpeg version 3.2.5-1 bpo8+1.

  EDIT : I’ve found that some guy had problem receiving metadata from the server, could it be the problem ? http://lists.live555.com/pipermail/live-devel/2013-May/017033.html

• Combine Audio and Images in Stream

  19 décembre 2017, par SenorContento

  I would like to be able to create images on the fly and also create audio on the fly too and be able to combine them together into an rtmp stream (for Twitch or YouTube). The goal is to accomplish this in Python 3 as that is the language my bot is written in. Bonus points for not having to save to disk.

  So far, I have figured out how to stream to rtmp servers using ffmpeg by loading a PNG image and playing it on loop as well as loading a mp3 and then combining them together in the stream. The problem is I have to load at least one of them from file.

  I know I can use Moviepy to create videos, but I cannot figure out whether or not I can stream the video from Moviepy to ffmpeg or directly to rtmp. I think that I have to generate a lot of really short clips and send them, but I want to know if there’s an existing solution.

  There’s also OpenCV which I hear can stream to rtmp, but cannot handle audio.

  A redacted version of an ffmpeg command I have successfully tested with is

  ffmpeg -loop 1 -framerate 15 -i ScreenRover.png -i "Song-Stereo.mp3" -c:v libx264 -preset fast -pix_fmt yuv420p -threads 0 -f flv rtmp://SITE-SUCH-AS-TWITCH/.../STREAM-KEY

  or

  cat Song-Stereo.mp3 | ffmpeg -loop 1 -framerate 15 -i ScreenRover.png -i - -c:v libx264 -preset fast -pix_fmt yuv420p -threads 0 -f flv rtmp://SITE-SUCH-AS-TWITCH/.../STREAM-KEY

  I know these commands are not set up properly for smooth streaming, the result manages to screw up both Twitch’s and Youtube’s player and I will have to figure out how to fix that.

  The problem with this is I don’t think I can stream both the image and the audio at once when creating them on the spot. I have to load one of them from the hard drive. This becomes a problem when trying to react to a command or user chat or anything else that requires live reactions. I also do not want to destroy my hard drive by constantly saving to it.

  As for the python code, what I have tried so far in order to create a video is the following code. This still saves to the HD and is not responsive in realtime, so this is not very useful to me. The video itself is okay, with the one exception that as time passes on, the clock the qr code says versus the video’s clock start to spread apart farther and farther as the video gets closer to the end. I can work around that limitation if it shows up while live streaming.

  def make_frame(t):
  
    img = qrcode.make("Hello! The second is %s!" % t)
  
    return numpy.array(img.convert("RGB"))
  
  
  
  clip = mpy.VideoClip(make_frame, duration=120)
  
  clip.write_gif("test.gif",fps=15)
  
  
  
  gifclip = mpy.VideoFileClip("test.gif")
  
  gifclip.set_duration(120).write_videofile("test.mp4",fps=15)

  My goal is to be able to produce something along the psuedo-code of

  original_video = qrcode_generator("I don't know, a clock, pyotp, today's news sources, just anything that can be generated on the fly!")
  
  original_video.overlay_text(0,0,"This is some sample text, the left two are coordinates, the right three are font, size, and color", Times_New_Roman, 12, Blue)
  
  original_video.add_audio(sine_wave_generator(0,180,2)) # frequency min-max, seconds
  
  
  
  # NOTICE - I did not add any time measurements to the actual video itself. The whole point is this is a live stream and not a video clip, so the time frame would be now. The 2 seconds list above is for our psuedo sine wave generator to know how long the audio clip should be, not for the actual streaming library.
  
  
  
  stream.send_to_rtmp_server(original_video) # Doesn't matter if ffmpeg or some native library

  The above example is what I am looking for in terms of video creation in Python and then streaming. I am not trying to create a clip and then stream it later, I am trying to have the program be able to respond to outside events and then update it’s stream to do whatever it wants. It is sort of like a chat bot, but with video instead of text.

  def track_movement(...):
  
    ...
  
    return ...
  
  
  
  original_video = user_submitted_clip(chat.lastVideoMessage)
  
  original_video.overlay_text(0,0,"The robot watches the user's movements and puts a blue square around it.", Times_New_Roman, 12, Blue)
  
  original_video.add_audio(sine_wave_generator(0,180,2)) # frequency min-max, seconds
  
  
  
  # It would be awesome if I could also figure out how to perform advance actions such as tracking movements or pulling a face out of a clip and then applying effects to it on the fly. I know OpenCV can track movements and I hear that it can work with streams, but I cannot figure out how that works. Any help would be appreciated! Thanks!

  Because I forgot to add the imports, here are some useful imports I have in my file !

  import pyotp
  
  import qrcode
  
  from io import BytesIO
  
  from moviepy import editor as mpy

  The library, pyotp, is for generating one time pad authenticator codes, qrcode is for the qr codes, BytesIO is used for virtual files, and moviepy is what I used to generate the GIF and MP4. I believe BytesIO might be useful for piping data to the streaming service, but how that happens, depends entirely on how data is sent to the service, whether it be ffmpeg over command line (from subprocess import Popen, PIPE) or it be a native library.