Recherche avancée

Sur d’autres sites (10261)

• Encoding audio_common messages to OPUS

  14 juin 2023, par djangbahevans

  ---

  


  I am trying to stream microphone and camera data to Amazon KVS WebRTC. I'm able to make video work using this package (adapted for noetic) however I am struggling to make audio work. I'm using the audio_capture package to get mp3 frames. I'm trying to convert this to OPUS frames before streaming to KVS, but I'm unsure how to do this. I wrote this bit of code based on the small resources I can find on using ffmpeg, but it's not working. avcodec_fill_audio_frame is returning -22.

  


  #include "opus_encoder.h"

OPUSEncoder::OPUSEncoder() {
  av_register_all();
  codecContext == nullptr;
}

OPUSEncoder::~OPUSEncoder() {
  if (codecContext != nullptr) {
    avcodec_free_context(&codecContext);
  }
}

int OPUSEncoder::Initialize(int Fs, int channels) {
  AVCodec *codec = avcodec_find_encoder(AV_CODEC_ID_OPUS);
  if (!codec) {
    printf("Codec not found\n");
    return -1;
  }

  codecContext = avcodec_alloc_context3(codec);
  if (!codecContext) {
    printf("Could not allocate audio codec context\n");
    return -1;
  }

  codecContext->sample_fmt = AV_SAMPLE_FMT_S16;
  codecContext->bit_rate = 128000;
  codecContext->sample_rate = Fs;
  codecContext->channel_layout = av_get_default_channel_layout(channels);
  codecContext->channels = channels;

  if (avcodec_open2(codecContext, codec, nullptr) < 0) {
    printf("Could not open codec\n");
    return -1;
  }

  return 0;
}

int OPUSEncoder::Encode(const uint8_t *audio_data, int frameSize,
                        uint8_t *out) {
  AVPacket pkt;
  av_init_packet(&pkt);
  pkt.data = nullptr;
  pkt.size = 0;

  AVFrame *frame = av_frame_alloc();
  frame->nb_samples = frameSize;
  frame->format = codecContext->sample_fmt;
  frame->channel_layout = codecContext->channel_layout;

  int ret = avcodec_fill_audio_frame(frame, codecContext->channels,
                                     codecContext->sample_fmt, audio_data,
                                     frameSize * 2, 0);
  if (ret < 0) {
    printf("Error filling audio frame: %d\n", ret);
    return -1;
  }

  ret = avcodec_send_frame(codecContext, frame);
  if (ret < 0) {
    printf("Error sending the frame to the encoder\n");
    return -1;
  }

  while (ret >= 0) {
    ret = avcodec_receive_packet(codecContext, &pkt);
    if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
      return 0;
    } else if (ret < 0) {
      printf("Error encoding audio frame\n");
      return -1;
    }

    memcpy(out, pkt.data, pkt.size);
    out += pkt.size;
    av_packet_unref(&pkt);
  }

  av_frame_free(&frame);

  return 0;
}


  


• Ream-time watermarking with MPEG-DASH

  14 juillet 2016, par Calvin W.

  In the system, I want to add a unique watermark (e.g. IP address of client and time stamp) into the video that he/she want to watch.

  But when I handled it with OpenCV, it spent 25 minute with a 15-min video. And I need to transcode to mp4 with ffmpeg.

  Now I’m trying the watermark function of ffmpeg, bit it still needs some time.

  It it possible to send the video to client side with MPEG-DASH while transcoding it with ffmpeg ?

  System spec :(Amazon EC2 c3.xlarge)
  Intel Xeon E5-2680 v2 (Ivy Bridge) - 4 vCPU
  7.5G RAM
  40GB SSD
  Ubuntu 14.04 LTS
  OpenCV2.4.13
  ffmpeg 3.1.1
  

  code :

  import cv2
  
  import sys
  
  import time
  
  from datetime import datetime as dt
  
  
  
  # frame of input video
  
  fps = float(sys.argv[4])
  
  # encode to AVC
  
  fourcc = cv2.cv.CV_FOURCC('A', 'V', 'C', '1')
  
  # transparency of text
  
  alpha = 0.1
  
  beta = 1 - alpha
  
  
  
  # input video
  
  cap = cv2.VideoCapture(sys.argv[3])
  
  
  
  # current frame index, start from 0
  
  frameIndex = 0
  
  
  
  # get input video's width/height
  
  width = int(cap.get(cv2.cv.CV_CAP_PROP_FRAME_WIDTH))
  
  height = int(cap.get(cv2.cv.CV_CAP_PROP_FRAME_HEIGHT))
  
  
  
  # config output (error using .mp4)
  
  out = cv2.VideoWriter('output.avi', fourcc, fps, (width, height))
  
  
  
  # access time
  
  timeStr = dt.fromtimestamp(time.time()).strftime('%Y-%m-%d %H:%M:%S')
  
  
  
  requestIP = sys.argv[1]
  
  username = sys.argv[2]
  
  text = "%s %s %s" % (requestIP, username, timeStr)
  
  
  
  
  
  # start loading video
  
  while(cap.isOpened()):
  
      ret, frame = cap.read()
  
      if ret:
  
          # add text between 10s - 20s
  
          if frameIndex > time10 and frameIndex < time20:
  
              # clone a new frame to add text
  
              overlay = frame.copy()
  
              cv2.putText(overlay, text, (100, 100), cv2.FONT_HERSHEY_PLAIN, 0.5, (255, 255, 255))
  
              # combine both frame and make text transparent
  
              cv2.addWeighted(overlay, alpha, frame, beta, 0, frame)
  
          # write frame to output
  
          out.write(frame)
  
          frameIndex += 1
  
      # wait for next frame
  
      if cap.get(cv2.cv.CV_CAP_PROP_POS_FRAMES) == cap.get(cv2.cv.CV_CAP_PROP_FRAME_COUNT):
  
          break
  
  # End of video
  
  # release
  
  cap.release()
  
  out.release()

• How to get the thumbnail of base64 encoded video file in Nodejs ?

  3 octobre 2018, par Wai Yan Hein

  I am developing a web application using Nodejs. I am using Amazon S3 bucket to store files. What I am doing now is that when I upload a video file (mp4) to the S3 bucket, I will get the thumbnail photo of the video file from the lambda function. For fetching the thumbnail photo of the video file, I am using this package - https://www.npmjs.com/package/ffmpeg. I tested the package locally on my laptop and it is working.

  Here is my code tested on my laptop

  var ffmpeg = require('ffmpeg');
  
  
  
  module.exports.createVideoThumbnail = function(req, res)
  
  {
  
      try {
  
          var process = new ffmpeg('public/lalaland.mp4');
  
          process.then(function (video) {
  
  
  
              video.fnExtractFrameToJPG('public', {
  
                  frame_rate : 1,
  
                  number : 5,
  
                  file_name : 'my_frame_%t_%s'
  
              }, function (error, files) {
  
                  if (!error)
  
                      console.log('Frames: ' + files);
  
                  else
  
                      console.log(error)
  
              });
  
  
  
          }, function (err) {
  
              console.log('Error: ' + err);
  
          });
  
      } catch (e) {
  
          console.log(e.code);
  
          console.log(e.msg);
  
      }
  
      res.json({ status : true , message: "Video thumbnail created." });
  
  }

  The above code works well. It gave me the thumbnail photos of the video file (mp4). Now, I am trying to use that code in the AWS lambda function. The issue is the above code is using video file path as the parameter to fetch the thumbnails. In the lambda function, I can only fetch the base 64 encoded format of the file. I can get id (s3 path) of the file, but I cannot use it as the parameter (file path) to fetch the thumbnails as my s3 bucket does not allow public access.

  So, what I tried to do was that I tried to save the base 64 encoded video file locally in the lambda function project itself and then passed the file path as the parameter for fetching the thumbnails. But the issue was that AWS lamda function file system is read-only. So I cannot write any file to the file system. So what I am trying to do right now is to retrieve the thumbnails directly from the base 64 encoded video file. How can I do it ?