Sur d’autres sites (14723)

• How to merge video and camera recording together in browser (Chrome especially) ?

  5 juillet 2021, par lzl124631x

  Goal

  


  I want to record/generate a video in browser (Chrome especially) with a custom video (e.g. .mp4, .webm) and camera recording side-by-side.

  


  --------------------------------------------------
|                        |                       |
|  Some Custom Video     |       My Camera       |
|                        |                       |
--------------------------------------------------


  


  What is working

  


  I can use MediaRecorder to record my camera, and play the recording side-by-side with my video, and download the recorded video as a webm.

  


  Challenge

  


  I'm facing difficulty merging the video and camera recording into a single video file side-by-side.

  


  My investigation

  


  MultiStreamMixer

  


  I first looked into MultiStreamMixer and built a demo with it (See codepen).

  


  

  


  The issue with it is that it stretches the video content to fit them in the same size. I can specify different width/height for those two streams but it doesn't work as expected. My camera got cropped.

  


  

  


  Custom Mixer

  


  I took a look at the source code of MultiStreamMixer and found the issue was because of its simple layout logic. So I took its source code as a reference and build my custom mixer. See codepen.

  


  The way it works :

  


  

  • We first render the streams one by one to an offscreen canvas.

  


  • Capture the stream from the canvas as the output video stream

  


  • Audio stream is generated separately using AudioContext, createMediaStreamSource, createMediaStreamDestination etc.

  


  • Merge the audio and video streams and output as a single stream.

  


  • Use MediaRecorder to record the mixed stream.

  


  


  It adds black margins to video/camera and won't stretch the videos.

  


  

  


  However, I found the recording is very blurry if you wave your hand in front of your camera while recording.

  


  

  


  Initially I thought it was because I didn't set some setting correctly to the canvas. But later I found that even the my MultiStreamMixer demo or the WebRTC demo (You can't see the text on the teapot clearly in the recording) generates blurry video with canvas.

  


  I'm asking in webrtc group to see if I can get around this issue. Meanwhile I looked into ffmpeg.js

  


  ffmpeg.js

  


  I think this would "work" but the file is too large. It's impratical to let the customer wait for this 23MB JS file to be downloaded.

  


  Other ways that I haven't tried

  


  The above are my investigations thus far.

  


  Another idea is to play the video and recorded video side-by-side and use screen recording API to record the merged version. (Example). But this would require the customer to wait for the same amount of time as the initial recording to get the screen/tab recorded.

  


  Uploading the video to server and doing the work in server would be my last resort.

  


• Video editing multi customer host in server instances

  1er août 2020, par General Omosco

  I created a multi conference live stream web app using rtc and ffmpeg C api for media stream, videos/images/texts files mixed up together in realtime sending the output to rtmp multiple destinations, but unfortunately my server could not handle more than 2 conference room smoothly.

  


  The server which couldn't handle more than two conference room running ffmpeg api

  


  It is Comfort plan in OVH
Type : vps,
Processor : 4 vCore,
Ram : 8gb

  


  Usage
CPU : 100% //playing smoothly or not.
Ram : 3%

  


  My question here is that. Is it possible to be creating self isolated instance on fly and automatically deploy the app in the instance for each customer ? And the host provider that can accept that.

  


• Strange results when converting rgb yuv using ffmpeg in python

  20 avril 2020, par user3569998

  I have JPG images and I am trying to do the following :

  



  

  1. resize the input and save the result into PNG

  


  2. convert PNG images to YUV (for instance yuv444p10le) in AVI container

  


  3. convert AVI back to PNG image.

  


  



  I am using python 3 and ffmpeg Linux version installed on Colab

  



  I printed the difference between resized_png and restored_png. I have values of 255 which doesn't make sense at all.

  



   [[[  0 255   1]
  [  0   0   1]
  [  0 255   1]
  ...
  [  1   0   0]
  [  1   0   0]
  [  2   0   1]]

 [[  0 255   1]
  [  0 255   1]
  [  1   0   0]
  ...
  [  1   0   0]
  [  1   0   0]
  [  1   0   0]]

 [[  0   0   1]
  [  0   0   1]
  [  0   0   1]
  ...
  [  1 255   0]
  [  1 255   1]
  [  1   0   0]]

 ...

 [[255   0   0]
  [255   0   1]
  [255   0   1]
  ...
  [  0   0   0]
  [  0   0   1]
  [255   0   1]]

 [[255   0   1]
  [255   0   1]
  [255   0   1]
  ...
  [  0   0   1]
  [  0   0   1]
  [  0   0   0]]

 [[255   0   1]
  [255   0   1]
  [255   0   1]
  ...
  [255   1   0]
  [255   0   0]
  [255   0   1]]]


  



  input_image = '/content/drive/My Drive/Colab Notebooks/adv_dnn/datasets/im2.jpg'
output_resized = '/content/drive/My Drive/Colab Notebooks/adv_dnn/datasets/im2.png'
folder_path = '/content/drive/My Drive/Colab Notebooks/adv_dnn/datasets/'

#do resize
# code = subprocess.call('cd /usr/bin/ffmpeg',  shell=True)
# print(code)
cmd_resize = ['ffmpeg', '-y', '-i',  input_image,'-vf', 'scale=224:224', output_resized]
process = subprocess.Popen(cmd_resize, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = process.communicate()
print(out)
print(err)

fmt = ['yuv420p', 'yuv420p10le', 'yuv444p', 'yuv444p10le']
for f in fmt:
  cmd2YUV = ['ffmpeg', '-y', '-i', output_resized, '-c:v', 'libx264', '-preset', 'placebo',\
             '-qp', '0', '-x264-params', "keyint=15:no-deblock=1", '-pix_fmt', f, \
             '-sws_flags', 'spline+accurate_rnd+full_chroma_int', \
             '-vf', "colorspace=bt709:iall=bt601-6-625:fast=1", '-color_range', '1', '-colorspace', '1', 
             '-color_primaries', '1', '-color_trc', '1', folder_path+'im2_'+f+'.avi']
  process = subprocess.Popen(cmd2YUV, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  out, err = process.communicate()
  print(out)
  print(err)

  cmd2PNG = ['ffmpeg', '-y', '-i',  folder_path+'im2_'+f+'.avi', '-compression_level', '10', '-pred', 'mixed', \
             '-pix_fmt', 'rgb24', '-sws_flags', '+accurate_rnd+full_chroma_int', \
             folder_path+'im2_'+f+'.png']
  process = subprocess.Popen(cmd2PNG, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  out, err = process.communicate()
  print(out)
  print(err)

im1=pilim.open('/content/drive/My Drive/Colab Notebooks/adv_dnn/datasets/im2.png')
im2=pilim.open('/content/drive/My Drive/Colab Notebooks/adv_dnn/datasets/im2_yuv444p10le.png')
im1_2d = np.asarray(im1)
im2_2d = np.asarray(im2)
print(im2_2d-im1_2d)