Recherche avancée

Sur d’autres sites (10475)

• Rails 5 - Concurrent large Video uploads using Carrierwave eats up the server memory/space

  22 mars 2020, par Milind

  I have a working Rails 5 apps using Reactjs for frontend and React dropzone uploader to upload video files using carrierwave.

  So far, what is working great is listed below -

  1. User can upload videos and videos are encoded based on the selection made by user - HLS or MPEG-DASH for online streaming.
  2. Once the video is uploaded on the server, it starts streaming it by :-
     • FIRST,copying video on /tmp folder.
     • Running a bash script that uses ffmpeg to transcode uploaded video using predefined commands to produce new fragments of videos inside /tmp folder.
     • Once the background job is done, all the videos are uploaded on AWS S3, which is how the default carrierwave works
  3. So, when multiple videos are uploaded, they are all copied in /tmp folder and then transcoded and eventually uploaded to S3.

  My questions, where i am looking some help are listed below -

  1- The above process is good for small videos, BUT what if there are many concurrent users uploading 2GB of videos ? I know this will kill my server as my /tmp folder will keep on increasing and consume all the memory, making it to die hard.How can I allow concurrent videos to upload videos without effecting my server’s memory consumption ?

  2- Is there a way where I can directly upload the videos on AWS-S3 first, and then use one more proxy server/child application to encode videos from S3, download it to the child server, convert it and again upload it to the destination ? but this is almost the same but doing it on cloud, where memory consumption can be on-demand but will be not cost-effective.

  3- Is there some easy and cost-effective way by which I can upload large videos, transcode them and upload it to AWS S3, without effecting my server memory. Am i missing some technical architecture here.

  4- How Youtube/Netflix works, I know they do the same thing in a smart way but can someone help me to improve this ?

  Thanks in advance.

• avformat/matroskaenc : Don't fail if reserved Cues space doesn't suffice

  30 décembre 2019, par Andreas Rheinhardt

  avformat/matroskaenc : Don't fail if reserved Cues space doesn't suffice
  When the user opted to write the Cues at the beginning, the Cues were
  
  simply written without checking in advance whether enough space has been
  
  reserved for them. If it wasn't enough, the data following the space
  
  reserved for the Cues was simply overwritten, corrupting the file.
  This commit changes this by checking whether enough space has been
  
  reserved for the Cues before outputting anything. If it isn't enough,
  
  no Cues will be output at all and the file will be finalized normally,
  
  yet writing the trailer will nevertheless return an error to notify
  
  the user that his wish of having Cues at the front of the file hasn't
  
  been fulfilled.
  This change opens new usecases for this option : It is now safe to use
  
  this option to e.g. record live streams or to use it when muxing the
  
  output of an expensive encoding, because when the reserved space turns
  
  out to be insufficient, one ends up with a file that just lacks Cues
  
  but is otherwise fine.
  Signed-off-by : Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
  

  • [DH] doc/muxers.texi
  • [DH] libavformat/matroskaenc.c

• avformat/movenc : Simplify reserving space for tfrf tags

  22 septembre 2021, par Andreas Rheinhardt

  avformat/movenc : Simplify reserving space for tfrf tags
  Signed-off-by : Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
  

  • [DH] libavformat/movenc.c