Recherche avancée

Sur d’autres sites (8279)

• Create Panorama from Non-Sequential Video Frames

  6 mai 2021, par M.Innat

  There is a similar question (not that detailed and no exact solution).

  


  ---

  


  I want to create a single panorama image from video frames. And for that, I need to get minimum non-sequential video frames at first. A demo video file is uploaded here.

  


  What I Need

  


  A mechanism that can produce not-only non-sequential video frames but also in such a way that can be used to create a panorama image. A sample is given below. As we can see to create a panorama image, all the input samples must contain minimum overlap regions to each other otherwise it can not be done.

  


  

  


  So, if I have the following video frame's order

  


  A, A, A, B, B, B, B, C, C, A, A, C, C, C, B, B, B ...


  


  To create a panorama image, I need to get something as follows - reduced sequential frames (or adjacent frames) but with minimum overlapping.

  


       [overlap]  [overlap]  [overlap] [overlap]  [overlap]
 A,    A,B,       B,C,       C,A,       A,C,      C,B,  ...


  


  What I've Tried and Stuck

  


  A demo video clip is given above. To get non-sequential video frames, I primarily rely on ffmpeg software.

  


  Trial 1 Ref.

  


  ffmpeg -i check.mp4 -vf mpdecimate,setpts=N/FRAME_RATE/TB -map 0:v out.mp4


  


  After that, on the out.mp4, I applied slice the video frames using opencv

  


  import cv2, os 
from pathlib import Path

vframe_dir = Path("vid_frames/")
vframe_dir.mkdir(parents=True, exist_ok=True)

vidcap = cv2.VideoCapture('out.mp4')
success,image = vidcap.read()
count = 0

while success:
    cv2.imwrite(f"{vframe_dir}/frame%d.jpg" % count, image)     
    success,image = vidcap.read()
    count += 1


  


  Next, I rotated these saved images horizontally (as my video is a vertical view).

  


  vframe_dir = Path("out/")
vframe_dir.mkdir(parents=True, exist_ok=True)

vframe_dir_rot = Path("vframe_dir_rot/")
vframe_dir_rot.mkdir(parents=True, exist_ok=True)

for i, each_img in tqdm(enumerate(os.listdir(vframe_dir))):
    image = cv2.imread(f"{vframe_dir}/{each_img}")[:, :, ::-1] # Read (with BGRtoRGB)
    
    image = cv2.rotate(image,cv2.cv2.ROTATE_180)
    image = cv2.rotate(image,cv2.ROTATE_90_CLOCKWISE)

    cv2.imwrite(f"{vframe_dir_rot}/{each_img}", image[:, :, ::-1]) # Save (with RGBtoBGR)


  


  The output is ok for this method (with ffmpeg) but inappropriate for creating the panorama image. Because it didn't give some overlapping frames sequentially in the results. Thus panorama can't be generated.

  
 


  Trail 2 - Ref

  


  ffmpeg -i check.mp4 -vf decimate=cycle=2,setpts=N/FRAME_RATE/TB -map 0:v out.mp4


  


  didn't work at all.

  


  Trail 3

  


  ffmpeg -i check.mp4 -ss 0 -qscale 0 -f image2 -r 1 out/images%5d.png


  


  No luck either. However, I've found this last ffmpeg command was close by far but wasn't enough. Comparatively to others, this gave me a small amount of non-duplicate frames (good) but the bad thing is still do not need frames, and I kinda manually pick some desired frames, and then the opecv stitching algorithm works. So, after picking some frames and rotating (as mentioned before) :

  


  stitcher = cv2.Stitcher.create()
status, pano = stitcher.stitch(images) # images: manually picked video frames -_- 


  
 


  Update

  


  After some trials, I am kinda adopting the non-programming solution. But would love to see an efficient programmatic approach.

  


  On the given demo video, I used Adobe products (premiere pro and photoshop) to do this task, video instruction. But the issue was, I kind of took all video frames at first (without dropping to any frames and that will computationally cost further) via premier and use photoshop to stitching them (according to the youtube video instruction). It was too heavy for these editor tools and didn't look better way but the output was better than anything until now. Though I took few (400+ frames) video frames only out of 1200+.

  


  

  


  ---

  


  Here are some big challenges. The original video clips have some conditions though, and it's too serious. Unlike the given demo video clips :

  


  

  • It's not straight forward, i.e. camera shaking

  


  • Lighting condition, i.e. causes different visual look at the same spot

  


  • Cameral flickering or banding

  


  


  This scenario is not included in the given demo video. And this brings additional and heavy challenges to create panorama images from such videos. Even with the non-programming way (using adobe tools) I couldn't make it any good.

  


  ---

  


  However, for now, all I'm interest to get a panorama image from the given demo video which is without the above condition. But I would love to know any comment or suggestion on that.

  


• PHP and ffmpeg - Converting to H.264 on the fly for JW Player

  10 mai 2012, par vertigoelectric

  So the project I'm working on has tons of MP4 files already on the server that need to be played in an embedded player. Ideally we'd like to use the JW Player which is already set up for most of the videos. Unfortunately, a lot of the older videos are MPEG-4 encoded and require additional plugins to play in the browser.

  What I'd like to do is be able to transcode these videos on the fly to a format that JW Player supports, such as H.264. Now, I'm part way there because I can get it to convert the old MP4 files, but I'm having difficulty passing the data directly to the player.

  This is my current script doing the converting (this is media_converted.php) :

  <?php
  
      error_reporting(0);
  
      if (!isset($_GET["MediaName"])) {
  
          $_GET["MediaName"] = "GSX238";
  
      }       
  
      require_once("php/media_center_functions.php");
  
      $MediaURL = ConstructVideoURLInternap($_GET["MediaName"]);  
  
      header("Content-Transfer-Encoding: binary");
  
      //header('Content-type: video/mp4');
  
      //$calculatedFileSize = filesize($MediaURL);
  
      //header("Content-Length: {$calculatedFileSize}");
  
  
  
      $cmd = 'ffmpeg -i "'.$MediaURL.'"  -vcodec libx264 -acodec copy -f mp4  -moov_size 32000 "_temp/'.time().'_temp.mp4"  2> _temp/media_converter_output.txt';
  
      passthru($cmd);
  
  
  
  ?>

  I'm trying to take the above file and pass it to the JW Player as the video source : in the JW Player's variable scripting, like so :

  so.addVariable('file','media_converted.php?MediaName=12345');

  While I believe this should work, it's just not.

  Even when the conversion seems to work and be sending to the player, it's hard to tell because I think the browser is trying to download the entire file before playing anything. Obviously I can't make the user wait like that.

  I've looked into qt-faststart, and I was able to use it on my computer to modify the file so that it can start playing while it's being loaded. Unfortunately, this requires two steps. I first have to completely convert the file, then when it's done I have to use qt-faststart.exe on it. Again, this doesn't isn't really an on-the-fly solution. I'm sure there has to be some way to do produce similar results in a single command to be output to the player... right ?

  There is more video data on the server than I currently have space for on my hard drive, so to permanently re-encode each video, even in batches, would be troublesome, because I'd have to download some, convert, reupload, download more, etc. If I have to do that, I will, but I would prefer to leave them as is and be able to transcode on the fly.

  Any ideas ?

• Playing local (m3u8) file in ijkplayer can't play videos faster, using protocol_whitelist with 'tcp, http, https, udp, file, crypto, concat'

  3 août 2020, par gh05t

  I've been trying to find a solution to play m3u8 files in flutter and have the ability to change quality and speed of the videos, and I've reached a point where I'm able to do one or the other but not both. To change streams of m3u8, I'm splitting the files locally and essentially creating different files with each file only having one quality thus achieving quality change, but to play local files I have to use format option of ffmpeg protocol_whitelist and for some reason it's messing with my ability to play videos faster than 1.0, now the problem is not with the files I tried to connect to same file using google drive and since it's not a local file I don't need to set the protocol_whitelist. I don't know if this is an ffmpeg issue or an ijk player issue or perhaps even just a flutter ijk player issue, but if someone knows or has any ideas on how to fix please let me know, I also tried to use networkdataSource and give url as 'file://path/filename.m3u8' but no luck it requires the protocol_whitelist as well.

  


  The player by default sets the fflags with 'fastseek' and I suspect that setting protocol_whitelist, is causing some issue with this flag and that is what is causing the problem.(fast seek helps with seeking to a point ahead of buffer in m3u8 and has nothing to do with player speed I've learned)

  


  So basically can I either fix this issue somehow ? (either by changing qualities from the original master m3u8 file with the player and not doing all this stuff with local file ?)

  


  Or is there a way to give a local file to player such that it thinks it came from http/https and it plays it without causing an issue ?
(Wait did I answer my own question here ? making a http server and serving my file from my phone and using that should that be enough but localhost will be same as using file :/// essentially or not ? It's not and it fixes the issue ! XD)

  


  In case someone just knows some other solution perhaps ?
I do know that it's possible to add a function in c in ffmpeg to play different streams from master m3u8 but then I would have to compile 4-5 versions of this for different mobile processors and then interface to Java with JNI/JNA and to objective C, and then to flutter, which seems out of scope of my ability for the time being.