Recherche avancée

Sur d’autres sites (7925)

• How to save frames to memory from video with ffmpeg gpu encoding ?

  6 juillet 2021, par Spetra

  I'm trying to extract frames from video and save them to memory(ram).
With CPU encoding, I don't have any problems :

  


  ffmpeg -i input -s 224x224 -pix_fmt bgr24 -vcodec rawvideo -an -sn -f image2pipe -


  


  But when I'm trying to use some NVIDIA GPU encoding, I'm always getting noisy images.
I tried to use different commands, but the result was always the same, on Windows and Ubuntu.

  


  ffmpeg -hwaccel cuda -i 12.mp4 -s 224x224 -f image2pipe - -vcodec rawvideo


  


  With saving JPG's on disk, I don't have any problems.

  


  ffmpeg -hwaccel cuvid -c:v h264_cuvid -resize 224x224 -i {input_video} \
     -vf thumbnail_cuda=2,hwdownload,format=nv12 {output_dir}/%d.jpg


  


  There was my python code for testing these commands :

  


  import cv2
import subprocess as sp
import numpy

IMG_W = 224
IMG_H = 224
input = '12.mp4'

ffmpeg_cmd = [ 'ffmpeg','-i', input,'-s', '224x224','-pix_fmt', 'bgr24',  '-vcodec', 'rawvideo', '-an','-sn', '-f', 'image2pipe', '-']


#ffmpeg_cmd = ['ffmpeg','-hwaccel' ,'cuda' ,'-i' ,'12.mp4','-s', '224x224','-f' , 'image2pipe' ,'-' , '-vcodec' ,'rawvideo']

pipe = sp.Popen(ffmpeg_cmd, stdout = sp.PIPE, bufsize=10)
images = []
encode_param = [int(cv2.IMWRITE_JPEG_QUALITY), 95]
cnt = 0
while True:
    cnt += 1
    raw_image = pipe.stdout.read(IMG_W*IMG_H*3)
    image =  numpy.fromstring(raw_image, dtype='uint8')     # convert read bytes to np
    if image.shape[0] == 0:
        del images
        break   
    else:
        image = image.reshape((IMG_H,IMG_W,3))
        

    cv2.imshow('test',image)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

pipe.stdout.flush()
cv2.destroyAllWindows()


  


• Save FFMpeg conversion to PHP variable vs. File System for use with Whisper API ?

  13 avril 2023, par SScotti

  I just started working on a little demo to transalte audio captured from the front-end as audio/webm using JS and then sent the back-end in a Laravel App. I guess there are JS libraries that can handle the conversion, but I'd rather use a server side solution with FFMPEG, which I am doing.

  


  The backend code is below. It seems to be working after playing around with the PHP composer package that I'm using vs. one for Laravel that is also there. I'd rather use this one because I have other PHP apps that are not Laravel.

  


  Questions :

  


  


  1. With the FFMpeg library, is there a way to capture the converted .mp3 file to a PHP variable in the script rather than saving it to the file system and then reading it back in later ?

     


  


  2. For the OpenAI call, I'd like to catch exceptions there also. I just sort of have a placeholder there for now.

     


     protected function whisper(Request $request) {

    $yourApiKey = getenv('OPENAI_API_KEY');
    $client = OpenAI::client($yourApiKey);

    $file = $request->file('file');
    $mimeType = $request->file('file')->getMimeType();
    $audioContents = $file->getContent();

    try {

        FFMpeg::open($file)
        ->export()
        ->toDisk('public')
        ->inFormat(new \FFMpeg\Format\Audio\Mp3)
        ->save('song_converted.mp3');
    }
    catch (EncodingException $exception) {
        $command = $exception->getCommand();
        $errorLog = $exception->getErrorOutput();
    }

    $mp3 = Storage::disk('public')->path('song_converted.mp3');
    try {
    $response = $client->audio()->transcribe([
    'model' => 'whisper-1',
    'file' =>  fopen($mp3, 'r'),
    'response_format' => 'verbose_json',
    ]);
    }
    catch (EncodingException $exception) {
        $command = $exception->getCommand();
        $errorLog = $exception->getErrorOutput();
    }

 echo json_encode($response);

}


     


  


  


• FFMPEG : How to extract multichannel track from m4v, mix it down and save the stereo downmix as "left" and "right" ?

  30 septembre 2015, par chillynilly

  just like the title already says : I want to extract a multichannel track (5.1) from an .m4v, mix this track down and save the output as separate files, so in the end I want to have something like ’downmix_left.wav’ and ’downmix_right.wav’
  I know how to do a downmix and I know how to split the audio, but I do not know how to do it in one step, which would save me a lot of time.

  This is the command I use for splitting :

  ffmpeg -i "video.m4v" -vn -filter_complex \
  
  "[0:2]channelsplit=channel_layout=5.1(side)[FL][FR][FC][LFE][SL][SR]" \ 
  
  -map "[FL]" video_left.wav \ 
  
  -map "[FR]" video_right.wav \ 
  
  -map "[FC]" video_center.wav \ 
  
  -map "[LFE]" video_lfe.wav \ 
  
  -map "[SL]" video_back_left.wav \ 
  
  -map "[SR]" video_back_right.wav

  And this is the command for the downmix of a multichannel track :

  ffmpeg -i "video.m4v" -vn -map 0:2 -ac 2 \ 
  
  -af "aresample=matrix_encoding=dplii" video_downmix.wav

  Is it possible to combine these and if so, how can it be done :D ? I would appreciate it very much if you could help me out here.