Recherche avancée

Sur d’autres sites (9942)

• Save FFMpeg conversion to PHP variable vs. File System for use with Whisper API ?

  13 avril 2023, par SScotti

  I just started working on a little demo to transalte audio captured from the front-end as audio/webm using JS and then sent the back-end in a Laravel App. I guess there are JS libraries that can handle the conversion, but I'd rather use a server side solution with FFMPEG, which I am doing.

  


  The backend code is below. It seems to be working after playing around with the PHP composer package that I'm using vs. one for Laravel that is also there. I'd rather use this one because I have other PHP apps that are not Laravel.

  


  Questions :

  


  


  1. With the FFMpeg library, is there a way to capture the converted .mp3 file to a PHP variable in the script rather than saving it to the file system and then reading it back in later ?

     


  


  2. For the OpenAI call, I'd like to catch exceptions there also. I just sort of have a placeholder there for now.

     


     protected function whisper(Request $request) {

    $yourApiKey = getenv('OPENAI_API_KEY');
    $client = OpenAI::client($yourApiKey);

    $file = $request->file('file');
    $mimeType = $request->file('file')->getMimeType();
    $audioContents = $file->getContent();

    try {

        FFMpeg::open($file)
        ->export()
        ->toDisk('public')
        ->inFormat(new \FFMpeg\Format\Audio\Mp3)
        ->save('song_converted.mp3');
    }
    catch (EncodingException $exception) {
        $command = $exception->getCommand();
        $errorLog = $exception->getErrorOutput();
    }

    $mp3 = Storage::disk('public')->path('song_converted.mp3');
    try {
    $response = $client->audio()->transcribe([
    'model' => 'whisper-1',
    'file' =>  fopen($mp3, 'r'),
    'response_format' => 'verbose_json',
    ]);
    }
    catch (EncodingException $exception) {
        $command = $exception->getCommand();
        $errorLog = $exception->getErrorOutput();
    }

 echo json_encode($response);

}


     


  


  


• How to save frames to memory from video with ffmpeg gpu encoding ?

  6 juillet 2021, par Spetra

  I'm trying to extract frames from video and save them to memory(ram).
With CPU encoding, I don't have any problems :

  


  ffmpeg -i input -s 224x224 -pix_fmt bgr24 -vcodec rawvideo -an -sn -f image2pipe -


  


  But when I'm trying to use some NVIDIA GPU encoding, I'm always getting noisy images.
I tried to use different commands, but the result was always the same, on Windows and Ubuntu.

  


  ffmpeg -hwaccel cuda -i 12.mp4 -s 224x224 -f image2pipe - -vcodec rawvideo


  


  With saving JPG's on disk, I don't have any problems.

  


  ffmpeg -hwaccel cuvid -c:v h264_cuvid -resize 224x224 -i {input_video} \
     -vf thumbnail_cuda=2,hwdownload,format=nv12 {output_dir}/%d.jpg


  


  There was my python code for testing these commands :

  


  import cv2
import subprocess as sp
import numpy

IMG_W = 224
IMG_H = 224
input = '12.mp4'

ffmpeg_cmd = [ 'ffmpeg','-i', input,'-s', '224x224','-pix_fmt', 'bgr24',  '-vcodec', 'rawvideo', '-an','-sn', '-f', 'image2pipe', '-']


#ffmpeg_cmd = ['ffmpeg','-hwaccel' ,'cuda' ,'-i' ,'12.mp4','-s', '224x224','-f' , 'image2pipe' ,'-' , '-vcodec' ,'rawvideo']

pipe = sp.Popen(ffmpeg_cmd, stdout = sp.PIPE, bufsize=10)
images = []
encode_param = [int(cv2.IMWRITE_JPEG_QUALITY), 95]
cnt = 0
while True:
    cnt += 1
    raw_image = pipe.stdout.read(IMG_W*IMG_H*3)
    image =  numpy.fromstring(raw_image, dtype='uint8')     # convert read bytes to np
    if image.shape[0] == 0:
        del images
        break   
    else:
        image = image.reshape((IMG_H,IMG_W,3))
        

    cv2.imshow('test',image)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

pipe.stdout.flush()
cv2.destroyAllWindows()


  


• Save first an last frame as a picture

  7 octobre 2020, par Oscar Thees

  I try to cut out the last and the first frame of videos. The following commands work for specific files :

  


  ffmpeg -i TLC00000.AVI -vframes 1 first.jpeg    
ffmpeg -sseof -2 -i TLC00000.AVI -update 1 -q:v 1 last.jpg
    


  


  However, when I start looping them over several files in a directory I am starting to run into problems.

  


  SET "source=C:\Users\...\...\%%~nF"
SET "first=C:\Users\...\...\first%%~nF"
SET "last=C:\Users\...\...\last\%%~nF"

for %%F in (*.AVI) do (
    If not Exist source MkDir first
    ffmpeg -i %%F -vframes 1 first\%%~nF-%%3d.first.jpg)

for %%F in (*.AVI) do (
    If not Exist source MkDir last
    ffmpeg -sseof -2 -i %%F -update 1 -q:v 1 last\%%~nF-%%3d.last.jpg)
Pause


  


  Any help with how to make this loop working is warmly welcome.