Recherche avancée

Sur d’autres sites (9943)

• Why is my .mp4 file created using cv2.VideoWriter not syncing up with the audio when i combine the video and audio using ffmpeg [closed]

  27 décembre 2024, par joeS125

  The aim of the script is to take text from a text file and put it onto a stock video with an ai reading the text. Similar to those reddit stories on social media with parkour minecraft in the background.

  


  import cv2
import time
from ffpyplayer.player import MediaPlayer
from Transcription import newTranscribeAudio
from pydub import AudioSegment

#get a gpt text generation to create a story based on a prompt, for example sci-fi story and spread it over 3-4 parts
#get stock footage, like minecraft parkour etc
#write text of script on the footage
#create video for each part
#have ai voiceover to read the transcript
cap = cv2.VideoCapture("Stock_Videos\Minecraft_Parkour.mp4")
transcription = newTranscribeAudio("final_us.wav")
player = MediaPlayer("final_us.mp3")
audio = AudioSegment.from_file("final_us.mp3")
story = open("Story.txt", "r").read()
story_split = story.split("||")
fps = cap.get(cv2.CAP_PROP_FPS)
frame_count = int(cap.get(cv2.CAP_PROP_FRAME_COUNT))
video_duration = frame_count / fps  # Duration of one loop of the video
fourcc = cv2.VideoWriter_fourcc(*"mp4v")
audio_duration = len(audio) / 1000  # Duration in seconds
video_writer = cv2.VideoWriter(f"CompletedVideo.mp4", fourcc, fps, (1080, 1920))

choice = 0#part of the story choice
part_split = story_split[choice].split(" ")
with open("Segment.txt", "w") as file:
    file.write(story_split[choice])
start_time = time.time()
length = len(part_split) - 1
next_text = []
for j in range(0, length):
    temp = part_split[j].replace("\n", "")
    next_text.append([temp])
index = 0
word_index = 0
frame_size_x = 1080
frame_size_y = 1920
audio_duration = len(audio) / 1000  # Duration in seconds
start_time = time.time()
wait_time = 1 / fps
while (time.time() - start_time) < audio_duration:
    cap.set(cv2.CAP_PROP_POS_FRAMES, 0)  # Restart video
    elapsed_time = time.time() - start_time
    print(video_writer)
    if index >= len(transcription):
        break
    while cap.isOpened():
        # Capture frames in the video 
        ret, frame = cap.read()
        if not ret:
            break
        audio_frame, val = player.get_frame()
        if val == 'eof':  # End of file
            print("Audio playback finished.")
            break
        if index >= len(transcription):
            break
        
        if frame_size_x == -1:
            frame_size_x = frame.shape[1]
            frame_size_y = frame.shape[0]

        elapsed_time = time.time() - start_time

        # describe the type of font 
        # to be used. 
        font = cv2.FONT_HERSHEY_SIMPLEX 
        trans = transcription[index]["words"]
        end_time = trans[word_index]["end"]
        if trans[word_index]["start"] < elapsed_time < trans[word_index]["end"]:
            video_text = trans[word_index]["text"]
        elif elapsed_time >= trans[word_index]["end"]:
            #index += 1
            word_index += 1
        if (word_index >= len(trans)):
            index += 1
            word_index = 0
        # get boundary of this text
        textsize = cv2.getTextSize(video_text, font, 3, 6)[0]
        # get coords based on boundary
        textX = int((frame.shape[1] - textsize[0]) / 2)
        textY = int((frame.shape[0] + textsize[1]) / 2)
        
        cv2.putText(frame,  
                    video_text,  
                    (textX, textY),  
                    font, 3,  
                    (0, 255, 255),  
                    6,  
                    cv2.LINE_4)
        
        # Define the resize scale
        scale_percent = 50  # Resize to 50% of the original size
        # Get new dimensions
        width = 1080
        height = 1920
        new_size = (width, height)

        # Resize the frame
        resized_frame = cv2.resize(frame, new_size)
        video_writer.write(resized_frame)
        cv2.imshow('video', resized_frame)
        cv2.waitKey(wait_time)
        if cv2.waitKey(1) & 0xFF == ord('q'): 
            break
cv2.destroyAllWindows()
video_writer.release()
cap.release()



  


  When I run this script the audio matches the text in the video perfectly and it runs for the correct amount of time to match with the audio (2 min 44 sec). However, the saved video CompletedVideo.mp4 only lasts for 1 min 10 sec. I am unsure why the video has sped up. The fps is 60 fps. If you require any more information please let me know and thanks in advance.

  


  I have tried changing the fps, changing the wait_time after writing each frame. I am expecting the CompletedVideo.mp4 to be 2 min 44 sec long not 1 min 10 sec long.

  


• error with ffmpeg, when trying to upload a video

  14 juillet 2021, par James

  I recently downloaded a script, which is similar to a social network. And all items in this script related to publish, for example. The button to publish images, video and polls are included. However the video upload button has a problem, the host I'm hosting this site doesn't accept ffmpeg conversion, and so far so good because in the script if there isn't you can leave the ffmpeg field (in the adm panel) empty , and I did that, the problem is that when I try to upload a video, I always get an error saying that it was not possible to relazie the operation, I went to take a look at the script to process the video, and found that it uploads because the video goes to the uploads folder, but it doesn't deliver anything to the bank and doesn't even preview the video on the publish page, that is, the ffmpeg code is running even with the host not allowing it, giving an error soon.

  


  I've tried to do everything like deleting the ffmpeg part of the code, trying to change the input names and many other small attempts like changing some name there or here, but the error still persists.

  


  The code is right below, could someone tell me what I can change in this code so that it uploads the normal video ? (this host allows video uploading, but without ffmpeg). And what could be wrong with him ?

  


  I tested it on localhost but everything worked fine

  


  the whole code

  


  <?php 

if (empty($cl['is_logged'])) {
    $data         = array(
        'code'    => 401,
        'data'    => array(),
        'message' => 'Unauthorized Access'
    );
}

else {
    $post_data  = $me['draft_post'];
    $media_type = fetch_or_get($_POST["type"], false);

    if (empty($media_type) || in_array($media_type, array("image", "video")) != true) {
        $data['code']    = 400;
        $data['message'] = "Media file type is missing or invalid";
        $data['data']    = array();
    }

    else {
        if ($media_type == "image") {
            if (not_empty($_FILES['file']) && not_empty($_FILES['file']['tmp_name'])) {
                if (empty($post_data)) {
                    $post_id   = cl_create_orphan_post($me['id'], "image");
                    $post_data = cl_get_orphan_post($post_id);

                    cl_update_user_data($me['id'], array(
                        'last_post' => $post_id
                    ));
                }
                
                if (not_empty($post_data) && $post_data["type"] == "image") {
                    if (empty($post_data['media']) || count($post_data['media']) < 10) {
                        $file_info      =  array(
                            'file'      => $_FILES['file']['tmp_name'],
                            'size'      => $_FILES['file']['size'],
                            'name'      => $_FILES['file']['name'],
                            'type'      => $_FILES['file']['type'],
                            'file_type' => 'image',
                            'folder'    => 'images',
                            'slug'      => 'original',
                            'crop'      => array('width' => 300, 'height' => 300),
                            'allowed'   => 'jpg,png,jpeg,gif'
                        );

                        $file_upload = cl_upload($file_info);

                        if (not_empty($file_upload['filename'])) {
          
                            $img_id      = cl_db_insert(T_PUBMEDIA, array(
                                "pub_id" => $post_data["id"],
                                "type"   => "image",
                                "src"    => $file_upload['filename'],
                                "time"   => time(),
                                "json_data" => json(array(
                                    "image_thumb" => $file_upload['cropped']
                                ),true)
                            ));

                            if (is_posnum($img_id)) {
                                $data['message'] = 'Media file uploaded successfully';
                                $data['code']    = 200;
                                $data['data']    = array(
                                    "media_id"   => $img_id, 
                                    "url"        => cl_get_media($file_upload['cropped']),
                                    "type"       => "Image"
                                );
                            }
                        }
                        else {
                            $data['code']    = 400;
                            $data['message'] = "Something went wrong while saving a uploaded media file. Please check your details and try again";
                            $data['data']    = array();
                        }
                    }
                    else {
                        $data['code']    = 400;
                        $data['message'] = "You cannot attach more than 10 images to a post";
                        $data['data']    = array();
                    }
                }
                else {
                    cl_delete_orphan_posts($me['id']);
                    cl_update_user_data($me['id'],array(
                        'last_post' => 0
                    ));

                    $data['code']    = 500;
                    $data['message'] = "An error occurred while processing your request. Please try again later.";
                    $data['data']    = array();
                }
            }
            else {
                $data['code']    = 500;
                $data['message'] = "Media file is missing or invalid";
                $data['data']    = array();
            }
        }

        else if($media_type == "video") {
            if (not_empty($_FILES['file']) && not_empty($_FILES['file']['tmp_name'])) {
                if (empty($post_data)) {
                    $post_id   = cl_create_orphan_post($me['id'], "video");
                    $post_data = cl_get_orphan_post($post_id);

                    cl_update_user_data($me['id'],array(
                        'last_post' => $post_id
                    ));
                }

                if (not_empty($post_data) && $post_data["type"] == "video") {
                    if (empty($post_data['media'])) {
                        $file_info      =  array(
                            'file'      => $_FILES['file']['tmp_name'],
                            'size'      => $_FILES['file']['size'],
                            'name'      => $_FILES['file']['name'],
                            'type'      => $_FILES['file']['type'],
                            'file_type' => 'video',
                            'folder'    => 'videos',
                            'slug'      => 'original',
                            'allowed'   => 'mp4,mov,3gp,webm',
                        );

                        $file_upload = cl_upload($file_info);
                        $upload_fail = false;
                        $post_id     = $post_data['id'];

                        if (not_empty($file_upload['filename'])) {
                            try {
                                require_once(cl_full_path("core/libs/ffmpeg-php/vendor/autoload.php"));

                                $ffmpeg         = new FFmpeg(cl_full_path($config['ffmpeg_binary']));
                                $thumb_path     = cl_gen_path(array(
                                    "folder"    => "images",
                                    "file_ext"  => "jpeg",
                                    "file_type" => "image",
                                    "slug"      => "poster",
                                ));

                                $ffmpeg->input($file_upload['filename']);
                                $ffmpeg->set('-ss','3');
                                $ffmpeg->set('-vframes','1');
                                $ffmpeg->set('-f','mjpeg');
                                $ffmpeg->output($thumb_path)->ready();
                            } 

                            catch (Exception $e) {
                                $upload_fail = true;
                            }

                            if (empty($upload_fail)) {
                                $vid_id      = cl_db_insert(T_PUBMEDIA, array(
                                    "pub_id" => $post_id,
                                    "type"   => "video",
                                    "src"    => $file_upload['filename'],
                                    "time"   => time(),
                                    "json_data" => json(array(
                                        "poster_thumb" => $thumb_path
                                    ),true)
                                ));

                                if (is_posnum($vid_id)) {
                                    $data['message'] = 'Media file uploaded successfully';
                                    $data['code']    = 200;
                                    $data['data']    = array(
                                        "media_id"   => $vid_id, 
                                        "type"       => "Video",
                                        "source"     => cl_get_media($file_upload['filename']),
                                        "poster"     => cl_get_media($thumb_path),
                                    );
                                }
                            }

                            else {
                                $data['code']    = 400;
                                $data['message'] = "Something went wrong while saving a uploaded media file. Please check your details and try again";
                                $data['data']    = array();
                            }
                        }
                    }
                    else {
                        $data['code']    = 400;
                        $data['message'] = "You cannot attach more than 1 video to a post";
                        $data['data']    = array();
                    }
                }
                else {
                    cl_delete_orphan_posts($me['id']);
                    cl_update_user_data($me['id'], array(
                        'last_post' => 0
                    ));
                }
            }

            else {
                $data['code']    = 500;
                $data['message'] = "Media file is missing or invalid";
                $data['data']    = array();
            }
        }
    }
}


  


  I dosen't made this code so, i dont know nothing about him

  


• ffmpeg API : handle frame loss in hevc encoding

  8 janvier 2024, par Mario

  Everything works fine until the introduction of frame->pts increment due to frame loss.

  


  Below is the regular progression without frame->pts increments :

  


  


  frame->pts=8 pkt->pts=512 pkt->dts=-512 pkt->flags=1
  
frame->pts=9 pkt->pts=2560 pkt->dts=0 pkt->flags=0
  
frame->pts=10 pkt->pts=1536 pkt->dts=512 pkt->flags=0
  
frame->pts=11 pkt->pts=1024 pkt->dts=1024 pkt->flags=0
  
frame->pts=12 pkt->pts=2048 pkt->dts=1536 pkt->flags=0
  
frame->pts=13 pkt->pts=4608 pkt->dts=2048 pkt->flags=0
  
frame->pts=14 pkt->pts=3584 pkt->dts=2560 pkt->flags=0
  
frame->pts=15 pkt->pts=3072 pkt->dts=3072 pkt->flags=0
  
frame->pts=16 pkt->pts=4096 pkt->dts=3584 pkt->flags=0
  
frame->pts=17 pkt->pts=6656 pkt->dts=4096 pkt->flags=0
  
frame->pts=18 pkt->pts=5632 pkt->dts=4608 pkt->flags=0

  


  


  When I introduce the frame->pts increment it happens :

  


  


  frame->pts=15 pkt->pts=512 pkt->dts=-512 pkt->flags=1
  
frame->pts=17 pkt->pts=4608 pkt->dts=2048 pkt->flags=0
  
frame->pts=19 pkt->pts=2560 pkt->dts=1536 pkt->flags=0
  
[mp4 @ 0x7eff842222c0] Application provided invalid, non monotonically increasing dts to muxer in stream 0 : 2048 >= 1536

  


  


  So I wrote the following code as a "quick" solution (between av_packet_rescale_ts() and av_interleaved_write_frame()) :

  


     av_packet_rescale_ts(pkt, c->time_base, st->time_base);  
   ...
   if (pkt->dts<=previous_dts)  
     {  
      if (pkt->pts<=previous_pts)  
        {  
         pkt->pts=previous_dts+1+pkt->pts-pkt->dts;  
        }  
       pkt->dts=previous_dts+1;  
     }  
   previous_dts=pkt->dts;  
   previous_pts=pkt->pts;  
   ...
   ret = av_interleaved_write_frame(fmt_ctx, pkt);  


  


  Now I no longer have the error, but the values are :

  


  


  frame->pts=15 pkt->pts=512 pkt->dts=-512 pkt->flags=1
  
changed frame->pts=15 pkt->pts=512 pkt->dts=1 pkt->flags=1
  
frame->pts=17 pkt->pts=4608 pkt->dts=2048 pkt->flags=0
  
frame->pts=19 pkt->pts=2560 pkt->dts=1536 pkt->flags=0
  
changed frame->pts=19 pkt->pts=3073 pkt->dts=2049 pkt->flags=0
  
frame->pts=21 pkt->pts=1536 pkt->dts=1536 pkt->flags=0
  
changed frame->pts=21 pkt->pts=2050 pkt->dts=2050 pkt->flags=0
  
frame->pts=23 pkt->pts=4096 pkt->dts=3584 pkt->flags=0
  
frame->pts=25 pkt->pts=8704 pkt->dts=6144 pkt->flags=0
  
frame->pts=27 pkt->pts=6656 pkt->dts=5632 pkt->flags=0
  
changed frame->pts=27 pkt->pts=7169 pkt->dts=6145 pkt->flags=0
  
frame->pts=29 pkt->pts=5632 pkt->dts=5632 pkt->flags=0
  
changed frame->pts=29 pkt->pts=6146 pkt->dts=6146 pkt->flags=0
  
frame->pts=31 pkt->pts=7680 pkt->dts=7168 pkt->flags=0
  
frame->pts=33 pkt->pts=12800 pkt->dts=10240 pkt->flags=0
  
frame->pts=35 pkt->pts=10752 pkt->dts=9728 pkt->flags=0
  
changed frame->pts=35 pkt->pts=11265 pkt->dts=10241 pkt->flags=0
  
frame->pts=37 pkt->pts=9728 pkt->dts=9728 pkt->flags=0
  
changed frame->pts=37 pkt->pts=10242 pkt->dts=10242 pkt->flags=0

  


  


  What is the correct way to handle frame loss scenario ?
Is there a way to inform the encoder about frame loss ?

  


  The encoder is "hevc_qsv" and the output format is mov (.mp4).