Recherche avancée

Sur d’autres sites (3996)

• ffmpeg : Preventing Unecessary Duplication of Frames In Overlay

  10 avril 2018, par SuperUser_Novice

  I’m using ffmpeg to overlay two videos. The overlay which sits in the top left corner is a collection of png images that act as a frame-counter and time-stamp for the video that the frames have been created around.

  The problem I’m having is that the frames have been generated to exactly match the number of frames the video it is overlaid to has, that way the counter is accurate and frames can be analyzed later. Right now :

  "ffmpeg", "-i", fmt.Sprintf("%s%s", CCME.PathsToUse[0], CCME.UnderlayVideo),
  
                     "-i", fmt.Sprintf("%s%s", CCME.PathsToUse[1], CCME.VideoName),
  
                     "-filter_complex", fmt.Sprintf("overlay=%s", CCME.OverlayPosition),
  
                     "-vsync", "0", "-y", "-strict", "-2",
  
                     fmt.Sprintf("%s%s.%s", CCME.PathsToUse[2], CCME.FinalVideoName, CCME.VidFormat

  (This is a go program running ffmpeg through exec thus the weird format) produces exactly the number of frames expected, but upon analyzing it, every third frame in the overlay video is dropped and the frame before it is duplicated, taking the correct frames place.

  I’ve analyzed the overlay video on its own and it does not have any duplicated frames and the underlay videos frames do not duplicate either.

  This behavior is confusing the rest of the program which is expecting an ordered set of frames and is instead getting repeats of one frame then a jump ahead to the frame it’s supposed to be at (It is also confusing me). Can anyone explain this behavior and perhaps let me know if there is a way to prevent this from happening ? Is there a problem with the arguments I’m passing to ffmpeg in the snippet that may contribute to this behavior ?

  TimeStamps For Videos
  

  Videos Durations
  

  Command :

  ffprobe -f lavfi -i "movie=inputvid.format,fps=fps=24[out0]" -show_frames -show_entries frame=pkt_pts_time -of csv=p=0

  Please note the 1000fps reported by the videos window is wrong. Running ffprobe shows the video is actually 24 fps or 24/1.

  Videos + Result of Overlay

• MoviePy write_videofile is very slow. Why ?

  5 novembre 2024, par RukshanJS

  I've seen multiple questions on SO relating with this, but couldn't find a solid answer. The following is my code.

  


  async def write_final_video(clip, output_path, results_dir):
    cpu_count = psutil.cpu_count(logical=False)
    threads = max(1, min(cpu_count - 1, 16))

    os.makedirs(results_dir, exist_ok=True)

    output_params = {
        "codec": await detect_hardware_encoder(),
        "audio_codec": "aac",
        "fps": 24,
        "threads": threads,
        "preset": "medium",
        "bitrate": "5000k",
        "audio_bitrate": "192k",
    }

    logger.info(f"Starting video writing process with codec: {output_params['codec']}")
    try:
        await asyncio.to_thread(
            clip.write_videofile,
            output_path,
            **output_params,
        )
    except Exception as e:
        logger.error(f"Error during video writing with {output_params['codec']}: {str(e)}")
        logger.info("Falling back to libx264 software encoding")
        output_params["codec"] = "libx264"
        output_params["preset"] = "medium"
        try:
            await asyncio.to_thread(
                clip.write_videofile,
                output_path,
                **output_params,
            )
        except Exception as e:
            logger.error(f"Error during fallback video writing: {str(e)}")
            raise
    finally:
        logger.info("Video writing process completed")

    # Calculate and return the relative path
    relative_path = os.path.relpath(output_path, start=os.path.dirname(ARTIFACTS_DIR))
    return relative_path


  


  and the helper function to get encoder is below

  


  async def detect_hardware_encoder():
    try:
        result = await asyncio.to_thread(
            subprocess.run,
            ["ffmpeg", "-encoders"],
            capture_output=True,
            text=True
        )

        # Check for hardware encoders in order of preference
        if "h264_videotoolbox" in result.stdout:
            return "h264_videotoolbox"
        elif "h264_nvenc" in result.stdout:
            return "h264_nvenc"
        elif "h264_qsv" in result.stdout:
            return "h264_qsv"

        return "libx264"  # Default software encoder
    except Exception:
        logger.warning("Failed to check for hardware acceleration. Using default encoder.")
        return "libx264"


  


  This code makes the rendering of a 15s video around 6min+ which is not acceptable.

  


  t:  62%|██████▏   | 223/361 [04:40<03:57,  1.72s/it, now=None]

  


  My config is MPS (Apple Silicon Metal Performance Shader), but also should work with NVIDIA CUDA.

  


  How can I reduce the time to write the video ? How to actually reduce this long time to write the file ?

  


  The question is about optimizing MoviePy's video processing pipeline, not just FFmpeg :

  


  

  1. clip is a MoviePy VideoFileClip object handling video processing before FFmpeg encoding

  


  2. write_videofile() is MoviePy's method that manages the entire rendering pipeline including :

     

     • Frame extraction and processing

     


     • Memory management

     


     • Multiprocessing coordination

     


     • Audio synchronization

     


     


  


  


  Source - https://zulko.github.io/moviepy/getting_started/videoclips.html#video-files-mp4-webm-ogv

  


  Other places where this is asked,

  


  

  • https://github.com/Zulko/moviepy/issues/2039

  


  


• Convert H.264 Annex B to MPEG-TS

  4 juin 2014, par Lane

  SO...

  I have RAW H.264 video data captured via RTSP in a local file and I am attempting to playback the video in a Java FX application. In order to do this, I need to use Http Live Streaming.

  I have successfully prototyped a Java FX architecture that can play a video via HLS with a local server using a local folder containing a .m3u8 (HLS index) file and collection of .ts (MPEG-TS) files. The last piece for me is to replace the .ts files with .264 / .h264 files and in the local server, perform the conversion / wrapping of the H.264 Annex B data into MPEG-TS.

  I am having trouble figuring out what is required to get H.264 Annex B into MPEG-TS. I have found the following information...

  "Annex B is commonly used in live and streaming formats such as
  transport streams..."

  szatmary.org/blog/25

  "Annex B of of the document specifies one such format, which wraps NAL
  units in a format resembling a traditional MPEG video elementary
  stream, thus making it suitable for use with containers like MPEG
  PS/TS unable to provide the required framing..."

  wiki.multimedia.cx/ ?title=H.264

  "Java FX supports a number of different media types. A media type is
  considered to be the combination of a container format and one or more
  encodings. In some cases the container format might simply be an
  elementary stream containing the encoded data."

  docs.oracle.com/javafx/2/api/javafx/scene/media/package-summary.html

  "Use the CODECS attribute of the EXT-X-STREAM-INF tag. When this
  attribute is present, it must include all codecs and profiles required
  to play back the stream..."

  developer.apple.com/library/ios/documentation/networkinginternet/conceptual/streamingmediaguide/FrequentlyAskedQuestions/FrequentlyAskedQuestions.html

  It seems like I am missing something simple around Elementary and Transport Streams. I have used ffmpeg to convert my H.264 file into a TS file and try to understand the differences. I have an idea of the approximate format differences, but I am still lacking on the details to do it. Does anyone have a link showcasing this or know something simple about how to serve H.264 Annex B data over MPEG-TS ?

  I am not looking to use a tool, I need to have a custom file format locally where I parse out the H.264 Annex B data and perform the format change in memory, on the fly. I know of a way to use ffmpeg with pipes to accomplish this, but I do not want to have any dependencies and performance is important.