Recherche avancée

Sur d’autres sites (11388)

• ffmpeg_kit_flutter_full : Video created from ffmpeg not showing in the web

  16 décembre 2024, par Đoàn Thanh Thảo

  I am implementing a screen capture widget to create a video. The video was successfully created in MP4 format and I can view it in my app and then I posted it to my web. However, on the web, I cannot see the video preview, only when I download the video and view it on my phone can I see it. I think the problem is that there is some missing format that makes the video not playable on Https. Here is my code :

  


      final command = '-y -f image2 -i $imagesPath '
    '${fps != null ? "-r $fps" : ""} '
    '-vf "scale=2000:5000" '
    // '-vf "scale=iw*$scaleMultiplier:ih*$scaleMultiplier" '
    '-c:v mpeg4 -q:v $qValue -pix_fmt yuv420p -movflags +faststart $outputVideoPath';


  


  I try to use libx264 nhưng bị báo lỗi

  


• Video generation and ffmpeg and locally stored images [closed]

  3 juillet, par Rahul Patil

  I am facing issue in ffmpeg while running the below code

  


  I'm working on a Flask application that generates a video by combining a sequence of images from a folder and a synthesized audio track using Bark (suno/bark-small). The idea is to use FFmpeg to stitch the images into a video, apply padding and scaling, and then merge it with the generated audio. I'm triggering the /generate-video endpoint with a simple curl POST request, passing a script that gets converted to audio. While the image and audio processing work as expected, FFmpeg fails during execution, and the server returns a 500 error. I've added error logging to capture FFmpeg’s stderr output, which suggests something is going wrong either with the generated input.txt file or the format of the inputs passed to FFmpeg. I'm not sure if the issue is related to file paths, the concat demuxer formatting, or possibly audio/video duration mismatch. Any insights on how to debug or correct the FFmpeg command would be appreciated.

  


  the curl request is

  


  curl -X POST http://localhost:5000/generate-video \
     -H "Content-Type: application/json" \
     -d '{"script": "Hello, this is a test script to generate a video."}' \
     --output output_video.mp4


  


  import os
import uuid
import subprocess
from pathlib import Path
import numpy as np
from flask import Flask, request, jsonify, send_file
from transformers import AutoProcessor, AutoModelForTextToWaveform
from scipy.io.wavfile import write as write_wav
import torch

# ========== CONFIG ==========
IMAGE_FOLDER = "./images"
OUTPUT_FOLDER = "./output"
RESOLUTION = (1280, 720)
IMAGE_DURATION = 3  # seconds per image
SAMPLE_RATE = 24000

app = Flask(__name__)
os.makedirs(OUTPUT_FOLDER, exist_ok=True)

# Load Bark-small model and processor
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoProcessor.from_pretrained("suno/bark-small")
model = AutoModelForTextToWaveform.from_pretrained("suno/bark-small").to(device)


# ========== UTILS ==========
def run_ffmpeg(cmd):
    result = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    if result.returncode != 0:
        print("[FFmpeg ERROR]\n", result.stderr.decode())
        raise RuntimeError("FFmpeg failed.")
    else:
        print("[FFmpeg] Success.")


def find_images(folder):
    return sorted([
        f for f in Path(folder).glob("*")
        if f.suffix.lower() in {".jpg", ".jpeg", ".png"}
    ])


def create_ffmpeg_input_list(images, list_file_path):
    with open(list_file_path, "w") as f:
        for img in images:
            f.write(f"file '{img.resolve()}'\n")
            f.write(f"duration {IMAGE_DURATION}\n")
        # Repeat last image to avoid cutoff
        f.write(f"file '{images[-1].resolve()}'\n")


# ========== FLASK ROUTE ==========
@app.route('/generate-video', methods=['POST'])
def generate_video():
    data = request.get_json()
    script = data.get("script")
    if not script:
        return jsonify({"error": "No script provided"}), 400

    images = find_images(IMAGE_FOLDER)
    if not images:
        return jsonify({"error": "No images found in ./images"}), 400

    # Generate audio
    print("[1/3] Generating audio with Bark...")
    inputs = processor(script, return_tensors="pt").to(device)
    with torch.no_grad():
        audio_values = model.generate(**inputs)

    audio_np = audio_values[0].cpu().numpy().squeeze()
    audio_np = np.clip(audio_np, -1.0, 1.0)
    audio_int16 = (audio_np * 32767).astype(np.int16)

    audio_path = os.path.join(OUTPUT_FOLDER, f"{uuid.uuid4()}.wav")
    write_wav(audio_path, SAMPLE_RATE, audio_int16)

    # Create FFmpeg concat file
    print("[2/3] Preparing image list for FFmpeg...")
    list_file = os.path.join(OUTPUT_FOLDER, "input.txt")
    create_ffmpeg_input_list(images, list_file)

    # Final video path
    final_video_path = os.path.join(OUTPUT_FOLDER, f"{uuid.uuid4()}.mp4")

    # Run FFmpeg
    print("[3/3] Running FFmpeg to create video...")
    ffmpeg_cmd = [
        "ffmpeg", "-y",
        "-f", "concat", "-safe", "0", "-i", list_file,
        "-i", audio_path,
        "-vf", f"scale={RESOLUTION[0]}:{RESOLUTION[1]}:force_original_aspect_ratio=decrease,"
               f"pad={RESOLUTION[0]}:{RESOLUTION[1]}:(ow-iw)/2:(oh-ih)/2:color=black",
        "-c:v", "libx264", "-pix_fmt", "yuv420p",
        "-c:a", "aac", "-b:a", "192k",
        "-shortest", "-movflags", "+faststart",
        final_video_path
    ]

    try:
        run_ffmpeg(ffmpeg_cmd)
    except RuntimeError:
        return jsonify({"error": "FFmpeg failed. Check server logs."}), 500

    return send_file(final_video_path, as_attachment=True)


# ========== RUN APP ==========
if __name__ == '__main__':
    app.run(debug=True)


  


• How to write a video stream to a server ?

  14 août, par The Mask

  Recently been playing with FFmpeg and it's powerful abilities. Working on a cool project where I'm trying to create a live video stream using FFmpeg. The client (reactJs) and server (nodeJS) are connected via web-socket. The client sends the byte packets to server and the server then spawns an FFmpeg process and serve it to an nginx server.

  


  Client(live-stream.js) :

  


  const stream = await navigator.mediaDevices.getUserMedia({
    video: true,
    audio: true,
  });
  videoRef.current.srcObject = stream;

  const ingestUrl = `ws://localhost:8081/ws`
  const socket = new WebSocket(ingestUrl);
  socket.binaryType = "arraybuffer";
  socket.onopen = () => {
    console.log("✅ WebSocket connection established");
    socket.send(JSON.stringify({ type: "start", stream_key: streamKey }));
    mediaRecorderRef.current.start(500);
  };
  socketRef.current = socket;

  socket.onerror = (error) => {
    console.error("❌ WebSocket error:", error);
  };

  mediaRecorderRef.current = new MediaRecorder(stream, {
    mimeType: "video/webm;codecs=vp8,opus",
    videoBitsPerSecond: 1000000,
    audioBitsPerSecond: 128000
  });
  mediaRecorderRef.current.ondataavailable = (event) => {
    if (event.data.size > 0 && socket.readyState === WebSocket.OPEN) {
      event.data.arrayBuffer().then((buffer) => socket.send(buffer));
    }
  };


  


  Server(index.js) :

  


  const http = require('http');
const WebSocket = require('ws');
const { spawn } = require('child_process');
const fs = require('fs');


const server = new WebSocket.Server({ server:wss, path:'/ws'});

const startFFmpeg = (stream_key) => {
  return ffmpeg = spawn("ffmpeg", [
    "-re",
    "-f", "matroska",
    "-i", "pipe:0",
    "-map", "0:v:0",
    "-map", "0:a:0",
    "-c:v", "libx264",
    "-c:a", "aac ",
    "-b:v", "6000k",
    "-maxrate", "6000k ",
    "-bufsize", "6000k ",
    "-pix_fmt", "yuv420p ",
    "-f", "flv",
    `rtmp://localhost/live/${stream_key}`,
  ]);
}
server.on('connection', (ws) => {
  console.log('📡 New WebSocket connection');

  let ffmpeg = null;
  let buffer = Buffer.alloc(0);
  let streamStarted = false;

  ws.on('message', (msg, isBinary) => {
    if (!isBinary) {
      const parsed = JSON.parse(msg);
      if (parsed.type === "start") {
        const { stream_key } = parsed;
        console.log(`🔑 Stream key: ${stream_key}`);
        console.log(`🎥 Starting ingest for stream key: ${stream_key}`);

        ffmpeg = startFFmpeg(stream_key)
        ffmpeg.stdin.on("error", (e) => {
          console.error("FFmpeg stdin error:", e.message);
        });

        ffmpeg.stderr.on("data", (data) => {
          console.log(`FFmpeg Data: ${data}`);
        });

        ffmpeg.on("close", (code) => {
          console.log(`FFmpeg exited with code ${code}`);
        });

        ffmpeg.on("exit", (code, signal) => {
          console.log(`FFmpeg exited with code: ${code}, signal: ${signal}`);
          if (signal === 'SIGSEGV') {
            console.log('🔄 FFmpeg segfaulted, attempting restart...');
            setTimeout(() => {
              if (ws.readyState === WebSocket.OPEN) {
                startFFmpeg(stream_key);
              }
            }, 1000);
          }
        });

        streamStarted = true;
      }
    } else if (isBinary && ffmpeg && ffmpeg.stdin.writable) {
        try {
        // Convert to Buffer if it's an ArrayBuffer
        let data;
        if (msg instanceof ArrayBuffer) {
          data = Buffer.from(msg);
        } else {
          data = Buffer.from(msg);
        }

        // Buffer the data
        buffer = Buffer.concat([buffer, data]);
        
        // Write in larger chunks to reduce overhead
        if (buffer.length >= 8192) { // 8KB threshold
          console.log(`📥 Writing ${buffer.length} bytes to FFmpeg`);
          
          if (ffmpeg.stdin.write(buffer)) {
            buffer = Buffer.alloc(0);
          } else {
            // Handle backpressure
            ffmpeg.stdin.once('drain', () => {
              buffer = Buffer.alloc(0);
              ffmpeg.stdin.setMaxListeners(20); // or a safe upper bound
            });
          }
        }
      } catch (e) {
        console.error("FFmpeg write error:", e);
      }
    }
  });
  
  ws.on('close', () => {
    console.log('❌ WebSocket closed');
    streamStarted = false;

    if (ffmpeg){     // Write any remaining buffer
      if (buffer.length > 0 && ffmpeg.stdin.writable) {
        console.log(`📥 Writing final ${buffer.length} bytes to FFmpeg`);
        ffmpeg.stdin.write(buffer);
      }
            
      // Gracefully close FFmpeg
      if (ffmpeg.stdin.writable) {
        ffmpeg.stdin.end();
      }
      
      setTimeout(() => {
        if (ffmpeg && !ffmpeg.killed) {
          ffmpeg.kill('SIGTERM');
          setTimeout(() => {
            if (ffmpeg && !ffmpeg.killed) {
              ffmpeg.kill('SIGKILL');
            }
          }, 5000);
        }
      }, 1000);
    }
  });
});

wss.listen(8081, "localhost", () => {
  console.log("🛰️ Server listening on http://localhost:8081/ws");
});


  


  The problem statment :
Been facing error like pixels drops in the video, bad quality. FFmpeg is crashing with error :

  


  FFmpeg Data: Input #0, matroska,webm, from 'pipe:0':

FFmpeg Data:   Metadata:
    encoder         : Chrome
  Duration: N/A, start: 0.000000, bitrate: N/A
  Stream #0:0(eng): Audio: opus, 48000 Hz, mono, fltp (default)

FFmpeg Data:   Stream #0:1(eng): Video: vp8, yuv420p(progressive), 640x480, SAR 1:1 DAR 4:3, 
FFmpeg Data: 1k tbr, 1k tbn (default)
    Metadata:
      alpha_mode      : 1

FFmpeg Data: Unknown pixel format requested: yuv420p .

FFmpeg stdin error: write EPIPE
FFmpeg exited with code: 1, signal: null
FFmpeg exited with code 1