Recherche avancée

Sur d’autres sites (5111)

• lavc/vvc : Ensure subpictures don't overlap

  22 février, par Frank Plowman

  lavc/vvc : Ensure subpictures don't overlap
  This is essentially a re-implementation of
  
  https://patchwork.ffmpeg.org/project/ffmpeg/patch/20241005223955.54158-1-post@frankplowman.com/
  That patch was not applied last time.  Instead we opted to identify
  
  issues which could be caused by invalid subpicture layouts and remedy
  
  those issues where they manifest, either through error detection or code
  
  hardening.  This was primarily implemented in the set
  
  https://patchwork.ffmpeg.org/project/ffmpeg/list/?series=13381.
  This has worked to some degree, however issues with subpicture layouts
  
  continue to crop up from the fuzzer and I've fixed a number of bugs
  
  related to subpicture layouts since then.  I think it's best to return
  
  to the initial plan and simply check if the subpicture layout is valid
  
  initially.
  This implementation is also lighter than the first time — by doing a
  
  bit more logic in pps_subpic_less_than_one_tile_slice, we are able to
  
  store a tile_in_subpic map rather than a ctu_in_subpic map.  This
  
  reduces the size of the map to the point it becomes possible to allocate
  
  it on the stack.  Similar to 8bd66a8c9587af61c7b46558be3c4ee317c1af5a,
  
  the layout is also validated in the slice map construction code, rather
  
  than in the CBS, which avoids duplicating some logic.
  Signed-off-by : Frank Plowman <post@frankplowman.com>
  

  • [DH] libavcodec/vvc/ps.c

• Bash script to automate FFmpeg operations fails when calling the command, but copy-pasting the generated command into the terminal works [duplicate]

  28 février, par GaboScharff99

  I wrote a bash script which automates a number of conversion operations on video files using FFmpeg. Oddly enough, the FFmpeg call itself now fails when running the script, with a very confusing error message, I might add, but when I copy the command generated by the script into the terminal and run it, it works flawlessly. I'm sorry to insert such a long code block here, but considering how strange this error is, it might be anywhere in the script, so here it is :

  


  #!/bin/bash

audioTrack=1
subSource=1
subTrack=0
transcodeVideo=1
transcodeAudio=1
volumeMultiplier=1
degradeToStereo=0
subLanguage="Japanese"

while getopts "t:ns:vam:dl:h" opt; do
    case "$opt" in
        t) audioTrack=${OPTARG};;
        n) subSource=0;;
        s) subTrack=${OPTARG};;
        v) transcodeVideo=0;;
        a) transcodeAudio=0;;
        m) volumeMultiplier=${OPTARG};;
        d) degradeToStereo=1;;
        l) subLanguage=${OPTARG};;
        h)
            echo "Options:"
            echo "-t [integer]: Audio track number. Default: 1."
            echo "-n: If included, subtitles will be taken from internal source."
            echo "-s [integer]: Subtitles track number. Default: 0."
            echo "-v: If included, video source will be copied without transcoding."
            echo "-a: If included, audio source will be copied without transcoding."
            echo "-m [number]: Volume multiplier. If 1, volume is unaffected. Default: 1"
            echo "-d: If included, audio will be degraded to stereo."
            echo "-l [language]: Subtitles language. Only used for external subtitles source. Default: Japanese."
            exit 0
        ;;
    esac
done

echo "Audio track: $audioTrack."
echo "Subtitles track: $subTrack."
params="-map 0:0 -map 0:$audioTrack -map $subSource:$subTrack -c:v"

if [[ $transcodeVideo -eq 1 ]]; then
    echo "Video will be transcoded."
    params="$params hevc"
elif [[ $transcodeVideo -eq 0 ]]; then
    echo "Video will be copied without transcoding."
    params="$params copy"
fi

params="$params -c:a"

if [[ $transcodeAudio -eq 1 ]]; then
    echo "Audio will be transcoded."
    params="$params libopus"
elif [[ $transcodeAudio -eq 0 ]]; then
    echo "Audio will be copied without transcoding."
    params="$params copy"
fi

if [[ $volumeMultiplier -ne 1 ]]; then
    echo "Volume will be multiplied by a factor of $volumeMultiplier."
    params="$params -filter:a 'volume=$volumeMultiplier'"
else
    echo "Volume will be unaffected."
fi

if [[ $degradeToStereo -eq 1 ]]; then
    echo "Audio will be degraded to stereo."
    params="$params -ac 2"
elif [[ $degradeToStereo -eq 0 ]]; then
    echo "Audio will not be degraded to stereo."
fi

params="$params -c:s copy"

if [[ $subSource -eq 1 ]]; then
    echo "Subtitles source is external."
    echo "Subtitles language is $subLanguage."
    params="$params -metadata:s:s:0 title='' -metadata:s:s:0 language='$subLanguage'"
else
    echo "Subtitles source is internal."
fi

if [[ -f titles.txt ]]; then
    echo "A titles.txt file was found. Titles will be changed according to it."
    echo "Please check titles.txt to make sure the titles are correct."
    changeTitles=1
    counter=0
else
    echo "A titles.txt file was not found. Titles will not be changed."
    changeTitles=0
fi

read -p "Are these options correct? (y/n) " choice

case "$choice" in
    y|Y)
        echo "Initiating conversion sequence. This may take a while..."

        mkdir output
        currentParams=""

        shopt -s nullglob
        for i in *.mp4 *.mkv; do
            currentParams=$params
            fileNameNoExtension=$(echo $i | rev | cut -f 2- -d '.' | rev)

            if [[ $subSource -eq 1 ]]; then
                currentParams="-f srt -i $fileNameNoExtension.srt $currentParams"
            fi

            if [[ $changeTitles -eq 1 ]]; then
                ((counter++))
                currentParams="$currentParams -metadata title='$(awk "NR==$counter" titles.txt)'"
            fi

            ffmpeg -i "$i" $currentParams "output/$fileNameNoExtension.mkv"
        done

        echo "Conversion finished!"
    ;;
    n|N) echo "Operation canceled. Exiting.";;
    *) echo "Invalid input. Try again.";;
esac


  


  The directory I'm running this in contains six video files :

  


  

  1. E1 - The Pirates of Orion.mkv

  


  2. E2 - Bem.mkv

  


  3. E3 - The Practical Joker.mkv

  


  4. E4 - Albatross.mkv

  


  5. E5 - How Sharper Than a Serpent's Tooth.mkv

  


  6. E6 - The Counter-Clock Incident.mkv

  


  


  Here's the titles.txt file, for completion's sake :

  


  Star Trek: The Animated Series - Season 2, Episode 1 - The Pirates of Orion
Star Trek: The Animated Series - Season 2, Episode 2 - Bem
Star Trek: The Animated Series - Season 2, Episode 3 - The Practical Joker
Star Trek: The Animated Series - Season 2, Episode 4 - Albatross
Star Trek: The Animated Series - Season 2, Episode 5 - How Sharper Than a Serpent's Tooth
Star Trek: The Animated Series - Season 2, Episode 6 - The Counter-Clock Incident


  


  And finally, here's the error message given by FFmpeg on the terminal for every video file when running the command :

  


  Unable to find a suitable output format for 'Trek:'
Trek:: Invalid argument


  


  Maybe there are better ways to handle all of this, but first and foremost, I would like to figure out why the command fails with such a confusing error message. The only place where the string 'Trek :' is found is in the title taken from titles.txt, but I don't understand why that's seemingly being passed to the name of the output file instead of the title, and apparently only when running the script.

  


  Thanks a lot for your answers ! I know this is quite a bit of text, so I really appreciate you taking your time to read through this.

  


• How to improve web camera streaming latency to v4l2loopback device with ffmpeg ?

  11 mars, par Made by Moses

  I'm trying to stream my iPhone camera to my PC on LAN.

  


  What I've done :

  


  


  1. HTTP server with html page and streaming script :

     


     I use WebSockets here and maybe WebRTC is better choice but it seems like network latency is good enough

     


  


  


  async function beginCameraStream() {
  const mediaStream = await navigator.mediaDevices.getUserMedia({
    video: { facingMode: "user" },
  });

  websocket = new WebSocket(SERVER_URL);

  websocket.onopen = () => {
    console.log("WS connected");

    const options = { mimeType: "video/mp4", videoBitsPerSecond: 1_000_000 };
    mediaRecorder = new MediaRecorder(mediaStream, options);

    mediaRecorder.ondataavailable = async (event) => {
      // to measure latency I prepend timestamp to the actual video bytes chunk
      const timestamp = Date.now();
      const timestampBuffer = new ArrayBuffer(8);
      const dataView = new DataView(timestampBuffer);
      dataView.setBigUint64(0, BigInt(timestamp), true);
      const data = await event.data.bytes();

      const result = new Uint8Array(data.byteLength + 8);
      result.set(new Uint8Array(timestampBuffer), 0);
      result.set(data, 8);

      websocket.send(result);
    };

    mediaRecorder.start(100); // Collect 100ms chunks
  };
}


  


  


  2. Server to process video chunks

     


  


  


  import { serve } from "bun";
import { Readable } from "stream";

const V4L2LOOPBACK_DEVICE = "/dev/video10";

export const setupFFmpeg = (v4l2device) => {
  // prettier-ignore
  return spawn("ffmpeg", [
    '-i', 'pipe:0',           // Read from stdin
    '-pix_fmt', 'yuv420p',    // Pixel format
    '-r', '30',               // Target 30 fps
    '-f', 'v4l2',             // Output format
    v4l2device, // Output to v4l2loopback device
  ]);
};

export class FfmpegStream extends Readable {
  _read() {
    // This is called when the stream wants more data
    // We push data when we get chunks
  }
}

function main() {
  const ffmpeg = setupFFmpeg(V4L2LOOPBACK_DEVICE);
  serve({
    port: 8000,
    fetch(req, server) {
      if (server.upgrade(req)) {
        return; // Upgraded to WebSocket
      }
    },
    websocket: {
      open(ws) {
        console.log("Client connected");
        const stream = new FfmpegStream();
        stream.pipe(ffmpeg?.stdin);

        ws.data = {
          stream,
          received: 0,
        };
      },
      async message(ws, message) {
        const view = new DataView(message.buffer, 0, 8);
        const ts = Number(view.getBigUint64(0, true));
        ws.data.received += message.byteLength;
        const chunk = new Uint8Array(message.buffer, 8, message.byteLength - 8);

        ws.data.stream.push(chunk);

        console.log(
          [
            `latency: ${Date.now() - ts} ms`,
            `chunk: ${message.byteLength}`,
            `total: ${ws.data.received}`,
          ].join(" | "),
        );
      },
    },
  });
}

main();


  


  After I try to open the v4l2loopback device

  


  cvlc v4l2:///dev/video10


  


  picture is delayed for at least 1.5 sec which is unacceptable for my project.

  


  Thoughts :

  


  

  • Problem doesn't seems to be with network latency

  


  


  latency: 140 ms | chunk: 661 Bytes | total: 661 Bytes
latency: 206 ms | chunk: 16.76 KB | total: 17.41 KB
latency: 141 ms | chunk: 11.28 KB | total: 28.68 KB
latency: 141 ms | chunk: 13.05 KB | total: 41.74 KB
latency: 199 ms | chunk: 11.39 KB | total: 53.13 KB
latency: 141 ms | chunk: 16.94 KB | total: 70.07 KB
latency: 139 ms | chunk: 12.67 KB | total: 82.74 KB
latency: 142 ms | chunk: 13.14 KB | total: 95.88 KB


  


   150ms is actually too much for 15KB on LAN but there can some issue with my router

  


  

  • As far as I can tell it neither ties to ffmpeg throughput :

  


  


  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'pipe:0':
  Metadata:
    major_brand     : iso5
    minor_version   : 1
    compatible_brands: isomiso5hlsf
    creation_time   : 2025-03-09T17:16:49.000000Z
  Duration: 00:00:01.38, start:
0.000000, bitrate: N/A
    Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc), 1280x720, 4012 kb/s, 57.14 fps, 29.83 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      rotate          : 90
      creation_time   : 2025-03-09T17:16:49.000000Z
      handler_name    : Core Media Video
    Side data:
      displaymatrix: rotation of -90.00 degrees

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))

[swscaler @ 0x55d8d0b83100] deprecated pixel format used, make sure you did set range correctly

Output #0, video4linux2,v4l2, to '/dev/video10':
  Metadata:
    major_brand     : iso5
    minor_version   : 1
    compatible_brands: isomiso5hlsf
    encoder         : Lavf58.45.100

Stream #0:0(und): Video: rawvideo (I420 / 0x30323449), yuv420p, 720x1280, q=2-31, 663552 kb/s, 60 fps, 60 tbn, 60 tbc (default)
    Metadata:
      encoder         : Lavc58.91.100 rawvideo
      creation_time   : 2025-03-09T17:16:49.000000Z
      handler_name    : Core Media Video
    Side data:
      displaymatrix: rotation of -0.00 degrees

frame=   99 fps=0.0 q=-0.0 size=N/A time=00:00:01.65 bitrate=N/A dup=50 drop=0 speed=2.77x
frame=  137 fps=114 q=-0.0 size=N/A time=00:00:02.28 bitrate=N/A dup=69 drop=0 speed=1.89x
frame=  173 fps= 98 q=-0.0 size=N/A time=00:00:02.88 bitrate=N/A dup=87 drop=0 speed=1.63x
frame=  210 fps= 86 q=-0.0 size=N/A time=00:00:03.50 bitrate=N/A dup=105 drop=0 speed=1.44x
frame=  249 fps= 81 q=-0.0 size=N/A time=00:00:04.15 bitrate=N/A dup=125 drop=0 speed=1.36
frame=  279 fps= 78 q=-0.0 size=N/A time=00:00:04.65 bitrate=N/A dup=139 drop=0 speed=1.31x


  


  


  • I also tried to write the video stream directly to video.mp4 file and immediately open it with vlc but it only can be successfully opened after 1.5 sec.

    


  


  • I've tried to use OBS v4l2 input source instead of vlc but the latency is the same

    


  


  


  Update №1

  


  When i try to stream actual .mp4 file to ffmpeg it works almost immediately with 0.2sec delay to spin up the ffmpeg itself :

  


  cat video.mp4 | ffmpeg -re -i pipe:0 -pix_fmt yuv420p -f v4l2 /dev/video10 & ; sleep 0.2 && cvlc v4l2:///dev/video10


  


  So the problem is apparently with streaming process