Recherche avancée

Sur d’autres sites (4349)

• FFmpeg decode video to lossless frames hardware acceleration CUDA

  29 décembre 2020, par Seva Safris

  I have 1 second H.265-encoded videos at 30 fps coming into a server for processing. The server needs to decode the videos into individual frames (lossless). These videos are coming in very quickly, so performance is of utmost importance. The server has a H.265 compatible Nvidia GPU, and I have built ffmpeg with support for CUDA. The following is the configuration output from ffmpeg :

  


  ffmpeg version N-100479-gd67c6c7f6f Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Ubuntu 8.4.0-3ubuntu2)
  configuration: --enable-nonfree --enable-cuda-nvcc --enable-nvenc --enable-opencl --enable-shared
                 --enable-pthreads --enable-version3 --enable-avresample --enable-ffplay --enable-gnutls
                 --enable-gpl --disable-libaom --disable-libbluray --disable-libdav1d 
                 --disable-libmp3lame --enable-libopus --disable-librav1e --enable-librubberband
                 --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora
                 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264
                 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig
                 --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb
                 --enable-libopencore-amrwb --disable-libopenjpeg --enable-librtmp --enable-libspeex
                 --enable-libsoxr --disable-videotoolbox --disable-libjack --disable-indev=jack
                 --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64


  


  I decode the videos into PNGs, and am using the following command :

  


  ffmpeg -y -vsync 0 -hwaccel cuvid -hwaccel_output_format cuda -hwaccel_device 0 -c:v hevc_cuvid \
       -i 0.mp4 -vf hwdownload,format=nv12 -q:v 1 -qmin 1 -qmax 1 -start_number 0 f%d.png


  


  This command successfully leverages the hardware acceleration for the H.265 decode. But, the PNG encode is done by the CPU.

  


  Does CUDA have support for encoding of lossless images ? The format does not need to be PNG, but it does need to be lossless. CUDA has a nvJPEG Library, but JPEG is a lossy format. Is there a similar image encoding library in CUDA for a lossless format (that is also integrated with ffmpeg) ?

  


  Edit : Some more context....

  


  I am currently using PNGs because of their compression-ability. These images are 2560x1280 in size, btw. On one hand, it is this compression that costs the CPU cycles. On the other hand, I am also limited by the throughput of how fast (and how much aggregate data) can I upload these frames to the upstream consumer. So it's basically a tradeoff between :

  


  

  1. We want to extract these frames as quickly as possible.

  


  2. We want efficiency regarding the image size.

  


  


• FFMPEG rtmp stream slow and only works on chrome

  18 juin 2023, par DrMeepso

  I am trying to stream to twitch from a Node.js script for fun ! so far i have got it working but its slow and seemingly limited to 10-12 fps.

  


  Here is my code, i am trying to stream a Node-Canvas to twitch so i can do stuff without the overhead of OBS and in a console :

  


  import { createCanvas } from 'canvas';
import { spawn } from 'child_process'
import fs from 'fs';

const canvas = createCanvas(1920, 1080);
const ctx = canvas.getContext('2d');
// create a ffmpeg process that will stream the canvas to a rtmp server
const ffmpeg = spawn('ffmpeg', [
    '-re', // read input at native frame rate
    '-f', 'image2pipe', // tell ffmpeg to expect raw image stream
    '-vcodec', 'png', // tell ffmpeg to expect jpeg encoded images
    '-r', '10', // tell ffmpeg to expect 60 fps
    '-s', '1920x1080', // tell ffmpeg to expect size 1920x1080
    '-i', '-', // tell ffmpeg to expect input from stdin
    //'-c:v', 'h264_amf', // tell ffmpeg to encode in h264 codec AMD
    '-c:v', 'libx264', // tell ffmpeg to encode in h264 codec
    '-pix_fmt', 'yuv420p', // tell ffmpeg to encode in yuv420p format
    '-preset', 'ultrafast', // tell ffmpeg to use ultrafast encoding preset
    '-g', '20', // set the keyframe interval to 2 seconds so twitch will not reject the stream
    '-f', 'flv', // tell ffmpeg that we are going to stream flv video
    "rtmp://live.restream.io/live/"
]);

ffmpeg.on('close', (code, signal) => {
    console.log('FFmpeg child process closed, code ' + code + ', signal ' + signal);
})

ffmpeg.on('error', (err) => {
    console.log('FFmpeg child process error: ' + err.message);
})

ffmpeg.stderr.on('data', (data) => {
    //console.log('FFmpeg stderr: ' + data);
    //ParseFFPMPEGOutput(data.toString());
})

ffmpeg.stdout.on('data', (data) => {
    console.log('FFmpeg stdout: ' + data);
})

var DrawingCatagorys: string[] = []
fs.readFileSync('categories.txt').toString().split('\n').forEach(function (line) { DrawingCatagorys.push(line); })

function ParseFFPMPEGOutput(data: string) {

    if (data.includes('frame=')) {
        let frame = data.split('frame=')[1].split('fps')[0].trim();
        let fps = data.split('fps=')[1].split('q=')[0].trim();
        let size = data.split('size=')[1].split('time=')[0].trim();
        let time = data.split('time=')[1].split('bitrate=')[0].trim();
        let bitrate = data.split('bitrate=')[1].split('speed=')[0].trim();

        console.log(`Frame: ${frame} FPS: ${fps} Size: ${size} Time: ${time} Bitrate: ${bitrate}`);
    }

}

// create a png stream
setInterval(() => {
    // convert canvas to png
    const buffer = canvas.toBuffer('image/png');
    // write png stream to stdin of ffmpeg process
    ffmpeg.stdin.write(buffer);
}, 2);


  


  Im thinking it has to do with Node-Canvas just being slow but i have no idea, also even on my gaming rig it cant pull 1x speed while streaming always sitting around 0.95 - 0.98 and about 9.2 fps. also it only works on chrome and not firefox ? i have no idea why probably a codec thing

  


• Create a demultiplexer for MPEG 2 TS in android

  17 novembre 2014, par anz

  I have a requirement where I need to extract ID3 tags from a MPEG2 TS(HLS STREAM). MPEG2 has a limited support in android in regards to playing the file. But my concern is to extract the ID3 tags(playing the file is not necessary). Hence I am not concerned with the codecs(encoding and decoding).

  I have explored the following options :

  libstagefright and OpenMax : A playback engine implemented by Google from Android 2.0.
  It has a MediaExtractor is responsible for retrieving track data and the corresponding meta data from the underlying file system or http stream. But according to this post Adding video codec to Android I need to build my own firmware or my own media player.I am hoping I don’t have to go down that path. More info on stagefright and openMax can be found here :

  An overview of Stagefright player

  Android’s Stagefright Media Player Architecture

  Custom Wrapper Codec Integration into Android

  How to integrate a decoder to multimedia framework

  Compiling and using FFMPEG : A complete, cross-platform solution to record, convert and stream audio and video. We can demultiplex ts files with this library as mentioned here :

  FFmpeg - Extracting video and audio from transport stream file (.ts).

  But I am not sure if I will be able to extract the ID3 tags from the HLS Stream. libavformat might be able to do this but I still need to come up with a mechanism for signaling the read metadata to my application.

  Compiling vlc for android : I have compiled vlc for android and made some modifications inside the transport module in demux component for extracting the tags, but it is not able to play all the streams that I am supplying to it.

  After looking through these options , I am still at a fix in how to achieve this. I don’t want to create a media player as I will not be playing the files nor do I want to build my own firmware. Using ffmpeg seems to be the most viable option, but I want to try this without using any third-party or open source library. My questions are :

  Is it even possible to create a demultiplexer from scratch that will work on android ?

  If possible then ,how to go about it ?

  Any options that I have missed ?

  I am new to this. Any help would be greatly appreciated..Thanks