Recherche avancée

Sur d’autres sites (3715)

• On the fly transcoding and HLS streaming with ffmpeg

  12 janvier 2023, par syfluqs

  I am building a web application that involves serving various kinds of video content. Web-friendly audio and video codecs are handled without any problems, but I am having trouble designing the delivery of video files incompatible with HTML5 video players like mkv containers or H265.

  



  What I have done till now, is use ffmpeg to transcode the video file on the server and make HLS master and VOD playlists and use hls.js on the frontend. The problem, however, is that ffmpeg treats the playlist as a live stream playlist until transcoding is complete on the whole file and then it changes the playlist to serve as VOD. So, the user can't seek until the transcoding is over, and that my server has unnecessarily transcoded the whole file if the user decides to seek the video file halfway ahead. I am using the following ffmpeg command line arguments

  



  ffmpeg -i sample.mkv \
       -c:v libx264 \
       -crf 18 \
       -preset ultrafast \
       -maxrate 4000k \
       -bufsize 8000k \
       -vf "scale=1280:-1,format=yuv420p" \
       -c:a copy -start_number 0 \
       -hls_time 10 \
       -hls_list_size 0 \
       -f hls \
file.m3u8


  



  Now to improve upon this system, I tried to generate the VOD playlist through my app and not ffmpeg, since the format is self explanatory. The webapp would generate the HLS master and VOD playlists beforehand using the video properties such as duration, resolution and bitrate (which are known to the server) and serve the master playlist to the client. The client then starts requesting the individual video segments at which point the server will individually transcode and generate each segment and serve them. Seeking would be possible as the client already has the complete VOD playlist and it can request the specific segment that the user seeks to. The benefit, as I see it, would be that my server would not have to transcode the whole file, if the user decides to seek forward and play the video halfway through.

  



  Now I tried manually creating segments (10s each) from my sample.mkv using the following command

  



  ffmpeg -ss 90 \
       -t 10 \
       -i sample.mkv \
       -g 52 \
       -strict experimental \
       -movflags +frag_keyframe+separate_moof+omit_tfhd_offset+empty_moov \
       -c:v libx264 \
       -crf 18 \
       -preset ultrafast \
       -maxrate 4000k \
       -bufsize 8000k \
       -vf "scale=1280:-1,format=yuv420p" \
       -c:a copy \
fileSequence0.mp4


  



  and so on for other segments, and the VOD playlist as

  



  #EXTM3U
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-TARGETDURATION:10
#EXT-X-VERSION:4
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:10.0,
fileSequence0.mp4
#EXTINF:10.0,
fileSequence1.mp4
...
... and so on 
...
#EXT-X-ENDLIST


  



  which plays the first segment just fine but not the subsequent ones.

  



  Now my questions,

  



  


  1. Why don't the subsequent segments play ? What am I doing wrong ?

  


  2. Is my technique even viable ? Would there be any problem with presetting the segment durations since segmenting is only possible after keyframes and whether ffmpeg can get around this ?

  


  



  My knowledge regarding video processing and generation borders on modest at best. I would greatly appreciate some pointers.

  


• FFMPEG SAR mismatch when concatenating videos

  6 septembre 2020, par marcman

  I'm trying to concatenate videos with different codecs using the concat filter. I'm also doing a bunch of processing to scale and pad the individual clips in this filter as well. However, I keep getting an error of this form :

  


  [Parsed_concat_14 @ 0x556b918ed580] Input link in1:v0 parameters (size 960x1280, SAR 0:1) do not match the corresponding output link in0:v0 parameters (1280x720, SAR 1:1)


  


  Here is my command :

  


  ffmpeg \
    -y \
    -loglevel warning \
    -stats \
    -i videos/vid0.mp4 \
    -i videos/vid1.mp4 \
    -i videos/vid2.mov \
    -filter_complex \
"[0:v]setpts=PTS-STARTPTS, scale=1920:1080, setsar=1, drawtext=expansion=strftime: basetime=$(date +%s -d'2020-07-10 16:04:44')000000 : fontcolor=white : text='%^b %d, %Y%n%l\\:%M%p' : fontsize=36 : y=1080-4*lh : x=1920-text_w-2*max_glyph_w; \
[1:v]setpts=PTS-STARTPTS, scale=810:1080, pad=width=1920:height=1080:x=555:y=0:color=black, setsar=1, drawtext=expansion=strftime: basetime=$(date +%s -d'2020-08-20 21:12:27')000000 : fontcolor=white : text='%^b %d, %Y%n%l\\:%M%p' : fontsize=36 : y=1080-4*lh : x=1365-text_w-2*max_glyph_w; \
[2:v]setpts=PTS-STARTPTS, scale=607:1080, pad=width=1920:height=1080:x=656:y=0:color=black, setsar=1, drawtext=expansion=strftime: basetime=$(date +%s -d'2020-08-27 16:42:26')000000 : fontcolor=white : text='%^b %d, %Y%n%l\\:%M%p' : fontsize=36 : y=1080-4*lh : x=1263-text_w-2*max_glyph_w; \
[0:v][0:a][1:v][1:a][2:v][2:a]concat=n=3:v=1:a=1[v][a]" \
    -map "[v]" \
    -map "[a]" \
    videos.mp4


  


  This gives the following error :

  


  [Parsed_concat_14 @ 0x563ab9d840c0] Input link in1:v0 parameters (size 960x1280, SAR 0:1) do not match the corresponding output link in0:v0 parameters (1280x720, SAR 1:1)
[Parsed_concat_14 @ 0x563ab9d840c0] Input link in2:v0 parameters (size 1080x1920, SAR 0:1) do not match the corresponding output link in0:v0 parameters (1280x720, SAR 1:1)


  


  The input videos have these resolutions :

  


  

  1. vid0.mp4 : (1280x720)

  


  2. vid1.mp4 : (960x1280)

  


  3. vid2.mp4 : (1080x1920)

  


  


  The output resolution of the concatenated output is to be 1920x1080.

  


  I've added the setsar=1 command after all my scaling and padding operations, as per this question and answer. I've also tried setdar=16/9 as in this answer, but it made no difference.

  


  What am I missing here ?

  


• NodeJS - efficiently and correctly convert from raw PCM to WAV at scale (without FFMPEG ?)

  13 juillet 2024, par Royi Bernthal

  I have a stream of raw PCM buffers I need to convert to playable WAV buffers.

  


  @ffmpeg.wasm can convert an individual buffer in the stream well, but it's limited to executing 1 command at a time, so it won't work in a real-life streaming scenario (streams x concurrent users). We can use it as a reference to a conversion that outputs a good playable wav.

  


  import { FFmpeg, createFFmpeg, fetchFile } from '@ffmpeg.wasm/main';

async pcmToWavFFMPEG(buffer: Buffer) {
    // bitDepth - PCM signed 16-bit little-endian
    const options = { sampleRate: '24k', channels: '1', bitDepth: 's16le' };

    this.ffmpeg.FS('writeFile', 'input.pcm', await fetchFile(buffer));

    await this.ffmpeg.run(
      '-f',
      options.bitDepth,
      '-ar',
      options.sampleRate,
      '-ac',
      options.channels,
      '-i',
      'input.pcm',
      'output.wav',
    );

    const wavBuffer = this.ffmpeg.FS('readFile', 'output.wav');

    this.ffmpeg.FS('unlink', `input.pcm`);
    this.ffmpeg.FS('unlink', `output.wav`);

    return Buffer.from(wavBuffer);
  }


  


  In order to get over the command execution limit, I've tried fluent-ffmpeg. I couldn't find a way to convert a single buffer, so I'm just passing the whole readable stream so that ffmpeg can convert all of its buffers to wav. The buffers I'm getting in on('data') aren't playable wav. The same is true for concatting the accumulated buffers in on('complete') - the result is not a playable wav.

  


  import ffmpeg from 'fluent-ffmpeg';
import internal from 'stream';

async pcmToWavFluentFFMPEG(
    readable: internal.Readable,
    callback: (chunk: Buffer) => void,
  ) {
    const options = { sampleRate: 24000, channels: 1, bitDepth: 's16le' };

    ffmpeg(readable)
      .inputFormat(options.bitDepth)
      .audioFrequency(options.sampleRate)
      .audioChannels(options.channels)
      .outputFormat('wav')
      .pipe()
      .on('data', callback);
  }


  


  I've also tried using node-wav to convert each buffer individually. It manages to convert everything to playable wavs that sound close to the desired result, however for some reason they're extremely loud and sound a bit weird.

  


  import wav from 'node-wav';

pcmToWavBad(buffer: Buffer) {
    const pcmData = new Int16Array(
      buffer.buffer,
      buffer.byteOffset,
      buffer.byteLength / Int16Array.BYTES_PER_ELEMENT,
    );

    const channelData = [pcmData]; // assuming mono channel

    return wav.encode(channelData, { sampleRate: 24000, bitDepth: 16 });
  }


  


  I've also tried wrapping the PCM as a WAV with wavefile without any actual conversion (which is redundant as PCM is contained as is in WAV), but it results in white noise :

  


  import { WaveFile } from 'wavefile';

pcmToWav(buffer: Buffer) {
    const wav = new WaveFile();

    wav.fromScratch(1, 24000, '16', buffer); // 's16le' is invalid

    return Buffer.from(wav.toBuffer());
  }