Recherche avancée

Sur d’autres sites (9590)

• Adding FFMPEG Layer to HLS streaming causes video playback issues

  25 juin 2023, par Moe

  I have been searching a lot about HLS streaming and have succeeded to create a simple HLS streaming server with nodejs, the problem now is I need to add a layer of ffmpeg encoding to the .ts chunks before streaming to the user, without this layer everything works fine and on my server only 3 requests are seen :

  


  manifest.m3u8
output_000.ts
output_000.ts
output_001.ts
output_002.ts


  


  But then when I add a simple ffmpeg layer that literally copies everything from the ts file and output the stream (I will add of course dynamic filters to each request, thats why I need this ffmpeg layer), the player goes insane and request the whole video in just 5 seconds or something :

  


  manifest.m3u8
output_000.ts
output_000.ts
output_001.ts
output_002.ts
output_001.ts
output_003.ts
output_002.ts
...
output_095.ts


  


  I have also notices that the numbers aren't increasing uniformly and suspect this is part of the issue, I have tried adding more ffmpeg options to not do anything to the .ts files that are being fed to it as they are a part of a bigger video.

  


  Here's my NodeJS server (NextJS API route) :

  


  
const fs = require(`fs`);
const path = require(`path`);
const {exec, spawn} = require(`child_process`);
const pathToFfmpeg = require(`ffmpeg-static`);

export default function handler(req, res) {
  
    const { filename } = req.query;
    console.log(filename);
    const filePath = path.join(process.cwd(), 'public', 'stream', `${filename}`);
    const inputStream = fs.createReadStream(filePath);

    // first check if that is ts file..
    if(filename.indexOf(`.ts`) != -1){
  
      const ffmpegProcess = spawn(pathToFfmpeg, [
        '-f', `mpegts`,
        '-i', 'pipe:0', // specify input as pipe
        '-c', 'copy', 
        '-avoid_negative_ts', '0',
        `-map_metadata`, `0`,  // copy without re-encoding
        '-f', 'mpegts', // output format
        'pipe:1'        // specify output as pipe
      ], {
        stdio: ['pipe', 'pipe', 'pipe'] // enable logging by redirecting stderr to stdout
      });
      res.status(200);
      res.setHeader('Content-Type', 'application/vnd.apple.mpegurl');
      res.setHeader('Cache-Control', 'no-cache');
      res.setHeader('Access-Control-Allow-Origin', '*');
 

      // ffmpegProcess.stderr.pipe(process.stdout); // log stderr to stdout
  
      inputStream.pipe(ffmpegProcess.stdin);
      ffmpegProcess.stdout.pipe(res);
  
      ffmpegProcess.on('exit', (code) => {
        if (code !== 0) {
          console.error(`ffmpeg process exited with code ${code}`);
        }
      });
    }else{
      // if not then stream whatever file as it is
      res.status(200);
      res.setHeader('Content-Type', 'application/vnd.apple.mpegurl');
      inputStream.pipe(res);
    }
  }


  


  I have tried to feed the request's player appropriate headers but that didn't work, I have also tried to add the '-re' option to the ffmpeg encoder itself and hoped for minimal performance hits, but that also caused playback issue due to being too slow.

  


• Is there a pure golang implementation of aac transcode opus ? [closed]

  15 février 2023, par fanzhang

  Please note : pure golang implementation, no ffmpeg-wrapper / c-go

  


  Is there a pure golang implementation of aac transcode opus ?

  


  I've written a streaming-WebRTC gateway app that can convert av streams from streaming devices to WebRTC via pion, but now there's a tricky problem, the audio encoding provided by these media devices is usually aac(WebRTC do not support aac), I can't find a library that implements aac -> opus (or pcm -> opus) in pure go, only some library based on c-go (like this one). The c-go based library has some limitations, e.g. it can't be self-contained, so there a pure golang implementation of aac transcode opus ?

  


  The code (snippet) below is my current implementation using Glimesh's fdk-aac and hraban's opus

  


  ...

    // 音频相关配置
    // https://github.com/Glimesh/go-fdkaac
    aacDecoder := fdkaac.NewAacDecoder()
    defer func() {
        _ = aacDecoder.Close()
    }()
    aacDecoderInitDone := false

    var opusEncoder *hrabanopus.Encoder
    minAudioSampleRate := 16000
    var opusAudioBuffer []byte
    opusBlockSize := 960
    opusBufferSize := 1000
    opusFramesSize := 120

...

            // 用 AAC 的元数据初始化 AAC 编码器
            // https://github.com/winlinvip/go-fdkaac
            if tag.AACPacketType == flvio.AAC_SEQHDR {
                if !aacDecoderInitDone {
                    if err := aacDecoder.InitRaw(tagData); err != nil {
                        return errors.Wrapf(err, "从 %s.%s 的(音频元数据 %s)标签初始化 AAC 解码器失败", flowControlGroup, streamKey, hex.EncodeToString(tagData))
                    }
                    aacDecoderInitDone = true

                    logrus.Infof("从 %s.%s 的(音频元数据 %s)标签初始化了 AAC 解码器 %p", flowControlGroup, streamKey, hex.EncodeToString(tagData), aacDecoder)
                }
            } else {
                tagDataString := hex.EncodeToString(tagData)
                logrus.Tracef("使用已初始化了的 AAC 解码器 %p 解码 %s.%s 的音频数据 %s", aacDecoder, flowControlGroup, streamKey, tagDataString)

                // 解码 AAC 为 PCM
                decodeResult, err := aacDecoder.Decode(tagData)
                if err != nil {
                    return errors.Wrapf(err, "从 %s.%s 的标签解码 PCM 数据失败", flowControlGroup, streamKey)
                }

                rate := aacDecoder.SampleRate()
                channels := aacDecoder.NumChannels()

                if rate < minAudioSampleRate {
                    logrus.Tracef("从 %s.%s 的标签解码 PCM 数据得到音频采样频 %d小于要求的最小值(%d),将忽略编码opus的操作", flowControlGroup, streamKey, rate, minAudioSampleRate)
                } else {
                    if opusEncoder == nil {
                        oEncoder, err := hrabanopus.NewEncoder(rate, channels, hrabanopus.AppAudio)
                        if err != nil {
                            return err
                        }
                        opusEncoder = oEncoder
                    }

                    // https://github.com/Glimesh/waveguide/blob/a7e7745be31d0a112aa6adb6437df03960c4a5c5/internal/inputs/rtmp/rtmp.go#L289
                    // https://github.com/xiangxud/rtmp-to-webrtc/blob/07d7da9197cedc3756a1c87389806c3670b9c909/rtmp.go#L168
                    for opusAudioBuffer = append(opusAudioBuffer, decodeResult...); len(opusAudioBuffer) >= opusBlockSize*4; opusAudioBuffer = opusAudioBuffer[opusBlockSize*4:] {
                        pcm16 := make([]int16, opusBlockSize*2)
                        pcm16len := len(pcm16)
                        for i := 0; i < pcm16len; i++ {
                            pcm16[i] = int16(binary.LittleEndian.Uint16(opusAudioBuffer[i*2:]))
                        }
                        opusData := make([]byte, opusBufferSize)
                        n, err := opusEncoder.Encode(pcm16, opusData)
                        if err != nil {
                            return err
                        }
                        opusOutput := opusData[:n]

                        // https://datatracker.ietf.org/doc/html/rfc6716#section-2.1.4
                        // Opus can encode frames of 2.5, 5, 10, 20, 40, or 60 ms.  It can also
                        // combine multiple frames into packets of up to 120 ms.  For real-time
                        // applications, sending fewer packets per second reduces the bitrate,
                        // since it reduces the overhead from IP, UDP, and RTP headers.
                        // However, it increases latency and sensitivity to packet losses, as
                        // losing one packet constitutes a loss of a bigger chunk of audio.
                        // Increasing the frame duration also slightly improves coding
                        // efficiency, but the gain becomes small for frame sizes above 20 ms.
                        // For this reason, 20 ms frames are a good choice for most
                        // applications.
                        sampleDuration := time.Duration(opusFramesSize) * time.Millisecond
                        sample := media.Sample{
                            Data:     opusOutput,
                            Duration: sampleDuration,
                        }
                        if err := audioTrack.WriteSample(sample); err != nil {
                            return err
                        }
                    }
                }
            }

...


  


  Also, is there a pure-go ffmpeg alternative ? not wrappers

  


• avcodec/bink : Fix off by 1 error in ref end

  11 janvier 2023, par Michael Niedermayer

  avcodec/bink : Fix off by 1 error in ref end
  Fixes : out of array access
  
  Fixes : 48567/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_BINK_fuzzer-6657932926517248
  Alterantivly to this it is possibly to allocate a bigger array
  Note : oss-fuzz assigned this issue to a unrelated theora bug so the bug number matches that
  Found-by : continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
  
  Signed-off-by : Michael Niedermayer <michael@niedermayer.cc>
  

  • [DH] libavcodec/bink.c