Newest 'ffmpeg' Questions - Stack Overflow

http://stackoverflow.com/questions/tagged/ffmpeg

Les articles publiés sur le site

  • Promise chaining and resolution

    23 janvier 2019, par Shourya Sharma

    I am trying to convert videos one by one with the help of promises. i am using ffmpeg for conversion and multer for uploading multiple files.

    multer uploads multiple files at once after which i have to chain the conversions one by one. as of now it just converts the 1st file.

    i thought chaining of promises ..like in an array should work but i am confused if i can define new promises in an array as ffmpeg also returns a promise

    My router:

    const router = require('express').Router();
const multer = require('multer');
const ffmpeg = require('ffmpeg');

let str;
const storage = multer.diskStorage({
  destination: (req, file, cb) => {
    cb(null, './uploads');
  },
  filename: (req, file, cb) => {
    str = file.originalname.replace(/\.[^/.]+$/, "");
    str = str.replace(/[^a-z0-9+]+/gi, '_') + '.' + file.originalname.replace(/^.*\./, '');
    cb(null, str);
  }
});

const upload = multer({ storage: storage }).array('files', 12);

router.post('/upload', (req, res, next) => {
  // req.files is an array of files
  // req.body will contain the text fields, if there were any
  function uploadFile() {
    return new Promise((resolve, reject) => {
      upload(req, res, (err) => {
        if (err) {
          res.send(err) // Pass errors to Express.
          reject(`Error: Something went wrong!`);
        } else if (req.files == undefined) {
          res.send(`No File selected.`);
          resolve();
        } else if (!err && req.files.length > 0) {
          res.status(201).send(`${req.files.length} File(s): ${req.files} uploaded successfully.`);
          console.log('uploaded');
          resolve();
        }
      });
    });
  }

  uploadFile().then(() => {
    try {
      var process = new ffmpeg('./uploads/' + str);
      process.then(function (video) {
        console.log('The video is ready to be processed');
        video.addCommand('-hide_banner', '');
        video.addCommand('-y', '');
        video.addCommand('-c:a', 'aac');
        video.addCommand('-ar', '48000');
        video.addCommand('-c:v', 'h264');
        video.addCommand('-profile:v', 'main');
        video.addCommand('-crf', '20');
        video.addCommand('-sc_threshold', '0');
        video.addCommand('-g', '50');
        video.addCommand('-keyint_min', '50');
        video.addCommand('-hls_time', '4');
        video.addCommand('-hls_playlist_type', 'vod');
        video.addCommand('-vf', 'scale=-2:720');
        video.addCommand('-b:v', '1400k');
        video.addCommand('-maxrate', '1498k');
        video.addCommand('-bufsize', '2100k');
        video.addCommand('-b:a', '128k');
        video.save('./converted/' + str, function (error, file) {
          if (!error)
            console.log('Video file: ' + file);
        });
        }, function (err) {
          console.log('Error: ' + err);
        });
    } catch (e) {
      console.log(e.code);
      console.log(e.msg);
    }
  }).catch((err) => {
    console.log(Error, err);
  });
});

module.exports = router;

  • Non-monotonous DTS when concatenate slowed down video

    23 janvier 2019, par whynotbealion

    Copy from Video StackExchange

    I try to concatenate multiple videos to achieve the following:

    All my videos are in mp4 format. One video is first slowed down, then mixed with audio Videos are concatenated in the following order: Normal video, normal video, slowed-down video (output.mp4), normal video ... The video is slowed down with the following command:

    ffmpeg -y -i raw.mp4 -filter:v 'setpts=1.3333*PTS' silent.mp4

    Audio is added with the following command:

    ffmpeg -y -i silent.mp4 -i audio.mp3 -map 0:0 -map 1:0 output.mp4

    When I concatenate using the Concat demuxer (https://trac.ffmpeg.org/wiki/Concatenate) the audio of the output is not in sync with the video. During the concatenation I get the "Non-monotonous DTS" error.

    I tried the following

    Following this suggestion (https://video.stackexchange.com/a/22941/24238) after I created my slowed down and mixed video (output.mp4) I convert all mp4 videos to mts format. After this I concatenate and with

    ffmpeg -i fullvideo.mts fullvideo.mp4

    I convert the resulting video back to mp4 format. This video plays just fine, but during the convert again I get the following error "Non-monotonous DTS in output stream 0:1; previous: 191803, current: 187633; changing to 191804. This may result in incorrect timestamps in the output file."

    My question

    Even regardless the error the video plays just fine, even in the browser. Does this solution still have a disadvantage or threat? Is there a better solution where I don't get any DTS error? I am thankful for any help as this is all new for me.

  • HLS Event Stream from Azure Blob Storage

    23 janvier 2019, par nemesis_567

    Alright, so I encode videos of around 30 seconds with ffmpeg, the following flags:

    ffmpeg.exe -i \"{filename}\"  -f hls -hls_segment_type fmp4 -hls_flags single_file -hls_time 5 -g 1 \"{outname}.m3u8\"

    This generates a single playlist with an m4s and m3u8 file:

    #EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:5
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-MAP:URI="1e02982a-c455-40f5-8229-1033a640c91c-0.mp4.m4s",BYTERANGE="818@0"
#EXTINF:4.781911,
#EXT-X-BYTERANGE:1398720@5953584
1e02982a-c455-40f5-8229-1033a640c91c-0.mp4.m4s
#EXTINF:5.063200,
#EXT-X-BYTERANGE:1490276@7352304
1e02982a-c455-40f5-8229-1033a640c91c-0.mp4.m4s
#EXTINF:5.063200,
#EXT-X-BYTERANGE:1484824@8842580
1e02982a-c455-40f5-8229-1033a640c91c-0.mp4.m4s
#EXTINF:5.063200,
#EXT-X-BYTERANGE:1468656@10327404
1e02982a-c455-40f5-8229-1033a640c91c-0.mp4.m4s
#EXTINF:1.969022,
#EXT-X-BYTERANGE:576408@11796060
1e02982a-c455-40f5-8229-1033a640c91c-0.mp4.m4s
#EXT-X-ENDLIST

    I take this playlist and append several others that are just the same as they come adding a DIscontinuity tag and between each of them and removing unneeded tags. While doing this I replace the filenames with a working SAS URL for the actual files.

    Which results in files of the following type:

    #EXTM3U
#EXT-X-PLAYLIST-TYPE:EVENT
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:10
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MAP:URI="https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/1-frag.m4s?sv=2018-03-28&sr=b&sig=el%2F529BGb4Xjeetiv9kxaHAxGYC%2BwogSYgvCivq1tlc%3D&st=2019-01-10T05%3A09%3A13Z&se=2021-01-10T05%3A14%3A13Z&sp=rw",BYTERANGE="818@0"
#EXTINF:4.654771,
#EXT-X-BYTERANGE:1474761@4854677
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/1-frag.m4s?sv=2018-03-28&sr=b&sig=el%2F529BGb4Xjeetiv9kxaHAxGYC%2BwogSYgvCivq1tlc%3D&st=2019-01-10T05%3A09%3A13Z&se=2021-01-10T05%3A14%3A13Z&sp=rw
#EXTINF:5.430566,
#EXT-X-BYTERANGE:1716825@6329438
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/1-frag.m4s?sv=2018-03-28&sr=b&sig=el%2F529BGb4Xjeetiv9kxaHAxGYC%2BwogSYgvCivq1tlc%3D&st=2019-01-10T05%3A09%3A13Z&se=2021-01-10T05%3A14%3A13Z&sp=rw
#EXTINF:4.654771,
#EXT-X-BYTERANGE:1479725@8046263
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/1-frag.m4s?sv=2018-03-28&sr=b&sig=el%2F529BGb4Xjeetiv9kxaHAxGYC%2BwogSYgvCivq1tlc%3D&st=2019-01-10T05%3A09%3A13Z&se=2021-01-10T05%3A14%3A13Z&sp=rw
#EXTINF:5.430566,
#EXT-X-BYTERANGE:1710125@9525988
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/1-frag.m4s?sv=2018-03-28&sr=b&sig=el%2F529BGb4Xjeetiv9kxaHAxGYC%2BwogSYgvCivq1tlc%3D&st=2019-01-10T05%3A09%3A13Z&se=2021-01-10T05%3A14%3A13Z&sp=rw
#EXTINF:3.103181,
#EXT-X-BYTERANGE:200@11236113
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/1-frag.m4s?sv=2018-03-28&sr=b&sig=el%2F529BGb4Xjeetiv9kxaHAxGYC%2BwogSYgvCivq1tlc%3D&st=2019-01-10T05%3A09%3A13Z&se=2021-01-10T05%3A14%3A13Z&sp=rw
#EXT-X-DISCONTINUITY
#EXT-X-MAP:URI="https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/2-frag.m4s?sv=2018-03-28&sr=b&sig=s3s3R1sjE6lZRgfmaYdOkOGCr7BYkOrygBy%2F90crwvM%3D&st=2019-01-10T05%3A09%3A53Z&se=2021-01-10T05%3A14%3A53Z&sp=rw",BYTERANGE="819@0"
#EXTINF:4.897959,
#EXT-X-BYTERANGE:1506220@6184008
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/2-frag.m4s?sv=2018-03-28&sr=b&sig=s3s3R1sjE6lZRgfmaYdOkOGCr7BYkOrygBy%2F90crwvM%3D&st=2019-01-10T05%3A09%3A53Z&se=2021-01-10T05%3A14%3A53Z&sp=rw
#EXTINF:4.897959,
#EXT-X-BYTERANGE:1496289@7690228
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/2-frag.m4s?sv=2018-03-28&sr=b&sig=s3s3R1sjE6lZRgfmaYdOkOGCr7BYkOrygBy%2F90crwvM%3D&st=2019-01-10T05%3A09%3A53Z&se=2021-01-10T05%3A14%3A53Z&sp=rw
#EXTINF:4.897959,
#EXT-X-BYTERANGE:1516854@9186517
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/2-frag.m4s?sv=2018-03-28&sr=b&sig=s3s3R1sjE6lZRgfmaYdOkOGCr7BYkOrygBy%2F90crwvM%3D&st=2019-01-10T05%3A09%3A53Z&se=2021-01-10T05%3A14%3A53Z&sp=rw
#EXTINF:4.897959,
#EXT-X-BYTERANGE:1492617@10703371
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/2-frag.m4s?sv=2018-03-28&sr=b&sig=s3s3R1sjE6lZRgfmaYdOkOGCr7BYkOrygBy%2F90crwvM%3D&st=2019-01-10T05%3A09%3A53Z&se=2021-01-10T05%3A14%3A53Z&sp=rw
#EXTINF:0.816327,
#EXT-X-BYTERANGE:219@12195988
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/2-frag.m4s?sv=2018-03-28&sr=b&sig=s3s3R1sjE6lZRgfmaYdOkOGCr7BYkOrygBy%2F90crwvM%3D&st=2019-01-10T05%3A09%3A53Z&se=2021-01-10T05%3A14%3A53Z&sp=rw
#EXT-X-DISCONTINUITY
#EXT-X-MAP:URI="https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/3-frag.m4s?sv=2018-03-28&sr=b&sig=O%2B%2FQBnVcyIzn2r3T8S7xq2dic9LKW3q7p2bGSYen%2BbI%3D&st=2019-01-10T05%3A10%3A43Z&se=2021-01-10T05%3A15%3A43Z&sp=rw",BYTERANGE="820@0"
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1352781@6631869
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/3-frag.m4s?sv=2018-03-28&sr=b&sig=O%2B%2FQBnVcyIzn2r3T8S7xq2dic9LKW3q7p2bGSYen%2BbI%3D&st=2019-01-10T05%3A10%3A43Z&se=2021-01-10T05%3A15%3A43Z&sp=rw
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1357320@7984650
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/3-frag.m4s?sv=2018-03-28&sr=b&sig=O%2B%2FQBnVcyIzn2r3T8S7xq2dic9LKW3q7p2bGSYen%2BbI%3D&st=2019-01-10T05%3A10%3A43Z&se=2021-01-10T05%3A15%3A43Z&sp=rw
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1360645@9341970
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/3-frag.m4s?sv=2018-03-28&sr=b&sig=O%2B%2FQBnVcyIzn2r3T8S7xq2dic9LKW3q7p2bGSYen%2BbI%3D&st=2019-01-10T05%3A10%3A43Z&se=2021-01-10T05%3A15%3A43Z&sp=rw
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1334946@10702615
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/3-frag.m4s?sv=2018-03-28&sr=b&sig=O%2B%2FQBnVcyIzn2r3T8S7xq2dic9LKW3q7p2bGSYen%2BbI%3D&st=2019-01-10T05%3A10%3A43Z&se=2021-01-10T05%3A15%3A43Z&sp=rw
#EXTINF:5.000000,
#EXT-X-BYTERANGE:238@12037561
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/3-frag.m4s?sv=2018-03-28&sr=b&sig=O%2B%2FQBnVcyIzn2r3T8S7xq2dic9LKW3q7p2bGSYen%2BbI%3D&st=2019-01-10T05%3A10%3A43Z&se=2021-01-10T05%3A15%3A43Z&sp=rw
#EXT-X-DISCONTINUITY
#EXT-X-MAP:URI="https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/4-frag.m4s?sv=2018-03-28&sr=b&sig=jPsSq8%2FfNzfjUx5bauJs7mkLu8w%2FYFKLaeLRBu8lPzg%3D&st=2019-01-10T05%3A11%3A25Z&se=2021-01-10T05%3A16%3A25Z&sp=rw",BYTERANGE="820@0"
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1359259@6647772
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/4-frag.m4s?sv=2018-03-28&sr=b&sig=jPsSq8%2FfNzfjUx5bauJs7mkLu8w%2FYFKLaeLRBu8lPzg%3D&st=2019-01-10T05%3A11%3A25Z&se=2021-01-10T05%3A16%3A25Z&sp=rw
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1353269@8007031
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/4-frag.m4s?sv=2018-03-28&sr=b&sig=jPsSq8%2FfNzfjUx5bauJs7mkLu8w%2FYFKLaeLRBu8lPzg%3D&st=2019-01-10T05%3A11%3A25Z&se=2021-01-10T05%3A16%3A25Z&sp=rw
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1359724@9360300
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/4-frag.m4s?sv=2018-03-28&sr=b&sig=jPsSq8%2FfNzfjUx5bauJs7mkLu8w%2FYFKLaeLRBu8lPzg%3D&st=2019-01-10T05%3A11%3A25Z&se=2021-01-10T05%3A16%3A25Z&sp=rw
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1339437@10720024
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/4-frag.m4s?sv=2018-03-28&sr=b&sig=jPsSq8%2FfNzfjUx5bauJs7mkLu8w%2FYFKLaeLRBu8lPzg%3D&st=2019-01-10T05%3A11%3A25Z&se=2021-01-10T05%3A16%3A25Z&sp=rw
#EXTINF:5.000000,
#EXT-X-BYTERANGE:238@12059461
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/4-frag.m4s?sv=2018-03-28&sr=b&sig=jPsSq8%2FfNzfjUx5bauJs7mkLu8w%2FYFKLaeLRBu8lPzg%3D&st=2019-01-10T05%3A11%3A25Z&se=2021-01-10T05%3A16%3A25Z&sp=rw
#EXT-X-DISCONTINUITY
#EXT-X-MAP:URI="https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/4-frag.m4s?sv=2018-03-28&sr=b&sig=yfgkYvCLFjt384tsDZZ2ahVfPI4XxoCyyWepmux4rvc%3D&st=2019-01-10T05%3A12%3A12Z&se=2021-01-10T05%3A17%3A12Z&sp=rw",BYTERANGE="820@0"
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1371742@6775319
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/4-frag.m4s?sv=2018-03-28&sr=b&sig=yfgkYvCLFjt384tsDZZ2ahVfPI4XxoCyyWepmux4rvc%3D&st=2019-01-10T05%3A12%3A12Z&se=2021-01-10T05%3A17%3A12Z&sp=rw
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1389742@8147061
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/4-frag.m4s?sv=2018-03-28&sr=b&sig=yfgkYvCLFjt384tsDZZ2ahVfPI4XxoCyyWepmux4rvc%3D&st=2019-01-10T05%3A12%3A12Z&se=2021-01-10T05%3A17%3A12Z&sp=rw
#EXTINF:5.000000,
#EXT-X-BYTERANGE:1377900@9536803
https://vsatstreams.blob.core.windows.net/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/75888d6c-41fb-4cb0-9573-0e2ab7a6dc4c/4-frag.m4s?sv=2018-03-28&sr=b&sig=yfgkYvCLFjt384tsDZZ2ahVfPI4XxoCyyWepmux4rvc%3D&st=2019-01-10T05%3A12%3A12Z&se=2021-01-10T05%3A17%3A12Z&sp=rw
#EXTINF:5.000000,

    Link for file is here

    This stream does not play in chrome with video.js, and only plays in bitmovin client, but it constantly fails with buffer append/stalled errors. In firefox it plays, but without much reliability.

    So what am I doing wrong in this process? I've checked the specification for HLS several times, can't see what I'm doing wrong, or what the player expects.

    Please check hls.js player, with the following URL which uses the stream above to see what I'm talking about. HLS.JS Demo Player With My Playlist

  • Dumping Subtitles From AVPicture into ppm Format File

    23 janvier 2019, par Rishabh Garg

    In FFMPEG sofftware, AVPicture is used to store image data using data pointer and linesizes. It means all subtitles are stored in the form of picture inside ffmpeg . Now I want to dump subtitles from AVPicture in .ppm format file.

    I have written this code to Dump subtitles with the help of data in AVPicture. May be my red,green,blue pixel data may be wrong in the loop.

    w = ctx->sub->rects[0]->w;
h = ctx->sub->rects[0]->h;
line_size = ctx->sub->rects[0]->linesize[0];
int m,n;
FILE *fp = fopen("first.ppm", "wb");
(void) fprintf(fp, "P3\n%d %d\n255\n", w, h);
for (m = 0; m < h; ++m)
{
    for (n = 0; n < line_size; ++n)
    {
            static int r,g,b,a,z;
            uint32_t *pallet = ctx->sub->rects[0]->data[1];
            z = ctx->sub->rects[0]->data[0][n];
            b = pallet[z]&0xFF;
            g = (pallet[z]>>8) & 0xFF;
            r = (pallet[z]>>16)& 0xFF;
            a = (pallet[z]>>24)& 0xFF; 
            fprintf(fp,"%d %d %d %d",b,g,r,a);
    }
    fprintf(fp,"\n");
    fflush(fp); 
}
(void) fclose(fp);

    The program is not dumping subtitles. Only 0 is coming in r,g,b,a by which subtitles are not dumping .

  • ffmpeg avcodec_send_packet/avcodec_receive_frame memory leak

    23 janvier 2019, par G Hamlin

    I'm attempting to decode frames, but memory usage grows with every frame (more specifically, with every call to avcodec_send_packet) until finally the code crashes with a bad_alloc. Here's the basic decode loop:

    int rfret = 0;
while((rfret = av_read_frame(inctx.get(), &packet)) >= 0){
    if (packet.stream_index == vstrm_idx) {

        //std::cout << "Sending Packet" << std::endl;
        int ret = avcodec_send_packet(ctx.get(), &packet);
        if (ret < 0 || ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
            std::cout << "avcodec_send_packet: " << ret << std::endl;
            break;
        }

        while (ret  >= 0) {
            //std::cout << "Receiving Frame" << std::endl;
            ret = avcodec_receive_frame(ctx.get(), fr);
            if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
                //std::cout << "avcodec_receive_frame: " << ret << std::endl;
                av_frame_unref(fr);
                // av_frame_free(&fr);
                break;
            }

            std::cout << "frame: " << ctx->frame_number << std::endl;

            // eventually do something with the frame here...

            av_frame_unref(fr);
            // av_frame_free(&fr);
        }
    }
    else {
        //std::cout << "Not Video" << std::endl;
    }
    av_packet_unref(&packet);
}

    Memory usage/leakage seems to scale with the resolution of the video I'm decoding. For example, for a 3840x2160 resolution video, the memory usage in windows task manager consistently jumps up by about 8mb (1 byte per pixel??) for each received frame. Do I need to do something besides call av_frame_unref to release the memory?

    (more) complete code below

    void AVFormatContextDeleter(AVFormatContext* ptr)
{
    if (ptr) {
        avformat_close_input(&ptr);
    }
}

void AVCodecContextDeleter(AVCodecContext* ptr)
{
    if (ptr) {
        avcodec_free_context(&ptr);
    }
}

typedef std::unique_ptr AVFormatContextPtr;
typedef std::unique_ptr AVCodecContextPtr;

AVCodecContextPtr createAvCodecContext(AVCodec *vcodec)
{
    AVCodecContextPtr ctx(avcodec_alloc_context3(vcodec), AVCodecContextDeleter);
    return ctx;
}

AVFormatContextPtr createFormatContext(const std::string& filename)
{
    AVFormatContext* inctxPtr = nullptr;
    int ret = avformat_open_input(&inctxPtr, filename.c_str(), nullptr, nullptr);
    //    int ret = avformat_open_input(&inctx, "D:/Videos/test.mp4", nullptr, nullptr);
    if (ret != 0) {
        inctxPtr = nullptr;
    }

    return AVFormatContextPtr(inctxPtr, AVFormatContextDeleter);
}

int testDecode()
{
    // open input file context
    AVFormatContextPtr inctx = createFormatContext("D:/Videos/Matt Chapman Hi Greg.MOV");

    if (!inctx) {
        // std::cerr << "fail to avforamt_open_input(\"" << infile << "\"): ret=" << ret;
        return 1;
    }

    // retrieve input stream information
    int ret = avformat_find_stream_info(inctx.get(), nullptr);
    if (ret < 0) {
        //std::cerr << "fail to avformat_find_stream_info: ret=" << ret;
        return 2;
    }

    // find primary video stream
    AVCodec* vcodec = nullptr;
    const int vstrm_idx = av_find_best_stream(inctx.get(), AVMEDIA_TYPE_VIDEO, -1, -1, &vcodec, 0);
    if (vstrm_idx < 0) {
        //std::cerr << "fail to av_find_best_stream: vstrm_idx=" << vstrm_idx;
        return 3;
    }

    AVCodecParameters* origin_par = inctx->streams[vstrm_idx]->codecpar;
    if (vcodec == nullptr) {  // is this even necessary?
        vcodec = avcodec_find_decoder(origin_par->codec_id);
        if (!vcodec) {
            // Can't find decoder
            return 4;
        }
    }

    AVCodecContextPtr ctx = createAvCodecContext(vcodec);
    if (!ctx) {
        return 5;
    }

    ret = avcodec_parameters_to_context(ctx.get(), origin_par);
    if (ret) {
        return 6;
    }

    ret = avcodec_open2(ctx.get(), vcodec, nullptr);
    if (ret < 0) {
        return 7;
    }

    //print input video stream informataion
    std::cout
            //<< "infile: " << infile << "\n"
            << "format: " << inctx->iformat->name << "\n"
            << "vcodec: " << vcodec->name << "\n"
            << "size:   " << origin_par->width << 'x' << origin_par->height << "\n"
            << "fps:    " << av_q2d(ctx->framerate) << " [fps]\n"
            << "length: " << av_rescale_q(inctx->duration, ctx->time_base, {1,1000}) / 1000. << " [sec]\n"
            << "pixfmt: " << av_get_pix_fmt_name(ctx->pix_fmt) << "\n"
            << "frame:  " << inctx->streams[vstrm_idx]->nb_frames << "\n"
            << std::flush;


    AVPacket packet;

    av_init_packet(&packet);
    packet.data = nullptr;
    packet.size = 0;

    AVFrame *fr = av_frame_alloc();
    if (!fr) {
        return 8;
    }

    int rfret = 0;
    while((rfret = av_read_frame(inctx.get(), &packet)) >= 0){
        if (packet.stream_index == vstrm_idx) {

            //std::cout << "Sending Packet" << std::endl;
            int ret = avcodec_send_packet(ctx.get(), &packet);
            if (ret < 0 || ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
                std::cout << "avcodec_send_packet: " << ret << std::endl;
                break;
            }

            while (ret  >= 0) {
                //std::cout << "Receiving Frame" << std::endl;
                ret = avcodec_receive_frame(ctx.get(), fr);
                if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
                    //std::cout << "avcodec_receive_frame: " << ret << std::endl;
                    av_frame_unref(fr);
                    // av_frame_free(&fr);
                    break;
                }

                std::cout << "frame: " << ctx->frame_number << std::endl;

                // do something with the frame here...

                av_frame_unref(fr);
                // av_frame_free(&fr);
            }
        }
        else {
            //std::cout << "Not Video" << std::endl;
        }
        av_packet_unref(&packet);
    }

    std::cout << "RFRET = " << rfret << std::endl;

    return 0;
}

    Update 1: (1/21/2019) Compiling on a different machine and running with different video files I am not seeing the memory usage growing without bound. I'll try to narrow down where the difference lies (compiler?, ffmpeg version?, or video encoding?)

    Update 2: (1/21/2019) Ok, it looks like there is some interaction occurring between ffmpeg and Qt's QCamera. In my application, I'm using Qt to manage the webcam, but decided to use ffmpeg libraries to handle decoding/encoding since Qt doesn't have as comprehensive support for different codecs. If I have the camera turned on (through Qt), ffmpeg decoding memory consumption grows without bound. If the camera is off, ffmpeg behaves fine. I've tried this both with a physical camera (Logitech C920) and with a virtual camera using OBS-Virtualcam, with the same result. So far I'm baffled as to how the two systems are interacting...

Navigation

Sites Web