Sur d’autres sites (9709)

• Can't upload folder with large amount of files to google storage. I using "@ffmpeg-installer/ffmpeg" and @google-cloud/storage

  20 juillet 2022, par Dmytro

  I upload file to google storage using "@ffmpeg-installer/ffmpeg" and @google-cloud/storage in my node.js App.
Step 1. file uploading to fs is in child processes - one process for each type of resolution (totaly six).
step 2. encription (converting to stream)
step 3. upload to google storage

  


  I use "Upload a directory to a bucket" in order to send the video from the client to the Google Cloud Storage bucket.

  


  This way is working fine only with small video.

  


  for example when I upload video with duration one hour it split on chunk and totally I get more three thousands files. But the problem occurs when there are more than 1500 files

  


  So actually i upload folder with large amount of files, but not all of this files are uploaded to cloud.

  


  maybe someone had the similar problem and helps fix it.

  


  


  


  const uploadFolder = async (bucketName, directoryPath, socketInstance) => {
    try {
      let dirCtr = 1;
      let itemCtr = 0;
      const fileList = [];

      const onComplete = async () => {
        const folderName = nanoid(46);

        await Promise.all(
          fileList.map(filePath => {
            const fileName = path.relative(directoryPath, filePath);
            const destination = `${ folderName }/${ fileName }`;

            return storage
              .bucket(bucketName)
              .upload(filePath, { destination })
              .then(
                uploadResp => ({ fileName: destination, status: uploadResp[0] }),
                err => ({ fileName: destination, response: err })
              );
          })
        );

        if (socketInstance) socketInstance.emit('uploadProgress', {
          message: `Added files to Google bucket`,
          last: false,
          part: false
        });

        return folderName;
      };

      const getFiles = async directory => {
        const items = await fs.readdir(directory);
        dirCtr--;
        itemCtr += items.length;
        for(const item of items) {
          const fullPath = path.join(directory, item);
          const stat = await fs.stat(fullPath);
          itemCtr--;
          if (stat.isFile()) {
            fileList.push(fullPath);
          } else if (stat.isDirectory()) {
            dirCtr++;
            await getFiles(fullPath);
          }
        }
      }

      await getFiles(directoryPath);

      return onComplete();
    } catch (e) {
      log.error(e.message);
      throw new Error('Can\'t store folder.');
    }
  };

  


  


  



• Redirect a livestream captured by FFmpeg to Azure Blob Storage in Azure Functions

  5 octobre 2022, par StupidUser3186

  The Goal

  


  I want to make a POC that allows me to upload a part of a RTMP stream to azure blob storage, using ffmpeg. In my case this will be done with Azure functions as a QueueTrigger background job.
  
Currently I manually close ffmpeg (pressing q in terminal) so the stream ends and can be fully written to the blob storage.

  


  I use Emulated Storage with Azurite and locally hosted Azure functions

  


  Progress

  


  I found enough info about how to make ffmpeg work with C#

  


  var process = new ProcessStartInfo();
process.FileName = "ffmpeg";
process.RedirectStandardOutput = true;
process.UseShellExecute = false;
process.Arguments = "-i rtmp://10.10.10.4/live/test -loglevel panic -f flv pipe:1";


  


  This should start ffmpeg with no logs -loglevel panic, correct format -f flvand pipe it to standard output pipe:1.

  


  Then I access the blob container and create a new file :

  


  uniqueFileName = string.Format(@"{0}" + ".flv", Guid.NewGuid());
blob = blobClient.GetBlobClient(uniqueFileName);


  


  Current problem

  


  Uploading to the blob storage. It seems there is no method that works.

  


  Attempt 1 : Upload Async

  


  I would start the process and tell the blobclient to upload whatever is on the stream.

  


  var runningProcess = Process.Start(process);
await blob.UploadAsync(runningProcess.StandardOutput.BaseStream, true);


  


  This results in the following error when accessing the StorageExplorer :
  
The first argument must be of type string or an instance of Buffer, ArrayBuffer, or Array or an Array-like Object. Received undefined
  
This also breaks the blob container, it must be deleted and another container must be made.

  


  Attempt 2 : OpenWriteAsync

  


  I would open the Blob's stream and write to it as the ffmpeg stream went on.

  


  using (var stream = await blob.OpenWriteAsync(true))
{
    log.LogInformation("Starting FFmpeg");
    var runningProcess = Process.Start(process);
    log.LogInformation("Started FFMPEG");


    while ((bytesRead = runningProcess.StandardOutput.BaseStream.Read(buffer)) != 0)
    {
        stream.Write(buffer, 0, bytesRead);
    }
    
}


  


  The same error occurred, though I seem closer to actually writing content to the blob as the live stream progresses.

  


  At this point I wonder if it is possible to do what I intend.
I looked at multiple other threads that worked with files, but all are using completed files, not stream that are still being written to.

  


• fftools/ffmpeg_mux_init : stop using OptionsContext as storage in copy_metadata()

  18 octobre 2022, par Anton Khirnov

  fftools/ffmpeg_mux_init : stop using OptionsContext as storage in copy_metadata()
  It should be input-only to this code. Will allow making it const in
  
  future commits.
  

  • [DH] fftools/ffmpeg.h
  • [DH] fftools/ffmpeg_mux_init.c