Recherche avancée

Sur d’autres sites (8766)

• Firebase Function using ffmpeg successful with emulator, out of memory when deployed

  25 septembre 2024, par flyingL123

  I need some help. I have a .mov file in Firebase Storage. The file is 25 seconds long and 106 MB. I wrote a callable Firebase function that uses ffmpeg to convert the file to a .mp4 file and save it to Firebase Storage. When I test the function using the Functions emulator, it works without issue. The function returns successfully and I see the converted file appear in storage. The converted video is about 6 MB and plays correctly when dowloaded.

  


  When I deploy this function and run it in production on the exact same video file, the function fails with :

  


  


  'Memory limit of 256 MiB exceeded with 407 MiB used. Consider
increasing the memory limit, see
https://cloud.google.com/functions/docs/configuring/memory'

  


  


  As a test, I edited the function and changed its allocated memory to 1 GiB. Then I test the function again in production. Now I receive the same error :

  


  


  'Memory limit of 1024 MiB exceeded with 1029 MiB used. Consider
increasing the memory limit, see
https://cloud.google.com/functions/docs/configuring/memory'

  


  


  This is my function code :

  


  const {initializeApp} = require("firebase-admin/app");
const {onCall} = require("firebase-functions/v2/https");
const { getStorage, getDownloadURL } = require('firebase-admin/storage');

initializeApp();

exports.convertVideo = onCall((request) => {
  const ffmpegPath = require('@ffmpeg-installer/ffmpeg').path;
  const ffmpeg = require('fluent-ffmpeg');
  ffmpeg.setFfmpegPath(ffmpegPath);
  const originalLocation = request.data.originalLocation;
  const convertedLocation = request.data.convertedLocation;
  const originalVideoFile = getStorage().bucket().file(originalLocation);
  const newVideoFile = getStorage().bucket().file(convertedLocation);

  return new Promise(async (resolve, reject) => {
    await originalVideoFile.download({destination: '/tmp/original'}).catch(console.error);
    
    ffmpeg('/tmp/original')
      .addOutputOptions('-movflags +frag_keyframe+separate_moof+omit_tfhd_offset+empty_moov')
      .format('mp4')
      .on('error', (err) => {
        console.log(err);
      })
      .pipe(newVideoFile.createWriteStream())
      .on('error', (err) => {
        console.log(err);
      })
      .on('close', async () => {
        fs.unlink('/tmp/original', (err) => {
          if (err) throw err;
        });
        const convertedUrl = await getDownloadURL(newVideoFile);
        resolve([convertedLocation, convertedUrl]);
      });
  });
});


  


  I am sending a test request to the Function emulator using curl :

  


  curl -d '{"data": {"originalLocation": "customer_videos/original_video.mov", "convertedLocation": "customer_videos/converted/original_video.mp4"}}' -H "Content-Type: application/json" http://127.0.0.1:5001/foo/bar/convertVideo


  


  This works correctly. I send the same request to the deployed function, and receive the out of memory error.

  


  curl -d '{"data": {"originalLocation": "customer_videos/original_video.mov", "convertedLocation": "customer_videos/converted/original_video.mp4"}}' -H "Content-Type: application/json" https://convertvideo-foobarbaz-uc.a.run.ap


  


  Can somebody please help me understand why this is happening ? I didn't think I was doing anything too memory intensive, especially since it works correctly using the emulator.

  


• Revision 4831 : on ajoute ts et mts comme documents vidéos avec des infos supp donc ...

  28 janvier 2011, par kent1 — Log

  on ajoute ts et mts comme documents vidéos avec des infos supp donc Petits ajouts également pour mieux styler les blocs de docs

• Convert ogg byte array to wav byte array Python

  2 février 2023, par Ramish Rasool

  I want to convert ogg byte array/bytes with Opus codec to wav byte array/bytes without saving to disk. I have downloaded audio from telegram api and it is in byte array format with .ogg extension. I do not want to save it to filesystem to eliminate filesystem io latencey.

  


  Currently what I am doing is after saving the audio file in .ogg format using code the below code using telegram api for reference https://docs.python-telegram-bot.org/en/stable/telegram.file.html#telegram.File.download_to_drive

  


  # listen for audio messages
async def audio(update, context):
    newFile = await context.bot.get_file(update.message.voice.file_id)
    await newFile.download_to_drive(output_path)


  


  I am using the code

  


  subprocess.call(["ffmpeg", "-i", output_path, output_path.replace(".ogg", ".wav"), '-y'], stderr=subprocess.DEVNULL, stdout=subprocess.DEVNULL)


  


  to convert ogg file to wav file. But this is not what I want.

  


  I want the code

  


  async def audio(update, context):
    newFile = await context.bot.get_file(update.message.voice.file_id)
    byte_array = await newFile.download_as_bytearray()


  


  to get byte_array and now I want this byte_array to be converted to wav without saving to disk and without using ffmpeg. Let me know in comments if something is unclear. Thanks !

  


  Note : I have setted up a telegram bot at the backend which listens for audios sent to private chat which I do manually for testing purposes.