Recherche avancée

Sur d’autres sites (5785)

• Limit number of Start-Process running in poweshell

  7 mars 2018, par Ahhhhhhhhhhhhhdfgbv

  I have tried to limit the number of Start-Process running from a Powershell, but I can’t seem to get it to work.

  I tried to follow this process : https://exchange12rocks.org/2015/05/24/how-to-limit-a-number-of-powershell-jobs-running-simultaneously/ and Run N parallel jobs in powershell

  But these are for Jobs not Processes, and I would like to remove the -Wait from the Start-Process

  My concern with the script is that if there are 1000 audio files in the folder, then FFMpeg would crash the system.

  ---

  # get the folder for conversion
  
  function mbAudioConvert {
  
      [Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms") | Out-Null
  
      [System.Windows.Forms.Application]::EnableVisualStyles()
  
  
  
      $fileBrowser = New-Object System.Windows.Forms.FolderBrowserDialog
  
      $fileBrowser.SelectedPath = "B:\"
  
      $fileBrowser.ShowNewFolderButton = $false
  
      $fileBrowser.Description = "Select the folder with the audio which you wish to convert to Avid DNxHD 120 25P 48kHz"
  
  
  
      $mbLoop     = $true
  
      $mbCount    = 0001
  
      $mbMaxJob   = 4
  
  
  
      while( $mbLoop ) {
  
          if( $fileBrowser.ShowDialog() -eq "OK" ) {
  
              $mbLoop     = $false
  
  
  
  
  
              $mbImage    = ( Get-Item -Path "C:\Users\user\Desktop\lib\AudioOnly.jpg" )
  
              $mbff32     = ( Get-Item -Path "C:\Users\user\Desktop\lib\ffmpeg32.exe" )
  
              $mbff64     = ( Get-Item -Path "C:\Users\user\Desktop\lib\ffmpeg64.exe" )
  
  
  
              $mbFolder   = $fileBrowser.SelectedPath
  
              $mbItemInc  = ( ls $mbFolder\* -Include *.mp3, *.MP3, *.wav*, *.WAV*, *.ogg, *.OGG, *.wma, *.WMA, *.flac, *.FLAC, *.m4a, *.M4a )
  
              $mbProgress = ( Get-ChildItem -Path $mbItemInc )
  
  
  
              $mbHasRaw   = ( $mbFolder + "\RAW" )
  
  
  
              if( !( Test-Path -Path $mbHasRaw ) ) {
  
                  # force create a RAW folder if it does not exist
  
                  New-Item -ItemType Directory -Force -Path "$mbHasRaw"
  
              }
  
  
  
  
  
              foreach( $mbItem in $mbItemInc ) {
  
  
  
                  $mbCheck    = $false
  
  
  
                  # output the progress
  
                  # Suggestion: You might want to consider updating this after starting the job and do the final update after running ex. Get-Job | Wait-Job to make the progress-bar stay until all processes are finished
  
                  #Write-Progress -Activity "Counting files for conversion" -status "Currently processing: $mbCount" -percentComplete ($mbCount / $mbItemInc.count*100)
  
  
  
                  # limit the run number
  
                  while ($mbCheck -eq $false) {
  
  
  
                      if( (Get-Job -State 'Running').count -lt $mbMaxJob) {
  
  
  
                          $mbScriptBlock = {
  
                              $mbItemName = $using:mbItem.BaseName
  
  
  
                              $mbNewItem  = ( $using:mbFolder + "\RAW\" + $mbItemName + ".mov" )
  
                              $mbArgs     = " -loop 1 -i $using:mbImage -i $using:mbItem -shortest -c:v dnxhd -b:v 120M -s 1920x1080 -pix_fmt yuv422p -r 25 -c:a pcm_s16le -ar 48k -af loudnorm=I=-12 $mbNewItem"
  
  
  
                              Start-Process -FilePath $using:mbff32 -ArgumentList $mbArgs -NoNewWindow -Wait
  
                          }
  
  
  
                          Start-Job -ScriptBlock $mbScriptBlock
  
  
  
                          #The job-thread doesn't know about $mbCount, better to increment it after starting the job
  
                          $mbCount++
  
                          $mbCheck  = $true          
  
                      }
  
  
  
                  }
  
              }
  
  
  
          } else {
  
  
  
              $mbResponse = [System.Windows.Forms.MessageBox]::Show("You have exited out of the automation process!", "User has cancelled")
  
              if( $mbResponse -eq "OK" ) {
  
                  return
  
              }
  
          }
  
      }
  
  
  
      $fileBrowser.SelectedPath
  
      $fileBrowser.Dispose()
  
  }
  
  
  
  # call to function
  
  mbAudioConvert

• Limit number of Start-Process running in powershell

  4 mai 2018, par Ahhhhhhhhhhhhhdfgbv

  I have tried to limit the number of Start-Process running from a Powershell, but I can’t seem to get it to work.

  I tried to follow this process : https://exchange12rocks.org/2015/05/24/how-to-limit-a-number-of-powershell-jobs-running-simultaneously/ and Run N parallel jobs in powershell

  But these are for Jobs not Processes, and I would like to remove the -Wait from the Start-Process

  My concern with the script is that if there are 1000 audio files in the folder, then FFMpeg would crash the system.

  ---

  # get the folder for conversion
  
  function mbAudioConvert {
  
      [Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms") | Out-Null
  
      [System.Windows.Forms.Application]::EnableVisualStyles()
  
  
  
      $fileBrowser = New-Object System.Windows.Forms.FolderBrowserDialog
  
      $fileBrowser.SelectedPath = "B:\"
  
      $fileBrowser.ShowNewFolderButton = $false
  
      $fileBrowser.Description = "Select the folder with the audio which you wish to convert to Avid DNxHD 120 25P 48kHz"
  
  
  
      $mbLoop     = $true
  
      $mbCount    = 0001
  
      $mbMaxJob   = 4
  
  
  
      while( $mbLoop ) {
  
          if( $fileBrowser.ShowDialog() -eq "OK" ) {
  
              $mbLoop     = $false
  
  
  
  
  
              $mbImage    = ( Get-Item -Path "C:\Users\user\Desktop\lib\AudioOnly.jpg" )
  
              $mbff32     = ( Get-Item -Path "C:\Users\user\Desktop\lib\ffmpeg32.exe" )
  
              $mbff64     = ( Get-Item -Path "C:\Users\user\Desktop\lib\ffmpeg64.exe" )
  
  
  
              $mbFolder   = $fileBrowser.SelectedPath
  
              $mbItemInc  = ( ls $mbFolder\* -Include *.mp3, *.MP3, *.wav*, *.WAV*, *.ogg, *.OGG, *.wma, *.WMA, *.flac, *.FLAC, *.m4a, *.M4a )
  
              $mbProgress = ( Get-ChildItem -Path $mbItemInc )
  
  
  
              $mbHasRaw   = ( $mbFolder + "\RAW" )
  
  
  
              if( !( Test-Path -Path $mbHasRaw ) ) {
  
                  # force create a RAW folder if it does not exist
  
                  New-Item -ItemType Directory -Force -Path "$mbHasRaw"
  
              }
  
  
  
  
  
              foreach( $mbItem in $mbItemInc ) {
  
  
  
                  $mbCheck    = $false
  
  
  
                  # output the progress
  
                  # Suggestion: You might want to consider updating this after starting the job and do the final update after running ex. Get-Job | Wait-Job to make the progress-bar stay until all processes are finished
  
                  #Write-Progress -Activity "Counting files for conversion" -status "Currently processing: $mbCount" -percentComplete ($mbCount / $mbItemInc.count*100)
  
  
  
                  # limit the run number
  
                  while ($mbCheck -eq $false) {
  
  
  
                      if( (Get-Job -State 'Running').count -lt $mbMaxJob) {
  
  
  
                          $mbScriptBlock = {
  
                              $mbItemName = $using:mbItem.BaseName
  
  
  
                              $mbNewItem  = ( $using:mbFolder + "\RAW\" + $mbItemName + ".mov" )
  
                              $mbArgs     = " -loop 1 -i $using:mbImage -i $using:mbItem -shortest -c:v dnxhd -b:v 120M -s 1920x1080 -pix_fmt yuv422p -r 25 -c:a pcm_s16le -ar 48k -af loudnorm=I=-12 $mbNewItem"
  
  
  
                              Start-Process -FilePath $using:mbff32 -ArgumentList $mbArgs -NoNewWindow -Wait
  
                          }
  
  
  
                          Start-Job -ScriptBlock $mbScriptBlock
  
  
  
                          #The job-thread doesn't know about $mbCount, better to increment it after starting the job
  
                          $mbCount++
  
                          $mbCheck  = $true          
  
                      }
  
  
  
                  }
  
              }
  
  
  
          } else {
  
  
  
              $mbResponse = [System.Windows.Forms.MessageBox]::Show("You have exited out of the automation process!", "User has cancelled")
  
              if( $mbResponse -eq "OK" ) {
  
                  return
  
              }
  
          }
  
      }
  
  
  
      $fileBrowser.SelectedPath
  
      $fileBrowser.Dispose()
  
  }
  
  
  
  # call to function
  
  mbAudioConvert

• Tiny node js video convert server, best way to manage the threads

  23 juin 2017, par efirvida

  I am developing a small server to convert videos for the web, I started with a TCP server (maybe it is not the best solution, but it is not the part of my problem right now) to which I send the path of videos to convert and then Add to a queue in a database system, in this case mysql, but I’m thinking of migrating to MongoDB when it works

  var net = require('net'),
  
      Sequelize = require('sequelize'),
  
      async = require('async'),
  
      JsonSocket = require('json-socket'),
  
      ffmpeg = require('fluent-ffmpeg');
  
  
  
  var configDB = require('./config/database.js');
  
  var sequelize = new Sequelize(configDB.url);
  
  
  
  var Videos    = sequelize.import('app/models/video');
  
  Videos.sync();
  
  
  
  var port = 9838;
  
  
  
  var server = net.createServer();
  
  server.listen(port);
  
  server.on('connection', function(socket) {
  
      socket = new JsonSocket(socket);
  
      socket.on('message', function(video) {
  
          console.log(video.video);
  
          db_data = {
  
              'name': video.video,
  
              'converted': false,
  
              'progress': 0,
  
              'status': 'Q',
  
          }
  
          Videos.create(db_data).then(function () {
  
              socket.sendMessage('New Video Added');
  
          }).catch(function () {
  
              console.log('Error adding Video');
  
              socket.sendMessage('Error adding video')
  
          });
  
      });
  
  });

  Once I have a list of videos on the database the process begin with this code :

  // Function to update the video status on the database
  
  var databaseUpdate = function(status, callback) {
  
      name = status.name;
  
      delete status["name"];
  
      Videos.update(status, {
  
          where: {
  
              name: name
  
          }
  
      }).then(function (video) {
  
          if (typeof callback === 'function' && callback) callback();
  
      });
  
  }
  
  
  
  //convert the video:
  
  // If the video status are:
  
  //   - 'Q' for in queue
  
  //   - 'R' for running conversion
  
  //   - 'C' for complete
  
  //   - 'E' for error on the conversion proccess
  
  function ffmpegConvert(video, output, callback)
  
  {
  
      var filename = video.split('/').reverse()[0].split('.')[0],
  
          output = output  + filename + '.mp4';  
  
  
  
      ffmpeg(video)
  
          .videoCodec('libx264')
  
          .audioCodec('libmp3lame')
  
          .on('error', function(err) {
  
              db_data = {
  
                  'name': video,
  
                  'status': 'E',   //<- set status if error ocurre
  
              }
  
              databaseUpdate(db_data)
  
              console.log('An error occurred: ' + err.message);
  
          })
  
          .on('end', function(stdout, stderr) {
  
              db_data = {
  
                  'name': video,
  
                  'converted': true,       //<- set status complete conversion
  
                  'status': 'C',           //<- set status complete conversion
  
              }
  
              databaseUpdate(db_data, function(){convertVideos(1)}) // <- put a next video in the queue to convert
  
          })
  
          .on('progress', function(progress) {
  
              db_data = {
  
                  'name': video,
  
                  'status': 'R',                //<- set status ass running conversion
  
                  'progress': progress.percent, //<- set the video processes %
  
              }
  
              databaseUpdate(db_data)
  
          })
  
          .save(output);
  
  }
  
  
  
  // get videos not converted `converted: false` and with
  
  // queque status `status: 'Q'` from the database 
  
  // the send to the ffmpegConvert function in a num of async task
  
  var convertVideos = function(num){
  
      Videos.findAll(
  
          {
  
              where: {
  
                  converted: false,
  
                  status: 'Q'
  
              }, 
  
              limit : num, 
  
              order: '"createdAt" DESC'
  
          }
  
      ).then(function (videos) {
  
          if (videos && videos.length){
  
              async.each(videos,
  
                  function(item){
  
                      ffmpegConvert(item.name, output_folder )
  
                  },
  
              function(err){
  
              });
  
          }
  
      });
  
  }
  
  
  
  
  
  // Start vidieo conversion with threads number equal to number of cpu    
  
  threads  = require('os').cpus().length,
  
  convertVideos(threads);

  So at this point All works great, and all the videos on the queue are converted, but my questions are :

  1 - How to put new videos to convert if they are added to the database after the first run on the server ?

  I can’t put it to run the socket.on('message', function(video) {} because if I send a new video when the queue is not completed this going to start a new separated thread. I also think that I have to find a way to monitor the async task to see how many task are running before send new one

  2 - How to manage uncompleted videos if server goes down ?

  If I turn off the server for some reason, the status of the current video in the database stay in ’R’ so the query to add it to the conversion prosses didn not see it, and left it corrupted.