Recherche avancée

Sur d’autres sites (8083)

• FFMpeg process created from Java on CentOS doesn't exit

  21 juin 2017, par Donz

  I need to convert a lot of wave files simultaneously. About 300 files in parallel. And new files come constantly. I use ffmpeg process call from my Java 1.8 app, which is running on CentOS. I know that I have to read error and input streams for making created process from Java possible to exit.

  My code after several expirements :

      private void ffmpegconverter(String fileIn, String fileOut){
  
      String[] comand = new String[]{"ffmpeg", "-v", "-8", "-i", fileIn, "-acodec", "pcm_s16le", fileOut};
  
  
  
      Process process = null;
  
      BufferedReader reader = null;
  
      try {
  
          ProcessBuilder pb = new ProcessBuilder(comand);
  
          pb.redirectErrorStream(true);
  
          process = pb.start();
  
  
  
          //Reading from error and standard output console buffer of process. Or it could halts because of nobody
  
          //reads its buffer
  
          reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
  
          String s;
  
          //noinspection StatementWithEmptyBody
  
          while ((s = reader.readLine()) != null) {
  
              log.info(Thread.currentThread().getName() + " with fileIn " + fileIn + " and fileOut " + fileOut + " writes " + s);
  
              //Ignored as we just need to empty the output buffer from process
  
          }
  
          log.info(Thread.currentThread().getName() + " ffmpeg process will be waited for");
  
          if (process.waitFor( 10, TimeUnit.SECONDS )) {
  
              log.info(Thread.currentThread().getName() + " ffmpeg process exited normally");
  
          } else {
  
              log.info(Thread.currentThread().getName() + " ffmpeg process timed out and will be killed");
  
          }
  
  
  
      } catch (IOException | InterruptedException e) {
  
          log.error(Thread.currentThread().getName() + "Error during ffmpeg process executing", e);
  
      } finally {
  
          if (process != null) {
  
              if (reader != null) {
  
                  try {
  
                      reader.close();
  
                  } catch (IOException e) {
  
                      log.error("Error during closing the process streams reader", e);
  
                  }
  
              }
  
              try {
  
                  process.getOutputStream().close();
  
              } catch (IOException e) {
  
                  log.error("Error during closing the process output stream", e);
  
              }
  
              process.destroyForcibly();
  
              log.info(Thread.currentThread().getName() + " ffmpeg process " + process + " must be dead now");
  
          }
  
      }
  
  }

  If I run separate test with this code it goes normally. But in my app I have hundreds of RUNNING deamon threads "process reaper" which are waiting for ffmpeg process finish. In my real app ffpmeg is started from timer thread. Also I have another activity in separate threads, but I don’t think that this is the problem. Max CPU consume is about 10%.

  Here is that I usual see in thread dump :

  "process reaper" #454 daemon prio=10 os_prio=0 tid=0x00007f641c007000 nid=0x5247 runnable [0x00007f63ec063000]
  
     java.lang.Thread.State: RUNNABLE
  
      at java.lang.UNIXProcess.waitForProcessExit(Native Method)
  
      at java.lang.UNIXProcess.lambda$initStreams$3(UNIXProcess.java:289)
  
      at java.lang.UNIXProcess$$Lambda$32/2113551491.run(Unknown Source)
  
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  
      at java.lang.Thread.run(Thread.java:745)

  What am I doing wrong ?

  UPD :
  My app accepts a lot of connects with voice traffic. So I have about 300-500 another "good" threads in every moment. Could it be the reason ? Deamon threads have low priority. But I don’t beleive that they really can’t do their jobs in one hour. Ususally it takes some tens of millis.

  UPD2 :
  My synthetic test that runs fine. I tried with new threads option and without it just with straigt calling of run method.

  import java.io.BufferedReader;
  
  import java.io.File;
  
  import java.io.IOException;
  
  import java.io.InputStreamReader;
  
  
  
  public class FFmpegConvert {
  
  
  
      public static void main(String[] args) throws Exception {
  
  
  
          FFmpegConvert f = new FFmpegConvert();
  
          f.processDir(args[0], args[1], args.length > 2);
  
      }
  
  
  
      private void processDir(String dirPath, String dirOutPath, boolean isNewThread) {
  
          File dir = new File(dirPath);
  
          File dirOut = new File(dirOutPath);
  
          if(!dirOut.exists()){
  
              dirOut.mkdir();
  
          }
  
          for (int i = 0; i < 1000; i++) {
  
              for (File f : dir.listFiles()) {
  
                  try {
  
                      System.out.println(f.getName());
  
                      FFmpegRunner fFmpegRunner = new FFmpegRunner(f.getAbsolutePath(), dirOut.getAbsolutePath() + "/" + System.currentTimeMillis() + f.getName());
  
                      if (isNewThread) {
  
                          new Thread(fFmpegRunner).start();
  
                      } else {
  
                          fFmpegRunner.run();
  
                      }
  
                  } catch (Exception e) {
  
                      e.printStackTrace();
  
                  }
  
              }
  
          }
  
      }
  
  
  
      class FFmpegRunner implements Runnable {
  
          private String fileIn;
  
          private String fileOut;
  
  
  
          FFmpegRunner(String fileIn, String fileOut) {
  
              this.fileIn = fileIn;
  
              this.fileOut = fileOut;
  
          }
  
  
  
          @Override
  
          public void run() {
  
              try {
  
                  ffmpegconverter(fileIn, fileOut);
  
              } catch (Exception e) {
  
                  e.printStackTrace();
  
              }
  
          }
  
  
  
          private void ffmpegconverter(String fileIn, String fileOut) throws Exception{
  
              String[] comand = new String[]{"ffmpeg", "-i", fileIn, "-acodec", "pcm_s16le", fileOut};
  
  
  
              Process process = null;
  
              try {
  
                  ProcessBuilder pb = new ProcessBuilder(comand);
  
                  pb.redirectErrorStream(true);
  
                  process = pb.start();
  
  
  
                  //Reading from error and standard output console buffer of process. Or it could halts because of nobody
  
                  //reads its buffer
  
                  BufferedReader reader =
  
                          new BufferedReader(new InputStreamReader(process.getInputStream()));
  
                  String line;
  
                  //noinspection StatementWithEmptyBody
  
                  while ((line = reader.readLine()) != null) {
  
                      System.out.println(line);
  
                      //Ignored as we just need to empty the output buffer from process
  
                  }
  
  
  
                  process.waitFor();
  
              } catch (IOException | InterruptedException e) {
  
                  throw e;
  
              } finally {
  
                  if (process != null)
  
                      process.destroy();
  
              }
  
          }
  
  
  
      }
  
  
  
  }

  UPD3 :
  Sorry, I forgot to notice that I see the work of all these process - they created new converted files but anyway don’t exit.

• Piwik PRO is hiring a Project Coordinator (Job description)

  18 février 2015, par Matthieu Aubry — Jobs

  At Piwik and Piwik PRO we develop the leading open source web analytics platform, used by more than one million websites worldwide. Our vision is to build the best open alternative to Google Universal Analytics. We are growing and now looking for a Project Coordinator !

  What will you be doing ?

  • Participating in calls with Piwik PRO clients and analyzing requirements for enterprise customers
  • Planning and coordinating the implementation of projects
  • Helping customers to improve and adjust data reporting methods to suit their needs
  • Functioning as the voice of the customer and provide internal feedback on how Piwik PRO can better serve our customers

  What do we expect from you ?

  • Fluent command of English (both in writing and speaking), confidence to communicate in formal conversations and a good knowledge of business English
  • Previous experience in working with corporate clients
  • Great communication skills
  • A proactive attitude and the ability to use your initiative
  • Ability to achieve goals without detailed instructions

  Possessing these assets would be an advantage :

  • Technical knowledge associated with using data analytics platforms
  • Data analysis and interpretation skills
  • Knowledge of German or French
  • Knowledge of Piwik Analytics

  What can you expect from us ?

  • Flexible cooperation
  • An attractive salary
  • The opportunity to develop your skills and gain more experience by working on exceptional projects
  • Multisport Card
  • Access to a regularly updated resource library and the opportunity to contribute to it
  • Flexible working hours
  • Unforgettable parties and integration trips
  • A completely unique work atmosphere – we really like to keep things informal

  Location

  We have offices in Poland and New Zealand, and Remote work is possible.

  Ideally you will be located in the USA or in Europe where most of our clients are based.

  Apply online

  To apply for this position, please Apply online here. We look forward to receiving your applications !

• Piwik PRO is hiring a Project Coordinator (Job description)

  18 février 2015, par Matthieu Aubry — Jobs

  At Piwik and Piwik PRO we develop the leading open source web analytics platform, used by more than one million websites worldwide. Our vision is to build the best open alternative to Google Universal Analytics. We are growing and now looking for a Project Coordinator !

  What will you be doing ?

  • Participating in calls with Piwik PRO clients and analyzing requirements for enterprise customers
  • Planning and coordinating the implementation of projects
  • Helping customers to improve and adjust data reporting methods to suit their needs
  • Functioning as the voice of the customer and provide internal feedback on how Piwik PRO can better serve our customers

  What do we expect from you ?

  • Fluent command of English (both in writing and speaking), confidence to communicate in formal conversations and a good knowledge of business English
  • Previous experience in working with corporate clients
  • Great communication skills
  • A proactive attitude and the ability to use your initiative
  • Ability to achieve goals without detailed instructions

  Possessing these assets would be an advantage :

  • Technical knowledge associated with using data analytics platforms
  • Data analysis and interpretation skills
  • Knowledge of German or French
  • Knowledge of Piwik Analytics

  What can you expect from us ?

  • Flexible cooperation
  • An attractive salary
  • The opportunity to develop your skills and gain more experience by working on exceptional projects
  • Multisport Card
  • Access to a regularly updated resource library and the opportunity to contribute to it
  • Flexible working hours
  • Unforgettable parties and integration trips
  • A completely unique work atmosphere – we really like to keep things informal

  Location

  We have offices in Poland and New Zealand, and Remote work is possible.

  Ideally you will be located in the USA or in Europe where most of our clients are based.

  Apply online

  To apply for this position, please Apply online here. We look forward to receiving your applications !