Recherche avancée

Sur d’autres sites (9863)

• FFMpeg process created from Java on CentOS doesn't exit

  21 juin 2017, par Donz

  I need to convert a lot of wave files simultaneously. About 300 files in parallel. And new files come constantly. I use ffmpeg process call from my Java 1.8 app, which is running on CentOS. I know that I have to read error and input streams for making created process from Java possible to exit.

  My code after several expirements :

      private void ffmpegconverter(String fileIn, String fileOut){
  
      String[] comand = new String[]{"ffmpeg", "-v", "-8", "-i", fileIn, "-acodec", "pcm_s16le", fileOut};
  
  
  
      Process process = null;
  
      BufferedReader reader = null;
  
      try {
  
          ProcessBuilder pb = new ProcessBuilder(comand);
  
          pb.redirectErrorStream(true);
  
          process = pb.start();
  
  
  
          //Reading from error and standard output console buffer of process. Or it could halts because of nobody
  
          //reads its buffer
  
          reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
  
          String s;
  
          //noinspection StatementWithEmptyBody
  
          while ((s = reader.readLine()) != null) {
  
              log.info(Thread.currentThread().getName() + " with fileIn " + fileIn + " and fileOut " + fileOut + " writes " + s);
  
              //Ignored as we just need to empty the output buffer from process
  
          }
  
          log.info(Thread.currentThread().getName() + " ffmpeg process will be waited for");
  
          if (process.waitFor( 10, TimeUnit.SECONDS )) {
  
              log.info(Thread.currentThread().getName() + " ffmpeg process exited normally");
  
          } else {
  
              log.info(Thread.currentThread().getName() + " ffmpeg process timed out and will be killed");
  
          }
  
  
  
      } catch (IOException | InterruptedException e) {
  
          log.error(Thread.currentThread().getName() + "Error during ffmpeg process executing", e);
  
      } finally {
  
          if (process != null) {
  
              if (reader != null) {
  
                  try {
  
                      reader.close();
  
                  } catch (IOException e) {
  
                      log.error("Error during closing the process streams reader", e);
  
                  }
  
              }
  
              try {
  
                  process.getOutputStream().close();
  
              } catch (IOException e) {
  
                  log.error("Error during closing the process output stream", e);
  
              }
  
              process.destroyForcibly();
  
              log.info(Thread.currentThread().getName() + " ffmpeg process " + process + " must be dead now");
  
          }
  
      }
  
  }

  If I run separate test with this code it goes normally. But in my app I have hundreds of RUNNING deamon threads "process reaper" which are waiting for ffmpeg process finish. In my real app ffpmeg is started from timer thread. Also I have another activity in separate threads, but I don’t think that this is the problem. Max CPU consume is about 10%.

  Here is that I usual see in thread dump :

  "process reaper" #454 daemon prio=10 os_prio=0 tid=0x00007f641c007000 nid=0x5247 runnable [0x00007f63ec063000]
  
     java.lang.Thread.State: RUNNABLE
  
      at java.lang.UNIXProcess.waitForProcessExit(Native Method)
  
      at java.lang.UNIXProcess.lambda$initStreams$3(UNIXProcess.java:289)
  
      at java.lang.UNIXProcess$$Lambda$32/2113551491.run(Unknown Source)
  
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
  
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
  
      at java.lang.Thread.run(Thread.java:745)

  What am I doing wrong ?

  UPD :
  My app accepts a lot of connects with voice traffic. So I have about 300-500 another "good" threads in every moment. Could it be the reason ? Deamon threads have low priority. But I don’t beleive that they really can’t do their jobs in one hour. Ususally it takes some tens of millis.

  UPD2 :
  My synthetic test that runs fine. I tried with new threads option and without it just with straigt calling of run method.

  import java.io.BufferedReader;
  
  import java.io.File;
  
  import java.io.IOException;
  
  import java.io.InputStreamReader;
  
  
  
  public class FFmpegConvert {
  
  
  
      public static void main(String[] args) throws Exception {
  
  
  
          FFmpegConvert f = new FFmpegConvert();
  
          f.processDir(args[0], args[1], args.length > 2);
  
      }
  
  
  
      private void processDir(String dirPath, String dirOutPath, boolean isNewThread) {
  
          File dir = new File(dirPath);
  
          File dirOut = new File(dirOutPath);
  
          if(!dirOut.exists()){
  
              dirOut.mkdir();
  
          }
  
          for (int i = 0; i < 1000; i++) {
  
              for (File f : dir.listFiles()) {
  
                  try {
  
                      System.out.println(f.getName());
  
                      FFmpegRunner fFmpegRunner = new FFmpegRunner(f.getAbsolutePath(), dirOut.getAbsolutePath() + "/" + System.currentTimeMillis() + f.getName());
  
                      if (isNewThread) {
  
                          new Thread(fFmpegRunner).start();
  
                      } else {
  
                          fFmpegRunner.run();
  
                      }
  
                  } catch (Exception e) {
  
                      e.printStackTrace();
  
                  }
  
              }
  
          }
  
      }
  
  
  
      class FFmpegRunner implements Runnable {
  
          private String fileIn;
  
          private String fileOut;
  
  
  
          FFmpegRunner(String fileIn, String fileOut) {
  
              this.fileIn = fileIn;
  
              this.fileOut = fileOut;
  
          }
  
  
  
          @Override
  
          public void run() {
  
              try {
  
                  ffmpegconverter(fileIn, fileOut);
  
              } catch (Exception e) {
  
                  e.printStackTrace();
  
              }
  
          }
  
  
  
          private void ffmpegconverter(String fileIn, String fileOut) throws Exception{
  
              String[] comand = new String[]{"ffmpeg", "-i", fileIn, "-acodec", "pcm_s16le", fileOut};
  
  
  
              Process process = null;
  
              try {
  
                  ProcessBuilder pb = new ProcessBuilder(comand);
  
                  pb.redirectErrorStream(true);
  
                  process = pb.start();
  
  
  
                  //Reading from error and standard output console buffer of process. Or it could halts because of nobody
  
                  //reads its buffer
  
                  BufferedReader reader =
  
                          new BufferedReader(new InputStreamReader(process.getInputStream()));
  
                  String line;
  
                  //noinspection StatementWithEmptyBody
  
                  while ((line = reader.readLine()) != null) {
  
                      System.out.println(line);
  
                      //Ignored as we just need to empty the output buffer from process
  
                  }
  
  
  
                  process.waitFor();
  
              } catch (IOException | InterruptedException e) {
  
                  throw e;
  
              } finally {
  
                  if (process != null)
  
                      process.destroy();
  
              }
  
          }
  
  
  
      }
  
  
  
  }

  UPD3 :
  Sorry, I forgot to notice that I see the work of all these process - they created new converted files but anyway don’t exit.

• How to port signal() to sigaction() ?

  28 septembre 2016, par Shark

  due to recent problems discovered with NDK12 and NDK13b2, i’m thinking of ’porting’ libx264’s use of signal() (and missing bsd_signal() in ndk12) to use sigaction() instead.

  The problem is, I’m not quite sure what’s the simple&fastest way to replace signal() calls with sigaction() ones.

  For all i see, it’s mainly used in x264-snapshot/common/cpu.c in the following manner :

  using the following signal handler :

  static void sigill_handler( int sig )
  
  {
  
      if( !canjump )
  
      {
  
          signal( sig, SIG_DFL );
  
          raise( sig );
  
      }
  
  
  
      canjump = 0;
  
      siglongjmp( jmpbuf, 1 );
  
  }

  This is the problematic x264_cpu_detect function... currently, i’m guessing i only need to tackle the ARM version, but i’ ; ; still have to replace all occurances of signal() with sigaction() so i might just cover both of them to get the thing building...

  FYI - the NDK13 beta2 still has "unstable" libc and the build doesn’t fail on this part, but rather the first invocation of the rand() function somewhere else... So i’m out of luck and replacing the signal() calls might be better than just waiting for the official NDK13 release. I’m doing this to get rid of text-relocations so i can run the library (and doubango) on API 24 (Android N)

  the problematic part of function that invokes signal() :

  #elif SYS_LINUX
  
  
  
  uint32_t x264_cpu_detect( void )
  
  {
  
      static void (*oldsig)( int );
  
  
  
      oldsig = signal( SIGILL, sigill_handler );
  
      if( sigsetjmp( jmpbuf, 1 ) )
  
      {
  
          signal( SIGILL, oldsig );
  
          return 0;
  
      }
  
  
  
      canjump = 1;
  
      asm volatile( "mtspr 256, %0\n\t"
  
                    "vand 0, 0, 0\n\t"
  
                    :
  
                    : "r"(-1) );
  
      canjump = 0;
  
  
  
      signal( SIGILL, oldsig );
  
  
  
      return X264_CPU_ALTIVEC;
  
  }
  
  #endif
  
  
  
  #elif ARCH_ARM
  
  
  
  void x264_cpu_neon_test( void );
  
  int x264_cpu_fast_neon_mrc_test( void );
  
  
  
  uint32_t x264_cpu_detect( void )
  
  {
  
      int flags = 0;
  
  #if HAVE_ARMV6
  
      flags |= X264_CPU_ARMV6;
  
  
  
      // don't do this hack if compiled with -mfpu=neon
  
  #if !HAVE_NEON
  
      static void (* oldsig)( int );
  
      oldsig = signal( SIGILL, sigill_handler );
  
      if( sigsetjmp( jmpbuf, 1 ) )
  
      {
  
          signal( SIGILL, oldsig );
  
          return flags;
  
      }
  
  
  
      canjump = 1;
  
      x264_cpu_neon_test();
  
      canjump = 0;
  
      signal( SIGILL, oldsig );
  
  #endif
  
  
  
      flags |= X264_CPU_NEON;
  
  
  
      // fast neon -> arm (Cortex-A9) detection relies on user access to the
  
      // cycle counter; this assumes ARMv7 performance counters.
  
      // NEON requires at least ARMv7, ARMv8 may require changes here, but
  
      // hopefully this hacky detection method will have been replaced by then.
  
      // Note that there is potential for a race condition if another program or
  
      // x264 instance disables or reinits the counters while x264 is using them,
  
      // which may result in incorrect detection and the counters stuck enabled.
  
      // right now Apple does not seem to support performance counters for this test
  
  #ifndef __MACH__
  
      flags |= x264_cpu_fast_neon_mrc_test() ? X264_CPU_FAST_NEON_MRC : 0;
  
  #endif
  
      // TODO: write dual issue test? currently it's A8 (dual issue) vs. A9 (fast      mrc)
  
  #endif
  
      return flags;
  
  }
  
  
  
  #else
  
  
  
  uint32_t x264_cpu_detect( void )
  
  {
  
      return 0;
  
  }

  So the question is really this : what would be the quickest/easiest//fastest way to replace the signal() calls with sigaction() ones while preserving the current functionality ?

  EDIT :
  The reason i’m trying to get rid of signal() are these build errors :

  /home/devshark/SCRATCH/doubango/thirdparties/android/armv5te/lib/dist/libx264.a(cpu.o):cpu.c:function sigill_handler: error: undefined reference to 'bsd_signal'
  
  
  
  /home/devshark/SCRATCH/doubango/thirdparties/android/armv5te/lib/dist/libx264.a(cpu.o):cpu.c:function x264_cpu_detect: error: undefined reference to 'bsd_signal'
  
  /home/devshark/SCRATCH/doubango/thirdparties/android/armv5te/lib/dist/libx264.a(cpu.o):cpu.c:function x264_cpu_detect: error: undefined reference to 'bsd_signal'
  
  
  
  /home/devshark/SCRATCH/doubango/thirdparties/android/armv5te/lib/dist/libx264.a(cpu.o):cpu.c:function x264_cpu_detect: error: undefined reference to 'bsd_signal'

  I already know that this is a known NDK12 problem, that might be solved by bringing bsd_signal back to the libc in NDK13. However, in it’ beta state with it’s unstable libc - it’s currently missing the rand() function and simply waiting for it might not do the trick. But in the worst-case scenario, i guess i’ll just have to wait for it and retry after it’s release.

  But as it currently is, the prebuilt version of the library i want to use has text-relocations and is being rejected by phones running newer API / version of the android OS.

  EDIT2 :
  I also know that signal() usually works by using sigaction() under the hood, but maybe i won’t get bsd_signal related build-errors... since i’m suspecting that this one isn’t using it. It’s obviously using bsd_signal, which may or may not be the same underlying thing :/

• Merge commit '2170017a1cd033b6f28e16476921022712a522d8'

  13 avril 2017, par James Almer

  Merge commit '2170017a1cd033b6f28e16476921022712a522d8'
  * commit '2170017a1cd033b6f28e16476921022712a522d8' :
  
    avutil : fix data race in av_get_cpu_flags()
  This commit is a noop, see fed50c4304eecb352e29ce789cdb96ea84d6162f
  Merged-by : James Almer <jamrial@gmail.com>
  

  • [DH] libavutil/Makefile