Recherche avancée

Sur d’autres sites (7873)

• Resize and overlay image on video in android efficiently

  29 décembre 2016, par Bhushan Ladhe

  I want to resize image and video(2160 x 3840) to a same resolution(540 × 960) and then overlay image on video for whole time. This whole process is similar to capturing video and adding caption to it like snap-chat and saving it.

  Since i am successful in doing so using FFMPEG library but it consumes considerable amount of time in processing.

  I have searched the internet for other faster operations but no luck. I will post commands that i am using.

  //1st command (video size from 59MB to 4MB, about time 110secs)
  
  -i sdcard/VID_20161228_174315.mp4 -vf scale=540:960 sdcard/output540.mp4
  
  //2nd command (image file , size done, time : a sec hence ignored)
  
  -i sdcard/capture.jpg -vf scale=540:960 sdcard/capture540.png
  
  //3rd command (time 80sec)
  
  -i sdcard/output540.mp4 -i sdcard/capture540.png -filter_complex overlay=0:0 -r 30 sdcard/output_overlay.mp4

  So here using -preset and copying same audio from source did not work as expected.

  Questions :

  1) Should i use specific codec ?

  2) How to achieve Minimum time for processing ?

  3) Will applying multiple threads help ? If yes then how ? I want this process to work in background even if app is closed.

  4) All other options that can combine to make an efficient solution ?

  Log for 3rd command :

   SUCCESS with output : ffmpeg version n3.0.1 Copyright (c) 2000-2016 the FFmpeg developers
  
                                                                        built with gcc 4.8 (GCC)
  
                                                                        configuration: --target-os=linux --cross-prefix=/home/vagrant/SourceCode/ffmpeg-android/toolchain-android/bin/arm-linux-androideabi- --arch=arm --cpu=cortex-a8 --enable-runtime-cpudetect --sysroot=/home/vagrant/SourceCode/ffmpeg-android/toolchain-android/sysroot --enable-pic --enable-libx264 --enable-libass --enable-libfreetype --enable-libfribidi --enable-libmp3lame --enable-fontconfig --enable-pthreads --disable-debug --disable-ffserver --enable-version3 --enable-hardcoded-tables --disable-ffplay --disable-ffprobe --enable-gpl --enable-yasm --disable-doc --disable-shared --enable-static --pkg-config=/home/vagrant/SourceCode/ffmpeg-android/ffmpeg-pkg-config --prefix=/home/vagrant/SourceCode/ffmpeg-android/build/armeabi-v7a --extra-cflags='-I/home/vagrant/SourceCode/ffmpeg-android/toolchain-android/include -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fno-strict-overflow -fstack-protector-all' --extra-ldflags='-L/home/vagrant/SourceCode/ffmpeg-android/toolchain-android/lib -Wl,-z,relro -Wl,-z,now -pie' --extra-libs='-lpng -lexpat -lm' --extra-cxxflags=
  
                                                                        libavutil      55. 17.103 / 55. 17.103
  
                                                                        libavcodec     57. 24.102 / 57. 24.102
  
                                                                        libavformat    57. 25.100 / 57. 25.100
  
                                                                        libavdevice    57.  0.101 / 57.  0.101
  
                                                                        libavfilter     6. 31.100 /  6. 31.100
  
                                                                        libswscale      4.  0.100 /  4.  0.100
  
                                                                        libswresample   2.  0.101 /  2.  0.101
  
                                                                        libpostproc    54.  0.100 / 54.  0.100
  
                                                                      Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'sdcard/output540.mp4':
  
                                                                        Metadata:
  
                                                                          major_brand     : isom
  
                                                                          minor_version   : 512
  
                                                                          compatible_brands: isomiso2avc1mp41
  
                                                                          encoder         : Lavf57.25.100
  
                                                                        Duration: 00:00:10.09, start: 0.021333, bitrate: 2055 kb/s
  
                                                                          Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 540x960 [SAR 1:1 DAR 9:16], 1930 kb/s, 16.75 fps, 16.75 tbr, 17152 tbn, 33.50 tbc (default)
  
                                                                          Metadata:
  
                                                                            handler_name    : VideoHandler
  
                                                                          Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 119 kb/s (default)
  
                                                                          Metadata:
  
                                                                            handler_name    : SoundHandler
  
                                                                      Input #1, png_pipe, from 'sdcard/output_540.png':
  
                                                                        Duration: N/A, bitrate: N/A
  
                                                                          Stream #1:0: Video: png, rgba(pc), 540x960, 25 tbr, 25 tbn, 25 tbc
  
                                                                      [libx264 @ 0xaca425f0] using SAR=1/1
  
                                                                      [libx264 @ 0xaca425f0] using cpu capabilities: none!
  
                                                                      [libx264 @ 0xaca425f0] profile High, level 3.1
  
                                                                      [libx264 @ 0xaca425f0] 264 - core 148 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=9 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
  
                                                                      Output #0, mp4, to 'sdcard/output_overlay_forlog.mp4':
  
                                                                        Metadata:
  
                                                                          major_brand     : isom
  
                                                                          minor_version   : 512
  
                                                                          compatible_brands: isomiso2avc1mp41
  
                                                                          encoder         : Lavf57.25.100
  
                                                                          Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 540x960 [SAR 1:1 DAR 9:16], q=-1--1, 30 fps, 15360 tbn, 30 tbc (default)
  
                                                                          Metadata:
  
                                                                            encoder         : Lavc57.24.102 libx264
  
                                                                          Side data:
  
                                                                            unknown side data type 10 (24 bytes)
  
                                                                          Stream #0:1(eng): Audio: aac (LC) ([64][0][0][0] / 0x0040), 48000 Hz, stereo, fltp, 128 kb/s (default)
  
                                                                          Metadata:
  
                                                                            handler_name    : SoundHandler
  
                                                                            encoder         : Lavc57.24.102 aac
  
                                                                      Stream mapping:
  
                                                                        Stream #0:0 (h264) -> overlay:main (graph 0)
  
                                                                        Stream #1:0 (png) -> overlay:overlay (graph 0)
  
                                                                        overlay (graph 0) -> Stream #0:0 (libx264)
  
                                                                        Stream #0:1 -> #0:1 (aac (native) -> aac (native))
  
                                                                      Press [q] to stop, [?] for help
  
                                                                      frame=   19 fps=0.0 q=0.0 size=       0kB time=00:00:00.88 bit
  
  12-29 02:20:37.914 19213-19213/in.techzilla.happenning_client D/Ffmpeg: Finished command : ffmpeg [Ljava.lang.String;@1a223d4

  here one more txt file for full output

  https://drive.google.com/file/d/0B60iSiCrzaVkUXFpYVFpdGdKN2M/view?usp=drivesdk

• compile FFMpeg for Android (Ubuntu14) - cannot locate symbol

  11 mai 2015, par unichiduci

  I’m trying to compile FFMpeg for Android and I have troubles running the APK on Android 4 (on Android 5 I don’t get this shitty unsatisfied link error) :

  05-09 15:16:18.880  22160-22304/com.gpac.Osmo4 I/LibrariesLoader﹕ Loading library avcodec...
  
  05-09 15:16:18.910  22160-22304/com.gpac.Osmo4 E/dalvikvm﹕ dlopen("/data/app-lib/com.gpac.Osmo4-1/libavcodec.so") failed: Cannot load library: soinfo_relocate(linker.cpp:975): cannot locate symbol "log2f" referenced by "libavcodec.so"...
  
  05-09 15:16:18.920  22160-22304/com.gpac.Osmo4 E/LibrariesLoader﹕ Failed to load library : avcodec due to link error Cannot load library: soinfo_relocate(linker.cpp:975): cannot locate symbol "log2f" referenced by "libavcodec.so"...
  
      java.lang.UnsatisfiedLinkError: Cannot load library: soinfo_relocate(linker.cpp:975): cannot locate symbol "log2f" referenced by "libavcodec.so"...

  I’m using NDK 10d, toolchain 4.9 but I was trying with NDK 8 and 9 also and I was getting the same result.
  The only difference when I compile with older NDK versions is that I get a warning message :

  incompatible declaration of built-in function log2f

  I checked libm.so (where log2f should be) which is on the target device and of course there is no log2f function defined there but replacing the library by hand probably would crash some other stuff + I need to root the phone.

  I know this is a linker issue and it should not be that hard to fix but I ran out of ideas.

  EDIT :
  I’m trying to compile ffmpeg 2.4.3
  The script that I’m using to configure :

  #!/bin/bash
  
  
  
  if [ "$NDK" = "" ]; then
  
      echo NDK variable not set, assuming ${HOME}/android-ndk
  
      export NDK=${HOME}/android-ndk
  
  fi
  
  
  
  echo "Compiling with NDK located at: $NDK"
  
  
  
  ROOT_DIR=`cd ..; pwd`
  
  CUR_DIR=`pwd`
  
  
  
  echo "Fetching Android system headers"
  
  if [ ! -d "$ROOT_DIR/android-source/frameworks/base" ]; then
  
      git clone --depth=1 --branch cm-11.0 https://github.com/CyanogenMod/android_frameworks_base.git "$ROOT_DIR/android-source/frameworks/base"
  
  fi
  
  if [ ! -d "$ROOT_DIR/android-source/system/core" ]; then
  
      git clone --depth=1 --branch cm-11.0 https://github.com/CyanogenMod/android_system_core.git "$ROOT_DIR/android-source/system/core"
  
  fi
  
  if [ ! -d "$ROOT_DIR/android-source/frameworks/av" ]; then
  
      git clone --depth=1 --branch cm-11.0 https://github.com/CyanogenMod/android_frameworks_av "$ROOT_DIR/android-source/frameworks/av"
  
  fi
  
  if [ ! -d "$ROOT_DIR/android-source/hardware/libhardware" ]; then
  
      git clone --depth=1 --branch cm-11.0 https://github.com/CyanogenMod/android_hardware_libhardware.git "$ROOT_DIR/android-source/hardware/libhardware"
  
  fi
  
  if [ ! -d "$ROOT_DIR/android-source/frameworks/native" ]; then
  
      git clone --depth=1 --branch cm-11.0 https://github.com/CyanogenMod/android_frameworks_native.git "$ROOT_DIR/android-source/frameworks/native"
  
  fi
  
  
  
  echo "Fetching Android libraries for linking"
  
  # Libraries from any froyo/gingerbread device/emulator should work
  
  # fine, since the symbols used  should be available on most of them.
  
  if [ ! -d "$ROOT_DIR/android-libs" ]; then
  
      wget http://download.cyanogenmod.org/get/jenkins/65493/cm-10.2.1.3-serranoltexx.zip -P../
  
      unzip ../cm-10.2.1.3-serranoltexx.zip system/lib/* -d../
  
      mv ../system/lib "$ROOT_DIR/android-libs"
  
      rmdir ../system
  
      rm ../cm-10.2.1.3-serranoltexx.zip
  
  fi
  
  
  
  ANDROID_SOURCE="$ROOT_DIR/android-source"
  
  echo "ANDROID_SOURCE: $ANDROID_SOURCE"
  
  ANDROID_LIBS="$ROOT_DIR/android-libs"
  
  OBJS="$ROOT_DIR/objs"
  
  if [ "$DEST" = "" ]; then
  
      rm -rf $ROOT_DIR/build/stagefright
  
      mkdir -p $ROOT_DIR/build/stagefright
  
      DEST=$ROOT_DIR/build/stagefright
  
  fi
  
  
  
  #for ABI in "armeabi-v7a" "armeabi" "x86"; do
  
  for ABI in "armeabi-v7a" "armeabi"; do
  
      if [ "$ABI" = "x86" ]; then
  
          ARCH="x86"
  
          TOOLCHAIN=`echo $NDK/toolchains/x86-4.9/prebuilt/*-x86*`
  
      else
  
          ARCH="arm"
  
          TOOLCHAIN=`echo $NDK/toolchains/arm-linux-androideabi-4.9/prebuilt/*-x86*`
  
      fi 
  
      SYSROOT=$NDK/platforms/android-18/arch-$ARCH
  
      # Expand the prebuilt/* path into the correct one
  
      export PATH=$TOOLCHAIN/bin:$PATH
  
  
  
      FLAGS="--target-os=linux --arch=$ARCH"
  
      FLAGS="$FLAGS --sysroot=$SYSROOT"
  
      FLAGS="$FLAGS --enable-shared --disable-doc --disable-ffplay --disable-ffprobe --disable-ffserver --disable-symver"
  
      if [ "$ARCH" = "arm" ]; then
  
          FLAGS="$FLAGS --cross-prefix=$TOOLCHAIN/bin/arm-linux-androideabi- --cpu=armv7-a --enable-libstagefright-h264"
  
          #FLAGS="$FLAGS --cross-prefix=$TOOLCHAIN/bin/arm-linux-androideabi- --cpu=armv7-a"
  
      else
  
          FLAGS="$FLAGS --cross-prefix=$TOOLCHAIN/bin/i686-linux-android- --disable-asm"
  
      fi
  
  
  
      EXTRA_CFLAGS="-I$DEST/$ABI/include"
  
      EXTRA_CFLAGS="$EXTRA_CFLAGS -I$ANDROID_SOURCE/frameworks/base/include -I$ANDROID_SOURCE/system/core/include"
  
      EXTRA_CFLAGS="$EXTRA_CFLAGS -I$ANDROID_SOURCE/frameworks/native/include"
  
      EXTRA_CFLAGS="$EXTRA_CFLAGS -I$ANDROID_SOURCE/frameworks/native/include/media/openmax"
  
      EXTRA_CFLAGS="$EXTRA_CFLAGS -I$ANDROID_SOURCE/frameworks/av/include"
  
      EXTRA_CFLAGS="$EXTRA_CFLAGS -I$ANDROID_SOURCE/frameworks/av/media/libstagefright"
  
      EXTRA_CFLAGS="$EXTRA_CFLAGS -I$NDK/sources/cxx-stl/gnu-libstdc++/4.9/include -I$NDK/sources/cxx-stl/gnu-libstdc++/4.9/libs/$ABI/include"
  
      EXTRA_CFLAGS="$EXTRA_CFLAGS -I$ANDROID_SOURCE/hardware/libhardware/include"
  
      if [ "$ARCH" = "arm" ]; then
  
          EXTRA_CFLAGS="$EXTRA_CFLAGS -march=armv7-a -mfloat-abi=softfp -mfpu=neon -Wl,--no-undefined" #-Werror=implicit-function-declaration"
  
      fi
  
      EXTRA_LDFLAGS="-Wl,--fix-cortex-a8 -L$ANDROID_LIBS -Wl,-rpath-link,$ANDROID_LIBS -L$NDK/sources/cxx-stl/gnu-libstdc++/4.9/libs/$ABI"
  
      EXTRA_CXXFLAGS="-Wno-multichar -fno-exceptions -fno-rtti"
  
      FLAGS="$FLAGS --prefix=$DEST/$ABI"
  
  
  
      mkdir -p $DEST/$ABI
  
      mkdir -p $OBJS/$ABI
  
  
  
      echo $FLAGS --extra-cflags="$EXTRA_CFLAGS" --extra-ldflags="$EXTRA_LDFLAGS" --extra-cxxflags="$EXTRA_CXXFLAGS" > $DEST/$ABI/info.txt
  
      echo "Configuring ..."
  
      cd $OBJS/$ABI
  
      $ROOT_DIR/configure $FLAGS --extra-cflags="$EXTRA_CFLAGS" --extra-ldflags="$EXTRA_LDFLAGS" --extra-cxxflags="$EXTRA_CXXFLAGS" | tee $DEST/$ABI/configuration.txt
  
      [ $PIPESTATUS == 0 ] || exit 1
  
      echo "Making ..."
  
      #make clean
  
      make -j4 || exit 1
  
      make install || exit 1
  
      cd $CUR_DIR
  
  done

• Recording openCV processed frames on Android using CameraBridgeViewBase

  16 août 2018, par Razvan Ilin

  I basically want to do the same thing as here : Recording Live OpenCV Processing on Android

  I am able to preview the camera using openCV’s JavaCameraView, process the frames (finding color blobs) and then I got stuck when I had to record the video as well.

  I tried to implement the highest voted answer on the above question, but due to my limited knowledge with these technologies, I couldn’t understand exactly how to do it. Then I tried to use ffmpeg and used this as an example : https://github.com/bytedeco/javacv/blob/master/samples/RecordActivity.java

  The point where I got stuck here, was how to pass my CameraBridgeViewBase to the CameraView that is used to record the frames. I’m pretty sure I need to somehow modify my onCreate() method somehow, but I need some pointers on how to approach it.

  I’ll paste my code below :

  OpenCVActivity.java :

  package ch.hepia.iti.opencvnativeandroidstudio;
  
  
  
  import android.Manifest;
  
  import android.content.Context;
  
  import android.content.pm.PackageManager;
  
  import android.hardware.Camera;
  
  import android.media.AudioFormat;
  
  import android.media.AudioRecord;
  
  import android.media.CamcorderProfile;
  
  import android.media.MediaRecorder;
  
  import android.media.MediaScannerConnection;
  
  import android.os.Bundle;
  
  import android.os.Environment;
  
  import android.support.v4.app.ActivityCompat;
  
  import android.support.v7.app.AppCompatActivity;
  
  import android.util.Log;
  
  import android.view.Display;
  
  import android.view.KeyEvent;
  
  import android.view.LayoutInflater;
  
  import android.view.SurfaceHolder;
  
  import android.view.SurfaceView;
  
  import android.view.View;
  
  import android.view.WindowManager;
  
  import android.widget.Button;
  
  import android.widget.LinearLayout;
  
  import android.widget.RelativeLayout;
  
  import android.widget.Toast;
  
  
  
  import org.bytedeco.javacpp.avutil;
  
  import org.bytedeco.javacpp.opencv_core;
  
  import org.bytedeco.javacv.FFmpegFrameFilter;
  
  import org.bytedeco.javacv.FFmpegFrameRecorder;
  
  import org.bytedeco.javacv.Frame;
  
  import org.bytedeco.javacv.FrameFilter;
  
  import org.bytedeco.javacv.OpenCVFrameConverter;
  
  import org.opencv.android.BaseLoaderCallback;
  
  import org.opencv.android.CameraBridgeViewBase;
  
  import org.opencv.android.LoaderCallbackInterface;
  
  import org.opencv.android.OpenCVLoader;
  
  import org.opencv.core.Mat;
  
  import org.opencv.core.Scalar;
  
  
  
  import java.io.File;
  
  import java.io.IOException;
  
  import java.nio.ByteBuffer;
  
  import java.nio.ShortBuffer;
  
  import java.util.Collections;
  
  import java.util.Comparator;
  
  import java.util.List;
  
  
  
  public class OpenCVActivity extends AppCompatActivity implements CameraBridgeViewBase.CvCameraViewListener2, View.OnClickListener {
  
  
  
      private String ffmpeg_link = Environment.getExternalStorageDirectory().getAbsolutePath() + "/test2Video.mp4";
  
      private String LOG_TAG = "VideoTest";
  
  
  
      private static final String TAG = "OCVSample::Activity";
  
      private CameraBridgeViewBase _cameraBridgeViewBase;
  
      private Button recordButton;
  
  
  
      long startTime = 0;
  
      boolean recording = false;
  
  
  
      private FFmpegFrameRecorder recorder;
  
  
  
      private boolean isPreviewOn = false;
  
  
  
      /*Filter information, change boolean to true if adding a fitler*/
  
      private boolean addFilter = true;
  
      private String filterString = "";
  
      FFmpegFrameFilter filter;
  
  
  
      private int sampleAudioRateInHz = 44100;
  
      private int imageWidth = 320;
  
      private int imageHeight = 240;
  
      private int frameRate = 30;
  
  
  
      /* audio data getting thread */
  
      private AudioRecord audioRecord;
  
      private AudioRecordRunnable audioRecordRunnable;
  
      private Thread audioThread;
  
      volatile boolean runAudioThread = true;
  
  
  
      /* video data getting thread */
  
      private Camera cameraDevice;
  
      private CameraView cameraView;
  
  
  
      private Frame yuvImage = null;
  
  
  
      /* layout setting */
  
      private final int bg_screen_bx = 232;
  
      private final int bg_screen_by = 128;
  
      private final int bg_screen_width = 700;
  
      private final int bg_screen_height = 500;
  
      private final int bg_width = 1123;
  
      private final int bg_height = 715;
  
      private final int live_width = 640;
  
      private final int live_height = 480;
  
      private int screenWidth, screenHeight;
  
      private Button btnRecorderControl;
  
  
  
      /* The number of seconds in the continuous record loop (or 0 to disable loop). */
  
      final int RECORD_LENGTH = 0;
  
      Frame[] images;
  
      long[] timestamps;
  
      ShortBuffer[] samples;
  
      int imagesIndex, samplesIndex;
  
  
  
      private BaseLoaderCallback _baseLoaderCallback = new BaseLoaderCallback(this) {
  
          @Override
  
          public void onManagerConnected(int status) {
  
              switch (status) {
  
                  case LoaderCallbackInterface.SUCCESS: {
  
                      Log.i(TAG, "OpenCV loaded successfully");
  
                      // Load ndk built module, as specified in moduleName in build.gradle
  
                      // after opencv initialization
  
                      System.loadLibrary("native-lib");
  
                      _cameraBridgeViewBase.enableView();
  
                  }
  
                  break;
  
                  default: {
  
                      super.onManagerConnected(status);
  
                  }
  
              }
  
          }
  
      };
  
  
  
      @Override
  
      public void onCreate(Bundle savedInstanceState) {
  
          super.onCreate(savedInstanceState);
  
  
  
          Display display = ((WindowManager) getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay();
  
          screenWidth = display.getWidth();
  
          screenHeight = display.getHeight();
  
  
  
          getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
  
          setContentView(R.layout.activity_main);
  
  
  
          // Permissions for Android 6+
  
          ActivityCompat.requestPermissions(OpenCVActivity.this,
  
                  new String[]{Manifest.permission.CAMERA},
  
                  1);
  
  
  
          _cameraBridgeViewBase = (CameraBridgeViewBase) findViewById(R.id.main_surface);
  
          _cameraBridgeViewBase.setVisibility(SurfaceView.VISIBLE);
  
          _cameraBridgeViewBase.setCvCameraViewListener(this);
  
  
  
          RelativeLayout.LayoutParams layoutParam = null;
  
          LayoutInflater myInflate = null;
  
          myInflate = (LayoutInflater) getSystemService(Context.LAYOUT_INFLATER_SERVICE);
  
          RelativeLayout topLayout = new RelativeLayout(this);
  
          setContentView(topLayout);
  
          LinearLayout preViewLayout = (LinearLayout) myInflate.inflate(R.layout.activity_main, null);
  
          layoutParam = new RelativeLayout.LayoutParams(screenWidth, screenHeight);
  
          topLayout.addView(_cameraBridgeViewBase, layoutParam);
  
  
  
          recordButton = (Button) findViewById(R.id.recorder_control);
  
          recordButton.setText("Start");
  
          recordButton.setOnClickListener(this);
  
  
  
          /* add camera view */
  
          int display_width_d = (int) (1.0 * bg_screen_width * screenWidth / bg_width);
  
          int display_height_d = (int) (1.0 * bg_screen_height * screenHeight / bg_height);
  
          int prev_rw, prev_rh;
  
          if (1.0 * display_width_d / display_height_d > 1.0 * live_width / live_height) {
  
              prev_rh = display_height_d;
  
              prev_rw = (int) (1.0 * display_height_d * live_width / live_height);
  
          } else {
  
              prev_rw = display_width_d;
  
              prev_rh = (int) (1.0 * display_width_d * live_height / live_width);
  
          }
  
          layoutParam = new RelativeLayout.LayoutParams(prev_rw, prev_rh);
  
          layoutParam.topMargin = (int) (1.0 * bg_screen_by * screenHeight / bg_height);
  
          layoutParam.leftMargin = (int) (1.0 * bg_screen_bx * screenWidth / bg_width);
  
  
  
          cameraDevice = Camera.open();
  
          cameraView = new CameraView(this, cameraDevice);
  
          topLayout.addView(cameraView, layoutParam);
  
      }
  
  
  
      //---------------------------------------
  
      // initialize ffmpeg_recorder
  
      //---------------------------------------
  
      private void initRecorder() {
  
  
  
          Log.w(LOG_TAG,"init recorder");
  
  
  
          if (RECORD_LENGTH > 0) {
  
              imagesIndex = 0;
  
              images = new Frame[RECORD_LENGTH * frameRate];
  
              timestamps = new long[images.length];
  
              for (int i = 0; i < images.length; i++) {
  
                  images[i] = new Frame(imageWidth, imageHeight, Frame.DEPTH_UBYTE, 2);
  
                  timestamps[i] = -1;
  
              }
  
          } else if (yuvImage == null) {
  
              yuvImage = new Frame(imageWidth, imageHeight, Frame.DEPTH_UBYTE, 2);
  
              Log.i(LOG_TAG, "create yuvImage");
  
          }
  
  
  
          Log.i(LOG_TAG, "ffmpeg_url: " + ffmpeg_link);
  
          recorder = new FFmpegFrameRecorder(ffmpeg_link, imageWidth, imageHeight, 1);
  
          recorder.setFormat("mp4");
  
          recorder.setSampleRate(sampleAudioRateInHz);
  
          // Set in the surface changed method
  
          recorder.setFrameRate(frameRate);
  
  
  
          // The filterString  is any ffmpeg filter.
  
          // Here is the link for a list: https://ffmpeg.org/ffmpeg-filters.html
  
          filterString = "transpose=0";
  
          filter = new FFmpegFrameFilter(filterString, imageWidth, imageHeight);
  
  
  
          //default format on android
  
          filter.setPixelFormat(avutil.AV_PIX_FMT_NV21);
  
  
  
          Log.i(LOG_TAG, "recorder initialize success");
  
  
  
          audioRecordRunnable = new AudioRecordRunnable();
  
          audioThread = new Thread(audioRecordRunnable);
  
          runAudioThread = true;
  
      }
  
  
  
      public void startRecording() {
  
  
  
          initRecorder();
  
  
  
          try {
  
              recorder.start();
  
              startTime = System.currentTimeMillis();
  
              recording = true;
  
              audioThread.start();
  
  
  
              if(addFilter) {
  
                  filter.start();
  
              }
  
  
  
          } catch (FFmpegFrameRecorder.Exception | FrameFilter.Exception e) {
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      public void stopRecording() {
  
  
  
          runAudioThread = false;
  
          try {
  
              audioThread.join();
  
          } catch (InterruptedException e) {
  
              // reset interrupt to be nice
  
              Thread.currentThread().interrupt();
  
              return;
  
          }
  
          audioRecordRunnable = null;
  
          audioThread = null;
  
  
  
          if (recorder != null && recording) {
  
              if (RECORD_LENGTH > 0) {
  
                  Log.v(LOG_TAG,"Writing frames");
  
                  try {
  
                      int firstIndex = imagesIndex % samples.length;
  
                      int lastIndex = (imagesIndex - 1) % images.length;
  
                      if (imagesIndex <= images.length) {
  
                          firstIndex = 0;
  
                          lastIndex = imagesIndex - 1;
  
                      }
  
                      if ((startTime = timestamps[lastIndex] - RECORD_LENGTH * 1000000L) < 0) {
  
                          startTime = 0;
  
                      }
  
                      if (lastIndex < firstIndex) {
  
                          lastIndex += images.length;
  
                      }
  
                      for (int i = firstIndex; i <= lastIndex; i++) {
  
                          long t = timestamps[i % timestamps.length] - startTime;
  
                          if (t >= 0) {
  
                              if (t > recorder.getTimestamp()) {
  
                                  recorder.setTimestamp(t);
  
                              }
  
                              recorder.record(images[i % images.length]);
  
                          }
  
                      }
  
  
  
                      firstIndex = samplesIndex % samples.length;
  
                      lastIndex = (samplesIndex - 1) % samples.length;
  
                      if (samplesIndex <= samples.length) {
  
                          firstIndex = 0;
  
                          lastIndex = samplesIndex - 1;
  
                      }
  
                      if (lastIndex < firstIndex) {
  
                          lastIndex += samples.length;
  
                      }
  
                      for (int i = firstIndex; i <= lastIndex; i++) {
  
                          recorder.recordSamples(samples[i % samples.length]);
  
                      }
  
                  } catch (FFmpegFrameRecorder.Exception e) {
  
                      Log.v(LOG_TAG,e.getMessage());
  
                      e.printStackTrace();
  
                  }
  
              }
  
  
  
              recording = false;
  
              Log.v(LOG_TAG,"Finishing recording, calling stop and release on recorder");
  
              try {
  
                  recorder.stop();
  
                  recorder.release();
  
                  filter.stop();
  
                  filter.release();
  
              } catch (FFmpegFrameRecorder.Exception | FrameFilter.Exception e) {
  
                  e.printStackTrace();
  
              }
  
              recorder = null;
  
  
  
          }
  
      }
  
  
  
      @Override
  
      public boolean onKeyDown(int keyCode, KeyEvent event) {
  
  
  
          if (keyCode == KeyEvent.KEYCODE_BACK) {
  
              if (recording) {
  
                  stopRecording();
  
              }
  
  
  
              finish();
  
  
  
              return true;
  
          }
  
  
  
          return super.onKeyDown(keyCode, event);
  
      }
  
  
  
      //---------------------------------------------
  
      // audio thread, gets and encodes audio data
  
      //---------------------------------------------
  
      class AudioRecordRunnable implements Runnable {
  
  
  
          @Override
  
          public void run() {
  
              android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
  
  
  
              // Audio
  
              int bufferSize;
  
              ShortBuffer audioData;
  
              int bufferReadResult;
  
  
  
              bufferSize = AudioRecord.getMinBufferSize(sampleAudioRateInHz,
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
  
              audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleAudioRateInHz,
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
  
  
  
              if (RECORD_LENGTH > 0) {
  
                  samplesIndex = 0;
  
                  samples = new ShortBuffer[RECORD_LENGTH * sampleAudioRateInHz * 2 / bufferSize + 1];
  
                  for (int i = 0; i < samples.length; i++) {
  
                      samples[i] = ShortBuffer.allocate(bufferSize);
  
                  }
  
              } else {
  
                  audioData = ShortBuffer.allocate(bufferSize);
  
              }
  
  
  
              Log.d(LOG_TAG, "audioRecord.startRecording()");
  
              audioRecord.startRecording();
  
  
  
              /* ffmpeg_audio encoding loop */
  
              while (runAudioThread) {
  
                  if (RECORD_LENGTH > 0) {
  
                      audioData = samples[samplesIndex++ % samples.length];
  
                      audioData.position(0).limit(0);
  
                  }
  
                  //Log.v(LOG_TAG,"recording? " + recording);
  
                  bufferReadResult = audioRecord.read(audioData.array(), 0, audioData.capacity());
  
                  audioData.limit(bufferReadResult);
  
                  if (bufferReadResult > 0) {
  
                      Log.v(LOG_TAG,"bufferReadResult: " + bufferReadResult);
  
                      // If "recording" isn't true when start this thread, it never get's set according to this if statement...!!!
  
                      // Why?  Good question...
  
                      if (recording) {
  
                          if (RECORD_LENGTH <= 0) try {
  
                              recorder.recordSamples(audioData);
  
                              //Log.v(LOG_TAG,"recording " + 1024*i + " to " + 1024*i+1024);
  
                          } catch (FFmpegFrameRecorder.Exception e) {
  
                              Log.v(LOG_TAG,e.getMessage());
  
                              e.printStackTrace();
  
                          }
  
                      }
  
                  }
  
              }
  
              Log.v(LOG_TAG,"AudioThread Finished, release audioRecord");
  
  
  
              /* encoding finish, release recorder */
  
              if (audioRecord != null) {
  
                  audioRecord.stop();
  
                  audioRecord.release();
  
                  audioRecord = null;
  
                  Log.v(LOG_TAG,"audioRecord released");
  
              }
  
          }
  
      }
  
  
  
      //---------------------------------------------
  
      // camera thread, gets and encodes video data
  
      //---------------------------------------------
  
      class CameraView extends SurfaceView implements SurfaceHolder.Callback, Camera.PreviewCallback {
  
  
  
          private SurfaceHolder mHolder;
  
          private Camera mCamera;
  
  
  
          public CameraView(Context context, Camera camera) {
  
              super(context);
  
              Log.w("camera","camera view");
  
              mCamera = camera;
  
              mHolder = getHolder();
  
              mHolder.addCallback(CameraView.this);
  
              mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
  
              mCamera.setPreviewCallback(CameraView.this);
  
          }
  
  
  
          @Override
  
          public void surfaceCreated(SurfaceHolder holder) {
  
              try {
  
                  stopPreview();
  
                  mCamera.setPreviewDisplay(holder);
  
              } catch (IOException exception) {
  
                  mCamera.release();
  
                  mCamera = null;
  
              }
  
          }
  
  
  
          public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
  
              stopPreview();
  
  
  
              Camera.Parameters camParams = mCamera.getParameters();
  
              List sizes = camParams.getSupportedPreviewSizes();
  
              // Sort the list in ascending order
  
              Collections.sort(sizes, new Comparator() {
  
  
  
                  public int compare(final Camera.Size a, final Camera.Size b) {
  
                      return a.width * a.height - b.width * b.height;
  
                  }
  
              });
  
  
  
              // Pick the first preview size that is equal or bigger, or pick the last (biggest) option if we cannot
  
              // reach the initial settings of imageWidth/imageHeight.
  
              for (int i = 0; i < sizes.size(); i++) {
  
                  if ((sizes.get(i).width >= imageWidth && sizes.get(i).height >= imageHeight) || i == sizes.size() - 1) {
  
                      imageWidth = sizes.get(i).width;
  
                      imageHeight = sizes.get(i).height;
  
                      Log.v(LOG_TAG, "Changed to supported resolution: " + imageWidth + "x" + imageHeight);
  
                      break;
  
                  }
  
              }
  
              camParams.setPreviewSize(imageWidth, imageHeight);
  
  
  
              Log.v(LOG_TAG,"Setting imageWidth: " + imageWidth + " imageHeight: " + imageHeight + " frameRate: " + frameRate);
  
  
  
              camParams.setPreviewFrameRate(frameRate);
  
              Log.v(LOG_TAG,"Preview Framerate: " + camParams.getPreviewFrameRate());
  
  
  
              mCamera.setParameters(camParams);
  
  
  
              // Set the holder (which might have changed) again
  
              try {
  
                  mCamera.setPreviewDisplay(holder);
  
                  mCamera.setPreviewCallback(CameraView.this);
  
                  startPreview();
  
              } catch (Exception e) {
  
                  Log.e(LOG_TAG, "Could not set preview display in surfaceChanged");
  
              }
  
          }
  
  
  
          @Override
  
          public void surfaceDestroyed(SurfaceHolder holder) {
  
              try {
  
                  mHolder.addCallback(null);
  
                  mCamera.setPreviewCallback(null);
  
              } catch (RuntimeException e) {
  
                  // The camera has probably just been released, ignore.
  
              }
  
          }
  
  
  
          public void startPreview() {
  
              if (!isPreviewOn && mCamera != null) {
  
                  isPreviewOn = true;
  
                  mCamera.startPreview();
  
              }
  
          }
  
  
  
          public void stopPreview() {
  
              if (isPreviewOn && mCamera != null) {
  
                  isPreviewOn = false;
  
                  mCamera.stopPreview();
  
              }
  
          }
  
  
  
          @Override
  
          public void onPreviewFrame(byte[] data, Camera camera) {
  
              if (audioRecord == null || audioRecord.getRecordingState() != AudioRecord.RECORDSTATE_RECORDING) {
  
                  startTime = System.currentTimeMillis();
  
                  return;
  
              }
  
              if (RECORD_LENGTH > 0) {
  
                  int i = imagesIndex++ % images.length;
  
                  yuvImage = images[i];
  
                  timestamps[i] = 1000 * (System.currentTimeMillis() - startTime);
  
              }
  
  
  
  
  
              /* get video data */
  
              if (yuvImage != null && recording) {
  
                  ((ByteBuffer)yuvImage.image[0].position(0)).put(data);
  
  
  
                  if (RECORD_LENGTH <= 0) try {
  
                      Log.v(LOG_TAG,"Writing Frame");
  
                      long t = 1000 * (System.currentTimeMillis() - startTime);
  
                      if (t > recorder.getTimestamp()) {
  
                          recorder.setTimestamp(t);
  
                      }
  
  
  
                      if(addFilter) {
  
                          filter.push(yuvImage);
  
                          Frame frame2;
  
                          while ((frame2 = filter.pull()) != null) {
  
                              recorder.record(frame2, filter.getPixelFormat());
  
                          }
  
                      } else {
  
                          recorder.record(yuvImage);
  
                      }
  
                  } catch (FFmpegFrameRecorder.Exception | FrameFilter.Exception e) {
  
                      Log.v(LOG_TAG,e.getMessage());
  
                      e.printStackTrace();
  
                  }
  
              }
  
          }
  
      }
  
  
  
      @Override
  
      public void onClick(View v) {
  
          if (!recording) {
  
              startRecording();
  
              Log.w(LOG_TAG, "Start Button Pushed");
  
              recordButton.setText("Stop");
  
          } else {
  
              // This will trigger the audio recording loop to stop and then set isRecorderStart = false;
  
              stopRecording();
  
              Log.w(LOG_TAG, "Stop Button Pushed");
  
              recordButton.setText("Start");
  
          }
  
      }
  
  
  
      @Override
  
      public void onPause() {
  
          super.onPause();
  
          disableCamera();
  
      }
  
  
  
      @Override
  
      public void onResume() {
  
          super.onResume();
  
          if (!OpenCVLoader.initDebug()) {
  
              Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");
  
              OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this, _baseLoaderCallback);
  
          } else {
  
              Log.d(TAG, "OpenCV library found inside package. Using it!");
  
              _baseLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
  
          }
  
      }
  
  
  
      @Override
  
      public void onRequestPermissionsResult(int requestCode, String permissions[], int[] grantResults) {
  
          switch (requestCode) {
  
              case 1: {
  
                  // If request is cancelled, the result arrays are empty.
  
                  if (grantResults.length > 0
  
                          && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
  
                      // permission was granted, yay! Do the
  
                      // contacts-related task you need to do.
  
                  } else {
  
                      // permission denied, boo! Disable the
  
                      // functionality that depends on this permission.
  
                      Toast.makeText(OpenCVActivity.this, "Permission denied to read your External storage", Toast.LENGTH_SHORT).show();
  
                  }
  
                  return;
  
              }
  
              // other 'case' lines to check for other
  
              // permissions this app might request
  
          }
  
      }
  
  
  
      public void onDestroy() {
  
          super.onDestroy();
  
          disableCamera();
  
      }
  
  
  
      public void disableCamera() {
  
          if (_cameraBridgeViewBase != null)
  
              _cameraBridgeViewBase.disableView();
  
      }
  
  
  
      public void onCameraViewStarted(int width, int height) {
  
          setup();
  
      }
  
  
  
      public void onCameraViewStopped() {
  
      }
  
  
  
      public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
  
          Mat rgba = inputFrame.rgba();
  
          salt(rgba.getNativeObjAddr());
  
  
  
          return rgba;
  
      }
  
  
  
  
  
      public native void salt(long matAddrGray);
  
      public native void setup();
  
  }

  activity_main.xml :

  <?xml version="1.0" encoding="utf-8"?>
  
  <linearlayout>
  
  
  
      
  
  
  
      <button></button>
  
  
  
  </linearlayout>

  Could really use some help with this. Has anyone done this before ?