Recherche avancée

Sur d’autres sites (9742)

• Memcached protocol support

  15 novembre 2013, par Mikko Koppanen — Imagick

  For the past few days I’ve been adding Memcached binary protocol support to PECL memcached extension. The protocol handler provides a high-level abstraction for acting as a memcached server. There are quite a few things still missing and only binary protocol is supported at the moment, but the code seems to work reasonably well in small-scale testing.

  I am not sure whether this is useful for anyone, but at least it allows things such as quick prototyping of network servers, exposing sqlite database over memcached protocol etc.

  The code is quite simple and implementing a simple server responding to get and set would look roughly like the following :

  php

  < view plain text >

  1. < ?php
  2. // Create new server instance
  3. $server = new MemcachedServer() ;
  4.  
  5. // Create a simple storage class
  6. class Storage {
  7.   private $values = array () ;
  8.   
  9.   public function set ($key, $value, $expiration) {
  10.     $this->values [$key] = array (’value’ => $value,
  11.                    ’expires’ => time () + $expiration) ;
  12.   }
  13.  
  14.   public function get ($key) {
  15.     if (isset ($this->values [$key])) {
  16.       if ($this->values [$key] [’expires’] < time ()) {
  17.         unset ($this->values [$key]) ;
  18.         return null ;
  19.       }
  20.       return $this->values [$key] [’value’] ;
  21.     }
  22.     else
  23.       return null ;
  24.   }
  25. }
  26.  
  27. $storage = new Storage () ;
  28.  
  29. // Set callback for get command
  30. $server->on (Memcached: :ON_GET,
  31.        function ($client_id, $key, &$value, &$flags, &$cas) use ($storage) {
  32.          echo "Getting key=[$key]" . PHP_EOL ;
  33.          if (($value = $storage->get ($key))  != null)
  34.            return Memcached: :RESPONSE_SUCCESS ;
  35.  
  36.          return Memcached: :RESPONSE_KEY_ENOENT ;
  37.        }) ;
  38.  
  39. // Set callback for set command
  40. $server->on (Memcached: :ON_SET,
  41.        function ($client_id, $key, $value, $flags, $expiration, $cas, &$result_cas) use ($storage) {
  42.          echo "Setting key=[$key] value=[$value]" . PHP_EOL ;
  43.          $storage->set ($key, $value, $expiration) ;
  44.          return Memcached: :RESPONSE_SUCCESS ;
  45.        }) ;
  46.  
  47. // Run the server on localhost, port 3434. Will block
  48. $server->run ("127.0.0.1:3434") ;
  49.  ?>

  And the client that communicates with the server :

  php

  < view plain text >

  1. < ?php
  2.  
  3. $cache = new Memcached() ;
  4. $cache->setOption(Memcached: :OPT_BINARY_PROTOCOL, true) ;
  5. $cache->setOption(Memcached: :OPT_COMPRESSION, false) ;
  6. $cache->addServer(’localhost’, 3434) ;
  7.  
  8. $cache->set (’set_key1’, ’This is the first key’, 10) ;
  9. var_dump ($cache->get (’set_key1’)) ;
  10.  
  11. $cache->set (’set_key2’, ’This is the second key’, 2) ;
  12. var_dump ($cache->get (’set_key2’)) ;
  13.  ?>

  The code is still work in progress but it’s available in github : https://github.com/mkoppanen/php-memcached/tree/feature-server. Note that you need to compile libmemcached with –enable-libmemcachedprotocol and the PECL memcached extension with –enable-memcached-protocol.

• mpeg2 ts android ffmpeg openmax

  10 octobre 2014, par WLGfx

  The setup is as follows :

  1. Multicast server 1000Mbs, UDP, Mpeg2-TS Part 1 (H.222) streaming live TV-channels.
  2. Quad core 1.5Ghz Android 4.2.2 GLES 2.0 renderer.
  3. FFMpeg library.
  4. Eclipse Kepler, Android SDK/NDK, etc. Running on Windows 8.1.
  5. Output screen 1920 x 1080, I am using a texture of 2048 x 1024 and getting between 35 and 45 Frames per second.

  The app :

  1. Renderer thread runs continuously and updates a single texture by uploading segments to the gpu when media images are ready.
  2. Media handler thread, downloads and processes media from server/or local storage.
  3. Video thread(s), one for buffering the UDP packets and another for decoding the packets into frames.

  I am connecting ffmpeg to the UDP stream just fine and the packets are being buffered and seemingly decoded fine. The packet buffers are plenty, no under/over-flows. The problem I am facing is it appears to be chopping up frames (ie only playing back 1 out of every so many frames). I understand that I need to distinguish I/P/B frames, but at the moment, hands up, I ain’t got a clue. I’ve even tried a hack to detect I frames to no avail. Plus, I am only rendering the frames to less than a quarter of the screen. So I’m not using full screen decoding.

  The decoded frames are also stored in separate buffers to cut out page tearing. The number of buffers I’ve changed too, from 1 to 10 with no luck.

  From what I’ve found about OpenMax IL, is it only handles MPeg2-TS Part 3 (H.264 and AAC), but you can use your own decoder. I understand that you can add your own decode component to it. Would it be worth me trying this route or should I continue on with ffmpeg ?

  The frame decoder (only the renderer will convert and scale the frames when ready)
  /*
  * This function will run through the packets and keep decoding
  * until a frame is ready first, or out of packets
  */

  while (packetsUsed[decCurrent])
  
  {
  
      hack_for_i_frame:
  
      i = avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packets[decCurrent]);
  
      packetsUsed[decCurrent] = 0; // finished with this one
  
      i = packets[decCurrent].flags & 0x0001;
  
      decCurrent++;
  
      if (decCurrent >= MAXPACKETS) decCurrent = 0;
  
      if (frameFinished)
  
      {
  
          ready_pFrame = pFrame;
  
          frameReady = true;  // notify renderer
  
          frameCounter++;
  
          if (frameCounter>=MAXFRAMES) frameCounter = 0;
  
          pFrame = pFrames[frameCounter];
  
          return 0;
  
      }
  
      else if (i)
  
          goto hack_for_i_frame;
  
  }
  
  
  
  return 0;

  The packet reader (spawned as a pthread)
  void *mainPacketReader(void *voidptr)
  
  int res ;

  while ( threadState == TS_RUNNING )
  
  {
  
      if (packetsUsed[prCurrent])
  
      {
  
          LOGE("Packet buffer overflow, dropping packet...");
  
          av_read_frame( pFormatCtx, &packet );
  
      }
  
      else if ( av_read_frame( pFormatCtx, &packets[prCurrent] ) >= 0 )
  
      {
  
          if ( packets[prCurrent].stream_index == videoStream )
  
          {
  
              packetsUsed[prCurrent] = 1; // flag as used
  
              prCurrent++;
  
              if ( prCurrent >= MAXPACKETS )
  
              {
  
                  prCurrent = 0;
  
              }
  
          }
  
  
  
          // here check if the packet is audio and add to audio buffer
  
      }
  
  }
  
  return NULL;

  And the renderer just simply does this
  // texture has already been bound before calling this function

  if ( frameReady == false ) return;
  
  
  
  AVFrame *temp;  // set to frame 'not' currently being decoded
  
  temp = ready_pFrame;
  
  
  
  sws_scale(sws_ctx,(uint8_t const* const *)temp->data,
  
          temp->linesize, 0, pCodecCtx->height,
  
          pFrameRGB->data, pFrameRGB->linesize);
  
  
  
  glTexSubImage2D(GL_TEXTURE_2D, 0,
  
          XPOS, YPOS, WID, HGT,
  
          GL_RGBA, GL_UNSIGNED_BYTE, buffer);
  
  
  
  frameReady = false;

  In the past, libvlc had audio syncing problems too, so that is my decision for going with ffmpeg and doing all the donkey work from scratch.

  If anybody has any pointers of how to stop the choppiness of the video playback (works great in VLC player) or possibly another route to go down, it would be seriously appreciated.

  EDIT I removed the hack for the I-frame (completely useless). Move the sws_scale function from the renderer to the packet decoder. And I left the udp packet reader thread alone.

  In the meantime I’ve also changed the packet reader thread and the packet decoder threads priority to real-time. Since doing that I don’t get shed loads of dropped packets.

• Recording openCV processed frames on Android using CameraBridgeViewBase

  16 août 2018, par Razvan Ilin

  I basically want to do the same thing as here : Recording Live OpenCV Processing on Android

  I am able to preview the camera using openCV’s JavaCameraView, process the frames (finding color blobs) and then I got stuck when I had to record the video as well.

  I tried to implement the highest voted answer on the above question, but due to my limited knowledge with these technologies, I couldn’t understand exactly how to do it. Then I tried to use ffmpeg and used this as an example : https://github.com/bytedeco/javacv/blob/master/samples/RecordActivity.java

  The point where I got stuck here, was how to pass my CameraBridgeViewBase to the CameraView that is used to record the frames. I’m pretty sure I need to somehow modify my onCreate() method somehow, but I need some pointers on how to approach it.

  I’ll paste my code below :

  OpenCVActivity.java :

  package ch.hepia.iti.opencvnativeandroidstudio;
  
  
  
  import android.Manifest;
  
  import android.content.Context;
  
  import android.content.pm.PackageManager;
  
  import android.hardware.Camera;
  
  import android.media.AudioFormat;
  
  import android.media.AudioRecord;
  
  import android.media.CamcorderProfile;
  
  import android.media.MediaRecorder;
  
  import android.media.MediaScannerConnection;
  
  import android.os.Bundle;
  
  import android.os.Environment;
  
  import android.support.v4.app.ActivityCompat;
  
  import android.support.v7.app.AppCompatActivity;
  
  import android.util.Log;
  
  import android.view.Display;
  
  import android.view.KeyEvent;
  
  import android.view.LayoutInflater;
  
  import android.view.SurfaceHolder;
  
  import android.view.SurfaceView;
  
  import android.view.View;
  
  import android.view.WindowManager;
  
  import android.widget.Button;
  
  import android.widget.LinearLayout;
  
  import android.widget.RelativeLayout;
  
  import android.widget.Toast;
  
  
  
  import org.bytedeco.javacpp.avutil;
  
  import org.bytedeco.javacpp.opencv_core;
  
  import org.bytedeco.javacv.FFmpegFrameFilter;
  
  import org.bytedeco.javacv.FFmpegFrameRecorder;
  
  import org.bytedeco.javacv.Frame;
  
  import org.bytedeco.javacv.FrameFilter;
  
  import org.bytedeco.javacv.OpenCVFrameConverter;
  
  import org.opencv.android.BaseLoaderCallback;
  
  import org.opencv.android.CameraBridgeViewBase;
  
  import org.opencv.android.LoaderCallbackInterface;
  
  import org.opencv.android.OpenCVLoader;
  
  import org.opencv.core.Mat;
  
  import org.opencv.core.Scalar;
  
  
  
  import java.io.File;
  
  import java.io.IOException;
  
  import java.nio.ByteBuffer;
  
  import java.nio.ShortBuffer;
  
  import java.util.Collections;
  
  import java.util.Comparator;
  
  import java.util.List;
  
  
  
  public class OpenCVActivity extends AppCompatActivity implements CameraBridgeViewBase.CvCameraViewListener2, View.OnClickListener {
  
  
  
      private String ffmpeg_link = Environment.getExternalStorageDirectory().getAbsolutePath() + "/test2Video.mp4";
  
      private String LOG_TAG = "VideoTest";
  
  
  
      private static final String TAG = "OCVSample::Activity";
  
      private CameraBridgeViewBase _cameraBridgeViewBase;
  
      private Button recordButton;
  
  
  
      long startTime = 0;
  
      boolean recording = false;
  
  
  
      private FFmpegFrameRecorder recorder;
  
  
  
      private boolean isPreviewOn = false;
  
  
  
      /*Filter information, change boolean to true if adding a fitler*/
  
      private boolean addFilter = true;
  
      private String filterString = "";
  
      FFmpegFrameFilter filter;
  
  
  
      private int sampleAudioRateInHz = 44100;
  
      private int imageWidth = 320;
  
      private int imageHeight = 240;
  
      private int frameRate = 30;
  
  
  
      /* audio data getting thread */
  
      private AudioRecord audioRecord;
  
      private AudioRecordRunnable audioRecordRunnable;
  
      private Thread audioThread;
  
      volatile boolean runAudioThread = true;
  
  
  
      /* video data getting thread */
  
      private Camera cameraDevice;
  
      private CameraView cameraView;
  
  
  
      private Frame yuvImage = null;
  
  
  
      /* layout setting */
  
      private final int bg_screen_bx = 232;
  
      private final int bg_screen_by = 128;
  
      private final int bg_screen_width = 700;
  
      private final int bg_screen_height = 500;
  
      private final int bg_width = 1123;
  
      private final int bg_height = 715;
  
      private final int live_width = 640;
  
      private final int live_height = 480;
  
      private int screenWidth, screenHeight;
  
      private Button btnRecorderControl;
  
  
  
      /* The number of seconds in the continuous record loop (or 0 to disable loop). */
  
      final int RECORD_LENGTH = 0;
  
      Frame[] images;
  
      long[] timestamps;
  
      ShortBuffer[] samples;
  
      int imagesIndex, samplesIndex;
  
  
  
      private BaseLoaderCallback _baseLoaderCallback = new BaseLoaderCallback(this) {
  
          @Override
  
          public void onManagerConnected(int status) {
  
              switch (status) {
  
                  case LoaderCallbackInterface.SUCCESS: {
  
                      Log.i(TAG, "OpenCV loaded successfully");
  
                      // Load ndk built module, as specified in moduleName in build.gradle
  
                      // after opencv initialization
  
                      System.loadLibrary("native-lib");
  
                      _cameraBridgeViewBase.enableView();
  
                  }
  
                  break;
  
                  default: {
  
                      super.onManagerConnected(status);
  
                  }
  
              }
  
          }
  
      };
  
  
  
      @Override
  
      public void onCreate(Bundle savedInstanceState) {
  
          super.onCreate(savedInstanceState);
  
  
  
          Display display = ((WindowManager) getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay();
  
          screenWidth = display.getWidth();
  
          screenHeight = display.getHeight();
  
  
  
          getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
  
          setContentView(R.layout.activity_main);
  
  
  
          // Permissions for Android 6+
  
          ActivityCompat.requestPermissions(OpenCVActivity.this,
  
                  new String[]{Manifest.permission.CAMERA},
  
                  1);
  
  
  
          _cameraBridgeViewBase = (CameraBridgeViewBase) findViewById(R.id.main_surface);
  
          _cameraBridgeViewBase.setVisibility(SurfaceView.VISIBLE);
  
          _cameraBridgeViewBase.setCvCameraViewListener(this);
  
  
  
          RelativeLayout.LayoutParams layoutParam = null;
  
          LayoutInflater myInflate = null;
  
          myInflate = (LayoutInflater) getSystemService(Context.LAYOUT_INFLATER_SERVICE);
  
          RelativeLayout topLayout = new RelativeLayout(this);
  
          setContentView(topLayout);
  
          LinearLayout preViewLayout = (LinearLayout) myInflate.inflate(R.layout.activity_main, null);
  
          layoutParam = new RelativeLayout.LayoutParams(screenWidth, screenHeight);
  
          topLayout.addView(_cameraBridgeViewBase, layoutParam);
  
  
  
          recordButton = (Button) findViewById(R.id.recorder_control);
  
          recordButton.setText("Start");
  
          recordButton.setOnClickListener(this);
  
  
  
          /* add camera view */
  
          int display_width_d = (int) (1.0 * bg_screen_width * screenWidth / bg_width);
  
          int display_height_d = (int) (1.0 * bg_screen_height * screenHeight / bg_height);
  
          int prev_rw, prev_rh;
  
          if (1.0 * display_width_d / display_height_d > 1.0 * live_width / live_height) {
  
              prev_rh = display_height_d;
  
              prev_rw = (int) (1.0 * display_height_d * live_width / live_height);
  
          } else {
  
              prev_rw = display_width_d;
  
              prev_rh = (int) (1.0 * display_width_d * live_height / live_width);
  
          }
  
          layoutParam = new RelativeLayout.LayoutParams(prev_rw, prev_rh);
  
          layoutParam.topMargin = (int) (1.0 * bg_screen_by * screenHeight / bg_height);
  
          layoutParam.leftMargin = (int) (1.0 * bg_screen_bx * screenWidth / bg_width);
  
  
  
          cameraDevice = Camera.open();
  
          cameraView = new CameraView(this, cameraDevice);
  
          topLayout.addView(cameraView, layoutParam);
  
      }
  
  
  
      //---------------------------------------
  
      // initialize ffmpeg_recorder
  
      //---------------------------------------
  
      private void initRecorder() {
  
  
  
          Log.w(LOG_TAG,"init recorder");
  
  
  
          if (RECORD_LENGTH > 0) {
  
              imagesIndex = 0;
  
              images = new Frame[RECORD_LENGTH * frameRate];
  
              timestamps = new long[images.length];
  
              for (int i = 0; i < images.length; i++) {
  
                  images[i] = new Frame(imageWidth, imageHeight, Frame.DEPTH_UBYTE, 2);
  
                  timestamps[i] = -1;
  
              }
  
          } else if (yuvImage == null) {
  
              yuvImage = new Frame(imageWidth, imageHeight, Frame.DEPTH_UBYTE, 2);
  
              Log.i(LOG_TAG, "create yuvImage");
  
          }
  
  
  
          Log.i(LOG_TAG, "ffmpeg_url: " + ffmpeg_link);
  
          recorder = new FFmpegFrameRecorder(ffmpeg_link, imageWidth, imageHeight, 1);
  
          recorder.setFormat("mp4");
  
          recorder.setSampleRate(sampleAudioRateInHz);
  
          // Set in the surface changed method
  
          recorder.setFrameRate(frameRate);
  
  
  
          // The filterString  is any ffmpeg filter.
  
          // Here is the link for a list: https://ffmpeg.org/ffmpeg-filters.html
  
          filterString = "transpose=0";
  
          filter = new FFmpegFrameFilter(filterString, imageWidth, imageHeight);
  
  
  
          //default format on android
  
          filter.setPixelFormat(avutil.AV_PIX_FMT_NV21);
  
  
  
          Log.i(LOG_TAG, "recorder initialize success");
  
  
  
          audioRecordRunnable = new AudioRecordRunnable();
  
          audioThread = new Thread(audioRecordRunnable);
  
          runAudioThread = true;
  
      }
  
  
  
      public void startRecording() {
  
  
  
          initRecorder();
  
  
  
          try {
  
              recorder.start();
  
              startTime = System.currentTimeMillis();
  
              recording = true;
  
              audioThread.start();
  
  
  
              if(addFilter) {
  
                  filter.start();
  
              }
  
  
  
          } catch (FFmpegFrameRecorder.Exception | FrameFilter.Exception e) {
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      public void stopRecording() {
  
  
  
          runAudioThread = false;
  
          try {
  
              audioThread.join();
  
          } catch (InterruptedException e) {
  
              // reset interrupt to be nice
  
              Thread.currentThread().interrupt();
  
              return;
  
          }
  
          audioRecordRunnable = null;
  
          audioThread = null;
  
  
  
          if (recorder != null && recording) {
  
              if (RECORD_LENGTH > 0) {
  
                  Log.v(LOG_TAG,"Writing frames");
  
                  try {
  
                      int firstIndex = imagesIndex % samples.length;
  
                      int lastIndex = (imagesIndex - 1) % images.length;
  
                      if (imagesIndex <= images.length) {
  
                          firstIndex = 0;
  
                          lastIndex = imagesIndex - 1;
  
                      }
  
                      if ((startTime = timestamps[lastIndex] - RECORD_LENGTH * 1000000L) < 0) {
  
                          startTime = 0;
  
                      }
  
                      if (lastIndex < firstIndex) {
  
                          lastIndex += images.length;
  
                      }
  
                      for (int i = firstIndex; i <= lastIndex; i++) {
  
                          long t = timestamps[i % timestamps.length] - startTime;
  
                          if (t >= 0) {
  
                              if (t > recorder.getTimestamp()) {
  
                                  recorder.setTimestamp(t);
  
                              }
  
                              recorder.record(images[i % images.length]);
  
                          }
  
                      }
  
  
  
                      firstIndex = samplesIndex % samples.length;
  
                      lastIndex = (samplesIndex - 1) % samples.length;
  
                      if (samplesIndex <= samples.length) {
  
                          firstIndex = 0;
  
                          lastIndex = samplesIndex - 1;
  
                      }
  
                      if (lastIndex < firstIndex) {
  
                          lastIndex += samples.length;
  
                      }
  
                      for (int i = firstIndex; i <= lastIndex; i++) {
  
                          recorder.recordSamples(samples[i % samples.length]);
  
                      }
  
                  } catch (FFmpegFrameRecorder.Exception e) {
  
                      Log.v(LOG_TAG,e.getMessage());
  
                      e.printStackTrace();
  
                  }
  
              }
  
  
  
              recording = false;
  
              Log.v(LOG_TAG,"Finishing recording, calling stop and release on recorder");
  
              try {
  
                  recorder.stop();
  
                  recorder.release();
  
                  filter.stop();
  
                  filter.release();
  
              } catch (FFmpegFrameRecorder.Exception | FrameFilter.Exception e) {
  
                  e.printStackTrace();
  
              }
  
              recorder = null;
  
  
  
          }
  
      }
  
  
  
      @Override
  
      public boolean onKeyDown(int keyCode, KeyEvent event) {
  
  
  
          if (keyCode == KeyEvent.KEYCODE_BACK) {
  
              if (recording) {
  
                  stopRecording();
  
              }
  
  
  
              finish();
  
  
  
              return true;
  
          }
  
  
  
          return super.onKeyDown(keyCode, event);
  
      }
  
  
  
      //---------------------------------------------
  
      // audio thread, gets and encodes audio data
  
      //---------------------------------------------
  
      class AudioRecordRunnable implements Runnable {
  
  
  
          @Override
  
          public void run() {
  
              android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
  
  
  
              // Audio
  
              int bufferSize;
  
              ShortBuffer audioData;
  
              int bufferReadResult;
  
  
  
              bufferSize = AudioRecord.getMinBufferSize(sampleAudioRateInHz,
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
  
              audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleAudioRateInHz,
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
  
  
  
              if (RECORD_LENGTH > 0) {
  
                  samplesIndex = 0;
  
                  samples = new ShortBuffer[RECORD_LENGTH * sampleAudioRateInHz * 2 / bufferSize + 1];
  
                  for (int i = 0; i < samples.length; i++) {
  
                      samples[i] = ShortBuffer.allocate(bufferSize);
  
                  }
  
              } else {
  
                  audioData = ShortBuffer.allocate(bufferSize);
  
              }
  
  
  
              Log.d(LOG_TAG, "audioRecord.startRecording()");
  
              audioRecord.startRecording();
  
  
  
              /* ffmpeg_audio encoding loop */
  
              while (runAudioThread) {
  
                  if (RECORD_LENGTH > 0) {
  
                      audioData = samples[samplesIndex++ % samples.length];
  
                      audioData.position(0).limit(0);
  
                  }
  
                  //Log.v(LOG_TAG,"recording? " + recording);
  
                  bufferReadResult = audioRecord.read(audioData.array(), 0, audioData.capacity());
  
                  audioData.limit(bufferReadResult);
  
                  if (bufferReadResult > 0) {
  
                      Log.v(LOG_TAG,"bufferReadResult: " + bufferReadResult);
  
                      // If "recording" isn't true when start this thread, it never get's set according to this if statement...!!!
  
                      // Why?  Good question...
  
                      if (recording) {
  
                          if (RECORD_LENGTH <= 0) try {
  
                              recorder.recordSamples(audioData);
  
                              //Log.v(LOG_TAG,"recording " + 1024*i + " to " + 1024*i+1024);
  
                          } catch (FFmpegFrameRecorder.Exception e) {
  
                              Log.v(LOG_TAG,e.getMessage());
  
                              e.printStackTrace();
  
                          }
  
                      }
  
                  }
  
              }
  
              Log.v(LOG_TAG,"AudioThread Finished, release audioRecord");
  
  
  
              /* encoding finish, release recorder */
  
              if (audioRecord != null) {
  
                  audioRecord.stop();
  
                  audioRecord.release();
  
                  audioRecord = null;
  
                  Log.v(LOG_TAG,"audioRecord released");
  
              }
  
          }
  
      }
  
  
  
      //---------------------------------------------
  
      // camera thread, gets and encodes video data
  
      //---------------------------------------------
  
      class CameraView extends SurfaceView implements SurfaceHolder.Callback, Camera.PreviewCallback {
  
  
  
          private SurfaceHolder mHolder;
  
          private Camera mCamera;
  
  
  
          public CameraView(Context context, Camera camera) {
  
              super(context);
  
              Log.w("camera","camera view");
  
              mCamera = camera;
  
              mHolder = getHolder();
  
              mHolder.addCallback(CameraView.this);
  
              mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
  
              mCamera.setPreviewCallback(CameraView.this);
  
          }
  
  
  
          @Override
  
          public void surfaceCreated(SurfaceHolder holder) {
  
              try {
  
                  stopPreview();
  
                  mCamera.setPreviewDisplay(holder);
  
              } catch (IOException exception) {
  
                  mCamera.release();
  
                  mCamera = null;
  
              }
  
          }
  
  
  
          public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
  
              stopPreview();
  
  
  
              Camera.Parameters camParams = mCamera.getParameters();
  
              List sizes = camParams.getSupportedPreviewSizes();
  
              // Sort the list in ascending order
  
              Collections.sort(sizes, new Comparator() {
  
  
  
                  public int compare(final Camera.Size a, final Camera.Size b) {
  
                      return a.width * a.height - b.width * b.height;
  
                  }
  
              });
  
  
  
              // Pick the first preview size that is equal or bigger, or pick the last (biggest) option if we cannot
  
              // reach the initial settings of imageWidth/imageHeight.
  
              for (int i = 0; i < sizes.size(); i++) {
  
                  if ((sizes.get(i).width >= imageWidth && sizes.get(i).height >= imageHeight) || i == sizes.size() - 1) {
  
                      imageWidth = sizes.get(i).width;
  
                      imageHeight = sizes.get(i).height;
  
                      Log.v(LOG_TAG, "Changed to supported resolution: " + imageWidth + "x" + imageHeight);
  
                      break;
  
                  }
  
              }
  
              camParams.setPreviewSize(imageWidth, imageHeight);
  
  
  
              Log.v(LOG_TAG,"Setting imageWidth: " + imageWidth + " imageHeight: " + imageHeight + " frameRate: " + frameRate);
  
  
  
              camParams.setPreviewFrameRate(frameRate);
  
              Log.v(LOG_TAG,"Preview Framerate: " + camParams.getPreviewFrameRate());
  
  
  
              mCamera.setParameters(camParams);
  
  
  
              // Set the holder (which might have changed) again
  
              try {
  
                  mCamera.setPreviewDisplay(holder);
  
                  mCamera.setPreviewCallback(CameraView.this);
  
                  startPreview();
  
              } catch (Exception e) {
  
                  Log.e(LOG_TAG, "Could not set preview display in surfaceChanged");
  
              }
  
          }
  
  
  
          @Override
  
          public void surfaceDestroyed(SurfaceHolder holder) {
  
              try {
  
                  mHolder.addCallback(null);
  
                  mCamera.setPreviewCallback(null);
  
              } catch (RuntimeException e) {
  
                  // The camera has probably just been released, ignore.
  
              }
  
          }
  
  
  
          public void startPreview() {
  
              if (!isPreviewOn && mCamera != null) {
  
                  isPreviewOn = true;
  
                  mCamera.startPreview();
  
              }
  
          }
  
  
  
          public void stopPreview() {
  
              if (isPreviewOn && mCamera != null) {
  
                  isPreviewOn = false;
  
                  mCamera.stopPreview();
  
              }
  
          }
  
  
  
          @Override
  
          public void onPreviewFrame(byte[] data, Camera camera) {
  
              if (audioRecord == null || audioRecord.getRecordingState() != AudioRecord.RECORDSTATE_RECORDING) {
  
                  startTime = System.currentTimeMillis();
  
                  return;
  
              }
  
              if (RECORD_LENGTH > 0) {
  
                  int i = imagesIndex++ % images.length;
  
                  yuvImage = images[i];
  
                  timestamps[i] = 1000 * (System.currentTimeMillis() - startTime);
  
              }
  
  
  
  
  
              /* get video data */
  
              if (yuvImage != null && recording) {
  
                  ((ByteBuffer)yuvImage.image[0].position(0)).put(data);
  
  
  
                  if (RECORD_LENGTH <= 0) try {
  
                      Log.v(LOG_TAG,"Writing Frame");
  
                      long t = 1000 * (System.currentTimeMillis() - startTime);
  
                      if (t > recorder.getTimestamp()) {
  
                          recorder.setTimestamp(t);
  
                      }
  
  
  
                      if(addFilter) {
  
                          filter.push(yuvImage);
  
                          Frame frame2;
  
                          while ((frame2 = filter.pull()) != null) {
  
                              recorder.record(frame2, filter.getPixelFormat());
  
                          }
  
                      } else {
  
                          recorder.record(yuvImage);
  
                      }
  
                  } catch (FFmpegFrameRecorder.Exception | FrameFilter.Exception e) {
  
                      Log.v(LOG_TAG,e.getMessage());
  
                      e.printStackTrace();
  
                  }
  
              }
  
          }
  
      }
  
  
  
      @Override
  
      public void onClick(View v) {
  
          if (!recording) {
  
              startRecording();
  
              Log.w(LOG_TAG, "Start Button Pushed");
  
              recordButton.setText("Stop");
  
          } else {
  
              // This will trigger the audio recording loop to stop and then set isRecorderStart = false;
  
              stopRecording();
  
              Log.w(LOG_TAG, "Stop Button Pushed");
  
              recordButton.setText("Start");
  
          }
  
      }
  
  
  
      @Override
  
      public void onPause() {
  
          super.onPause();
  
          disableCamera();
  
      }
  
  
  
      @Override
  
      public void onResume() {
  
          super.onResume();
  
          if (!OpenCVLoader.initDebug()) {
  
              Log.d(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");
  
              OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_0_0, this, _baseLoaderCallback);
  
          } else {
  
              Log.d(TAG, "OpenCV library found inside package. Using it!");
  
              _baseLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
  
          }
  
      }
  
  
  
      @Override
  
      public void onRequestPermissionsResult(int requestCode, String permissions[], int[] grantResults) {
  
          switch (requestCode) {
  
              case 1: {
  
                  // If request is cancelled, the result arrays are empty.
  
                  if (grantResults.length > 0
  
                          && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
  
                      // permission was granted, yay! Do the
  
                      // contacts-related task you need to do.
  
                  } else {
  
                      // permission denied, boo! Disable the
  
                      // functionality that depends on this permission.
  
                      Toast.makeText(OpenCVActivity.this, "Permission denied to read your External storage", Toast.LENGTH_SHORT).show();
  
                  }
  
                  return;
  
              }
  
              // other 'case' lines to check for other
  
              // permissions this app might request
  
          }
  
      }
  
  
  
      public void onDestroy() {
  
          super.onDestroy();
  
          disableCamera();
  
      }
  
  
  
      public void disableCamera() {
  
          if (_cameraBridgeViewBase != null)
  
              _cameraBridgeViewBase.disableView();
  
      }
  
  
  
      public void onCameraViewStarted(int width, int height) {
  
          setup();
  
      }
  
  
  
      public void onCameraViewStopped() {
  
      }
  
  
  
      public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
  
          Mat rgba = inputFrame.rgba();
  
          salt(rgba.getNativeObjAddr());
  
  
  
          return rgba;
  
      }
  
  
  
  
  
      public native void salt(long matAddrGray);
  
      public native void setup();
  
  }

  activity_main.xml :

  <?xml version="1.0" encoding="utf-8"?>
  
  <linearlayout>
  
  
  
      
  
  
  
      <button></button>
  
  
  
  </linearlayout>

  Could really use some help with this. Has anyone done this before ?