Recherche avancée

Sur d’autres sites (2281)

• Play video using mse (media source extension) in google chrome

  23 août 2019, par liyuqihxc

  I’m working on a project that convert rtsp stream (ffmpeg) and play it on the web page (signalr + mse).

  So far it works pretty much as I expected on the latest version of edge and firefox, but not chrome.

  here’s the code

  public class WebmMediaStreamContext
  
  {
  
      private Process _ffProcess;
  
      private readonly string _cmd;
  
      private byte[] _initSegment;
  
      private Task _readMediaStreamTask;
  
      private CancellationTokenSource _cancellationTokenSource;
  
  
  
      private const string _CmdTemplate = "-i {0} -c:v libvpx -tile-columns 4 -frame-parallel 1 -keyint_min 90 -g 90 -f webm -dash 1 pipe:";
  
  
  
      public static readonly byte[] ClusterStart = { 0x1F, 0x43, 0xB6, 0x75, 0x01, 0x00, 0x00, 0x00 };
  
  
  
      public event EventHandler<clusterreadyeventargs> ClusterReadyEvent;
  
  
  
      public WebmMediaStreamContext(string rtspFeed)
  
      {
  
          _cmd = string.Format(_CmdTemplate, rtspFeed);
  
      }
  
  
  
      public async Task StartConverting()
  
      {
  
          if (_ffProcess != null)
  
              throw new InvalidOperationException();
  
  
  
          _ffProcess = new Process();
  
          _ffProcess.StartInfo = new ProcessStartInfo
  
          {
  
              FileName = "ffmpeg/ffmpeg.exe",
  
              Arguments = _cmd,
  
              UseShellExecute = false,
  
              CreateNoWindow = true,
  
              RedirectStandardOutput = true
  
          };
  
          _ffProcess.Start();
  
  
  
          _initSegment = await ParseInitSegmentAndStartReadMediaStream();
  
      }
  
  
  
      public byte[] GetInitSegment()
  
      {
  
          return _initSegment;
  
      }
  
  
  
      // Find the first cluster， and everything before it is the InitSegment
  
      private async Task ParseInitSegmentAndStartReadMediaStream()
  
      {
  
          Memory<byte> buffer = new byte[10 * 1024];
  
          int length = 0;
  
          while (length != buffer.Length)
  
          {
  
              length += await _ffProcess.StandardOutput.BaseStream.ReadAsync(buffer.Slice(length));
  
              int cluster = buffer.Span.IndexOf(ClusterStart);
  
              if (cluster >= 0)
  
              {
  
                  _cancellationTokenSource = new CancellationTokenSource();
  
                  _readMediaStreamTask = new Task(() => ReadMediaStreamProc(buffer.Slice(cluster, length - cluster).ToArray(), _cancellationTokenSource.Token), _cancellationTokenSource.Token, TaskCreationOptions.LongRunning);
  
                  _readMediaStreamTask.Start();
  
                  return buffer.Slice(0, cluster).ToArray();
  
              }
  
          }
  
  
  
          throw new InvalidOperationException();
  
      }
  
  
  
      private void ReadMoreBytes(Span<byte> buffer)
  
      {
  
          int size = buffer.Length;
  
          while (size > 0)
  
          {
  
              int len = _ffProcess.StandardOutput.BaseStream.Read(buffer.Slice(buffer.Length - size));
  
              size -= len;
  
          }
  
      }
  
  
  
      // Parse every single cluster and fire ClusterReadyEvent
  
      private void ReadMediaStreamProc(byte[] bytesRead, CancellationToken cancel)
  
      {
  
          Span<byte> buffer = new byte[5 * 1024 * 1024];
  
          bytesRead.CopyTo(buffer);
  
          int bufferEmptyIndex = bytesRead.Length;
  
  
  
          do
  
          {
  
              if (bufferEmptyIndex &lt; ClusterStart.Length + 4)
  
              {
  
                  ReadMoreBytes(buffer.Slice(bufferEmptyIndex, 1024));
  
                  bufferEmptyIndex += 1024;
  
              }
  
  
  
              int clusterDataSize = BitConverter.ToInt32(
  
                  buffer.Slice(ClusterStart.Length, 4)
  
                  .ToArray()
  
                  .Reverse()
  
                  .ToArray()
  
              );
  
              int clusterSize = ClusterStart.Length + 4 + clusterDataSize;
  
              if (clusterSize > buffer.Length)
  
              {
  
                  byte[] newBuffer = new byte[clusterSize];
  
                  buffer.Slice(0, bufferEmptyIndex).CopyTo(newBuffer);
  
                  buffer = newBuffer;
  
              }
  
  
  
              if (bufferEmptyIndex &lt; clusterSize)
  
              {
  
                  ReadMoreBytes(buffer.Slice(bufferEmptyIndex, clusterSize - bufferEmptyIndex));
  
                  bufferEmptyIndex = clusterSize;
  
              }
  
  
  
              ClusterReadyEvent?.Invoke(this, new ClusterReadyEventArgs(buffer.Slice(0, bufferEmptyIndex).ToArray()));
  
  
  
              bufferEmptyIndex = 0;
  
          } while (!cancel.IsCancellationRequested);
  
      }
  
  }
  
  </byte></byte></byte></clusterreadyeventargs>

  I use ffmpeg to convert the rtsp stream to vp8 WEBM byte stream and parse it to "Init Segment" (ebml head、info、tracks...) and "Media Segment" (cluster)， then send it to browser via signalR

  $(function () {
  
  
  
      var mediaSource = new MediaSource();
  
      var mimeCodec = 'video/webm; codecs="vp8"';
  
  
  
      var video = document.getElementById('video');
  
  
  
      mediaSource.addEventListener('sourceopen', callback, false);
  
      function callback(e) {
  
          var sourceBuffer = mediaSource.addSourceBuffer(mimeCodec);
  
          var queue = [];
  
  
  
          sourceBuffer.addEventListener('updateend', function () {
  
              if (queue.length === 0) {
  
                  return;
  
              }
  
  
  
              var base64 = queue[0];
  
              if (base64.length === 0) {
  
                  mediaSource.endOfStream();
  
                  queue.shift();
  
                  return;
  
              } else {
  
                  var buffer = new Uint8Array(atob(base64).split("").map(function (c) {
  
                      return c.charCodeAt(0);
  
                  }));
  
                  sourceBuffer.appendBuffer(buffer);
  
                  queue.shift();
  
              }
  
          }, false);
  
  
  
          var connection = new signalR.HubConnectionBuilder()
  
              .withUrl("/signalr-video")
  
              .configureLogging(signalR.LogLevel.Information)
  
              .build();
  
          connection.start().then(function () {
  
              connection.stream("InitVideoReceive")
  
                  .subscribe({
  
                      next: function(item) {
  
                          if (queue.length === 0 && !!!sourceBuffer.updating) {
  
                              var buffer = new Uint8Array(atob(item).split("").map(function (c) {
  
                                  return c.charCodeAt(0);
  
                              }));
  
                              sourceBuffer.appendBuffer(buffer);
  
                              console.log(blockindex++ + " : " + buffer.byteLength);
  
                          } else {
  
                              queue.push(item);
  
                          }
  
                      },
  
                      complete: function () {
  
                          queue.push('');
  
                      },
  
                      error: function (err) {
  
                          console.error(err);
  
                      }
  
                  });
  
          });
  
      }
  
      video.src = window.URL.createObjectURL(mediaSource);
  
  })

  chrome just play the video for 3 5 seconds and then stop for buffering, even though there are plenty of cluster transfered and inserted into SourceBuffer.

  here’s the information in chrome ://media-internals/

  Player Properties :

  render_id: 217
  
  player_id: 1
  
  origin_url: http://localhost:52531/
  
  frame_url: http://localhost:52531/
  
  frame_title: Home Page
  
  url: blob:http://localhost:52531/dcb25d89-9830-40a5-ba88-33c13b5c03eb
  
  info: Selected FFmpegVideoDecoder for video decoding, config: codec: vp8 format: 1 profile: vp8 coded size: [1280,720] visible rect: [0,0,1280,720] natural size: [1280,720] has extra data? false encryption scheme: Unencrypted rotation: 0°
  
  pipeline_state: kSuspended
  
  found_video_stream: true
  
  video_codec_name: vp8
  
  video_dds: false
  
  video_decoder: FFmpegVideoDecoder
  
  duration: unknown
  
  height: 720
  
  width: 1280
  
  video_buffering_state: BUFFERING_HAVE_NOTHING
  
  for_suspended_start: false
  
  pipeline_buffering_state: BUFFERING_HAVE_NOTHING
  
  event: PAUSE

  Log

  Timestamp       Property            Value
  
  00:00:00 00     origin_url          http://localhost:52531/
  
  00:00:00 00     frame_url           http://localhost:52531/
  
  00:00:00 00     frame_title         Home Page
  
  00:00:00 00     url                 blob:http://localhost:52531/dcb25d89-9830-40a5-ba88-33c13b5c03eb
  
  00:00:00 00     info                ChunkDemuxer: buffering by DTS
  
  00:00:00 35     pipeline_state      kStarting
  
  00:00:15 213    found_video_stream  true
  
  00:00:15 213    video_codec_name    vp8
  
  00:00:15 216    video_dds           false
  
  00:00:15 216    video_decoder       FFmpegVideoDecoder
  
  00:00:15 216    info                Selected FFmpegVideoDecoder for video decoding, config: codec: vp8 format: 1 profile: vp8 coded size: [1280,720] visible rect: [0,0,1280,720] natural size: [1280,720] has extra data? false encryption scheme: Unencrypted rotation: 0°
  
  00:00:15 216    pipeline_state      kPlaying
  
  00:00:15 213    duration            unknown
  
  00:00:16 661    height              720
  
  00:00:16 661    width               1280
  
  00:00:16 665    video_buffering_state       BUFFERING_HAVE_ENOUGH
  
  00:00:16 665    for_suspended_start         false
  
  00:00:16 665    pipeline_buffering_state    BUFFERING_HAVE_ENOUGH
  
  00:00:16 667    pipeline_state      kSuspending
  
  00:00:16 670    pipeline_state      kSuspended
  
  00:00:52 759    info                Effective playback rate changed from 0 to 1
  
  00:00:52 759    event               PLAY
  
  00:00:52 759    pipeline_state      kResuming
  
  00:00:52 760    video_dds           false
  
  00:00:52 760    video_decoder       FFmpegVideoDecoder
  
  00:00:52 760    info                Selected FFmpegVideoDecoder for video decoding, config: codec: vp8 format: 1 profile: vp8 coded size: [1280,720] visible rect: [0,0,1280,720] natural size: [1280,720] has extra data? false encryption scheme: Unencrypted rotation: 0°
  
  00:00:52 760    pipeline_state      kPlaying
  
  00:00:52 793    height              720
  
  00:00:52 793    width               1280
  
  00:00:52 798    video_buffering_state       BUFFERING_HAVE_ENOUGH
  
  00:00:52 798    for_suspended_start         false
  
  00:00:52 798    pipeline_buffering_state    BUFFERING_HAVE_ENOUGH
  
  00:00:56 278    video_buffering_state       BUFFERING_HAVE_NOTHING
  
  00:00:56 295    for_suspended_start         false
  
  00:00:56 295    pipeline_buffering_state    BUFFERING_HAVE_NOTHING
  
  00:01:20 717    event               PAUSE
  
  00:01:33 538    event               PLAY
  
  00:01:35 94     event               PAUSE
  
  00:01:55 561    pipeline_state      kSuspending
  
  00:01:55 563    pipeline_state      kSuspended

  Can someone tell me what’s wrong with my code, or dose chrome require some magic configuration to work ?

  Thanks 

  Please excuse my english :)

• Android studio + OpenCV + FFmpeg

  11 juillet 2018, par t0m

  I have problem with code, which is functional only for Genymotion device (Android 4.1.1), but for Genymotion device 5.0.1 and real device Huawei honor 4c Android 4.4.2 not.

  I have imported OpenCV 3.1 to Android studio by : https://stackoverflow.com/a/27421494/4244605
  I added JavaCV with FFmpeg by : https://github.com/bytedeco/javacv

  Android studio 1.5.1
  minSdkVersion 15
  compileSdkVersion 23

  Code is only for test.
  OpenCVCameraActivity.java :

  import android.app.Activity;
  
  import android.hardware.Camera;
  
  import android.media.AudioFormat;
  
  import android.media.AudioRecord;
  
  import android.media.MediaRecorder;
  
  import android.os.Bundle;
  
  import android.os.Environment;
  
  import android.util.Log;
  
  import android.view.Menu;
  
  import android.view.MenuItem;
  
  import android.view.MotionEvent;
  
  import android.view.SubMenu;
  
  import android.view.SurfaceView;
  
  import android.view.View;
  
  import android.view.WindowManager;
  
  import android.widget.Toast;
  
  
  
  import org.bytedeco.javacv.FFmpegFrameRecorder;
  
  import org.bytedeco.javacv.Frame;
  
  import org.opencv.android.BaseLoaderCallback;
  
  import org.opencv.android.CameraBridgeViewBase;
  
  import org.opencv.android.LoaderCallbackInterface;
  
  import org.opencv.android.OpenCVLoader;
  
  import org.opencv.core.Mat;
  
  
  
  import java.io.File;
  
  import java.nio.ShortBuffer;
  
  import java.text.SimpleDateFormat;
  
  import java.util.Date;
  
  import java.util.List;
  
  import java.util.ListIterator;
  
  
  
  @SuppressWarnings("ALL")
  
  public class OpenCVCameraActivity extends Activity implements
  
          CameraBridgeViewBase.CvCameraViewListener2,
  
          View.OnTouchListener {
  
  
  
      //name of activity, for DEBUGGING
  
      private static final String TAG = OpenCVCameraActivity.class.getSimpleName();
  
  
  
      private OpenCVCameraPreview mOpenCvCameraView;
  
      private List mResolutionList;
  
      private MenuItem[] mEffectMenuItems;
  
      private SubMenu mColorEffectsMenu;
  
      private MenuItem[] mResolutionMenuItems;
  
      private SubMenu mResolutionMenu;
  
  
  
      private static long frameCounter = 0;
  
  
  
      long startTime = 0;
  
      private Mat edgesMat;
  
      boolean recording = false;
  
      private int sampleAudioRateInHz = 44100;
  
      private int imageWidth = 1920;
  
      private int imageHeight = 1080;
  
      private int frameRate = 30;
  
      private Frame yuvImage = null;
  
      private File ffmpeg_link;
  
      private FFmpegFrameRecorder recorder;
  
  
  
      /* audio data getting thread */
  
      private AudioRecord audioRecord;
  
      private AudioRecordRunnable audioRecordRunnable;
  
      private Thread audioThread;
  
      volatile boolean runAudioThread = true;
  
      ShortBuffer[] samples;
  
  
  
  
  
      private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
  
          @Override
  
          public void onManagerConnected(int status) {
  
              switch (status) {
  
                  case LoaderCallbackInterface.SUCCESS:
  
                      Log.i(TAG, "OpenCV loaded successfully");
  
                      mOpenCvCameraView.enableView();
  
                      mOpenCvCameraView.setOnTouchListener(OpenCVCameraActivity.this);
  
                  break;
  
                  default:
  
                      super.onManagerConnected(status);
  
                  break;
  
              }
  
          }
  
      };
  
  
  
      @Override
  
      protected void onCreate(Bundle savedInstanceState) {
  
          super.onCreate(savedInstanceState);
  
          if(Static.DEBUG) Log.i(TAG, "onCreate()");
  
  
  
          getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
  
  
  
          try {
  
              setContentView(R.layout.activity_opencv);
  
  
  
              mOpenCvCameraView = (OpenCVCameraPreview) findViewById(R.id.openCVCameraPreview);
  
              mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
  
              mOpenCvCameraView.setCvCameraViewListener(this);
  
  
  
              //mOpenCvCameraView.enableFpsMeter();
  
  
  
              ffmpeg_link = new File(Environment.getExternalStorageDirectory(), "stream.mp4");
  
          } catch (Exception e){
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      @Override
  
      protected void onRestart() {
  
          if (Static.DEBUG) Log.i(TAG, "onRestart()");
  
          super.onRestart();
  
      }
  
  
  
      @Override
  
      protected void onStart() {
  
          if (Static.DEBUG) Log.i(TAG, "onStart()");
  
          super.onStart();
  
      }
  
  
  
      @Override
  
      protected void onResume() {
  
          if (Static.DEBUG) Log.i(TAG, "onResume()");
  
          super.onResume();
  
  
  
          if (!OpenCVLoader.initDebug()) {
  
              Log.i(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");
  
              OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_11, this, mLoaderCallback);
  
          } else {
  
              Log.i(TAG, "OpenCV library found inside package. Using it!");
  
              mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
  
          }
  
  
  
      }
  
  
  
      @Override
  
      public boolean onCreateOptionsMenu(Menu menu) {
  
          if (Static.DEBUG) Log.i(TAG, "onCreateOptionsMenu()");
  
          super.onCreateOptionsMenu(menu);
  
  
  
          List<string> effects = mOpenCvCameraView.getEffectList();
  
  
  
          if (effects == null) {
  
              Log.e(TAG, "Color effects are not supported by device!");
  
              return true;
  
          }
  
  
  
          mColorEffectsMenu = menu.addSubMenu("Color Effect");
  
          mEffectMenuItems = new MenuItem[effects.size()];
  
  
  
          int idx = 0;
  
          ListIterator<string> effectItr = effects.listIterator();
  
          while(effectItr.hasNext()) {
  
              String element = effectItr.next();
  
              mEffectMenuItems[idx] = mColorEffectsMenu.add(1, idx, Menu.NONE, element);
  
              idx++;
  
          }
  
  
  
          mResolutionMenu = menu.addSubMenu("Resolution");
  
          mResolutionList = mOpenCvCameraView.getResolutionList();
  
          mResolutionMenuItems = new MenuItem[mResolutionList.size()];
  
  
  
          ListIterator resolutionItr = mResolutionList.listIterator();
  
          idx = 0;
  
          while(resolutionItr.hasNext()) {
  
              Camera.Size element = resolutionItr.next();
  
              mResolutionMenuItems[idx] = mResolutionMenu.add(2, idx, Menu.NONE,
  
                      Integer.valueOf(element.width).toString() + "x" + Integer.valueOf(element.height).toString());
  
              idx++;
  
          }
  
  
  
          return true;
  
      }
  
  
  
      @Override
  
      protected void onPause() {
  
          if (Static.DEBUG) Log.i(TAG, "onPause()");
  
          super.onPause();
  
  
  
          if (mOpenCvCameraView != null)
  
              mOpenCvCameraView.disableView();
  
  
  
      }
  
  
  
      @Override
  
      protected void onStop() {
  
          if (Static.DEBUG) Log.i(TAG, "onStop()");
  
          super.onStop();
  
      }
  
  
  
      @Override
  
      protected void onDestroy() {
  
          if (Static.DEBUG) Log.i(TAG, "onDestroy()");
  
          super.onDestroy();
  
  
  
          if (mOpenCvCameraView != null)
  
              mOpenCvCameraView.disableView();
  
      }
  
  
  
      public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
  
  
  
          ++frameCounter;
  
          //Log.i(TAG, "Frame number: "+frameCounter);
  
  
  
          return inputFrame.rgba();
  
      }
  
  
  
  
  
      @Override
  
      public void onCameraViewStarted(int width, int height) {
  
          edgesMat = new Mat();
  
      }
  
  
  
      @Override
  
      public void onCameraViewStopped() {
  
          if (edgesMat != null)
  
              edgesMat.release();
  
  
  
          edgesMat = null;
  
      }
  
  
  
      public boolean onOptionsItemSelected(MenuItem item) {
  
          Log.i(TAG, "called onOptionsItemSelected; selected item: " + item);
  
          if (item.getGroupId() == 1)
  
          {
  
              mOpenCvCameraView.setEffect((String) item.getTitle());
  
              Toast.makeText(this, mOpenCvCameraView.getEffect(), Toast.LENGTH_SHORT).show();
  
          } else if (item.getGroupId() == 2) {
  
              int id = item.getItemId();
  
              Camera.Size resolution = mResolutionList.get(id);
  
              mOpenCvCameraView.setResolution(resolution);
  
              resolution = mOpenCvCameraView.getResolution();
  
              String caption = Integer.valueOf(resolution.width).toString() + "x" + Integer.valueOf(resolution.height).toString();
  
              Toast.makeText(this, caption, Toast.LENGTH_SHORT).show();
  
          }
  
  
  
          return true;
  
      }
  
  
  
      @Override
  
      public boolean onTouch(View v, MotionEvent event) {
  
          Log.i(TAG,"onTouch event");
  
          SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd_HH-mm-ss");
  
          String currentDateandTime = sdf.format(new Date());
  
          String fileName = Environment.getExternalStorageDirectory().getPath() +
  
                  "/sample_picture_" + currentDateandTime + ".jpg";
  
          mOpenCvCameraView.takePicture(fileName);
  
          Toast.makeText(this, fileName + " saved", Toast.LENGTH_SHORT).show();
  
          return false;
  
      }
  
  
  
      /**
  
       * Click to ImageButton to start recording.
  
       */
  
      public void onClickBtnStartRecord2(View v) {
  
          if (Static.DEBUG) Log.i(TAG, "onClickBtnStartRecord()");
  
  
  
          if(!recording)
  
              startRecording();
  
          else
  
              stopRecording();
  
      }
  
  
  
      private void startRecording() {
  
          if (Static.DEBUG) Log.i(TAG, "startRecording()");
  
          initRecorder();
  
  
  
          try {
  
              recorder.start();
  
              startTime = System.currentTimeMillis();
  
              recording = true;
  
              audioThread.start();
  
          } catch(FFmpegFrameRecorder.Exception e) {
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      private void stopRecording() {
  
          if (Static.DEBUG) Log.i(TAG, "stopRecording()");
  
  
  
          runAudioThread = false;
  
          try {
  
              audioThread.join();
  
          } catch(InterruptedException e) {
  
              e.printStackTrace();
  
          }
  
          audioRecordRunnable = null;
  
          audioThread = null;
  
  
  
          if(recorder != null &amp;&amp; recording) {
  
  
  
              recording = false;
  
              Log.v(TAG, "Finishing recording, calling stop and release on recorder");
  
              try {
  
                  recorder.stop();
  
                  recorder.release();
  
              } catch(FFmpegFrameRecorder.Exception e) {
  
                  e.printStackTrace();
  
              }
  
              recorder = null;
  
          }
  
      }
  
  
  
  
  
      //---------------------------------------
  
      // initialize ffmpeg_recorder
  
      //---------------------------------------
  
      private void initRecorder() {
  
  
  
          Log.w(TAG, "init recorder");
  
          try {
  
  
  
              if (yuvImage == null) {
  
                  yuvImage = new Frame(imageWidth, imageHeight, Frame.DEPTH_UBYTE, 2);
  
                  Log.i(TAG, "create yuvImage");
  
              }
  
  
  
              Log.i(TAG, "ffmpeg_url: " + ffmpeg_link.getAbsolutePath());
  
              Log.i(TAG, "ffmpeg_url: " + ffmpeg_link.exists());
  
              recorder = new FFmpegFrameRecorder(ffmpeg_link, imageWidth, imageHeight, 1);
  
              recorder.setFormat("mp4");
  
              recorder.setSampleRate(sampleAudioRateInHz);
  
              // Set in the surface changed method
  
              recorder.setFrameRate(frameRate);
  
  
  
              Log.i(TAG, "recorder initialize success");
  
  
  
              audioRecordRunnable = new AudioRecordRunnable();
  
              audioThread = new Thread(audioRecordRunnable);
  
              runAudioThread = true;
  
          } catch (Exception e){
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      //---------------------------------------------
  
      // audio thread, gets and encodes audio data
  
      //---------------------------------------------
  
      class AudioRecordRunnable implements Runnable {
  
  
  
          @Override
  
          public void run() {
  
              android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
  
  
  
              // Audio
  
              int bufferSize;
  
              ShortBuffer audioData;
  
              int bufferReadResult;
  
  
  
              bufferSize = AudioRecord.getMinBufferSize(sampleAudioRateInHz,
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
  
              audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleAudioRateInHz,
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
  
  
  
              audioData = ShortBuffer.allocate(bufferSize);
  
  
  
              Log.d(TAG, "audioRecord.startRecording()");
  
              audioRecord.startRecording();
  
  
  
              /* ffmpeg_audio encoding loop */
  
              while(runAudioThread) {
  
                  //Log.v(TAG,"recording? " + recording);
  
                  bufferReadResult = audioRecord.read(audioData.array(), 0, audioData.capacity());
  
                  audioData.limit(bufferReadResult);
  
                  if(bufferReadResult > 0) {
  
                      Log.v(TAG, "bufferReadResult: " + bufferReadResult);
  
                      // If "recording" isn't true when start this thread, it never get's set according to this if statement...!!!
  
                      // Why?  Good question...
  
                      if(recording) {
  
                              try {
  
                                  recorder.recordSamples(audioData);
  
                                  //Log.v(TAG,"recording " + 1024*i + " to " + 1024*i+1024);
  
                              } catch(FFmpegFrameRecorder.Exception e) {
  
                                  Log.v(TAG, e.getMessage());
  
                                  e.printStackTrace();
  
                              }
  
                      }
  
                  }
  
              }
  
              Log.v(TAG, "AudioThread Finished, release audioRecord");
  
  
  
              /* encoding finish, release recorder */
  
              if(audioRecord != null) {
  
                  audioRecord.stop();
  
                  audioRecord.release();
  
                  audioRecord = null;
  
                  Log.v(TAG, "audioRecord released");
  
              }
  
          }
  
      }
  
  }
  
  </string></string>

  OpenCVCameraPreview.java :

  import android.content.Context;
  
  import android.hardware.Camera;
  
  import android.util.AttributeSet;
  
  import android.util.Log;
  
  
  
  import org.opencv.android.JavaCameraView;
  
  
  
  import java.io.FileOutputStream;
  
  import java.util.List;
  
  
  
  public class OpenCVCameraPreview extends JavaCameraView implements Camera.PictureCallback {
  
  
  
      private static final String TAG =  OpenCVCameraPreview.class.getSimpleName();
  
      private String mPictureFileName;
  
  
  
      public OpenCVCameraPreview(Context context, AttributeSet attrs) {
  
          super(context, attrs);
  
      }
  
  
  
      public List<string> getEffectList() {
  
          return mCamera.getParameters().getSupportedColorEffects();
  
      }
  
  
  
      public boolean isEffectSupported() {
  
          return (mCamera.getParameters().getColorEffect() != null);
  
      }
  
  
  
      public String getEffect() {
  
          return mCamera.getParameters().getColorEffect();
  
      }
  
  
  
      public void setEffect(String effect) {
  
          Camera.Parameters params = mCamera.getParameters();
  
          params.setColorEffect(effect);
  
          mCamera.setParameters(params);
  
      }
  
  
  
      public List getResolutionList() {
  
          return mCamera.getParameters().getSupportedPreviewSizes();
  
      }
  
  
  
      public void setResolution(Camera.Size resolution) {
  
          disconnectCamera();
  
          mMaxHeight = resolution.height;
  
          mMaxWidth = resolution.width;
  
          connectCamera(getWidth(), getHeight());
  
      }
  
  
  
      public Camera.Size getResolution() {
  
          return mCamera.getParameters().getPreviewSize();
  
      }
  
  
  
      public void takePicture(final String fileName) {
  
          Log.i(TAG, "Taking picture");
  
          this.mPictureFileName = fileName;
  
          // Postview and jpeg are sent in the same buffers if the queue is not empty when performing a capture.
  
          // Clear up buffers to avoid mCamera.takePicture to be stuck because of a memory issue
  
          mCamera.setPreviewCallback(null);
  
  
  
          // PictureCallback is implemented by the current class
  
          mCamera.takePicture(null, null, this);
  
      }
  
  
  
      @Override
  
      public void onPictureTaken(byte[] data, Camera camera) {
  
          Log.i(TAG, "Saving a bitmap to file");
  
          // The camera preview was automatically stopped. Start it again.
  
          mCamera.startPreview();
  
          mCamera.setPreviewCallback(this);
  
  
  
          // Write the image in a file (in jpeg format)
  
          try {
  
              FileOutputStream fos = new FileOutputStream(mPictureFileName);
  
  
  
              fos.write(data);
  
              fos.close();
  
  
  
          } catch (java.io.IOException e) {
  
              Log.e("PictureDemo", "Exception in photoCallback", e);
  
          }
  
  
  
      }
  
  }
  
  </string>

  Gradle :

  apply plugin: 'com.android.application'
  
  
  
  android {
  
      compileSdkVersion 23
  
      buildToolsVersion "23.0.2"
  
  
  
      defaultConfig {
  
          applicationId "co.example.example"
  
          minSdkVersion 15
  
          targetSdkVersion 23
  
          versionCode 1
  
          versionName "1.0"
  
      }
  
      buildTypes {
  
          release {
  
              minifyEnabled false
  
              proguardFiles getDefaultProguardFile('proguard-android.txt'), 'proguard-rules.pro'
  
          }
  
      }
  
  
  
      packagingOptions {
  
          exclude 'META-INF/maven/org.bytedeco.javacpp-presets/opencv/pom.properties'
  
          exclude 'META-INF/maven/org.bytedeco.javacpp-presets/opencv/pom.xml'
  
          exclude 'META-INF/maven/org.bytedeco.javacpp-presets/ffmpeg/pom.properties'
  
          exclude 'META-INF/maven/org.bytedeco.javacpp-presets/ffmpeg/pom.xml'
  
      }
  
  }
  
  
  
  repositories {
  
      mavenCentral()
  
  }
  
  
  
  dependencies {
  
      compile fileTree(include: ['*.jar'], dir: 'libs')
  
      testCompile 'junit:junit:4.12'
  
      compile 'com.android.support:appcompat-v7:23.1.1'
  
      compile 'com.google.android.gms:play-services-appindexing:8.1.0'
  
  
  
      compile group: 'org.bytedeco', name: 'javacv', version: '1.1'
  
      compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.0.0-1.1', classifier: 'android-arm'
  
      compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.0.0-1.1', classifier: 'android-x86'
  
      compile group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '2.8.1-1.1', classifier: 'android-arm'
  
      compile group: 'org.bytedeco.javacpp-presets', name: 'ffmpeg', version: '2.8.1-1.1', classifier: 'android-x86'
  
  
  
      compile project(':openCVLibrary310')
  
  }

  proguard-rules.pro
  Edited by : link

  jniLibs :
  app/src/main/jniLibs :

  armeabi armeabi-v7a arm64-v8a mips mips64 x86 x86_64

  Problem

  02-19 11:57:37.684 1759-1759/ I/OpenCVCameraActivity: onClickBtnStartRecord()
  
  02-19 11:57:37.684 1759-1759/ I/OpenCVCameraActivity: startRecording()
  
  02-19 11:57:37.684 1759-1759/ W/OpenCVCameraActivity: init recorder
  
  02-19 11:57:37.691 1759-1759/ I/OpenCVCameraActivity: create yuvImage
  
  02-19 11:57:37.691 1759-1759/ I/OpenCVCameraActivity: ffmpeg_url: /storage/emulated/0/stream.mp4
  
  02-19 11:57:37.696 1759-1759/ I/OpenCVCameraActivity: ffmpeg_url: false
  
  02-19 11:57:37.837 1759-1759/ W/linker: libjniavutil.so: unused DT entry: type 0x1d arg 0x18cc3
  
  02-19 11:57:37.837 1759-1759/ W/linker: libjniavutil.so: unused DT entry: type 0x6ffffffe arg 0x21c30
  
  02-19 11:57:37.837 1759-1759/ W/linker: libjniavutil.so: unused DT entry: type 0x6fffffff arg 0x1
  
  02-19 11:57:37.838 1759-1759/co.example.example E/art: dlopen("/data/app/co.example.example-2/lib/x86/libjniavutil.so", RTLD_LAZY) failed: dlopen failed: cannot locate symbol "av_version_info" referenced by "libjniavutil.so"...
  
  02-19 11:57:37.843 1759-1759/co.example.example I/art: Rejecting re-init on previously-failed class java.lang.Class
  
  02-19 11:57:37.844 1759-1759/co.example.example E/AndroidRuntime: FATAL EXCEPTION: main
  
                                          Process: co.example.example, PID: 1759
  
                                          java.lang.IllegalStateException: Could not execute method of the activity
  
                                              at android.view.View$1.onClick(View.java:4020)
  
                                              at android.view.View.performClick(View.java:4780)
  
                                              at android.view.View$PerformClick.run(View.java:19866)
  
                                              at android.os.Handler.handleCallback(Handler.java:739)
  
                                              at android.os.Handler.dispatchMessage(Handler.java:95)
  
                                              at android.os.Looper.loop(Looper.java:135)
  
                                              at android.app.ActivityThread.main(ActivityThread.java:5254)
  
                                              at java.lang.reflect.Method.invoke(Native Method)
  
                                              at java.lang.reflect.Method.invoke(Method.java:372)
  
                                              at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:903)
  
                                              at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:698)
  
                                           Caused by: java.lang.reflect.InvocationTargetException
  
                                              at java.lang.reflect.Method.invoke(Native Method)
  
                                              at java.lang.reflect.Method.invoke(Method.java:372)
  
                                              at android.view.View$1.onClick(View.java:4015)
  
                                              at android.view.View.performClick(View.java:4780) 
  
                                              at android.view.View$PerformClick.run(View.java:19866) 
  
                                              at android.os.Handler.handleCallback(Handler.java:739) 
  
                                              at android.os.Handler.dispatchMessage(Handler.java:95) 
  
                                              at android.os.Looper.loop(Looper.java:135) 
  
                                              at android.app.ActivityThread.main(ActivityThread.java:5254) 
  
                                              at java.lang.reflect.Method.invoke(Native Method) 
  
                                              at java.lang.reflect.Method.invoke(Method.java:372) 
  
                                              at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:903) 
  
                                              at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:698) 
  
                                           Caused by: java.lang.UnsatisfiedLinkError: org.bytedeco.javacpp.avutil
  
                                              at java.lang.Class.classForName(Native Method)
  
                                              at java.lang.Class.forName(Class.java:309)
  
                                              at org.bytedeco.javacpp.Loader.load(Loader.java:413)
  
                                              at org.bytedeco.javacpp.Loader.load(Loader.java:381)
  
                                              at org.bytedeco.javacpp.avcodec$AVPacket.<clinit>(avcodec.java:1650)
  
                                              at org.bytedeco.javacv.FFmpegFrameRecorder.<init>(FFmpegFrameRecorder.java:149)
  
                                              at org.bytedeco.javacv.FFmpegFrameRecorder.<init>(FFmpegFrameRecorder.java:129)
  
                                              at co.example.example.OpenCVCameraActivity.initRecorder(OpenCVCameraActivity.java:320)
  
                                              at co.example.example.OpenCVCameraActivity.startRecording(OpenCVCameraActivity.java:266)
  
                                              at co.example.example.OpenCVCameraActivity.onClickBtnStartRecord2(OpenCVCameraActivity.java:259)
  
                                              at java.lang.reflect.Method.invoke(Native Method) 
  
                                              at java.lang.reflect.Method.invoke(Method.java:372) 
  
                                              at android.view.View$1.onClick(View.java:4015) 
  
                                              at android.view.View.performClick(View.java:4780) 
  
                                              at android.view.View$PerformClick.run(View.java:19866) 
  
                                              at android.os.Handler.handleCallback(Handler.java:739) 
  
                                              at android.os.Handler.dispatchMessage(Handler.java:95) 
  
                                              at android.os.Looper.loop(Looper.java:135) 
  
                                              at android.app.ActivityThread.main(ActivityThread.java:5254) 
  
                                              at java.lang.reflect.Method.invoke(Native Method) 
  
                                              at java.lang.reflect.Method.invoke(Method.java:372) 
  
                                              at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:903) 
  
                                              at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:698) 
  
  </init></init></clinit>

• How to verify user permissions – Introducing the Piwik Platform

  9 novembre 2014, par Thomas Steur — Development

  This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to make your plugin multilingual). This time you’ll learn how to verify user permissions. For this tutorial you will need to have basic knowledge of PHP and the Piwik platform.

  When should a plugin verify permissions ?

  Usually you want to do this before executing any action – such as deleting or fetching data – and before rendering any sensitive information that should not be accessible by everyone. For instance in an API method or Controller action. You sometimes also need to verify permissions before registering menu items or widgets.

  How does Piwik’s user management work ?

  It is quite simple as it only differentiates between a few roles : View permission, Admin permission and Super User permission. If you manage multiple websites with Piwik a user can be assigned to different roles as a user might have no permission for some websites but view or admin permission for another set of websites.

  Worth mentioning is that roles inherit from each other. This means the role admin automatically includes the role view and a super user automatically covers the view and admin role.

  Getting started

  In this post, we assume that you have already set up your development environment and created a plugin. If not, visit the Piwik Developer Zone where you’ll find the tutorial Setting up Piwik and other Guides that help you to develop a plugin.

  Verifying user permissions

  To protect your data the platform offers many convenient methods in the \Piwik\Piwik class. There you will find methods that either start with check, is or has. While methods that start with check throw an exception in case a condition is not met, the other methods return a boolean true or false.

  Use methods that throw an exception if you want to stop any further execution in case a user does not have an appropriate role. The platform will catch the exception and display an error message or ask the user to log in.

  public function deleteAllMessages()
  {
      // delete messages only if user has super user access, otherwise show an error message
      Piwik::checkUserSuperUserAccess();
   
      $this->getModel()->deleteAllMessages();
  }
  Télécharger

  Use methods that return a boolean for instance when registering menu items or widgets.

  public function configureAdminMenu(MenuAdmin $menu)
  {
      if (Piwik::hasUserSuperUserAccess()) {
          $menu->addPlatformItem('Plugins', $this->urlForDefaultAction());
      }
  }
  Télécharger

  It is important to be aware that just because the menu item won’t be displayed in the UI a user can still open the registered URL manually. Therefore you have to check for permissions in the actual controller action as well.

  View permission

  A user having a view permission should be only able to view reports but not make any changes apart from his personal settings. The methods that end with UserHasSomeViewAccess make sure a user has at least view permission for one website whereas the methods *UserHasViewAccess($idSites = array(1,2,3)) check whether a user has view access for all of the given websites.

  Piwik::checkUserHasSomeViewAccess();
   
  Piwik::checkUserHasViewAccess($idSites = array(1,2,3));
  Télécharger

  As a plugin developer you would usually use the latter example to verify the permissions for specific websites. Use the first example in case you develop something like an “All Websites Dashboard” where you only want to make sure the user has a view permission for at least one website.

  Admin permission

  A user having an admin permission cannot only view reports but also change website related settings. The methods to check for this role are similar to the ones before, just swap the term View with Admin.

  Piwik::checkUserHasSomeAdminAccess();
   
  Piwik::checkUserHasAdminAccess($idSites = array(1,2,3));
  Télécharger

  Super user permission

  A user having the super user permission is allowed to access all of the data stored in Piwik and change any settings. To check if a user has this role use one of the methods that end with UserSuperUserAccess.

  Piwik::checkUserHasSuperUserAccess();

  As a plugin developer you would check for this permission for instance in places where your plugin shows an activity log over all users or where it offers the possibility to change any system wide settings.

  Getting information about the currently logged in user

  Sometimes you might want to know which user is currently logged in. This can be useful if you want to persist user related information in the database or if you want to send an email to the currently logged in user. You can easily get this information by calling the following methods :

  $login = Piwik::getCurrentUserLogin()
  $email = Piwik::getCurrentUserEmail()
  Télécharger

  Advanced features

  Of course there is more that you can do. For instance you can verify whether a user is an anonymous user or whether a user has a specific role. You can also perform any operation in the context of a super user even if the current user does not have this role. Would you like to know more about those features ? Check out the Piwik class reference, the Security guide and the Manage Users user guide.

  If you have any feedback regarding our APIs or our guides in the Developer Zone feel free to send it to us.