Recherche avancée

Sur d’autres sites (12983)

• Merge videos and images into a new video in Android [on hold]

  10 septembre 2013, par vijay

  I have to create an android app that is similar to PicPlayPost.

  The app should be able to do the following things :

  • Load images and videos into the corresponding blocks in the frame.

  • Resize the image.

  • Add audio to the block that contains image.

  • Play two or more videos in parallel.

  • Save the whole frame as a video clip.

  I thought of using FFmpeg concatenation for this but it will not play all the videos in parallel.

  I have tried to use FFmpeg with android in my earlier project and ended up with lot of issues when building latest FFmpeg with android NDK.The below link explains the usage of pre-compiled FFmpeg and OpenCV for Android.

  https://code.google.com/p/javacv/#Quick_Start_for_OpenCV_and_FFmpeg

  but i could not find ffmpeg-1.2*.jar there.

  Is there any package to process video and playback in android other than FFmpeg ?

  Thanks.

• Video Recording on Android from Frames using javaCV

  23 juin 2016, par jawad bin zafar

  I am using this library javaCV to record video on android. They have provided a sample VideoRecording Activity But There is some bug which I could not figure out what I am doing it wrong or what is missing which causing this bug.

  package org.bytedeco.javacv.recordactivity;
  
  
  
  import android.app.Activity;
  
  import android.content.Context;
  
  import android.content.pm.ActivityInfo;
  
  import android.hardware.Camera;
  
  import android.hardware.Camera.PreviewCallback;
  
  import android.media.AudioFormat;
  
  import android.media.AudioRecord;
  
  import android.media.MediaRecorder;
  
  import android.os.Bundle;
  
  import android.os.PowerManager;
  
  import android.util.Log;
  
  import android.view.Display;
  
  import android.view.KeyEvent;
  
  import android.view.LayoutInflater;
  
  import android.view.SurfaceHolder;
  
  import android.view.SurfaceView;
  
  import android.view.View;
  
  import android.view.View.OnClickListener;
  
  import android.view.WindowManager;
  
  import android.widget.Button;
  
  import android.widget.LinearLayout;
  
  import android.widget.RelativeLayout;
  
  
  
  import java.io.IOException;
  
  import java.nio.ShortBuffer;
  
  
  
  import org.bytedeco.javacv.FFmpegFrameRecorder;
  
  
  
  import static org.bytedeco.javacpp.opencv_core.*;
  
  
  
  public class RecordActivity extends Activity implements OnClickListener {
  
  
  
      private final static String CLASS_LABEL = "RecordActivity";
  
      private final static String LOG_TAG = CLASS_LABEL;
  
  
  
      private PowerManager.WakeLock mWakeLock;
  
  
  
      private String ffmpeg_link = "/mnt/sdcard/stream.flv";
  
  
  
      long startTime = 0;
  
      boolean recording = false;
  
  
  
      private volatile FFmpegFrameRecorder recorder;
  
  
  
      private boolean isPreviewOn = false;
  
  
  
      private int sampleAudioRateInHz = 44100;
  
      private int imageWidth = 320;
  
      private int imageHeight = 240;
  
      private int frameRate = 30;
  
  
  
      /* audio data getting thread */
  
      private AudioRecord audioRecord;
  
      private AudioRecordRunnable audioRecordRunnable;
  
      private Thread audioThread;
  
      volatile boolean runAudioThread = true;
  
  
  
      /* video data getting thread */
  
      private Camera cameraDevice;
  
      private CameraView cameraView;
  
  
  
      private IplImage yuvIplimage = null;
  
  
  
      /* layout setting */
  
      private final int bg_screen_bx = 232;
  
      private final int bg_screen_by = 128;
  
      private final int bg_screen_width = 700;
  
      private final int bg_screen_height = 500;
  
      private final int bg_width = 1123;
  
      private final int bg_height = 715;
  
      private final int live_width = 640;
  
      private final int live_height = 480;
  
      private int screenWidth, screenHeight;
  
      private Button btnRecorderControl;
  
  
  
      /** The number of seconds in the continuous record loop (or 0 to disable loop). */
  
      final int RECORD_LENGTH = 10;
  
      IplImage[] images;
  
      long[] timestamps;
  
      ShortBuffer[] samples;
  
      int imagesIndex, samplesIndex;
  
  
  
      @Override
  
      public void onCreate(Bundle savedInstanceState) {
  
          super.onCreate(savedInstanceState);
  
          setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);
  
  
  
          setContentView(R.layout.main);
  
  
  
          PowerManager pm = (PowerManager) getSystemService(Context.POWER_SERVICE); 
  
          mWakeLock = pm.newWakeLock(PowerManager.SCREEN_BRIGHT_WAKE_LOCK, CLASS_LABEL); 
  
          mWakeLock.acquire(); 
  
  
  
          initLayout();
  
      }
  
  
  
  
  
      @Override
  
      protected void onResume() {
  
          super.onResume();
  
  
  
          if (mWakeLock == null) {
  
             PowerManager pm = (PowerManager) getSystemService(Context.POWER_SERVICE);
  
             mWakeLock = pm.newWakeLock(PowerManager.SCREEN_BRIGHT_WAKE_LOCK, CLASS_LABEL);
  
             mWakeLock.acquire();
  
          }
  
      }
  
  
  
      @Override
  
      protected void onPause() {
  
          super.onPause();
  
  
  
          if (mWakeLock != null) {
  
              mWakeLock.release();
  
              mWakeLock = null;
  
          }
  
      }
  
  
  
      @Override
  
      protected void onDestroy() {
  
          super.onDestroy();
  
  
  
          recording = false;
  
  
  
          if (cameraView != null) {
  
              cameraView.stopPreview();
  
          }
  
  
  
          if(cameraDevice != null) {
  
             cameraDevice.stopPreview();
  
             cameraDevice.release();
  
             cameraDevice = null;
  
          }
  
  
  
          if (mWakeLock != null) {
  
              mWakeLock.release();
  
              mWakeLock = null;
  
          }
  
      }
  
  
  
  
  
      private void initLayout() {
  
  
  
          /* get size of screen */
  
          Display display = ((WindowManager) getSystemService(Context.WINDOW_SERVICE)).getDefaultDisplay();
  
          screenWidth = display.getWidth();
  
          screenHeight = display.getHeight();
  
          RelativeLayout.LayoutParams layoutParam = null; 
  
          LayoutInflater myInflate = null; 
  
          myInflate = (LayoutInflater) getSystemService(Context.LAYOUT_INFLATER_SERVICE);
  
          RelativeLayout topLayout = new RelativeLayout(this);
  
          setContentView(topLayout);
  
          LinearLayout preViewLayout = (LinearLayout) myInflate.inflate(R.layout.main, null);
  
          layoutParam = new RelativeLayout.LayoutParams(screenWidth, screenHeight);
  
          topLayout.addView(preViewLayout, layoutParam);
  
  
  
          /* add control button: start and stop */
  
          btnRecorderControl = (Button) findViewById(R.id.recorder_control);
  
          btnRecorderControl.setText("Start");
  
          btnRecorderControl.setOnClickListener(this);
  
  
  
          /* add camera view */
  
          int display_width_d = (int) (1.0 * bg_screen_width * screenWidth / bg_width);
  
          int display_height_d = (int) (1.0 * bg_screen_height * screenHeight / bg_height);
  
          int prev_rw, prev_rh;
  
          if (1.0 * display_width_d / display_height_d > 1.0 * live_width / live_height) {
  
              prev_rh = display_height_d;
  
              prev_rw = (int) (1.0 * display_height_d * live_width / live_height);
  
          } else {
  
              prev_rw = display_width_d;
  
              prev_rh = (int) (1.0 * display_width_d * live_height / live_width);
  
          }
  
          layoutParam = new RelativeLayout.LayoutParams(prev_rw, prev_rh);
  
          layoutParam.topMargin = (int) (1.0 * bg_screen_by * screenHeight / bg_height);
  
          layoutParam.leftMargin = (int) (1.0 * bg_screen_bx * screenWidth / bg_width);
  
  
  
          cameraDevice = Camera.open();
  
          Log.i(LOG_TAG, "cameara open");
  
          cameraView = new CameraView(this, cameraDevice);
  
          topLayout.addView(cameraView, layoutParam);
  
          Log.i(LOG_TAG, "cameara preview start: OK");
  
      }
  
  
  
      //---------------------------------------
  
      // initialize ffmpeg_recorder
  
      //---------------------------------------
  
      private void initRecorder() {
  
  
  
          Log.w(LOG_TAG,"init recorder");
  
  
  
          if (RECORD_LENGTH > 0) {
  
              imagesIndex = 0;
  
              images = new IplImage[RECORD_LENGTH * frameRate];
  
              timestamps = new long[images.length];
  
              for (int i = 0; i < images.length; i++) {
  
                  images[i] = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_8U, 2);
  
                  timestamps[i] = -1;
  
              }
  
          } else if (yuvIplimage == null) {
  
              yuvIplimage = IplImage.create(imageWidth, imageHeight, IPL_DEPTH_8U, 2);
  
              Log.i(LOG_TAG, "create yuvIplimage");
  
          }
  
  
  
          Log.i(LOG_TAG, "ffmpeg_url: " + ffmpeg_link);
  
          recorder = new FFmpegFrameRecorder(ffmpeg_link, imageWidth, imageHeight, 1);
  
          recorder.setFormat("flv");
  
          recorder.setSampleRate(sampleAudioRateInHz);
  
          // Set in the surface changed method
  
          recorder.setFrameRate(frameRate);
  
  
  
          Log.i(LOG_TAG, "recorder initialize success");
  
  
  
          audioRecordRunnable = new AudioRecordRunnable();
  
          audioThread = new Thread(audioRecordRunnable);
  
          runAudioThread = true;
  
      }
  
  
  
      public void startRecording() {
  
  
  
          initRecorder();
  
  
  
          try {
  
              recorder.start();
  
              startTime = System.currentTimeMillis();
  
              recording = true;
  
              audioThread.start();
  
  
  
          } catch (FFmpegFrameRecorder.Exception e) {
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      public void stopRecording() {
  
  
  
          runAudioThread = false;
  
          try {
  
              audioThread.join();
  
          } catch (InterruptedException e) {
  
              e.printStackTrace();
  
          }
  
          audioRecordRunnable = null;
  
          audioThread = null;
  
  
  
          if (recorder != null && recording) {
  
              if (RECORD_LENGTH > 0) {
  
                  Log.v(LOG_TAG,"Writing frames");
  
                  try {
  
                      int firstIndex = imagesIndex % samples.length;
  
                      int lastIndex = (imagesIndex - 1) % images.length;
  
                      if (imagesIndex <= images.length) {
  
                          firstIndex = 0;
  
                          lastIndex = imagesIndex - 1;
  
                      }
  
                      if ((startTime = timestamps[lastIndex] - RECORD_LENGTH * 1000000L) < 0) {
  
                          startTime = 0;
  
                      }
  
                      if (lastIndex < firstIndex) {
  
                          lastIndex += images.length;
  
                      }
  
                      for (int i = firstIndex; i <= lastIndex; i++) {
  
                          long t = timestamps[i % timestamps.length] - startTime;
  
                          if (t >= 0) {
  
                              if (t > recorder.getTimestamp()) {
  
                                  recorder.setTimestamp(t);
  
                              }
  
                              recorder.record(images[i % images.length]);
  
                          }
  
                      }
  
  
  
                      firstIndex = samplesIndex % samples.length;
  
                      lastIndex = (samplesIndex - 1) % samples.length;
  
                      if (samplesIndex <= samples.length) {
  
                          firstIndex = 0;
  
                          lastIndex = samplesIndex - 1;
  
                      }
  
                      if (lastIndex < firstIndex) {
  
                          lastIndex += samples.length;
  
                      }
  
                      for (int i = firstIndex; i <= lastIndex; i++) {
  
                          recorder.record(samples[i % samples.length]);
  
                      }
  
                  } catch (FFmpegFrameRecorder.Exception e) {
  
                      Log.v(LOG_TAG,e.getMessage());
  
                      e.printStackTrace();
  
                  }
  
              }
  
  
  
              recording = false;
  
              Log.v(LOG_TAG,"Finishing recording, calling stop and release on recorder");
  
              try {
  
                  recorder.stop();
  
                  recorder.release();
  
              } catch (FFmpegFrameRecorder.Exception e) {
  
                  e.printStackTrace();
  
              }
  
              recorder = null;
  
  
  
          }
  
      }
  
  
  
      @Override
  
      public boolean onKeyDown(int keyCode, KeyEvent event) {
  
  
  
          if (keyCode == KeyEvent.KEYCODE_BACK) {
  
              if (recording) {
  
                  stopRecording();
  
              }
  
  
  
              finish();
  
  
  
              return true;
  
          }
  
  
  
          return super.onKeyDown(keyCode, event);
  
      }
  
  
  
  
  
      //---------------------------------------------
  
      // audio thread, gets and encodes audio data
  
      //---------------------------------------------
  
      class AudioRecordRunnable implements Runnable {
  
  
  
          @Override
  
          public void run() {
  
              android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
  
  
  
              // Audio
  
              int bufferSize;
  
              ShortBuffer audioData;
  
              int bufferReadResult;
  
  
  
              bufferSize = AudioRecord.getMinBufferSize(sampleAudioRateInHz, 
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
  
              audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleAudioRateInHz, 
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
  
  
  
              if (RECORD_LENGTH > 0) {
  
                  samplesIndex = 0;
  
                  samples = new ShortBuffer[RECORD_LENGTH * sampleAudioRateInHz * 2 / bufferSize + 1];
  
                  for (int i = 0; i < samples.length; i++) {
  
                      samples[i] = ShortBuffer.allocate(bufferSize);
  
                  }
  
              } else {
  
                  audioData = ShortBuffer.allocate(bufferSize);
  
              }
  
  
  
              Log.d(LOG_TAG, "audioRecord.startRecording()");
  
              audioRecord.startRecording();
  
  
  
              /* ffmpeg_audio encoding loop */
  
              while (runAudioThread) {
  
                  if (RECORD_LENGTH > 0) {
  
                      audioData = samples[samplesIndex++ % samples.length];
  
                      audioData.position(0).limit(0);
  
                  }
  
                  //Log.v(LOG_TAG,"recording? " + recording);
  
                  bufferReadResult = audioRecord.read(audioData.array(), 0, audioData.capacity());
  
                  audioData.limit(bufferReadResult);
  
                  if (bufferReadResult > 0) {
  
                      Log.v(LOG_TAG,"bufferReadResult: " + bufferReadResult);
  
                      // If "recording" isn't true when start this thread, it never get's set according to this if statement...!!!
  
                      // Why?  Good question...
  
                      if (recording) {
  
                          if (RECORD_LENGTH <= 0) try {
  
                              recorder.record(audioData);
  
                              //Log.v(LOG_TAG,"recording " + 1024*i + " to " + 1024*i+1024);
  
                          } catch (FFmpegFrameRecorder.Exception e) {
  
                              Log.v(LOG_TAG,e.getMessage());
  
                              e.printStackTrace();
  
                          }
  
                      }
  
                  }
  
              }
  
              Log.v(LOG_TAG,"AudioThread Finished, release audioRecord");
  
  
  
              /* encoding finish, release recorder */
  
              if (audioRecord != null) {
  
                  audioRecord.stop();
  
                  audioRecord.release();
  
                  audioRecord = null;
  
                  Log.v(LOG_TAG,"audioRecord released");
  
              }
  
          }
  
      }
  
  
  
      //---------------------------------------------
  
      // camera thread, gets and encodes video data
  
      //---------------------------------------------
  
      class CameraView extends SurfaceView implements SurfaceHolder.Callback, PreviewCallback {
  
  
  
          private SurfaceHolder mHolder;
  
          private Camera mCamera;
  
  
  
          public CameraView(Context context, Camera camera) {
  
              super(context);
  
              Log.w("camera","camera view");
  
              mCamera = camera;
  
              mHolder = getHolder();
  
              mHolder.addCallback(CameraView.this);
  
              mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
  
              mCamera.setPreviewCallback(CameraView.this);
  
          }
  
  
  
          @Override
  
          public void surfaceCreated(SurfaceHolder holder) {
  
              try {
  
                  stopPreview();
  
                  mCamera.setPreviewDisplay(holder);
  
              } catch (IOException exception) {
  
                  mCamera.release();
  
                  mCamera = null;
  
              }
  
          }
  
  
  
          public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
  
              Log.v(LOG_TAG,"Setting imageWidth: " + imageWidth + " imageHeight: " + imageHeight + " frameRate: " + frameRate);
  
              Camera.Parameters camParams = mCamera.getParameters();
  
              camParams.setPreviewSize(imageWidth, imageHeight);
  
  
  
              Log.v(LOG_TAG,"Preview Framerate: " + camParams.getPreviewFrameRate());
  
  
  
              camParams.setPreviewFrameRate(frameRate);
  
              mCamera.setParameters(camParams);
  
              startPreview();
  
          }
  
  
  
          @Override
  
          public void surfaceDestroyed(SurfaceHolder holder) {
  
              try {
  
                  mHolder.addCallback(null);
  
                  mCamera.setPreviewCallback(null);
  
              } catch (RuntimeException e) {
  
                  // The camera has probably just been released, ignore.
  
              }
  
          }
  
  
  
          public void startPreview() {
  
              if (!isPreviewOn && mCamera != null) {
  
                  isPreviewOn = true;
  
                  mCamera.startPreview();
  
              }
  
          }
  
  
  
          public void stopPreview() {
  
              if (isPreviewOn && mCamera != null) {
  
                  isPreviewOn = false;
  
                  mCamera.stopPreview();
  
              }
  
          }
  
  
  
          @Override
  
          public void onPreviewFrame(byte[] data, Camera camera) {
  
              if (audioRecord == null || audioRecord.getRecordingState() != AudioRecord.RECORDSTATE_RECORDING) {
  
                  startTime = System.currentTimeMillis();
  
                  return;
  
              }
  
              if (RECORD_LENGTH > 0) {
  
                  int i = imagesIndex++ % images.length;
  
                  yuvIplimage = images[i];
  
                  timestamps[i] = 1000 * (System.currentTimeMillis() - startTime);
  
              }
  
              /* get video data */
  
              if (yuvIplimage != null && recording) {
  
                  yuvIplimage.getByteBuffer().put(data);
  
  
  
                  if (RECORD_LENGTH <= 0) try {
  
                      Log.v(LOG_TAG,"Writing Frame");
  
                      long t = 1000 * (System.currentTimeMillis() - startTime);
  
                      if (t > recorder.getTimestamp()) {
  
                          recorder.setTimestamp(t);
  
                      }
  
                      recorder.record(yuvIplimage);
  
                  } catch (FFmpegFrameRecorder.Exception e) {
  
                      Log.v(LOG_TAG,e.getMessage());
  
                      e.printStackTrace();
  
                  }
  
              }
  
          }
  
      }
  
  
  
      @Override
  
      public void onClick(View v) {
  
          if (!recording) {
  
              startRecording();
  
              Log.w(LOG_TAG, "Start Button Pushed");
  
              btnRecorderControl.setText("Stop");
  
          } else {
  
              // This will trigger the audio recording loop to stop and then set isRecorderStart = false;
  
              stopRecording();
  
              Log.w(LOG_TAG, "Stop Button Pushed");
  
              btnRecorderControl.setText("Start");
  
          }
  
      }
  
  }

  I am getting this error

  The type org.bytedeco.javacpp.avutil$AVFrame cannot be resolved. It is indirectly referenced from required .class files

  at following line in the above code

  if (RECORD_LENGTH <= 0) try {
  
      recorder.record(audioData);
  
      //Log.v(LOG_TAG,"recording " + 1024*i + " to " + 1024*i+1024);
  
   }

  at recorder.record(audioData). I don’t know what I am doing wrong here. New to JavaCV. Any help will be appreciated.

• Android-How to pass back frames from FFmpeg back to Android

  23 octobre 2013, par yarin

  It is an architecture question-i am really interesting about the answer

  I building an app with following goals :

  1.record video with effect in real time(using FFmpeg)

  2.display the customized video in real time for the user while he recording

  So,after 1 month of working...i decide to remember that goal number 2 is worth to thinking about :)
  I have a ready skeleton app that record video with effect in real time.
  but i have to preview this customized frame back to the user.

  My options (and this is my question) :

  1.Each frame that pass from onPreviewFrame(byte[] video_frame_data, Camera camera) to ffmpeg with JNI to encode-will sending back to android through the same JNI after i apply the effects(i mean : onPreviewFrame->JNI to FFMPEG->immediately apply effect->send the costumed frame back to android side for display->encode the costumed frame).

  Advantages : it is look like is the most easy to use.

  Disadvantages : use the JNI twice or the passing back the frame could consume time(i really don't now if it really big price to pay,cuz it is only byte array or int array per frame to send to android side)

  2.I heard about openGL on ndk,but i think that the surface it self created on the android side-so is it really going to be better ?
  i prefer to use other surface that i using now in java

  3.Create an video player on FFmpeg to preview each customized frame in real time.

  Thank for your helping,i hope that the first solution is available and not consume to much expensive time in terms of real time processing