Recherche avancée

Sur d’autres sites (10822)

• Recording a video using MediaRecorder

  21 juillet 2016, par Cédric Portmann

  I am currently using the TextureFromCameraActivity from Grafika to record a video in square ( 1:1 ) resolution. Therefor I the GLES20.glViewport so that the video gets moved to the top and it appears to be squared. Now I would like to record this square view using the MediaRecorder or at least record the camera with normal resolutiona and then crop it using FFmpeg. However I get the same error over and over again and I cant figure out why.

  The error I get :

  start called in an invalid state : 4

  And yes I added all the necessary permissions.

  android.permission.WRITE_EXTERNAL_STORAGE android.permission.CAMERA
  android.permission.RECORD_VIDEO android.permission.RECORD_AUDIO
  android.permission.STORAGE android.permission.READ_EXTERNAL_STORAGE

  Here the modified code :

  https://github.com/google/grafika

  Thanks for your help :D

  package com.android.grafika;
  
  
  
  import android.graphics.SurfaceTexture;
  
  import android.hardware.Camera;
  
  import android.media.CamcorderProfile;
  
  import android.media.MediaRecorder;
  
  import android.opengl.GLES20;
  
  import android.opengl.Matrix;
  
  import android.os.Bundle;
  
  import android.os.Environment;
  
  import android.os.Handler;
  
  import android.os.Looper;
  
  import android.os.Message;
  
  import android.util.Log;
  
  import android.view.MotionEvent;
  
  import android.view.Surface;
  
  import android.view.SurfaceHolder;
  
  import android.view.SurfaceView;
  
  import android.view.View;
  
  import android.widget.Button;
  
  import android.widget.SeekBar;
  
  import android.widget.TextView;
  
  import android.app.Activity;
  
  import android.widget.Toast;
  
  
  
  import com.android.grafika.gles.Drawable2d;
  
  import com.android.grafika.gles.EglCore;
  
  import com.android.grafika.gles.GlUtil;
  
  import com.android.grafika.gles.Sprite2d;
  
  import com.android.grafika.gles.Texture2dProgram;
  
  import com.android.grafika.gles.WindowSurface;
  
  
  
  import java.io.File;
  
  import java.io.IOException;
  
  import java.lang.ref.WeakReference;
  
  
  
  
  
  public class TextureFromCameraActivity extends Activity implements View.OnClickListener, SurfaceHolder.Callback,
  
          SeekBar.OnSeekBarChangeListener {
  
  
  
  
  
      private static final int DEFAULT_ZOOM_PERCENT = 0;      // 0-100
  
      private static final int DEFAULT_SIZE_PERCENT = 80;     // 0-100
  
      private static final int DEFAULT_ROTATE_PERCENT = 75;    // 0-100
  
  
  
      // Requested values; actual may differ.
  
      private static final int REQ_CAMERA_WIDTH = 720;
  
      private static final int REQ_CAMERA_HEIGHT = 720;
  
      private static final int REQ_CAMERA_FPS = 30;
  
  
  
      // The holder for our SurfaceView.  The Surface can outlive the Activity (e.g. when
  
      // the screen is turned off and back on with the power button).
  
      //
  
      // This becomes non-null after the surfaceCreated() callback is called, and gets set
  
      // to null when surfaceDestroyed() is called.
  
      private static SurfaceHolder sSurfaceHolder;
  
  
  
      // Thread that handles rendering and controls the camera.  Started in onResume(),
  
      // stopped in onPause().
  
      private RenderThread mRenderThread;
  
  
  
      // Receives messages from renderer thread.
  
      private MainHandler mHandler;
  
  
  
      // User controls.
  
      private SeekBar mZoomBar;
  
      private SeekBar mSizeBar;
  
      private SeekBar mRotateBar;
  
  
  
      // These values are passed to us by the camera/render thread, and displayed in the UI.
  
      // We could also just peek at the values in the RenderThread object, but we'd need to
  
      // synchronize access carefully.
  
      private int mCameraPreviewWidth, mCameraPreviewHeight;
  
      private float mCameraPreviewFps;
  
      private int mRectWidth, mRectHeight;
  
      private int mZoomWidth, mZoomHeight;
  
      private int mRotateDeg;
  
      SurfaceHolder sh;
  
      MediaRecorder recorder;
  
      SurfaceHolder holder;
  
      boolean recording = false;
  
  
  
      public static final String TAG = "VIDEOCAPTURE";
  
  
  
      private static final File OUTPUT_DIR = Environment.getExternalStorageDirectory();
  
  
  
  
  
      @Override
  
      protected void onCreate(Bundle savedInstanceState) {
  
          super.onCreate(savedInstanceState);
  
  
  
          recorder = new MediaRecorder();
  
  
  
  
  
  
  
          setContentView(R.layout.activity_texture_from_camera);
  
  
  
          mHandler = new MainHandler(this);
  
  
  
          SurfaceView cameraView = (SurfaceView) findViewById(R.id.cameraOnTexture_surfaceView);
  
          sh = cameraView.getHolder();
  
          cameraView.setClickable(true);// make the surface view clickable
  
          sh.addCallback(this);
  
  
  
  
  
          //prepareRecorder();
  
  
  
  
  
          mZoomBar = (SeekBar) findViewById(R.id.tfcZoom_seekbar);
  
          mSizeBar = (SeekBar) findViewById(R.id.tfcSize_seekbar);
  
          mRotateBar = (SeekBar) findViewById(R.id.tfcRotate_seekbar);
  
          mZoomBar.setProgress(DEFAULT_ZOOM_PERCENT);
  
          mSizeBar.setProgress(DEFAULT_SIZE_PERCENT);
  
          mRotateBar.setProgress(DEFAULT_ROTATE_PERCENT);
  
          mZoomBar.setOnSeekBarChangeListener(this);
  
          mSizeBar.setOnSeekBarChangeListener(this);
  
          mRotateBar.setOnSeekBarChangeListener(this);
  
  
  
          Button record_btn = (Button)findViewById(R.id.button);
  
          record_btn.setOnClickListener(this);
  
          initRecorder();
  
  
  
  
  
          updateControls();
  
  
  
  
  
  
  
  
  
      }
  
  
  
  
  
  
  
  
  
  
  
      @Override
  
      protected void onResume() {
  
          Log.d(TAG, "onResume BEGIN");
  
          super.onResume();
  
  
  
          mRenderThread = new RenderThread(mHandler);
  
          mRenderThread.setName("TexFromCam Render");
  
          mRenderThread.start();
  
          mRenderThread.waitUntilReady();
  
  
  
          RenderHandler rh = mRenderThread.getHandler();
  
          rh.sendZoomValue(mZoomBar.getProgress());
  
          rh.sendSizeValue(mSizeBar.getProgress());
  
          rh.sendRotateValue(mRotateBar.getProgress());
  
  
  
          if (sSurfaceHolder != null) {
  
              Log.d(TAG, "Sending previous surface");
  
              rh.sendSurfaceAvailable(sSurfaceHolder, false);
  
          } else {
  
              Log.d(TAG, "No previous surface");
  
          }
  
          Log.d(TAG, "onResume END");
  
      }
  
  
  
      @Override
  
      protected void onPause() {
  
          Log.d(TAG, "onPause BEGIN");
  
          super.onPause();
  
  
  
          RenderHandler rh = mRenderThread.getHandler();
  
          rh.sendShutdown();
  
          try {
  
              mRenderThread.join();
  
          } catch (InterruptedException ie) {
  
              // not expected
  
              throw new RuntimeException("join was interrupted", ie);
  
          }
  
          mRenderThread = null;
  
          Log.d(TAG, "onPause END");
  
      }
  
  
  
      @Override   // SurfaceHolder.Callback
  
      public void surfaceCreated(SurfaceHolder holder) {
  
          Log.d(TAG, "surfaceCreated holder=" + holder + " (static=" + sSurfaceHolder + ")");
  
          if (sSurfaceHolder != null) {
  
              throw new RuntimeException("sSurfaceHolder is already set");
  
          }
  
  
  
          sSurfaceHolder = holder;
  
  
  
          if (mRenderThread != null) {
  
              // Normal case -- render thread is running, tell it about the new surface.
  
              RenderHandler rh = mRenderThread.getHandler();
  
              rh.sendSurfaceAvailable(holder, true);
  
          } else {
  
              // Sometimes see this on 4.4.x N5: power off, power on, unlock, with device in
  
              // landscape and a lock screen that requires portrait.  The surface-created
  
              // message is showing up after onPause().
  
              //
  
              // Chances are good that the surface will be destroyed before the activity is
  
              // unpaused, but we track it anyway.  If the activity is un-paused and we start
  
              // the RenderThread, the SurfaceHolder will be passed in right after the thread
  
              // is created.
  
              Log.d(TAG, "render thread not running");
  
          }
  
  
  
          recorder.setPreviewDisplay(holder.getSurface());
  
  
  
      }
  
  
  
      @Override   // SurfaceHolder.Callback
  
      public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
  
          Log.d(TAG, "surfaceChanged fmt=" + format + " size=" + width + "x" + height +
  
                  " holder=" + holder);
  
  
  
          if (mRenderThread != null) {
  
              RenderHandler rh = mRenderThread.getHandler();
  
              rh.sendSurfaceChanged(format, width, height);
  
          } else {
  
              Log.d(TAG, "Ignoring surfaceChanged");
  
              return;
  
          }
  
      }
  
  
  
      @Override   // SurfaceHolder.Callback
  
      public void surfaceDestroyed(SurfaceHolder holder) {
  
          // In theory we should tell the RenderThread that the surface has been destroyed.
  
          if (mRenderThread != null) {
  
              RenderHandler rh = mRenderThread.getHandler();
  
              rh.sendSurfaceDestroyed();
  
          }
  
          Log.d(TAG, "surfaceDestroyed holder=" + holder);
  
          sSurfaceHolder = null;
  
      }
  
  
  
      @Override   // SeekBar.OnSeekBarChangeListener
  
      public void onProgressChanged(SeekBar seekBar, int progress, boolean fromUser) {
  
          if (mRenderThread == null) {
  
              // Could happen if we programmatically update the values after setting a listener
  
              // but before starting the thread.  Also, easy to cause this by scrubbing the seek
  
              // bar with one finger then tapping "recents" with another.
  
              Log.w(TAG, "Ignoring onProgressChanged received w/o RT running");
  
              return;
  
          }
  
          RenderHandler rh = mRenderThread.getHandler();
  
  
  
          // "progress" ranges from 0 to 100
  
          if (seekBar == mZoomBar) {
  
              //Log.v(TAG, "zoom: " + progress);
  
              rh.sendZoomValue(progress);
  
          } else if (seekBar == mSizeBar) {
  
              //Log.v(TAG, "size: " + progress);
  
              rh.sendSizeValue(progress);
  
          } else if (seekBar == mRotateBar) {
  
              //Log.v(TAG, "rotate: " + progress);
  
              rh.sendRotateValue(progress);
  
          } else {
  
              throw new RuntimeException("unknown seek bar");
  
          }
  
  
  
          // If we're getting preview frames quickly enough we don't really need this, but
  
          // we don't want to have chunky-looking resize movement if the camera is slow.
  
          // OTOH, if we get the updates too quickly (60fps camera?), this could jam us
  
          // up and cause us to run behind.  So use with caution.
  
          rh.sendRedraw();
  
      }
  
  
  
      @Override   // SeekBar.OnSeekBarChangeListener
  
      public void onStartTrackingTouch(SeekBar seekBar) {}
  
      @Override   // SeekBar.OnSeekBarChangeListener
  
      public void onStopTrackingTouch(SeekBar seekBar) {}
  
      @Override
  
  
  
      /**
  
       * Handles any touch events that aren't grabbed by one of the controls.
  
       */
  
      public boolean onTouchEvent(MotionEvent e) {
  
          float x = e.getX();
  
          float y = e.getY();
  
  
  
          switch (e.getAction()) {
  
              case MotionEvent.ACTION_MOVE:
  
              case MotionEvent.ACTION_DOWN:
  
                  //Log.v(TAG, "onTouchEvent act=" + e.getAction() + " x=" + x + " y=" + y);
  
                  if (mRenderThread != null) {
  
                      RenderHandler rh = mRenderThread.getHandler();
  
                      rh.sendPosition((int) x, (int) y);
  
  
  
                      // Forcing a redraw can cause sluggish-looking behavior if the touch
  
                      // events arrive quickly.
  
                      //rh.sendRedraw();
  
                  }
  
                  break;
  
              default:
  
                  break;
  
          }
  
  
  
          return true;
  
      }
  
  
  
      /**
  
       * Updates the current state of the controls.
  
       */
  
      private void updateControls() {
  
          String str = getString(R.string.tfcCameraParams, mCameraPreviewWidth,
  
                  mCameraPreviewHeight, mCameraPreviewFps);
  
          TextView tv = (TextView) findViewById(R.id.tfcCameraParams_text);
  
          tv.setText(str);
  
  
  
          str = getString(R.string.tfcRectSize, mRectWidth, mRectHeight);
  
          tv = (TextView) findViewById(R.id.tfcRectSize_text);
  
          tv.setText(str);
  
  
  
          str = getString(R.string.tfcZoomArea, mZoomWidth, mZoomHeight);
  
          tv = (TextView) findViewById(R.id.tfcZoomArea_text);
  
          tv.setText(str);
  
      }
  
  
  
      @Override
  
      public void onClick(View view) {
  
  
  
          if (recording) {
  
              recorder.stop();
  
              recording = false;
  
  
  
              // Let's initRecorder so we can record again
  
              initRecorder();
  
              prepareRecorder();
  
          } else {
  
              recording = true;
  
              recorder.start();
  
          }
  
      }
  
  
  
  
  
      private void initRecorder() {
  
          recorder.setAudioSource(MediaRecorder.AudioSource.DEFAULT);
  
          recorder.setVideoSource(MediaRecorder.VideoSource.DEFAULT);
  
  
  
          CamcorderProfile cpHigh = CamcorderProfile
  
                  .get(CamcorderProfile.QUALITY_HIGH);
  
          recorder.setProfile(cpHigh);
  
          String path = Environment.getExternalStorageDirectory() + File.separator
  
                  + Environment.DIRECTORY_DCIM + File.separator + "AlphaRun";
  
  
  
          recorder.setOutputFile(path);
  
          recorder.setMaxDuration(50000); // 50 seconds
  
          recorder.setMaxFileSize(5000000); // Approximately 5 megabytes
  
  
  
      }
  
  
  
      private void prepareRecorder() {
  
  
  
  
  
          try {
  
              recorder.prepare();
  
          } catch (IllegalStateException e) {
  
              e.printStackTrace();
  
              finish();
  
          } catch (IOException e) {
  
              e.printStackTrace();
  
              finish();
  
          }
  
      }
  
  
  
  
  
  
  
  
  
      /**
  
       * Thread that handles all rendering and camera operations.
  
       */
  
      private static class RenderThread extends Thread implements
  
              SurfaceTexture.OnFrameAvailableListener {
  
          // Object must be created on render thread to get correct Looper, but is used from
  
          // UI thread, so we need to declare it volatile to ensure the UI thread sees a fully
  
          // constructed object.
  
          private volatile RenderHandler mHandler;
  
  
  
          // Used to wait for the thread to start.
  
          private Object mStartLock = new Object();
  
          private boolean mReady = false;
  
  
  
          private MainHandler mMainHandler;
  
  
  
          private Camera mCamera;
  
          private int mCameraPreviewWidth, mCameraPreviewHeight;
  
  
  
          private EglCore mEglCore;
  
          private WindowSurface mWindowSurface;
  
          private int mWindowSurfaceWidth;
  
          private int mWindowSurfaceHeight;
  
  
  
          // Receives the output from the camera preview.
  
          private SurfaceTexture mCameraTexture;
  
  
  
          // Orthographic projection matrix.
  
          private float[] mDisplayProjectionMatrix = new float[16];
  
  
  
          private Texture2dProgram mTexProgram;
  
          private final ScaledDrawable2d mRectDrawable =
  
                  new ScaledDrawable2d(Drawable2d.Prefab.RECTANGLE);
  
          private final Sprite2d mRect = new Sprite2d(mRectDrawable);
  
  
  
          private int mZoomPercent = DEFAULT_ZOOM_PERCENT;
  
          private int mSizePercent = DEFAULT_SIZE_PERCENT;
  
          private int mRotatePercent = DEFAULT_ROTATE_PERCENT;
  
          private float mPosX, mPosY;
  
  
  
  
  
          /**
  
           * Constructor.  Pass in the MainHandler, which allows us to send stuff back to the
  
           * Activity.
  
           */
  
          public RenderThread(MainHandler handler) {
  
              mMainHandler = handler;
  
  
  
          }
  
  
  
          /**
  
           * Thread entry point.
  
           */
  
          @Override
  
          public void run() {
  
              Looper.prepare();
  
  
  
              // We need to create the Handler before reporting ready.
  
              mHandler = new RenderHandler(this);
  
              synchronized (mStartLock) {
  
                  mReady = true;
  
                  mStartLock.notify();    // signal waitUntilReady()
  
              }
  
  
  
              // Prepare EGL and open the camera before we start handling messages.
  
              mEglCore = new EglCore(null, 0);
  
              openCamera(REQ_CAMERA_WIDTH, REQ_CAMERA_HEIGHT, REQ_CAMERA_FPS);
  
  
  
              Looper.loop();
  
  
  
              Log.d(TAG, "looper quit");
  
              releaseCamera();
  
              releaseGl();
  
              mEglCore.release();
  
  
  
              synchronized (mStartLock) {
  
                  mReady = false;
  
              }
  
          }
  
  
  
          /**
  
           * Waits until the render thread is ready to receive messages.
  
           * <p>
  
           * Call from the UI thread.
  
           */
  
          public void waitUntilReady() {
  
              synchronized (mStartLock) {
  
                  while (!mReady) {
  
                      try {
  
                          mStartLock.wait();
  
                      } catch (InterruptedException ie) { /* not expected */ }
  
                  }
  
              }
  
          }
  
  
  
          /**
  
           * Shuts everything down.
  
           */
  
          private void shutdown() {
  
              Log.d(TAG, "shutdown");
  
              Looper.myLooper().quit();
  
          }
  
  
  
          /**
  
           * Returns the render thread's Handler.  This may be called from any thread.
  
           */
  
          public RenderHandler getHandler() {
  
              return mHandler;
  
          }
  
  
  
          /**
  
           * Handles the surface-created callback from SurfaceView.  Prepares GLES and the Surface.
  
           */
  
          private void surfaceAvailable(SurfaceHolder holder, boolean newSurface) {
  
  
  
              Surface surface = holder.getSurface();
  
              mWindowSurface = new WindowSurface(mEglCore, surface, false);
  
              mWindowSurface.makeCurrent();
  
  
  
              // Create and configure the SurfaceTexture, which will receive frames from the
  
              // camera.  We set the textured rect's program to render from it.
  
              mTexProgram = new Texture2dProgram(Texture2dProgram.ProgramType.TEXTURE_EXT);
  
              int textureId = mTexProgram.createTextureObject();
  
              mCameraTexture = new SurfaceTexture(textureId);
  
              mRect.setTexture(textureId);
  
  
  
              if (!newSurface) {
  
                  // This Surface was established on a previous run, so no surfaceChanged()
  
                  // message is forthcoming.  Finish the surface setup now.
  
                  //
  
                  // We could also just call this unconditionally, and perhaps do an unnecessary
  
                  // bit of reallocating if a surface-changed message arrives.
  
                  mWindowSurfaceWidth = mWindowSurface.getWidth();
  
                  mWindowSurfaceHeight = mWindowSurface.getWidth();
  
                  finishSurfaceSetup();
  
              }
  
  
  
              mCameraTexture.setOnFrameAvailableListener(this);
  
  
  
  
  
  
  
          }
  
  
  
          /**
  
           * Releases most of the GL resources we currently hold (anything allocated by
  
           * surfaceAvailable()).
  
           * </p><p>
  
           * Does not release EglCore.
  
           */
  
          private void releaseGl() {
  
              GlUtil.checkGlError("releaseGl start");
  
  
  
              if (mWindowSurface != null) {
  
                  mWindowSurface.release();
  
                  mWindowSurface = null;
  
              }
  
              if (mTexProgram != null) {
  
                  mTexProgram.release();
  
                  mTexProgram = null;
  
              }
  
              GlUtil.checkGlError("releaseGl done");
  
  
  
              mEglCore.makeNothingCurrent();
  
          }
  
  
  
          /**
  
           * Handles the surfaceChanged message.
  
           * </p><p>
  
           * We always receive surfaceChanged() after surfaceCreated(), but surfaceAvailable()
  
           * could also be called with a Surface created on a previous run.  So this may not
  
           * be called.
  
           */
  
          private void surfaceChanged(int width, int height) {
  
              Log.d(TAG, "RenderThread surfaceChanged " + width + "x" + height);
  
  
  
              mWindowSurfaceWidth = width;
  
              mWindowSurfaceHeight = width;
  
              finishSurfaceSetup();
  
          }
  
  
  
          /**
  
           * Handles the surfaceDestroyed message.
  
           */
  
          private void surfaceDestroyed() {
  
              // In practice this never appears to be called -- the activity is always paused
  
              // before the surface is destroyed.  In theory it could be called though.
  
              Log.d(TAG, "RenderThread surfaceDestroyed");
  
              releaseGl();
  
          }
  
  
  
          /**
  
           * Sets up anything that depends on the window size.
  
           * </p><p>
  
           * Open the camera (to set mCameraAspectRatio) before calling here.
  
           */
  
          private void finishSurfaceSetup() {
  
              int width = mWindowSurfaceWidth;
  
              int height = mWindowSurfaceHeight;
  
              Log.d(TAG, "finishSurfaceSetup size=" + width + "x" + height +
  
                      " camera=" + mCameraPreviewWidth + "x" + mCameraPreviewHeight);
  
  
  
              // Use full window.
  
              GLES20.glViewport(0, 700, width, height);
  
  
  
              // Simple orthographic projection, with (0,0) in lower-left corner.
  
              Matrix.orthoM(mDisplayProjectionMatrix, 0, 0, width, 0, height, -1, 1);
  
  
  
              // Default position is center of screen.
  
              mPosX = width / 2.0f;
  
              mPosY = height / 2.0f;
  
  
  
              updateGeometry();
  
  
  
              // Ready to go, start the camera.
  
              Log.d(TAG, "starting camera preview");
  
              try {
  
                  mCamera.setPreviewTexture(mCameraTexture);
  
  
  
              } catch (IOException ioe) {
  
                  throw new RuntimeException(ioe);
  
              }
  
              mCamera.startPreview();
  
          }
  
  
  
          /**
  
           * Updates the geometry of mRect, based on the size of the window and the current
  
           * values set by the UI.
  
           */
  
          private void updateGeometry() {
  
              int width = mWindowSurfaceWidth;
  
              int height = mWindowSurfaceHeight;
  
  
  
  
  
              int smallDim = Math.min(width, height);
  
              // Max scale is a bit larger than the screen, so we can show over-size.
  
              float scaled = smallDim * (mSizePercent / 100.0f) * 1.25f;
  
              float cameraAspect = (float) mCameraPreviewWidth / mCameraPreviewHeight;
  
              int newWidth = Math.round(scaled * cameraAspect);
  
              int newHeight = Math.round(scaled);
  
  
  
              float zoomFactor = 1.0f - (mZoomPercent / 100.0f);
  
              int rotAngle = Math.round(360 * (mRotatePercent / 100.0f));
  
  
  
              mRect.setScale(newWidth, newHeight);
  
              mRect.setPosition(mPosX, mPosY);
  
              mRect.setRotation(rotAngle);
  
              mRectDrawable.setScale(zoomFactor);
  
  
  
              mMainHandler.sendRectSize(newWidth, newHeight);
  
              mMainHandler.sendZoomArea(Math.round(mCameraPreviewWidth * zoomFactor),
  
                      Math.round(mCameraPreviewHeight * zoomFactor));
  
              mMainHandler.sendRotateDeg(rotAngle);
  
          }
  
  
  
          @Override   // SurfaceTexture.OnFrameAvailableListener; runs on arbitrary thread
  
          public void onFrameAvailable(SurfaceTexture surfaceTexture) {
  
              mHandler.sendFrameAvailable();
  
          }
  
  
  
          /**
  
           * Handles incoming frame of data from the camera.
  
           */
  
          private void frameAvailable() {
  
              mCameraTexture.updateTexImage();
  
  
  
              draw();
  
          }
  
  
  
          /**
  
           * Draws the scene and submits the buffer.
  
           */
  
          private void draw() {
  
              GlUtil.checkGlError("draw start");
  
  
  
              GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
  
              GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT);
  
              mRect.draw(mTexProgram, mDisplayProjectionMatrix);
  
              mWindowSurface.swapBuffers();
  
  
  
              GlUtil.checkGlError("draw done");
  
          }
  
  
  
          /**
  
           * Opens a camera, and attempts to establish preview mode at the specified width
  
           * and height with a fixed frame rate.
  
           * </p><p>
  
           * Sets mCameraPreviewWidth / mCameraPreviewHeight.
  
           */
  
          private void openCamera(int desiredWidth, int desiredHeight, int desiredFps) {
  
              if (mCamera != null) {
  
                  throw new RuntimeException("camera already initialized");
  
              }
  
  
  
              Camera.CameraInfo info = new Camera.CameraInfo();
  
  
  
              // Try to find a front-facing camera (e.g. for videoconferencing).
  
              int numCameras = Camera.getNumberOfCameras();
  
              for (int i = 0; i &lt; numCameras; i++) {
  
                  Camera.getCameraInfo(i, info);
  
                  if (info.facing == Camera.CameraInfo.CAMERA_FACING_BACK) {
  
                      mCamera = Camera.open(i);
  
                      break;
  
                  }
  
              }
  
              if (mCamera == null) {
  
                  Log.d(TAG, "No front-facing camera found; opening default");
  
                  mCamera = Camera.open();    // opens first back-facing camera
  
              }
  
              if (mCamera == null) {
  
                  throw new RuntimeException("Unable to open camera");
  
              }
  
  
  
              Camera.Parameters parms = mCamera.getParameters();
  
  
  
              CameraUtils.choosePreviewSize(parms, desiredWidth, desiredHeight);
  
              parms.setFocusMode(Camera.Parameters.FOCUS_MODE_CONTINUOUS_PICTURE);
  
              // Try to set the frame rate to a constant value.
  
              int thousandFps = CameraUtils.chooseFixedPreviewFps(parms, desiredFps * 1000);
  
  
  
              // Give the camera a hint that we're recording video.  This can have a big
  
              // impact on frame rate.
  
              parms.setRecordingHint(true);
  
  
  
              mCamera.setParameters(parms);
  
  
  
              int[] fpsRange = new int[2];
  
              Camera.Size mCameraPreviewSize = parms.getPreviewSize();
  
              parms.getPreviewFpsRange(fpsRange);
  
              String previewFacts = mCameraPreviewSize.width + "x" + mCameraPreviewSize.height;
  
              if (fpsRange[0] == fpsRange[1]) {
  
                  previewFacts += " @" + (fpsRange[0] / 1000.0) + "fps";
  
              } else {
  
                  previewFacts += " @[" + (fpsRange[0] / 1000.0) +
  
                          " - " + (fpsRange[1] / 1000.0) + "] fps";
  
              }
  
              Log.i(TAG, "Camera config: " + previewFacts);
  
  
  
              mCameraPreviewWidth = mCameraPreviewSize.width;
  
              mCameraPreviewHeight = mCameraPreviewSize.height;
  
              mMainHandler.sendCameraParams(mCameraPreviewWidth, mCameraPreviewHeight,
  
                      thousandFps / 1000.0f);
  
          }
  
  
  
          /**
  
           * Stops camera preview, and releases the camera to the system.
  
           */
  
          private void releaseCamera() {
  
              if (mCamera != null) {
  
                  mCamera.stopPreview();
  
                  mCamera.release();
  
                  mCamera = null;
  
                  Log.d(TAG, "releaseCamera -- done");
  
              }
  
          }
  
      }
  
  
  
  }
  
  </p>

• ffmpeg - concatenation produces corrupted video

  24 novembre 2020, par codefox

  I want to copy scene from one video to another video. In order to make it fast, I want to use stream copy. But scene do not begin with keyframe, so i have to create two videos that contain parts of the scene : first video - using encoding - from start frame to frame preceding nearest keyframe, and second video - using stream copy - from keyframe to last frame of the scene. Then these two videos are concatenating using demuxer concat (with text file). For video created with encoding, I used encoder's settings as similar as possible to those in original video. Unfortunatelly, concatenation always produces highly corrupted video, with gray screen and uneven play, especially on second part. Interestingly, when concat order is reversed (part 1 after part 2), the resulting video is correct. Also, both videos concatenate with themselves flawlessly. What is wrong with these vidoes ?

  


  Some probed details for videos :

  


  Original video :

  


   Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.71.100
  Duration: 00:00:25.27, start: 0.032993, bitrate: 759 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 652 kb/s, 30 fps, 30 tbr, 30k tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 96 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=Main
codec_type=video
codec_time_base=1/60
codec_tag_string=avc1
codec_tag=0x31637661
width=1280
height=720
coded_width=1280
coded_height=720
closed_captions=0
has_b_frames=1
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv420p
level=31
color_range=tv
color_space=bt709
color_transfer=bt709
color_primaries=bt709
chroma_location=left
field_order=unknown
timecode=N/A
refs=3
is_avc=true
nal_length_size=4
id=N/A
r_frame_rate=30/1
avg_frame_rate=30/1
time_base=1/30000
start_pts=990
start_time=0.033000
duration_ts=758000
duration=25.266667
bit_rate=652591
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=758
nb_read_frames=758
nb_read_packets=758
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
TAG:language=und
TAG:handler_name=VideoHandler
[/STREAM]



  


  Creating part 1 and part 2 with scene

  


  ffmpeg -ss 11.966400 -to 13.666433 -i C:\vid\orig.mp4 -video_track_timescale 90k -an -y -r 30 -c:v libx264 -profile:v main -pix_fmt yuv420p -crf 20 -vf scale=out_color_matrix=bt709 -color_primaries bt709 -color_trc bt709 -colorspace bt709 C:/vid/p1.mp4
ffmpeg -ss 13.699667 -i C:\vid\orig.mp4 -video_track_timescale 90k -an -c copy -y C:/vid/p2.mp4


  


  Part 1 video

  


    Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.49.100
  Duration: 00:00:01.73, start: 0.000000, bitrate: 1007 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 1000 kb/s, 30 fps, 30 tbr, 90k tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=Main
codec_type=video
codec_time_base=1/60
codec_tag_string=avc1
codec_tag=0x31637661
width=1280
height=720
coded_width=1280
coded_height=720
closed_captions=0
has_b_frames=2
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv420p
level=31
color_range=tv
color_space=bt709
color_transfer=bt709
color_primaries=bt709
chroma_location=left
field_order=unknown
timecode=N/A
refs=4
is_avc=true
nal_length_size=4
id=N/A
r_frame_rate=30/1
avg_frame_rate=30/1
time_base=1/90000
start_pts=0
start_time=0.000000
duration_ts=156000
duration=1.733333
bit_rate=1000813
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=52
nb_read_frames=52
nb_read_packets=52
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
TAG:language=und
TAG:handler_name=VideoHandler
[/STREAM]


  


  Part 2 video

  


  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.49.100
  Duration: 00:00:11.60, start: 0.000000, bitrate: 533 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 528 kb/s, 30 fps, 30 tbr, 90k tbn, 60 tbc (default)
    Metadata:
      handler_name    : VideoHandler
[STREAM]
index=0
codec_name=h264
codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
profile=Main
codec_type=video
codec_time_base=1/60
codec_tag_string=avc1
codec_tag=0x31637661
width=1280
height=720
coded_width=1280
coded_height=720
closed_captions=0
has_b_frames=1
sample_aspect_ratio=1:1
display_aspect_ratio=16:9
pix_fmt=yuv420p
level=31
color_range=tv
color_space=bt709
color_transfer=bt709
color_primaries=bt709
chroma_location=left
field_order=unknown
timecode=N/A
refs=3
is_avc=true
nal_length_size=4
id=N/A
r_frame_rate=30/1
avg_frame_rate=5220000/174001
time_base=1/90000
start_pts=0
start_time=0.000000
duration_ts=1041120
duration=11.568000
bit_rate=528799
max_bit_rate=N/A
bits_per_raw_sample=8
nb_frames=348
nb_read_frames=347
nb_read_packets=348
DISPOSITION:default=1
DISPOSITION:dub=0
DISPOSITION:original=0
DISPOSITION:comment=0
DISPOSITION:lyrics=0
DISPOSITION:karaoke=0
DISPOSITION:forced=0
DISPOSITION:hearing_impaired=0
DISPOSITION:visual_impaired=0
DISPOSITION:clean_effects=0
DISPOSITION:attached_pic=0
DISPOSITION:timed_thumbnails=0
TAG:language=und
TAG:handler_name=VideoHandler
[/STREAM]


  


  Concatenating

  


  ffmpeg -f concat -i c:/vid/list.txt -y -an -c:v copy -video_track_timescale 90k c:/vid/out.mp4 -hide_banner


  


• ffmpeg failed to call avcodec_send_packet

  23 juin 2017, par Gizak

  Windows 10 x64,
  ffmpeg : 3.1, prebuilt dlls

  Absolute beginner here.

  I was trying out a very basic case to decode frames from a video file but desperately failed at every attempts. The final code snippet looks like this :

  #define __STDC_CONSTANT_MACROS
  
  extern "C" {
  
  #include<libavutil></libavutil>avutil.h>
  
  #include<libavutil></libavutil>imgutils.h>
  
  #include <libavformat></libavformat>avformat.h>
  
  #include <libavcodec></libavcodec>avcodec.h>
  
  #include <libswscale></libswscale>swscale.h>
  
  }
  
  
  
  #include <iostream>
  
  
  
  
  
  int main(int argc, char **argv) {
  
      AVFormatContext* ctx_format = nullptr;
  
      AVCodecContext* ctx_codec = nullptr;
  
      AVCodec* codec = nullptr;
  
      AVFrame* frame = av_frame_alloc();
  
      int stream_idx;
  
      SwsContext* ctx_sws = nullptr;
  
      const char* fin = argv[1];
  
      AVStream *vid_stream = nullptr;
  
      AVPacket* pkt = av_packet_alloc();
  
  
  
      av_register_all();
  
  
  
      if (int ret = avformat_open_input(&amp;ctx_format, fin, nullptr, nullptr) != 0) {
  
          std::cout &lt;&lt; 1 &lt;&lt; std::endl;
  
          return ret;
  
      }
  
      if (avformat_find_stream_info(ctx_format, nullptr) &lt; 0) {
  
          std::cout &lt;&lt; 2 &lt;&lt; std::endl;
  
          return -1; // Couldn't find stream information
  
      }
  
      av_dump_format(ctx_format, 0, fin, false);
  
  
  
      for (int i = 0; i &lt; ctx_format->nb_streams; i++)
  
          if (ctx_format->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO) {
  
              stream_idx = i;
  
              vid_stream = ctx_format->streams[i];
  
              break;
  
      }
  
      if (vid_stream == nullptr) {
  
          std::cout &lt;&lt; 4 &lt;&lt; std::endl;
  
          return -1;
  
      }
  
  
  
      codec = avcodec_find_decoder(vid_stream->codecpar->codec_id);
  
      if (!codec) {
  
          fprintf(stderr, "codec not found\n");
  
          exit(1);
  
      }
  
      ctx_codec = avcodec_alloc_context3(codec);
  
  
  
      if(avcodec_parameters_to_context(ctx_codec, vid_stream->codecpar)&lt;0)
  
          std::cout &lt;&lt; 512;
  
      if (avcodec_open2(ctx_codec, codec, nullptr)&lt;0) {
  
          std::cout &lt;&lt; 5;
  
          return -1;
  
      }
  
  
  
      //av_new_packet(pkt, pic_size);
  
  
  
      while(av_read_frame(ctx_format, pkt) >= 0){
  
          int ret = avcodec_send_packet(ctx_codec, pkt);
  
          if (ret &lt; 0 || ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
  
              std::cout &lt;&lt; "avcodec_send_packet: " &lt;&lt; ret &lt;&lt; std::endl;
  
              break;
  
          }
  
          if (pkt->stream_index == stream_idx) {
  
              while (ret  >= 0) {
  
                  ret = avcodec_receive_frame(ctx_codec, frame);
  
                  if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) {
  
                      //std::cout &lt;&lt; "avcodec_receive_frame: " &lt;&lt; ret &lt;&lt; std::endl;
  
                      break;
  
                  }
  
                  std::cout &lt;&lt; "frame: " &lt;&lt; ctx_codec->frame_number &lt;&lt; std::endl;
  
              }
  
          }
  
          av_packet_unref(pkt);
  
      }
  
  
  
      avformat_close_input(&amp;ctx_format);
  
      av_packet_unref(pkt);
  
      avcodec_free_context(&amp;ctx_codec);
  
      avformat_free_context(ctx_format);
  
      return 0;
  
  }
  
  </iostream>

  It basically leaves everything by its defaults and try to read packets and decode frames from them. The avcodec_send_packet always returned a negative value at the first or second round of calling in the while loop.

  Output for flv :

  > .\streamer_test.exe .\SampleVideo_360x240_5mb.flv
  
  Input #0, flv, from '.\SampleVideo_360x240_5mb.flv':
  
    Metadata:
  
      encoder         : Lavf53.24.2
  
    Duration: 00:00:53.32, start: 0.000000, bitrate: 787 kb/s
  
      Stream #0:0: Audio: aac (LC), 48000 Hz, 5.1, fltp, 384 kb/s
  
      Stream #0:1: Video: flv1, yuv420p, 320x240, 500 kb/s, 1k fps, 25 tbr, 1k tbn
  
  frame: 1
  
  [flv @ 000001545edb66c0] Bad picture start code
  
  [flv @ 000001545edb66c0] header damaged
  
  avcodec_send_packet: -1094995529

  Output for mp4 :

  > .\streamer_test.exe .\SampleVideo_360x240_10mb.mp4
  
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '.\SampleVideo_360x240_10mb.mp4':
  
    Metadata:
  
      major_brand     : isom
  
      minor_version   : 512
  
      compatible_brands: isomiso2avc1mp41
  
      creation_time   : 1970-01-01T00:00:00.000000Z
  
      encoder         : Lavf53.24.2
  
    Duration: 00:02:05.95, start: 0.000000, bitrate: 669 kb/s
  
      Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p, 320x240 [SAR 1:1 DAR 4:3], 282 kb/s, 15 fps, 15 t
  
  br, 15360 tbn, 30 tbc (default)
  
      Metadata:
  
        creation_time   : 1970-01-01T00:00:00.000000Z
  
        handler_name    : VideoHandler
  
      Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s (default)
  
      Metadata:
  
        creation_time   : 1970-01-01T00:00:00.000000Z
  
        handler_name    : SoundHandler
  
  [h264 @ 000002e2446a6cc0] Invalid NAL unit size (17191968 > 1007).
  
  [h264 @ 000002e2446a6cc0] Error splitting the input into NAL units.
  
  avcodec_send_packet: -1094995529

  Output for mkv :

  > .\streamer_test.exe .\SampleVideo_1280x720_2mb.mkv
  
  Input #0, matroska,webm, from '.\SampleVideo_1280x720_2mb.mkv':
  
    Metadata:
  
      ENCODER         : Lavf53.24.2
  
    Duration: 00:00:10.65, bitrate: 1575 kb/s
  
      Stream #0:0: Video: mpeg4 (Simple Profile), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 1k tbn, 25 tbc (de
  
  fault)
  
      Stream #0:1: Audio: aac (LC), 48000 Hz, 5.1, fltp (default)
  
  frame: 1
  
  [mpeg4 @ 000001166798ede0] header damaged
  
  avcodec_send_packet: -1

  So I guess there must be something wrong around avcodec_send_packet call but could not figured out what is missing. Any ideas about the causes ? Much appreciate your help !