Recherche avancée

Sur d’autres sites (10737)

• Texture rendering on iOS using OpenGL ES in Unity project

  30 mars 2016, par Time1ess

  I’m working on a project, part of which is to stream video to my iPhone. I use my laptop to create the video stream to my iPhone with ffmpeg.

  The stream code in shell is below :

  ffmpeg \
  
      -f avfoundation -i "1" -s 1280*720 -r 29.97 \
  
      -c:v mpeg2video -q:v 20 -pix_fmt yuv420p -g 1 -threads 4\
  
      -f mpegts udp://192.168.1.102:6666

  with this, I successfully create my video stream.

  In Unity, I want to decode the video stream to create a texture. After I have gone through some ffmpeg tutorial and Unity tutorial, I created my link library. Some of these codes are below (ask me if more is needed) :

  In my library :

  buffer alloc :

  uint8_t *buffer;
  
  int buffer_size;
  
  buffer_size = avpicture_get_size(AV_PIX_FMT_RGBA, VIEW_WIDTH, VIEW_HEIGHT);
  
  
  
  buffer = (uint8_t *) av_malloc(buffer_size*sizeof(uint8_t));
  
  
  
  avpicture_fill((AVPicture *) pFrameRGB, buffer, AV_PIX_FMT_RGBA,
  
                 VIEW_WIDTH, VIEW_HEIGHT);

  getContext :

      is->sws_ctx = sws_getContext
  
      (
  
       is->video_st->codec->width,
  
       is->video_st->codec->height,
  
       is->video_st->codec->pix_fmt,
  
       VIEW_WIDTH,
  
       VIEW_HEIGHT,
  
       AV_PIX_FMT_RGBA,
  
       SWS_BILINEAR,
  
       NULL,
  
       NULL,
  
       NULL
  
       );

  sws_scale :

  sws_scale(
  
            is->sws_ctx,
  
            (uint8_t const * const *)pFrame->data,
  
            pFrame->linesize,
  
            0,
  
            is->video_st->codec->height,
  
            pFrameRGB->data,
  
            pFrameRGB->linesize
  
            );

  texture render :

  static void UNITY_INTERFACE_API OnRenderEvent(int texID)
  
  {
  
      GLuint gltex = (GLuint)(size_t)(texID);
  
  
  
      glBindTexture(GL_TEXTURE_2D, gltex);
  
  
  
      glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, VIEW_WIDTH, VIEW_HEIGHT,
  
                      GL_RGBA, GL_UNSIGNED_BYTE, pFrameRGB->data[0]);
  
  
  
      glGetError();
  
      return;
  
  }
  
  
  
  extern "C" UnityRenderingEvent UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API GetRenderEventFunc()
  
  {
  
      return OnRenderEvent;
  
  }

  In Unity :

  texture created :

      private Texture2D texture;
  
      private int texID;
  
      texture = new Texture2D (width, height, TextureFormat.RGBA32, false);
  
      texture.filterMode = FilterMode.Point;
  
      texture.Apply ();
  
      GetComponent<renderer> ().material.mainTexture = texture;
  
      texID = texture.GetNativeTexturePtr ().ToInt32();
  
  </renderer>

  update func :

      void Update ()
  
      {
  
          GL.IssuePluginEvent(GetRenderEventFunc(), texID);
  
      }

  Video stream info :

  Input #0, mpegts, from 'udp://0.0.0.0:6666':
  
    Duration: N/A, start: 2.534467, bitrate: N/A
  
    Program 1 
  
      Metadata:
  
        service_name    : Service01
  
        service_provider: FFmpeg
  
      Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], max. 104857 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

  Leave other details, my library works fine on the Unity simulator, but when I compiled all my libraries for arm64 and used the xcode project that Unity created to build my app and ran it, I couldn’t get any texture rendered in my iPhone, I checked my network and I’m sure that data had been sent to my iPhone and the Debug log showed me that frame has been successfully decoded also the OnRenderEvent function had been called.

  FYI :

  Unity 5.3.2f1 Personal

  Xcode 7.2.1

  iOS 9.2.1

  ffmpeg 3.0

• Revision 30118 : protéger des erreurs si pas configuré

  23 juillet 2009, par kent1@… — Log

  protéger des erreurs si pas configuré

• FFmpeg Time Lapse from Sources with Long Frozen Tail End

  5 juillet 2019, par Rich_F

  My source for inputs into FFmpeg is either one AVI file or a concat of many of them. Either way my resulting timelapse file has a long tail of frames that are a repeat of a single frame. It’s like a very long freeze frame is resulting on the end of my output file.

  I’m on an older Mac Pro so I can’t update my FFmpeg. I have a laptop that has a newer version and I get the same there as well. I’m not sure if it’s because my source files are AVI or not.

  ffmpeg version 4.1.1 Copyright (c) 2000-2019 the FFmpeg developers
  
      built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  
      configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags=-I/System/Library/Frameworks/JavaVM.framework/Versions/Current/Headers/ --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr
  
      libavutil      56. 22.100 / 56. 22.100
  
      libavcodec     58. 35.100 / 58. 35.100
  
      libavformat    58. 20.100 / 58. 20.100
  
      libavdevice    58.  5.100 / 58.  5.100
  
      libavfilter     7. 40.101 /  7. 40.101
  
      libavresample   4.  0.  0 /  4.  0.  0
  
      libswscale      5.  3.100 /  5.  3.100
  
      libswresample   3.  3.100 /  3.  3.100
  
      libpostproc    55.  3.100 / 55.  3.100
  
  [avi @ 0x7fa82080c800] sample size (1) != block align (2)
  
  Guessed Channel Layout for Input Stream #0.1 : mono
  
  Input #0, avi, from 'concat:16460001.AVI|16460002.AVI|16460003.AVI|16460004.AVI|16460005.AVI|16460006.AVI|16460007.AVI|16460008.AVI|16460009.AVI|16460010.AVI|16460011.AVI|16460012.AVI|16460013.AVI|16460014.AVI|16460015.AVI|16460016.AVI|16460017.AVI|16460018.AVI|16460019.AVI|16460020.AVI|16460021.AVI':
  
      Duration: 00:10:02.00, start: 0.000000, bitrate: 365923 kb/s
  
          Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), yuvj420p(pc, bt470bg/unknown/unknown), 1280x720, 30 fps, 30 tbr, 30 tbn, 30 tbc
  
          Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 32000 Hz, mono, s16, 512 kb/s
  
  Stream mapping:
  
      Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
  
      Stream #0:1 -> #0:1 (pcm_s16le (native) -> aac (native))
  
  Press [q] to stop, [?] for help
  
  [libx264 @ 0x7fa82082cc00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.1 Cache64
  
  [libx264 @ 0x7fa82082cc00] profile High, level 3.1
  
  [libx264 @ 0x7fa82082cc00] 264 - core 155 r2917 0a84d98 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=16 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
  
  Output #0, mp4, to 'out.mp4':
  
      Metadata:
  
          encoder         : Lavf58.20.100
  
          Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc, progressive), 1280x720, q=-1--1, 16 fps, 16384 tbn, 16 tbc
  
          Metadata:
  
              encoder         : Lavc58.35.100 libx264
  
          Side data:
  
              cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
  
          Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 32000 Hz, mono, fltp, 69 kb/s
  
          Metadata:
  
              encoder         : Lavc58.35.100 aac
  
  frame= 1962 fps=1.2 q=-1.0 Lsize=  136725kB time=03:24:07.00 bitrate=  91.5kbits/s dup=0 drop=365448 speed=7.41x
  
  video:31548kB audio:103624kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.149437%
  
  [libx264 @ 0x7fa82082cc00] frame I:10    Avg QP:18.74  size: 62176
  
  [libx264 @ 0x7fa82082cc00] frame P:514   Avg QP:21.24  size: 30528
  
  [libx264 @ 0x7fa82082cc00] frame B:1438  Avg QP:22.74  size: 11121
  
  [libx264 @ 0x7fa82082cc00] consecutive B-frames:  1.4%  2.0%  2.0% 94.6%
  
  [libx264 @ 0x7fa82082cc00] mb I  I16..4:  2.1% 97.2%  0.7%
  
  [libx264 @ 0x7fa82082cc00] mb P  I16..4:  2.0% 33.3%  0.1%  P16..4: 37.7% 12.0% 10.4%  0.0%  0.0%    skip: 4.5%
  
  [libx264 @ 0x7fa82082cc00] mb B  I16..4:  0.7% 11.5%  0.0%  B16..8: 29.1%  4.8%  1.3%  direct:12.0%  skip:40.6%  L0:47.5% L1:42.9% BI: 9.6%
  
  [libx264 @ 0x7fa82082cc00] 8x8 transform intra:94.3% inter:83.0%
  
  [libx264 @ 0x7fa82082cc00] coded y,uvDC,uvAC intra: 73.4% 66.7% 8.5% inter: 25.9% 46.8% 2.2%
  
  [libx264 @ 0x7fa82082cc00] i16 v,h,dc,p: 17% 33% 26% 24%
  
  [libx264 @ 0x7fa82082cc00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 13% 23% 51%  1%  2%  1%  4%  1%  2%
  
  [libx264 @ 0x7fa82082cc00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 17% 21% 19%  5%  8%  7% 13%  5%  5%
  
  [libx264 @ 0x7fa82082cc00] i8c dc,h,v,p: 46% 29% 23%  2%
  
  [libx264 @ 0x7fa82082cc00] Weighted P-Frames: Y:47.5% UV:19.3%
  
  [libx264 @ 0x7fa82082cc00] ref P L0: 45.0% 13.8% 20.6% 15.7%  5.0%
  
  [libx264 @ 0x7fa82082cc00] ref B L0: 61.1% 30.4%  8.5%
  
  [libx264 @ 0x7fa82082cc00] ref B L1: 85.8% 14.2%
  
  [libx264 @ 0x7fa82082cc00] kb/s:2107.54
  
  [aac @ 0x7fa82081ea00] Qavg: 130.084

  I’ve read this thread and tried to follow it :

  Create time lapse video from other video

  Here are some trials I’ve attempted before, all with the same output :

  # ffmpeg -y -i $mov -vf framestep=10,setpts=N/FRAME_RATE/TB,fps=2 -r 30 $out
  
  # ffmpeg -y -i $mov -vf framestep=10,setpts=.01*PTS -r 30 $out
  
  # ffmpeg -y -i $mov -vf framestep=10,setpts=.1*PTS -r 30 $out
  
  # ffmpeg -y -i "concat:16460001.AVI|16460002.AVI|16460003.AVI|16460004.AVI|16460005.AVI|16460006.AVI|16460007.AVI|16460008.AVI|16460009.AVI|16460010.AVI|16460011.AVI|16460012.AVI|16460013.AVI|16460014.AVI|16460015.AVI|16460016.AVI|16460017.AVI|16460018.AVI|16460019.AVI|16460020.AVI|16460021.AVI" -vf framestep=10,setpts=.05*PTS -r 30 $out
  
  ffmpeg -y -i "concat:16460001.AVI|16460002.AVI|16460003.AVI|16460004.AVI|16460005.AVI|16460006.AVI|16460007.AVI|16460008.AVI|16460009.AVI|16460010.AVI|16460011.AVI|16460012.AVI|16460013.AVI|16460014.AVI|16460015.AVI|16460016.AVI|16460017.AVI|16460018.AVI|16460019.AVI|16460020.AVI|16460021.AVI" -r 16 -filter:v "setpts=0.01*PTS" out.mp4

  Am I overlooking something ? I’m trying to speed up the inputs into a single file to quickly review incoming security footage. How can I do this without the super long useless tail at the end ?