Recherche avancée

Sur d’autres sites (7912)

• Texture rendering on iOS using OpenGL ES in Unity project

  30 mars 2016, par Time1ess

  I’m working on a project, part of which is to stream video to my iPhone. I use my laptop to create the video stream to my iPhone with ffmpeg.

  The stream code in shell is below :

  ffmpeg \
  
      -f avfoundation -i "1" -s 1280*720 -r 29.97 \
  
      -c:v mpeg2video -q:v 20 -pix_fmt yuv420p -g 1 -threads 4\
  
      -f mpegts udp://192.168.1.102:6666

  with this, I successfully create my video stream.

  In Unity, I want to decode the video stream to create a texture. After I have gone through some ffmpeg tutorial and Unity tutorial, I created my link library. Some of these codes are below (ask me if more is needed) :

  In my library :

  buffer alloc :

  uint8_t *buffer;
  
  int buffer_size;
  
  buffer_size = avpicture_get_size(AV_PIX_FMT_RGBA, VIEW_WIDTH, VIEW_HEIGHT);
  
  
  
  buffer = (uint8_t *) av_malloc(buffer_size*sizeof(uint8_t));
  
  
  
  avpicture_fill((AVPicture *) pFrameRGB, buffer, AV_PIX_FMT_RGBA,
  
                 VIEW_WIDTH, VIEW_HEIGHT);

  getContext :

      is->sws_ctx = sws_getContext
  
      (
  
       is->video_st->codec->width,
  
       is->video_st->codec->height,
  
       is->video_st->codec->pix_fmt,
  
       VIEW_WIDTH,
  
       VIEW_HEIGHT,
  
       AV_PIX_FMT_RGBA,
  
       SWS_BILINEAR,
  
       NULL,
  
       NULL,
  
       NULL
  
       );

  sws_scale :

  sws_scale(
  
            is->sws_ctx,
  
            (uint8_t const * const *)pFrame->data,
  
            pFrame->linesize,
  
            0,
  
            is->video_st->codec->height,
  
            pFrameRGB->data,
  
            pFrameRGB->linesize
  
            );

  texture render :

  static void UNITY_INTERFACE_API OnRenderEvent(int texID)
  
  {
  
      GLuint gltex = (GLuint)(size_t)(texID);
  
  
  
      glBindTexture(GL_TEXTURE_2D, gltex);
  
  
  
      glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, VIEW_WIDTH, VIEW_HEIGHT,
  
                      GL_RGBA, GL_UNSIGNED_BYTE, pFrameRGB->data[0]);
  
  
  
      glGetError();
  
      return;
  
  }
  
  
  
  extern "C" UnityRenderingEvent UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API GetRenderEventFunc()
  
  {
  
      return OnRenderEvent;
  
  }

  In Unity :

  texture created :

      private Texture2D texture;
  
      private int texID;
  
      texture = new Texture2D (width, height, TextureFormat.RGBA32, false);
  
      texture.filterMode = FilterMode.Point;
  
      texture.Apply ();
  
      GetComponent<renderer> ().material.mainTexture = texture;
  
      texID = texture.GetNativeTexturePtr ().ToInt32();
  
  </renderer>

  update func :

      void Update ()
  
      {
  
          GL.IssuePluginEvent(GetRenderEventFunc(), texID);
  
      }

  Video stream info :

  Input #0, mpegts, from 'udp://0.0.0.0:6666':
  
    Duration: N/A, start: 2.534467, bitrate: N/A
  
    Program 1 
  
      Metadata:
  
        service_name    : Service01
  
        service_provider: FFmpeg
  
      Stream #0:0[0x100]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], max. 104857 kb/s, 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc

  Leave other details, my library works fine on the Unity simulator, but when I compiled all my libraries for arm64 and used the xcode project that Unity created to build my app and ran it, I couldn’t get any texture rendered in my iPhone, I checked my network and I’m sure that data had been sent to my iPhone and the Debug log showed me that frame has been successfully decoded also the OnRenderEvent function had been called.

  FYI :

  Unity 5.3.2f1 Personal

  Xcode 7.2.1

  iOS 9.2.1

  ffmpeg 3.0

• how to fix the delay when play a ts stream via udp

  4 juin 2013, par user2223169

  I want to play udp ts stream on iPhone using ffmepg, but the video delayed almost 1s, so my question is how to fix the delay, currently I think I can specify the fifo_size, just like udp ://@:1234 ?fifo_size=1024, but the player can't play with this configuration, anybody knows how to reduce the buffer size to reduce the delay ?
  Thanks :)

• ffmpeg ios, decoding rtsp stream causes memory leak

  5 août 2013, par Simone M

  i have an ios application that stream a rtsp video via network using ffmpeg library.
  In random time i have a crash without error message, the only message in console is "Memory warning".
  Using instrument i detect some memory leaks, this link a screenshot about error : http://vincentvega.it/iphone/video.png, i think the problem is in the point of code where i make uiimage to show on screen :

  - (void)convertFrameToRGB
  
  {
  
      sws_scale(img_convert_ctx,
  
                pFrame->data,
  
                pFrame->linesize,
  
                0,
  
                pCodecCtx->height,
  
                picture.data,
  
                picture.linesize);
  
  
  
  }
  
  
  
  - (UIImage *)imageFromAVPicture:(AVPicture)pict width:(int)width height:(int)height
  
  {
  
      CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault;
  
      CFDataRef data = CFDataCreateWithBytesNoCopy(kCFAllocatorDefault, pict.data[0], pict.linesize[0]*height,kCFAllocatorNull);
  
      CGDataProviderRef provider = CGDataProviderCreateWithCFData(data);
  
      CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
  
      CGImageRef cgImage = CGImageCreate(width, 
  
                                         height, 
  
                                         8, 
  
                                         24, 
  
                                         pict.linesize[0], 
  
                                         colorSpace, 
  
                                         bitmapInfo, 
  
                                         provider, 
  
                                         NULL, 
  
                                         NO, 
  
                                         kCGRenderingIntentDefault);
  
      CGColorSpaceRelease(colorSpace);
  
      UIImage *image = [[UIImage alloc] initWithCGImage:cgImage];
  
  
  
  
  
  
  
      return [image autorelease];
  
  }

  how can i detect the function that causes memory problem ?

  SOLVED ! I just neeed to add the following code in "imagefromavcapture" :

  CGImageRelease(cgImage);
  
      CGDataProviderRelease(provider);
  
      CFRelease(data);