Recherche avancée

Sur d’autres sites (5594)

• ffmpeg video to opengl texture

  23 avril 2017, par Infiniti Fizz

  I’m trying to render frames grabbed and converted from a video using ffmpeg to an OpenGL texture to be put on a quad. I’ve pretty much exhausted google and not found an answer, well I’ve found answers but none of them seem to have worked.

  Basically, I am using avcodec_decode_video2() to decode the frame and then sws_scale() to convert the frame to RGB and then glTexSubImage2D() to create an openGL texture from it but can’t seem to get anything to work.

  I’ve made sure the "destination" AVFrame has power of 2 dimensions in the SWS Context setup. Here is my code :

  SwsContext *img_convert_ctx = sws_getContext(pCodecCtx->width,
  
                  pCodecCtx->height, pCodecCtx->pix_fmt, 512,
  
                  256, PIX_FMT_RGB24, SWS_BICUBIC, NULL,
  
                  NULL, NULL);
  
  
  
  //While still frames to read
  
  while(av_read_frame(pFormatCtx, &packet)>=0) {
  
      glClear(GL_COLOR_BUFFER_BIT);
  
  
  
      //If the packet is from the video stream
  
      if(packet.stream_index == videoStream) {
  
          //Decode the video
  
          avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packet);
  
  
  
          //If we got a frame then convert it and put it into RGB buffer
  
          if(frameFinished) {
  
              printf("frame finished: %i\n", number);
  
              sws_scale(img_convert_ctx, pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize);
  
  
  
              glBindTexture(GL_TEXTURE_2D, texture);
  
              //gluBuild2DMipmaps(GL_TEXTURE_2D, 3, pCodecCtx->width, pCodecCtx->height, GL_RGB, GL_UNSIGNED_INT, pFrameRGB->data);
  
              glTexSubImage2D(GL_TEXTURE_2D, 0, 0,0, 512, 256, GL_RGB, GL_UNSIGNED_BYTE, pFrameRGB->data[0]);
  
              SaveFrame(pFrameRGB, pCodecCtx->width, pCodecCtx->height, number);
  
              number++;
  
          }
  
      }
  
  
  
      glColor3f(1,1,1);
  
      glBindTexture(GL_TEXTURE_2D, texture);
  
      glBegin(GL_QUADS);
  
          glTexCoord2f(0,1);
  
          glVertex3f(0,0,0);
  
  
  
          glTexCoord2f(1,1);
  
          glVertex3f(pCodecCtx->width,0,0);
  
  
  
          glTexCoord2f(1,0);
  
          glVertex3f(pCodecCtx->width, pCodecCtx->height,0);
  
  
  
          glTexCoord2f(0,0);
  
          glVertex3f(0,pCodecCtx->height,0);
  
  
  
      glEnd();

  As you can see in that code, I am also saving the frames to .ppm files just to make sure they are actually rendering, which they are.

  The file being used is a .wmv at 854x480, could this be the problem ? The fact I’m just telling it to go 512x256 ?

  P.S. I’ve looked at this Stack Overflow question but it didn’t help.

  Also, I have glEnable(GL_TEXTURE_2D) as well and have tested it by just loading in a normal bmp.

  EDIT

  I’m getting an image on the screen now but it is a garbled mess, I’m guessing something to do with changing things to a power of 2 (in the decode, swscontext and gluBuild2DMipmaps as shown in my code). I’m usually nearly exactly the same code as shown above, only I’ve changed glTexSubImage2D to gluBuild2DMipmaps and changed the types to GL_RGBA.

  Here is what the frame looks like :

  

  EDIT AGAIN

  Just realised I haven’t showed the code for how pFrameRGB is set up :

  //Allocate video frame for 24bit RGB that we convert to.
  
  AVFrame *pFrameRGB;
  
  pFrameRGB = avcodec_alloc_frame();
  
  
  
  if(pFrameRGB == NULL) {
  
      return -1;
  
  }
  
  
  
  //Allocate memory for the raw data we get when converting.
  
  uint8_t *buffer;
  
  int numBytes;
  
  numBytes = avpicture_get_size(PIX_FMT_RGB24, pCodecCtx->width, pCodecCtx->height);
  
  buffer = (uint8_t *) av_malloc(numBytes*sizeof(uint8_t));
  
  
  
  //Associate frame with our buffer
  
  avpicture_fill((AVPicture *) pFrameRGB, buffer, PIX_FMT_RGB24,
  
      pCodecCtx->width, pCodecCtx->height);

  Now that I ahve changed the PixelFormat in avgpicture_get_size to PIX_FMT_RGB24, I’ve done that in SwsContext as well and changed GluBuild2DMipmaps to GL_RGB and I get a slightly better image but it looks like I’m still missing lines and it’s still a bit stretched :

  

  Another Edit

  After following Macke’s advice and passing the actual resolution to OpenGL I get the frames nearly proper but still a bit skewed and in black and white, also it’s only getting 6fps now rather than 110fps :

  

  P.S.

  I’ve got a function to save the frames to image after sws_scale() and they are coming out fine as colour and everything so something in OGL is making it B&W.

  LAST EDIT

  Working ! Okay I have it working now, basically I am not padding out the texture to a power of 2 and just using the resolution the video is.

  I got the texture showing up properly with a lucky guess at the correct glPixelStorei()

  glPixelStorei(GL_UNPACK_ALIGNMENT, 2);

  Also, if anyone else has the subimage() showing blank problem like me, you have to fill the texture at least once with glTexImage2D() and so I use it once in the loop and then use glTexSubImage2D() after that.

  Thanks Macke and datenwolf for all your help.

• Saying Goodbye To Old Machines

  1er décembre 2014, par Multimedia Mike — General, powerpc, via

  I recently sent a few old machines off for recycling. Both had relevance to the early days of the FATE testing effort. As is my custom, I photographed them (poorly, of course).

  First, there’s the PowerPC-based Mac Mini I procured thanks to a Craigslist ad in late 2006. I had plans to develop automated FFmpeg building and testing and was already looking ahead toward testing multiple CPU architectures. Again, this was 2006 and PowerPC wasn’t completely on the outs yet– although Apple’s MacTel transition was in full swing, the entire new generation of video game consoles was based on PowerPC.

  
  

  

  Click for larger image

  
  

  I remember trying to find a Mac Mini PPC on Craigslist. Many were to be found, but all asked more than the price of even a new Mac Mini Intel, always because the seller was leaving all of last year’s applications and perhaps including a monitor, neither of which I needed. Fortunately, I found this bare Mac Mini. Also fortunate was the fact that it was far easier to install Linux on it than the first PowerPC machine I owned.

  After FATE operation transitioned away from me, I still kept the machine in service as an edge server and automated backup machine. That is, until the hard drive failed on reboot one day. Thus, when it was finally time to recycle the computer, I felt it necessary to disassemble the machine and remove the hard drive for possible salvage and then for destruction.
  

  If you’ve ever attempted to upgrade or otherwise service this style of Mac Mini, you will no doubt recognize the pictured paint scraper tool as standard kit. I have had that tool since I first endeavored to upgrade the RAM to 1 GB from the standard 1/2 GB. Performing such activities on a Mac Mini is tedious, but only if you care about putting it back together afterwards.

  The next machine is a bit older. I put it together nearly a decade ago, early in 2005. This machine’s original duty was “download agent”– this would be more specifically called a BitTorrent machine in modern tech parlance. Back then, I placed it on someone else’s woefully underutilized home broadband connection (with their permission, of course) when I was too cheap to upgrade from dialup.

  
  

  

  Click for larger image

  
  

  This is a small form factor system from VIA that was clearly designed with home theater PC (HTPC) use cases in mind. It has a VIA C3 x86-compatible CPU (according to my notes, Centaur VIA Samuel 2 stepping 03, flags : fpu de tsc msr cx8 mtrr pge mmx 3dnow) and 128 MB of RAM (initially ; I upgraded it to 512 MB some years later, just for the sake of doing it). And then there was the 120 GB PATA HD for all that downloaded goodness.

  
  

  

  Click for larger image

  
  

  I have specific memories of a time when my main computer at home wasn’t working correctly for one reason or another. Instead, I logged into this machine remotely via SSH to make several optimizations and fixes on FFmpeg’s VP3/Theora video decoder, all from the terminal, without being able to see the decoded images with my own eyes (which is why I insist that even blind people could work on video codecs).

  By the time I got my own broadband, I had become inspired to attempt the automated build and test system for FFmpeg. This was the machine I used for prototyping early brainstorms of FATE. By the time I put a basic build/test system into place in early 2008, I had much faster computers that could build and test the project– obvious limitation of this machine is that it could take at least 1/2 hour to build the entire codebase, and that was the project from 8 years ago.

  So the machine got stuffed in a closet somewhere along the line. The next time I pulled it out was in 2010 when I wanted to toy with Dreamcast programming once more (the machine appears in one of the photos in this post). This was the only machine I still owned which still had an RS-232 serial port (I didn’t know much about USB serial converters yet), plus it still had a bunch of pre-compiled DC homebrew binaries (I was having trouble getting the toolchain to work right).

  The next time I dusted off this machine was late last year when I was trying some experiments with the Microsoft Xbox’s IDE drive (a photo in that post also shows the machine ; this thing shows up a lot on this blog). The VIA machine was the only machine I still owned which had 40-pin IDE connectors which was crucial to my experiment.

  At this point, I was trying to make the machine more useful which meant replacing the ancient Gentoo Linux distribution as well as simply interacting with it via a keyboard and mouse. I have a long Evernote entry documenting a comedy of errors revolving around this little box. The interaction troubles were due to the fact that I didn’t have any PS/2 keyboards left and I couldn’t make a USB keyboard work with it. Diego was able to explain that I needed to flip a bit in the BIOS to address this which worked. As for upgrading the OS, I tried numerous Linux distributions large and small, mostly focusing on the small. None worked. I eventually learned that, while I was trying to use i686 distributions, this machine did not actually qualify as an i686 CPU ; installations usually booted but failed because the default kernel required the cmov instruction. I was advised to try i386 distros instead. My notes don’t indicate whether I had any luck on this front before I gave up and moved on.

  I just made the connection that this VIA machine has two 40-pin IDE connectors which means that the thing was technically capable of supporting up to 4 IDE devices. Obviously, the computer couldn’t really accommodate that in terms of space or power. When I wanted to try installing a new OS, I needed take off the top and connect a rather bulky IDE CD-ROM drive. This computer’s casing was supposed to be able to support a slimline optical drive (perhaps like the type found in laptops), but I could never quite visualize how that was supposed to work, space-wise. When I disassembled the PowerPC Mac Mini, I realized I might be able to repurpose that machines optical drive for this computer. Obviously, I thought better of trying since both machines are off to the recycle pile.

  I would still like to work on the Xbox project a bit more, but I procured a different, unused, much more powerful yet still old computer that has a motherboard with 1 PATA connector in addition to 6 SATA connectors. If I ever get around to toying with Linux kernel development, this should be a much more appropriate platform to use.

  I thought about turning this machine into an old Windows XP (and lower, down to Windows 3.1) gaming platform ; the capabilities of the machine would probably be perfect for a huge portion of my Windows game collection. But I think the lack of an optical drive renders this idea intractable. External USB drives are likely out of the question since there is very little chance that this motherboard featured USB 2.0 (the specs don’t mention 2.0, so the USB ports are probably 1.1).

  So it is with fond memories that I send off both machines, sans hard drives, to the recycle pile. I’m still deciding on an appropriate course of action for failed hard drives, though.

• can't compile ffmpeg on Solaris 10 sparc

  9 juin 2014, par Raoul

  Has anyone compiled ffmpeg 0.6.1 for Solaris 10 sparc ? I’m getting the following errors :

      uname -a
  
  SunOS SERVERNAME 5.10 Generic_118833-36 sun4u sparc SUNW,Sun-Fire-V440
  
  
  
  bwddmadm@bwddmliv>/export/home/USERID/ffmpeg-0.6.1/configure --prefix=/export/home/USERID/ffmpegX --extra-cflags="-fPIC" --disable-mmx --disable-protocol=udp --disable-encoder=nellymoser
  
  Broken shell detected.  Trying alternatives.
  
  Trying shell bash
  
  grep: illegal option -- q
  
  Usage: grep -hblcnsviw pattern file . . .
  
  grep: illegal option -- q
  
  Usage: grep -hblcnsviw pattern file . . .
  
  grep: illegal option -- q
  
  Usage: grep -hblcnsviw pattern file . . .
  
  grep: illegal option -- q
  
  Usage: grep -hblcnsviw pattern file . . .
  
  grep: illegal option -- q
  
  Usage: grep -hblcnsviw pattern file . . .
  
  grep: illegal option -- q
  
  Usage: grep -hblcnsviw pattern file . . .
  
  grep: illegal option -- q
  
  Usage: grep -hblcnsviw pattern file . . .
  
  grep: illegal option -- q
  
  Usage: grep -hblcnsviw pattern file . . .
  
  Unknown C compiler gcc
  
  awk: syntax error near line 1
  
  awk: illegal statement near line 1
  
  grep: illegal option -- q
  
  Usage: grep -hblcnsviw pattern file . . .
  
  install prefix            /export/home/USERID/ffmpegX
  
  source path               /export/home/USERID/ffmpeg-0.6.1
  
  C compiler                gcc
  
  .align is power-of-two    no
  
  ARCH                      sparc (generic)
  
  big-endian                no
  
  runtime cpu detection     no
  
  VIS enabled               yes
  
  gprof enabled             no
  
  debug symbols             yes
  
  strip symbols             yes
  
  optimizations             yes
  
  static                    yes
  
  shared                    no
  
  postprocessing support    no
  
  new filter support        no
  
  filters using lavformat   no
  
  network support           yes
  
  threading support         no
  
  SDL support               no
  
  Sun medialib support      no
  
  AVISynth enabled          no
  
  libdc1394 support         no
  
  libdirac enabled          no
  
  libfaac enabled           no
  
  libfaad enabled           no
  
  libfaad dlopened          no
  
  libgsm enabled            no
  
  libmp3lame enabled        no
  
  libnut enabled            no
  
  libopencore-amrnb support no
  
  libopencore-amrwb support no
  
  libopenjpeg enabled       no
  
  librtmp enabled           no
  
  libschroedinger enabled   no
  
  libspeex enabled          no
  
  libtheora enabled         no
  
  libvorbis enabled         no
  
  libvpx enabled            no
  
  libx264 enabled           no
  
  libxvid enabled           no
  
  zlib enabled              no
  
  bzlib enabled             no
  
  
  
  Enabled decoders:
  
  pr:  -- empty file
  
  
  
  Enabled encoders:
  
  pr:  -- empty file
  
  
  
  Enabled hwaccels:
  
  pr:  -- empty file
  
  
  
  Enabled parsers:
  
  pr:  -- empty file
  
  
  
  Enabled demuxers:
  
  pr:  -- empty file
  
  
  
  Enabled muxers:
  
  pr:  -- empty file
  
  
  
  Enabled protocols:
  
  pr:  -- empty file
  
  
  
  Enabled filters:
  
  pr:  -- empty file
  
  
  
  Enabled bsfs:
  
  pr:  -- empty file
  
  
  
  Enabled indevs:
  
  pr:  -- empty file
  
  
  
  Enabled outdevs:
  
  pr:  -- empty file
  
  
  
  License: LGPL version 2.1 or later
  
  Creating config.mak and config.h...
  
  bwddmadm@bwddmliv> gmake
  
  /export/home/USERID/ffmpeg-0.6.1/version.sh: syntax error at line 4: `revision=$' unexpected
  
  CC      libavdevice/alldevices.o
  
  libavdevice/alldevices.c: In function `avdevice_register_all':
  
  libavdevice/alldevices.c:42: error: `CONFIG_ALSA_OUTDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:42: error: (Each undeclared identifier is reported only once
  
  libavdevice/alldevices.c:42: error: for each function it appears in.)
  
  libavdevice/alldevices.c:42: error: `CONFIG_ALSA_INDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:43: error: `CONFIG_AUDIO_BEOS_OUTDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:43: error: `CONFIG_AUDIO_BEOS_INDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:44: error: `CONFIG_BKTR_INDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:45: error: `CONFIG_DV1394_INDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:46: error: `CONFIG_JACK_INDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:47: error: `CONFIG_OSS_OUTDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:47: error: `CONFIG_OSS_INDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:48: error: `CONFIG_V4L2_INDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:49: error: `CONFIG_V4L_INDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:50: error: `CONFIG_VFWCAP_INDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:51: error: `CONFIG_X11_GRAB_DEVICE_INDEV' undeclared (first use in this function)
  
  libavdevice/alldevices.c:54: error: `CONFIG_LIBDC1394_INDEV' undeclared (first use in this function)
  
  gmake: *** [libavdevice/alldevices.o] Error 1
  
  bwddmadm@bwddmliv> exit
  
  
  
  script done on Fri Jan 14 11:34:05 2011