Recherche avancée

Sur d’autres sites (11217)

• Subtitling Sierra RBT Files

  2 juin 2016, par Multimedia Mike — Game Hacking

  This is part 2 of the adventure started in my Subtitling Sierra VMD Files post. After I completed the VMD subtitling, The Translator discovered a wealth of animation files in a format called RBT (this apparently stands for “Robot” but I think “Ribbit” format could be more fun). What are we going to do ? We had come so far by solving the VMD subtitling problem for Phantasmagoria. It would be a shame if the effort ground to a halt due to this.

  Fortunately, the folks behind the ScummVM project already figured out enough of the format to be able to decode the RBT files in Phantasmagoria.

  In the end, I was successful in creating a completely standalone tool that can take a Robot file and a subtitle file and create a new Robot file with subtitles. The source code is here (subtitle-rbt.c). Here’s what the final result looks like :

  
  
  “What’s in the refrigerator ?” I should note at this juncture that I am not sure if this particular Robot file even has sound or dialogue since I was conducting these experiments on a computer with non-working audio.
  

  The RBT Format
  I have created a new MultimediaWiki page describing the Robot Animation format based on the ScummVM source code. I have not worked with a format quite like this before. These are paletted animations which consist of a sequence of independent frames that are designed to be overlaid on top of static background. Because of these characteristics, each frame encodes its own unique dimensions and origin coordinate within the frame. While the Phantasmagoria VMD files are usually 288×144 (which are usually double-sized for the benefit of a 640×400 Super VGA canvas), these frames are meant to be plotted on a game field that was roughly 576×288 (288×144 doublesized).
  

  For example, 2 minimalist animation frames from a desk investigation Robot file :

  
  
  100×147

  
  101×149
  

  As for compression, my first impression was that the algorithm was the same as VMD. This is wrong. It evidently uses an unmodified version of a standard algorithm called Lempel-Ziv-Stac (LZS). It shows up in several RFCs and was apparently used in MS-DOS’s transparent disk compression scheme.

  Approach
  Thankfully, many of the lessons I learned from the previous project are applicable to this project, including : subtitle library interfacing, subtitling in the paletted colorspace, and replacing encoded frames from the original file instead of trying to create a new file.

  Here is the pitch for this project :

  • Create a C program that can traverse through an input file, piece by piece, and generate an output file. The result of this should be a bitwise identical file.
  • Adapt the LZS compression decoding algorithm from ScummVM into the new tool. Make the tool dump raw Portable NetMap (PNM) files of varying dimensions and ensure that they look correct.
  • Compress using LZS.
  • Stretch the frames and draw subtitles.
  • More compression. Find the minimum window for each frame.

  Compression
  Normally, my first goal is to decompress the video and store the data in a raw form. However, this turned out to be mathematically intractable. While the format does support both compressed and uncompressed frames (even though ScummVM indicates that the uncompressed path is yet unexercised), the goal of this project requires making the frames so large that they overflow certain parameters of the file.

  A Robot file has a sequence of frames and 2 tables describing the size of each frame. One table describes the entire frame size (audio + video) while the second table describes just the video frame size. Since these tables only use 16 bits to specify a size, the maximum frame size is 65536 bytes. Leaving space for the audio portion of the frame, this only leaves a per-frame byte budget of about 63000 bytes for the video. Expanding the frame to 576×288 (165,888 pixels) would overflow this limit.

  Anyway, the upshot is that I needed to compress the data up front.

  Fortunately, the LZS compressor is pretty straightforward, at least if you have experience writing VLC-oriented codecs. While the algorithm revolves around back references, my approach was to essentially write an RLE encoder. My compressor would search for runs of data (plentiful when I started to stretch the frame for subtitling purposes). When a run length of n=3 or more of the same pixel is found, encode the pixel by itself, and then store a back reference of offset -1 and length (n-1). It took a little while to iron out a few problems, but I eventually got it to work perfectly.

  I have to say, however, that the format is a little bit weird in how it codes very large numbers. The length encoding is somewhat Golomb-like, i.e., smaller values are encoded with fewer bits. However, when it gets to large numbers, it starts encoding counts of 15 as blocks of 1111. For example, 24 is bigger than 7. Thus, emit 1111 into the bitstream and subtract 8 from 23 -> 16. Still bigger than 15, so stuff another 1111 into the bitstream and subtract 15. Now we’re at 1, so stuff 0001. So 24 is 11111111 0001. 12 bits is not too horrible. But the total number of bytes (value / 30). So a value of 300 takes around 10 bytes (80 bits) to encode.

  Palette Slices
  As in the VMD subtitling project, I took the subtitle color offered in the subtitle spec file as a suggestion and used Euclidean distance to match to the closest available color in the palette. One problem, however, is that the palette is a lot smaller in these animations. According to my notes, for the set of animations I scanned, only about 80 colors were specified, starting at palette index 55. I hypothesize that different slices of the palette are reserved for different uses. E.g., animation, background, and user interface. Thus, there is a smaller number of colors to draw upon for subtitling purposes.

  Scaling
  One bit of residual weirdness in this format is the presence of a per-frame scale factor. While most frames set this to 100 (100% scale), I have observed 70%, 80%, and 90%. ScummVM is a bit unsure about how to handle these, so I am as well. However, I eventually realized I didn’t really need to care, at least not when decoding and re-encoding the frame. Just preserve the scale factor. I intend to modify the tool further to take scale factor into account when creating the subtitle.

  The Final Resolution
  Right around the time that I was composing this post, The Translator emailed me and notified me that he had found a better way to subtitle the Robot files by modifying the scripts, rendering my entire approach moot. The result is much cleaner :

  
  
  Turns out that the engine supported subtitles all along
  

  It’s a good thing that I enjoyed the challenge or I might be annoyed at this point.

  See Also

  • Subtitling Sierra VMD Files : My effort to subtitle the main FMV files found in Sierra games.

  The post Subtitling Sierra RBT Files first appeared on Breaking Eggs And Making Omelettes.

• libavfilter/libmpcodecs : sync existing filters with mplayer HEAD (ebcacb8b3ca91ef90ac...

  3 mai 2014, par mplayer developers

  libavfilter/libmpcodecs : sync existing filters with mplayer HEAD (ebcacb8b3ca91ef90acb93785b62fd8c5e5dae41)
  Authors from svn :
  
  cehoyos (2) :
  
        Support playback of JPEG 2000 digital cinema files.
  
        Add name for image format IMGFMT_440P.
  ib (1) :
  
        Get rid of VOCTRL_GUI_NOWINDOW.
  Matt Oliver (4) :
  
        Fix libmpcodecs inline asm on ICL.
  
        Use DECLARE_ALIGNED helper macros.
  
        Remove some superfluous commas from inline asm for better compatibility.
  
        Use numeric labels in inline asm for consistency and better     compatibility.
  reimar (10) :
  
        Fix bpp calculation for XYZ format.
  
        Avoid duplicating the mouse autohide code.
  
        Add NV12/NV21 support to some helper functions.
  
        Add support for rotating the video via OpenGL.
  
        Add options to determine where borders will be added when adjusting for aspect.
  
        Apply forgotten move of apply_border_pos function.
  
        Extract window creation code to common file.
  
        Make VDPAU support work again with latest FFmpeg.
  
        img_format : document why mp_get_chroma_shift does not work for NV12/NV21
  
        Minor spelling/grammar fixes.
  For detailed line by line authorship please see svn log of mplayer
  
  svn ://svn.mplayerhq.hu/mplayer/trunk
  Signed-off-by : Michael Niedermayer <michaelni@gmx.at>
  

  • [DH] libavfilter/libmpcodecs/img_format.c
  • [DH] libavfilter/libmpcodecs/img_format.h
  • [DH] libavfilter/libmpcodecs/libvo/video_out.h
  • [DH] libavfilter/libmpcodecs/mp_image.c
  • [DH] libavfilter/libmpcodecs/vf_fspp.c
  • [DH] libavfilter/libmpcodecs/vf_ilpack.c
  • [DH] libavfilter/libmpcodecs/vf_pp7.c
  • [DH] libavfilter/libmpcodecs/vf_uspp.c

• Crash at sws_scale when converting AVFrame to RGB32

  1er mars 2017, par WLGfx

  Crashing at sws_scale when converting an AVPicture

  At first I was using sws_scale to actually scale the frames up but the cpu overhead was too high, so I decided to just convert the frames and adjust the QImage size instead. Before it was working and I was getting the video displaying when rendered, but now it crashes at sws_scale.

  This is written in Qt for Android and using FFMpeg 3.1.4.

  Also, is there another way around not using the deprecated functions ?

  Does anybody know why I am getting the crash at sws_scale ?

  The class for the VideoFrameCopy

  class VideoFrameCopy {
  
  public:
  
      VideoFrameCopy() {}
  
      VideoFrameCopy(AVFrame *frame) { copyAVFrame(frame); }
  
      ~VideoFrameCopy();
  
      void copyAVFrame(AVFrame *frame); // copy essential data from AVFrame
  
  
  
      AVPicture picture;
  
  
  
      int64_t pkt_pts = -1; // show it hasn't been initialised
  
      int64_t best_pts;
  
      int interlaced_frame;
  
      int width = 0, height = 0;
  
      int format = -1;
  
  };

  The code that converts the frame to RGBA8888 QImage

  if (frame) {
  
  
  
      if (image->width() != vid_ctx->width || image->height() != vid_ctx->height) {
  
          QSize old_size(image->size());
  
  
  
          // block until renderer has finished with it
  
  
  
          while (parent->buffer_ready) {
  
              QThread::yieldCurrentThread();
  
          }
  
  
  
          delete image;
  
  
  
          image = new QImage(vid_ctx->width, vid_ctx->height, QImage::Format_RGBA8888);
  
          parent->image = image;
  
  
  
          if (scale_context) sws_freeContext(scale_context);
  
          scale_context = nullptr;
  
  
  
          qDebug() << "Video image size" << image->size() << "old" << old_size;
  
      }
  
  
  
      // the src width and height may need to change to use the context info instead
  
  
  
      if (!scale_context) { // create the scale context
  
          int src_width = vid_ctx->width;
  
          int src_height = vid_ctx->height;
  
          AVPixelFormat src_format = vid_ctx->pix_fmt;//(AVPixelFormat)frame->format;
  
  
  
          int dst_width = vid_ctx->width;
  
          int dst_height = vid_ctx->height;
  
          AVPixelFormat dst_format = AV_PIX_FMT_RGBA;
  
  
  
          scale_context = sws_getContext(src_width, src_height, src_format,
  
                                         dst_width, dst_height, dst_format,
  
                                         SWS_FAST_BILINEAR, NULL, NULL, NULL);
  
  
  
          av_image_fill_linesizes(scale_linesizes, dst_format, vid_ctx->width);
  
  
  
          qDebug() << "Created scale context" << scale_context;
  
      }
  
  
  
      if (scale_context) { // valid
  
          scale_data[0] = image->bits();
  
  
  
          sws_scale(scale_context,
  
                    frame->picture.data, // deprecated
  
                    frame->picture.linesize, // deprecated
  
                    0, image->height(),
  
                    scale_data,
  
                    scale_linesizes);
  
  
  
          qDebug() << "Frame converted";
  
      }
  
  
  
      //av_frame_unref(frame);
  
  
  
      //vid_frames_mutex.lock();
  
      //if (quit) av_frame_free(&frame);
  
      if (quit) delete frame;
  
      else vid_frames_unused.push_back(frame);
  
      //vid_frames_mutex.unlock();
  
  
  
      //qDebug() << "got frame" << clock_current_frame_last << "clock" << clock_current_time;
  
  }
  
  
  
  vid_frames_mutex.unlock();
  
  
  
  return frame != nullptr;

  Functions from the VideoFrameCopy class

  void VideoFrameCopy::copyAVFrame(AVFrame *frame) {
  
      if (pkt_pts != -1 &&
  
              (width != frame->width ||
  
               height != frame->height ||
  
               format != frame->format)
  
              ) { // picture changed?
  
          avpicture_free(&picture); // deprecated
  
          pkt_pts = -1;
  
      }
  
  
  
      width = frame->width;
  
      height = frame->height;
  
      format = frame->format;
  
      interlaced_frame = frame->interlaced_frame;
  
  
  
      if (pkt_pts == -1) { // alloc picture
  
          if (avpicture_alloc(&picture, (AVPixelFormat)format, width, height) < 0) return; // deprecated
  
          int size = avpicture_get_size((AVPixelFormat)format, width, height); // deprecated
  
          uint8_t *picture_data = (uint8_t*)av_malloc(size);
  
          avpicture_fill(&picture, picture_data, (AVPixelFormat)format, width, height); // deprecated
  
  
  
          qDebug() << "New frame" << width << "x" << height << format;
  
      }
  
  
  
      pkt_pts = frame->pkt_pts;
  
      best_pts = av_frame_get_best_effort_timestamp(frame);
  
  
  
      av_picture_copy(&picture, (AVPicture*)frame, (AVPixelFormat)format, width, height); // deprecated
  
  
  
      qDebug() << "picture" << picture.linesize[0] << picture.linesize[1]; // deprecated
  
  }
  
  
  
  VideoFrameCopy::~VideoFrameCopy() {
  
      if (pkt_pts != -1) {
  
          /*if (picture.data) {
  
              av_free(picture.data);
  
              picture.data = nullptr;
  
          }*/
  
          avpicture_free(&picture); // deprecated
  
      }
  
  }

  Sample output from the logcat

  D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:659 (void VideoFrameCopy::copyAVFrame(AVFrame*)): New frame 640 x 358 0
  
  D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:667 (void VideoFrameCopy::copyAVFrame(AVFrame*)): picture 640 320
  
  D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:586 (bool FFMpegFile::getVideoFrame()): Video image size QSize(640, 358) old QSize(500, 320)
  
  D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:659 (void VideoFrameCopy::copyAVFrame(AVFrame*)): New frame 640 x 358 0
  
  D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:606 (bool FFMpegFile::getVideoFrame()): Created scale context 0x4bb49060
  
  D/libcwengage2.so(20157): ../cwengage2/ffmpegfile.cpp:667 (void VideoFrameCopy::copyAVFrame(AVFrame*)): picture 640 320
  
  F/libc    (20157): Fatal signal 7 (SIGBUS) at 0x4e065008 (code=1), thread 20335 (QThread)
  
  I/DEBUG   (  116): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
  
  I/DEBUG   (  116): Build fingerprint: 'ODROID/odroidc/odroidc:4.4.2/KOT49H/odroidc-eng-s805_4.4.2_master-410:eng/test-keys'
  
  I/DEBUG   (  116): Revision: '10'
  
  I/DEBUG   (  116): pid: 20157, tid: 20335, name: QThread  >>> org.qtproject.example <<<
  
  I/DEBUG   (  116): signal 7 (SIGBUS), code 1 (BUS_ADRALN), fault addr 4e065008
  
  I/DEBUG   (  116):     r0 00000280  r1 00000166  r2 4e065008  r3 00000a00
  
  I/DEBUG   (  116):     r4 4d9bd030  r5 00000280  r6 4d9f4f28  r7 00000140
  
  I/DEBUG   (  116):     r8 00000280  r9 00000010  sl 4da02ee8  fp 4e065a08
  
  I/DEBUG   (  116):     ip 4d9bd2a0  sp 4c1948f8  lr 4a8f1d2c  pc 4a8f4dc0  cpsr 280f0010
  
  I/DEBUG   (  116):     d0  004a004a004a004a  d1  0081ffccffe70066
  
  I/DEBUG   (  116):     d2  004a004a004a004a  d3  0000000000000000
  
  I/DEBUG   (  116):     d4  0000000000000000  d5  0000000000000000
  
  I/DEBUG   (  116):     d6  0000000001010101  d7  0000000001010101
  
  I/DEBUG   (  116):     d8  0000000001010101  d9  ffffffffffffffff
  
  I/DEBUG   (  116):     d10 0000000000000000  d11 0000000000000000
  
  I/DEBUG   (  116):     d12 0000000000000000  d13 ffffffffffffffff
  
  I/DEBUG   (  116):     d14 004a004a004a004a  d15 004a004a004a004a
  
  I/DEBUG   (  116):     d16 0000000000000000  d17 0000000000000000
  
  I/DEBUG   (  116):     d18 0000000000000000  d19 0000000000000000
  
  I/DEBUG   (  116):     d20 0000000000000000  d21 0000000000000000
  
  I/DEBUG   (  116):     d22 0000000000000000  d23 0000000000000000
  
  I/DEBUG   (  116):     d24 0000000000000000  d25 0000000000000000
  
  I/DEBUG   (  116):     d26 0000000000000000  d27 0000000000000000
  
  I/DEBUG   (  116):     d28 004a004a004a004a  d29 0000000000000000
  
  I/DEBUG   (  116):     d30 0000000000000000  d31 0000000000000000
  
  I/DEBUG   (  116):     scr 20000010
  
  I/DEBUG   (  116): 
  
  I/DEBUG   (  116): backtrace:
  
  I/DEBUG   (  116):     #00  pc 0000edc0  /data/app-lib/org.qtproject.example-1/libswscale-4.so
  
  I/DEBUG   (  116):     #01  pc 0000bd28  /data/app-lib/org.qtproject.example-1/libswscale-4.so
  
  I/DEBUG   (  116): 
  
  I/DEBUG   (  116): stack:
  
  I/DEBUG   (  116):          4c1948b8  0000004b  
  
  I/DEBUG   (  116):          4c1948bc  4bb2d270  
  
  I/DEBUG   (  116):          4c1948c0  0000013c  
  
  I/DEBUG   (  116):          4c1948c4  4011edbc  /system/lib/libc.so (dlmalloc+480)
  
  I/DEBUG   (  116):          4c1948c8  4c19494a  [stack:20335]
  
  I/DEBUG   (  116):          4c1948cc  4015e384  
  
  I/DEBUG   (  116):          4c1948d0  00000010  
  
  I/DEBUG   (  116):          4c1948d4  489033ef  /data/app-lib/org.qtproject.example-1/libQt5Core.so
  
  I/DEBUG   (  116):          4c1948d8  00001000  
  
  I/DEBUG   (  116):          4c1948dc  00000000  
  
  I/DEBUG   (  116):          4c1948e0  4bb2d470  
  
  I/DEBUG   (  116):          4c1948e4  4bb2d478  
  
  I/DEBUG   (  116):          4c1948e8  4bb2d478  
  
  I/DEBUG   (  116):          4c1948ec  4bb2d470  
  
  I/DEBUG   (  116):          4c1948f0  00000002  
  
  I/DEBUG   (  116):          4c1948f4  4012109c  /system/lib/libc.so (dlfree+996)
  
  I/DEBUG   (  116):     #00  4c1948f8  11111111  
  
  I/DEBUG   (  116):          ........  ........
  
  I/DEBUG   (  116):     #01  4c1948f8  11111111  
  
  I/DEBUG   (  116):          4c1948fc  3fa11111  
  
  I/DEBUG   (  116):          4c194900  40000000  
  
  I/DEBUG   (  116):          4c194904  40640d79  /system/lib/libskia.so
  
  I/DEBUG   (  116):          4c194908  00000000  
  
  I/DEBUG   (  116):          4c19490c  3ff00000  
  
  I/DEBUG   (  116):          4c194910  00000000  
  
  I/DEBUG   (  116):          4c194914  3ff00000  
  
  I/DEBUG   (  116):          4c194918  00000000  
  
  I/DEBUG   (  116):          4c19491c  3ff00000  
  
  I/DEBUG   (  116):          4c194920  00000000  
  
  I/DEBUG   (  116):          4c194924  3f800000  
  
  I/DEBUG   (  116):          4c194928  00000000  
  
  I/DEBUG   (  116):          4c19492c  00000000  
  
  I/DEBUG   (  116):          4c194930  00000000  
  
  I/DEBUG   (  116):          4c194934  00000000  
  
  I/DEBUG   (  116): 
  
  I/DEBUG   (  116): memory near r2:
  
  I/DEBUG   (  116):     4e064fe8 00000000 00000000 00000000 00000007  
  
  
  
  ...
  
  
  
  I/DEBUG   (  116): memory map around fault addr 4e065008:
  
  I/DEBUG   (  116):     4dd15000-4df15000 rw- /dev/mali
  
  I/DEBUG   (  116):     4df15000-4e199000 rw- 
  
  I/DEBUG   (  116):     4e676000-4e876000 rw- /dev/mali
  
  I/BootReceiver(  479): Copying /data/tombstones/tombstone_07 to DropBox (SYSTEM_TOMBSTONE)
  
  W/ActivityManager(  479):   Force finishing activity org.qtproject.example/org.qtproject.qt5.android.bindings.QtActivity
  
  I/WindowState(  479): WIN DEATH: Window{64cf3a40 u0 org.qtproject.example/org.qtproject.qt5.android.bindings.QtActivity}
  
  I/WindowState(  479): WIN DEATH: Window{64d0f6e0 u0 SurfaceView}
  
  I/UsageStats(  479): No package stats for pkg:org.qtproject.example
  
  I/art     (  118): Process 20157 terminated by signal (7)
  
  W/ActivityManager(  479): Exception thrown during pause
  
  W/ActivityManager(  479): android.os.DeadObjectException
  
  W/ActivityManager(  479):   at android.os.BinderProxy.transact(Native Method)
  
  W/ActivityManager(  479):   at android.app.ApplicationThreadProxy.schedulePauseActivity(ApplicationThreadNative.java:660)
  
  W/ActivityManager(  479):   at com.android.server.am.ActivityStack.startPausingLocked(ActivityStack.java:778)
  
  W/ActivityManager(  479):   at com.android.server.am.ActivityStack.finishActivityLocked(ActivityStack.java:2614)
  
  W/ActivityManager(  479):   at com.android.server.am.ActivityStack.finishTopRunningActivityLocked(ActivityStack.java:2488)
  
  W/ActivityManager(  479):   at com.android.server.am.ActivityStackSupervisor.finishTopRunningActivityLocked(ActivityStackSupervisor.java:2196)
  
  W/ActivityManager(  479):   at com.android.server.am.ActivityManagerService.handleAppCrashLocked(ActivityManagerService.java:9705)
  
  W/ActivityManager(  479):   at com.android.server.am.ActivityManagerService.makeAppCrashingLocked(ActivityManagerService.java:9598)
  
  W/ActivityManager(  479):   at com.android.server.am.ActivityManagerService.crashApplication(ActivityManagerService.java:10243)
  
  W/ActivityManager(  479):   at com.android.server.am.ActivityManagerService.handleApplicationCrashInner(ActivityManagerService.java:9794)
  
  W/ActivityManager(  479):   at com.android.server.am.NativeCrashListener$NativeCrashReporter.run(NativeCrashListener.java:86)
  
  D/ActivityManager(  479): resumeClassName is com.android.launcher2.Launcher
  
  D/ActivityManager(  479): resumePackageName is com.android.launcher
  
  I/ActivityManager(  479): Process org.qtproject.example (pid 20157) has died.
  
  D/ActivityManager(  479): send app_CRASH broadcast, packageName:org.qtproject.example

  Much appreciated if anyone can help...