Recherche avancée

Sur d’autres sites (11556)

• Opencv crosscompile with ffmpeg and other 3rdparty for arm board

  16 juin 2017, par Little Tooth

  After buliding Qt5.5.1 for my arm board sucessfully, I am taking up to crosscompile opencv. Here are the steps :

  First, I crosscompile 3rdparty :

  a.libz

  lmk@lmk-virtual-machine:/home/newdisk$ sudo tar -zvxf zlib-1.2.8.tar.gz
  
  lmk@lmk-virtual-machine:/home/newdisk$ cd zlib-1.2.8
  
  lmk@lmk-virtual-machine:/home/newdisk/zlib-1.2.8$ sudo ./configure --prefix=/home/newdisk/optnew/opencv-rely -shared
  
  lmk@lmk-virtual-machine:/home/newdisk/zlib-1.2.8$ sudo vi Makefile

  and etit Makefile in some details :

  #Makefile
  
  CC=/home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-none-linux-gnueabi-gcc
  
  LDSHARED= /home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-none-linux-gnueabi-gcc -shared -Wl,-soname,libz.so.1,--version-script,zlib.map
  
  AR=/home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-none-linux-gnueabi-ar
  
  RANLIB=arm-linux-ranlib

  the last step is run make and make install :

  lmk@lmk-virtual-machine:/home/newdisk/zlib-1.2.8$ sudo make
  
  lmk@lmk-virtual-machine:/home/newdisk/zlib-1.2.8$ sudo make install

  b.libjpeg

  lmk@lmk-virtual-machine:/home/newdisk$ sudo tar -zvxf jpegsrc.v9.tar.gz
  
  lmk@lmk-virtual-machine:/home/newdisk$ cd jpeg-9
  
  lmk@lmk-virtual-machine:/home/newdisk/jpeg-9$ sudo CC=/home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-none-linux-gnueabi-gcc ./configure --host=arm-linux --prefix=/home/newdisk/optnew/opencv-rely --enable-shared --enable-static
  
  lmk@lmk-virtual-machine:/home/newdisk/jpeg-9$ sudo make
  
  lmk@lmk-virtual-machine:/home/newdisk/jpeg-9$ sudo make install

  c.libpng

  lmk@lmk-virtual-machine:/home/newdisk$ sudo xz -d libpng-1.6.29.tar.xz
  
  lmk@lmk-virtual-machine:/home/newdisk$ sudo tar -xvf libpng-1.6.29.tar
  
  lmk@lmk-virtual-machine:/home/newdisk$ cd libpng-1.6.29
  
  lmk@lmk-virtual-machine:/home/newdisk/libpng-1.6.29$ sudo CC=/home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-none-linux-gnueabi-gcc ./configure --host=arm-linux --prefix=/home/newdisk/optnew/opencv-rely --enable-shared --enable-static
  
  lmk@lmk-virtual-machine:/home/newdisk/libpng-1.6.29$ sudo make
  
  lmk@lmk-virtual-machine:/home/newdisk/libpng-1.6.29$ sudo make install

  d.yasm

  lmk@lmk-virtual-machine:/home/newdisk$ sudo tar -zvxf yasm-1.3.0.tar.gz
  
  lmk@lmk-virtual-machine:/home/newdisk$ cd yasm-1.3.0
  
  lmk@lmk-virtual-machine:/home/newdisk/yasm-1.3.0$ sudo CC=/home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-none-linux-gnueabi-gcc ./configure --prefix=/home/newdisk/optnew/opencv-rely --host=arm-linux
  
  lmk@lmk-virtual-machine:/home/newdisk/yasm-1.3.0$ sudo make
  
  lmk@lmk-virtual-machine:/home/newdisk/yasm-1.3.0$ sudo make install

  e.libx264

  lmk@lmk-virtual-machine:/home/newdisk$ sudo tar -jxvf last_x264.tar.bz2
  
  lmk@lmk-virtual-machine:/home/newdisk$ cd x264-snapshot-20170612-2245 
  
  lmk@lmk-virtual-machine:/home/newdisk/x264-snapshot-20170612-2245$ sudo CC=/home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-none-linux-gnueabi-gcc ./configure --enable-shared --host=arm-linux --disable-asm --prefix=/home/newdisk/optnew/opencv-rely
  
  lmk@lmk-virtual-machine:/home/newdisk/x264-snapshot-20170612-2245$ sudo make
  
  lmk@lmk-virtual-machine:/home/newdisk/x264-snapshot-20170612-2245$ sudo make install

  f.libxvid

  lmk@lmk-virtual-machine:/home/newdisk$ cd xvidcore-1.3.3
  
  lmk@lmk-virtual-machine:/home/newdisk/xvidcore-1.3.3$ cd build/generic
  
  lmk@lmk-virtual-machine:/home/newdisk/xvidcore-1.3.3/build/generic$ sudo CC=/home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-none-linux-gnueabi-gcc ./configure --prefix=/home/newdisk/optnew/opencv-rely --host=arm-linux  --disable-assembly
  
  lmk@lmk-virtual-machine:/home/newdisk/xvidcore-1.3.3/build/generic$ sudo make
  
  lmk@lmk-virtual-machine:/home/newdisk/xvidcore-1.3.3/build/generic$ sudo make install

  g.ffmpeg

  lmk@lmk-virtual-machine:/home/newdisk$ sudo tar -jvxf ffmpeg-3.3.2.tar.bz2
  
  lmk@lmk-virtual-machine:/home/newdisk$ cd ffmpeg-3.3.2
  
  lmk@lmk-virtual-machine:/home/newdisk/ffmpeg-3.3.2$
  
  sudo ./configure --prefix=/home/newdisk/optnew/opencv-rely --enable-shared --disable-static --enable-gpl --enable-cross-compile  --arch=arm --disable-stripping --target-os=linux --enable-libx264 --enable-libxvid --cc=/home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-none-linux-gnueabi-gcc --enable-swscale --extra-ldflags=-L/home/newdisk/optnew/opencv-rely/lib --extra-cflags=-I/home/newdisk/optnew/opencv-rely/include
  
  lmk@lmk-virtual-machine:/home/newdisk/ffmpeg-3.3.2$ sudo make
  
  lmk@lmk-virtual-machine:/home/newdisk/ffmpeg-3.3.2$ sudo make install

  There is no problem in 3rdparty crosscompile.
  Next, opencv crosscompile :

  lmk@lmk-virtual-machine:/home/newdisk$ sudo unzip opencv-3.1.0.zip
  
  lmk@lmk-virtual-machine:/home/newdisk$ cd opencv-3.1.0
  
  lmk@lmk-virtual-machine:/home/newdisk/opencv-3.1.0$ sudo mkdir BuildOpencv
  
  lmk@lmk-virtual-machine:/home/newdisk/opencv-3.1.0$ cd BuildOpencv
  
  lmk@lmk-virtual-machine:/home/newdisk/opencv-3.1.0/BuildOpencv$ sudo vim toolchain.cmake
  
  #toolchain.cmake
  
  ###########user defined#############
  
  set( CMAKE_SYSTEM_NAME Linux )
  
  set( CMAKE_SYSTEM_PROCESSOR arm )
  
  set( CMAKE_C_COMPILER /home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-linux-gcc )
  
  set( CMAKE_CXX_COMPILER /home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-linux-g++ )
  
  ###########user defined#############
  
  set( CMAKE_FIND_ROOT_PATH /home/newdisk/optnew/opencv-rely)
  
  set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER )
  
  set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
  
  set( CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
  
  ######################################

  Then I run cmake :

  lmk@lmk-virtual-machine:/home/newdisk/opencv-3.1.0/BuildOpencv$ sudo cmake -DCMAKE_TOOLCHAIN_FILE=toolchain.cmake ../

  Here is the result：

  -- Detected version of GNU GCC: 45 (405)
  
  -- Found ZLIB: /home/newdisk/optnew/opencv-rely/lib/libz.so (found suitable version "1.2.8", minimum required is "1.2.3") 
  
  -- Could NOT find TIFF (missing:  TIFF_LIBRARY TIFF_INCLUDE_DIR) 
  
  -- Could NOT find Jasper (missing:  JASPER_LIBRARIES) (found version "1.900.1")
  
  -- Found ZLIB: /home/newdisk/optnew/opencv-rely/lib/libz.so (found version "1.2.8") 
  
  -- checking for module 'gtk+-3.0'
  
  --   package 'gtk+-3.0' not found
  
  -- checking for module 'gstreamer-base-1.0'
  
  --   package 'gstreamer-base-1.0' not found
  
  -- checking for module 'gstreamer-video-1.0'
  
  --   package 'gstreamer-video-1.0' not found
  
  -- checking for module 'gstreamer-app-1.0'
  
  --   package 'gstreamer-app-1.0' not found
  
  -- checking for module 'gstreamer-riff-1.0'
  
  --   package 'gstreamer-riff-1.0' not found
  
  -- checking for module 'gstreamer-pbutils-1.0'
  
  --   package 'gstreamer-pbutils-1.0' not found
  
  -- checking for module 'gstreamer-base-0.10'
  
  --   package 'gstreamer-base-0.10' not found
  
  -- checking for module 'gstreamer-video-0.10'
  
  --   package 'gstreamer-video-0.10' not found
  
  -- checking for module 'gstreamer-app-0.10'
  
  --   package 'gstreamer-app-0.10' not found
  
  -- checking for module 'gstreamer-riff-0.10'
  
  --   package 'gstreamer-riff-0.10' not found
  
  -- checking for module 'gstreamer-pbutils-0.10'
  
  --   package 'gstreamer-pbutils-0.10' not found
  
  -- Looking for linux/videodev.h
  
  -- Looking for linux/videodev.h - found
  
  -- Looking for linux/videodev2.h
  
  -- Looking for linux/videodev2.h - found
  
  -- Looking for sys/videoio.h
  
  -- Looking for sys/videoio.h - not found
  
  -- checking for module 'libavresample'
  
  --   package 'libavresample' not found
  
  -- Looking for libavformat/avformat.h
  
  -- Looking for libavformat/avformat.h - not found
  
  -- Looking for ffmpeg/avformat.h
  
  -- Looking for ffmpeg/avformat.h - not found
  
  -- checking for module 'libgphoto2'
  
   --   package 'libgphoto2' not found
  
  -- Could NOT find Doxygen (missing:  DOXYGEN_EXECUTABLE) 
  
  -- To enable PlantUML support, set PLANTUML_JAR environment variable or pass -DPLANTUML_JAR=<filepath> option to cmake
  
  -- Found PythonInterp: /usr/bin/python2.7 (found suitable version "2.7.6", minimum required is "2.7") 
  
  -- Could NOT find PythonLibs (missing:  PYTHON_LIBRARIES) (found suitable version "2.7.6", minimum required is "2.7")
  
  -- Cannot probe for Python/Numpy support (because we are cross-compiling OpenCV)
  
  -- If you want to enable Python/Numpy support, set the following variables:
  
  --   PYTHON2_INCLUDE_PATH
  
  --   PYTHON2_LIBRARIES
  
  --   PYTHON2_NUMPY_INCLUDE_DIRS
  
  --   PYTHON3_INCLUDE_PATH
  
  --   PYTHON3_LIBRARIES
  
  --   PYTHON3_NUMPY_INCLUDE_DIRS
  
  -- Found PythonInterp: /usr/bin/python3.4 (found suitable version "3.4.3", minimum required is "3.4") 
  
  -- Could NOT find PythonLibs (missing:  PYTHON_LIBRARIES) (Required is at least version "3.4")
  
  -- Cannot probe for Python/Numpy support (because we are cross-compiling OpenCV)
  
  -- If you want to enable Python/Numpy support, set the following variables:
  
  --   PYTHON2_INCLUDE_PATH
  
  --   PYTHON2_LIBRARIES
  
  --   PYTHON2_NUMPY_INCLUDE_DIRS
  
  --   PYTHON3_INCLUDE_PATH
  
  --   PYTHON3_LIBRARIES
  
  --   PYTHON3_NUMPY_INCLUDE_DIRS
  
  -- Could NOT find JNI (missing:  JAVA_AWT_LIBRARY JAVA_JVM_LIBRARY JAVA_INCLUDE_PATH JAVA_INCLUDE_PATH2 JAVA_AWT_INCLUDE_PATH) 
  
  -- Could NOT find Matlab (missing:  MATLAB_MEX_SCRIPT MATLAB_INCLUDE_DIRS MATLAB_ROOT_DIR MATLAB_LIBRARIES MATLAB_LIBRARY_DIRS MATLAB_MEXEXT MATLAB_ARCH MATLAB_BIN) 
  
  -- 
  
  -- General configuration for OpenCV 3.1.0 
  
  =====================================
  
  --   Version control:               unknown
  
  -- 
  
  --   Platform:
  
  --     Host:                        Linux 3.16.0-77-generic i686
  
  --     Target:                      Linux arm
  
  --     CMake:                       2.8.12.2
  
  --     CMake generator:             Unix Makefiles
  
  --     CMake build tool:            /usr/bin/make
  
  --     Configuration:               Release
  
  -- 
  
  --   C/C++:
  
  --     Built as dynamic libs?:      YES
  
  --     C++ Compiler:                /home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-linux-g++  (ver 4.5.1)
  
  --     C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
  
  --     C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
  
  --     C Compiler:                  /home/newdisk/optnew/opt/FriendlyARM/toolschain/4.5.1/bin/arm-linux-gcc
  
  --     C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
  
  --     C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
  
  --     Linker flags (Release):      
  
  --     Linker flags (Debug):        
  
  --     Precompiled headers:         YES
  
  --     Extra dependencies:          /home/newdisk/optnew/opencv-rely/lib/libjpeg.so /home/newdisk/optnew/opencv-rely/lib/libpng.so /home/newdisk/optnew/opencv-rely/lib/libz.so gtk-x11-2.0 gdk-x11-2.0 atk-1.0 gio-2.0 pangoft2-1.0 pangocairo-1.0 gdk_pixbuf-2.0 cairo pango-1.0 fontconfig gobject-2.0 freetype gthread-2.0 glib-2.0 dc1394 v4l1 v4l2 avcodec avformat avutil swscale dl m pthread rt
  
  --     3rdparty dependencies:       libwebp libtiff libjasper IlmImf
  
  -- 
  
  --   OpenCV modules:
  
  --     To be built:                 core flann imgproc ml photo video imgcodecs shape videoio highgui objdetect superres ts features2d calib3d stitching videostab
  
  --     Disabled:                    world
  
  --     Disabled by dependency:      -
  
  --     Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev java python2 python3 viz
  
  -- 
  
  --   GUI: 
  
  --     QT:                          NO
  
  --     GTK+ 2.x:                    YES (ver 2.24.23)
  
  --     GThread :                    YES (ver 2.40.2)
  
  --     GtkGlExt:                    NO
  
  --     OpenGL support:              NO
  
  --     VTK support:                 NO
  
  -- 
  
  --   Media I/O: 
  
  --     ZLib:                        /home/newdisk/optnew/opencv-rely/lib/libz.so (ver 1.2.8)
  
  --     JPEG:                        /home/newdisk/optnew/opencv-rely/lib/libjpeg.so (ver 90)
  
  --     WEBP:                        build (ver 0.3.1)
  
  --     PNG:                         /home/newdisk/optnew/opencv-rely/lib/libpng.so (ver 1.6.29)
  
  --     TIFF:                        build (ver 42 - 4.0.2)
  
  --     JPEG 2000:                   build (ver 1.900.1)
  
  --     OpenEXR:                     build (ver 1.7.1)
  
  --     GDAL:                        NO
  
  -- 
  
  --   Video I/O:
  
  --     DC1394 1.x:                  NO
  
  --     DC1394 2.x:                  YES (ver 2.2.1)
  
  --     FFMPEG:                      YES
  
  --       codec:                     YES (ver 54.35.1)
  
  --       format:                    YES (ver 54.20.4)
  
  --       util:                      YES (ver 52.3.0)
  
  --       swscale:                   YES (ver 2.1.1)
  
  --       resample:                  NO
  
  --       gentoo-style:              YES
  
  --     GStreamer:                   NO
  
  --     OpenNI:                      NO
  
  --     OpenNI PrimeSensor Modules:  NO
  
  --     OpenNI2:                     NO
  
  --     PvAPI:                       NO
  
  --     GigEVisionSDK:               NO
  
  --     UniCap:                      NO
  
  --     UniCap ucil:                 NO
  
  --     V4L/V4L2:                    Using libv4l1 (ver 0.8.8) / libv4l2 (ver 0.8.8)
  
  --     XIMEA:                       NO
  
  --     Xine:                        NO
  
  --     gPhoto2:                     NO
  
  -- 
  
  --   Parallel framework:            pthreads
  
  -- 
  
  --   Other third-party libraries:
  
  --     Use IPP:                     NO
  
  --     Use VA:                      NO
  
  --     Use Intel VA-API/OpenCL:     NO
  
  --     Use Eigen:                   NO
  
  --     Use Cuda:                    NO
  
  --     Use OpenCL:                  YES
  
  --     Use custom HAL:              NO
  
  -- 
  
  --   OpenCL:
  
  --     Version:                     dynamic
  
  --     Include path:                /home/newdisk/opencv-3.1.0/3rdparty/include/opencl/1.2
  
  --     Use AMDFFT:                  NO
  
  --      Use AMDBLAS:                 NO
  
  -- 
  
  --   Python 2:
  
  --     Interpreter:                 /usr/bin/python2.7 (ver 2.7.6)
  
  -- 
  
  --   Python 3:
  
  --     Interpreter:                 /usr/bin/python3.4 (ver 3.4.3)
  
  -- 
  
  --   Python (for build):            /usr/bin/python2.7
  
  -- 
  
  --   Java:
  
  --     ant:                         NO
  
  --     JNI:                         NO
  
  --     Java wrappers:               NO
  
  --     Java tests:                  NO
  
  -- 
  
  --   Matlab:                        Matlab not found or implicitly disabled
  
  -- 
  
  --   Documentation:
  
  --     Doxygen:                     NO
  
  --     PlantUML:                    NO
  
  -- 
  
  --   Tests and samples:
  
  --     Tests:                       YES
  
  --     Performance tests:           YES
  
  --     C/C++ Examples:              NO
  
  -- 
  
  --   Install path:                  /home/newdisk/opencv-3.1.0/BuildOpencv/install
  
  -- 
  
  --   cvconfig.h is in:              /home/newdisk/opencv-3.1.0/BuildOpencv
  
  -- -----------------------------------------------------------------
  
  -- 
  
  -- Configuring done
  
  -- Generating done
  
  -- Build files have been written to: /home/newdisk/opencv-3.1.0/BuildOpencv
  
  </filepath>

  After configuring and generating, run cmake-gui :

  lmk@lmk-virtual-machine:/home/newdisk/opencv-3.1.0/BuildOpencv$ sudo cmake-gui
  
  
  
  Src:/home/newdisk/opencv-3.1.0
  
  Build:/home/newdisk/opencv-3.1.0/BuildOpencv
  
  
  
  CMAKE_INSTALL_PREFIX:/home/newdisk/optnew/opencv-arm
  
  
  
  Don't choose these items:
  
  WITH_CUDA
  
  WITH_GTK
  
  WITH_1394
  
  WITH_GSTREAMER
  
  WITH_LIBV4L 
  
  WITH_TIFF
  
  BUILD_OPENEXR
  
  WITH_OPENEXR
  
  WITH_OPENCL
  
  
  
  Configure and generate.
  
  
  
  lmk@lmk-virtual-machine:/home/newdisk/opencv-3.1.0/BuildOpencv$ sudo vi CMakeCache.txt
  
  #CMakeCache.txt
  
  //Flags used by the linker.
  
  CMAKE_EXE_LINKER_FLAGS:STRING=-lpthread -lrt
  
  
  
  lmk@lmk-virtual-machine:/home/newdisk/opencv-3.1.0/BuildOpencv$ sudo make

  Fianlly,I got these errors :

  [ 27%] Building CXX object modules/videoio/CMakeFiles/opencv_videoio.dir/src/cap_mjpeg_decoder.cpp.o
  
  [ 27%] Building CXX object modules/videoio/CMakeFiles/opencv_videoio.dir/src/cap_v4l.cpp.o
  
  [ 27%] Building CXX object modules/videoio/CMakeFiles/opencv_videoio.dir/src/cap_ffmpeg.cpp.o
  
  In file included from /home/newdisk/opencv-3.1.0/modules/videoio/src/cap_ffmpeg_impl.hpp:65:0,
  
               from /home/newdisk/opencv-3.1.0/modules/videoio/src/cap_ffmpeg.cpp:45:
  
  /home/newdisk/opencv-3.1.0/modules/videoio/src/ffmpeg_codecs.hpp:77:36: fatal error: libavformat/avformat.h: No such file or directory
  
  compilation terminated.
  
  make[2]: *** [modules/videoio/CMakeFiles/opencv_videoio.dir/src/cap_ffmpeg.cpp.o] Error 1
  
  make[1]: *** [modules/videoio/CMakeFiles/opencv_videoio.dir/all] Error 2
  
  make: *** [all] Error 2

  While the file libavformat/avformat.h does exits in my directory
  /home/newdisk/optnew/opencv-rely/include

  Why can cmake find lib directory

  -- Found ZLIB: /home/newdisk/optnew/opencv-rely/lib/libz.so (found suitable version "1.2.8", minimum required is "1.2.3")

  but cannot find include directory ?

  /home/newdisk/optnew/opencv-rely/include

  Do you have any idea ?

• Saying Goodbye To Old Machines

  1er décembre 2014, par Multimedia Mike — General, powerpc, via

  I recently sent a few old machines off for recycling. Both had relevance to the early days of the FATE testing effort. As is my custom, I photographed them (poorly, of course).

  First, there’s the PowerPC-based Mac Mini I procured thanks to a Craigslist ad in late 2006. I had plans to develop automated FFmpeg building and testing and was already looking ahead toward testing multiple CPU architectures. Again, this was 2006 and PowerPC wasn’t completely on the outs yet– although Apple’s MacTel transition was in full swing, the entire new generation of video game consoles was based on PowerPC.

  
  

  

  Click for larger image

  
  

  I remember trying to find a Mac Mini PPC on Craigslist. Many were to be found, but all asked more than the price of even a new Mac Mini Intel, always because the seller was leaving all of last year’s applications and perhaps including a monitor, neither of which I needed. Fortunately, I found this bare Mac Mini. Also fortunate was the fact that it was far easier to install Linux on it than the first PowerPC machine I owned.

  After FATE operation transitioned away from me, I still kept the machine in service as an edge server and automated backup machine. That is, until the hard drive failed on reboot one day. Thus, when it was finally time to recycle the computer, I felt it necessary to disassemble the machine and remove the hard drive for possible salvage and then for destruction.
  

  If you’ve ever attempted to upgrade or otherwise service this style of Mac Mini, you will no doubt recognize the pictured paint scraper tool as standard kit. I have had that tool since I first endeavored to upgrade the RAM to 1 GB from the standard 1/2 GB. Performing such activities on a Mac Mini is tedious, but only if you care about putting it back together afterwards.

  The next machine is a bit older. I put it together nearly a decade ago, early in 2005. This machine’s original duty was “download agent”– this would be more specifically called a BitTorrent machine in modern tech parlance. Back then, I placed it on someone else’s woefully underutilized home broadband connection (with their permission, of course) when I was too cheap to upgrade from dialup.

  
  

  

  Click for larger image

  
  

  This is a small form factor system from VIA that was clearly designed with home theater PC (HTPC) use cases in mind. It has a VIA C3 x86-compatible CPU (according to my notes, Centaur VIA Samuel 2 stepping 03, flags : fpu de tsc msr cx8 mtrr pge mmx 3dnow) and 128 MB of RAM (initially ; I upgraded it to 512 MB some years later, just for the sake of doing it). And then there was the 120 GB PATA HD for all that downloaded goodness.

  
  

  

  Click for larger image

  
  

  I have specific memories of a time when my main computer at home wasn’t working correctly for one reason or another. Instead, I logged into this machine remotely via SSH to make several optimizations and fixes on FFmpeg’s VP3/Theora video decoder, all from the terminal, without being able to see the decoded images with my own eyes (which is why I insist that even blind people could work on video codecs).

  By the time I got my own broadband, I had become inspired to attempt the automated build and test system for FFmpeg. This was the machine I used for prototyping early brainstorms of FATE. By the time I put a basic build/test system into place in early 2008, I had much faster computers that could build and test the project– obvious limitation of this machine is that it could take at least 1/2 hour to build the entire codebase, and that was the project from 8 years ago.

  So the machine got stuffed in a closet somewhere along the line. The next time I pulled it out was in 2010 when I wanted to toy with Dreamcast programming once more (the machine appears in one of the photos in this post). This was the only machine I still owned which still had an RS-232 serial port (I didn’t know much about USB serial converters yet), plus it still had a bunch of pre-compiled DC homebrew binaries (I was having trouble getting the toolchain to work right).

  The next time I dusted off this machine was late last year when I was trying some experiments with the Microsoft Xbox’s IDE drive (a photo in that post also shows the machine ; this thing shows up a lot on this blog). The VIA machine was the only machine I still owned which had 40-pin IDE connectors which was crucial to my experiment.

  At this point, I was trying to make the machine more useful which meant replacing the ancient Gentoo Linux distribution as well as simply interacting with it via a keyboard and mouse. I have a long Evernote entry documenting a comedy of errors revolving around this little box. The interaction troubles were due to the fact that I didn’t have any PS/2 keyboards left and I couldn’t make a USB keyboard work with it. Diego was able to explain that I needed to flip a bit in the BIOS to address this which worked. As for upgrading the OS, I tried numerous Linux distributions large and small, mostly focusing on the small. None worked. I eventually learned that, while I was trying to use i686 distributions, this machine did not actually qualify as an i686 CPU ; installations usually booted but failed because the default kernel required the cmov instruction. I was advised to try i386 distros instead. My notes don’t indicate whether I had any luck on this front before I gave up and moved on.

  I just made the connection that this VIA machine has two 40-pin IDE connectors which means that the thing was technically capable of supporting up to 4 IDE devices. Obviously, the computer couldn’t really accommodate that in terms of space or power. When I wanted to try installing a new OS, I needed take off the top and connect a rather bulky IDE CD-ROM drive. This computer’s casing was supposed to be able to support a slimline optical drive (perhaps like the type found in laptops), but I could never quite visualize how that was supposed to work, space-wise. When I disassembled the PowerPC Mac Mini, I realized I might be able to repurpose that machines optical drive for this computer. Obviously, I thought better of trying since both machines are off to the recycle pile.

  I would still like to work on the Xbox project a bit more, but I procured a different, unused, much more powerful yet still old computer that has a motherboard with 1 PATA connector in addition to 6 SATA connectors. If I ever get around to toying with Linux kernel development, this should be a much more appropriate platform to use.

  I thought about turning this machine into an old Windows XP (and lower, down to Windows 3.1) gaming platform ; the capabilities of the machine would probably be perfect for a huge portion of my Windows game collection. But I think the lack of an optical drive renders this idea intractable. External USB drives are likely out of the question since there is very little chance that this motherboard featured USB 2.0 (the specs don’t mention 2.0, so the USB ports are probably 1.1).

  So it is with fond memories that I send off both machines, sans hard drives, to the recycle pile. I’m still deciding on an appropriate course of action for failed hard drives, though.

• Translating Return To Ringworld

  17 août 2016, par Multimedia Mike — Game Hacking

  As indicated in my previous post, the Translator has expressed interest in applying his hobby towards another DOS adventure game from the mid 1990s : Return to Ringworld (henceforth R2RW) by Tsunami Media. This represents significantly more work than the previous outing, Phantasmagoria.

  
  
  Return to Ringworld Title Screen
  

  I have been largely successful thus far in crafting translation tools. I have pushed the fruits of these labors to a Github repository named improved-spoon (named using Github’s random name generator because I wanted something more interesting than ‘game-hacking-tools’).

  Further, I have recorded everything I have learned about the game’s resource format (named RLB) at the XentaxWiki.

  New Challenges
  The previous project mostly involved scribbling subtitle text on an endless series of video files by leveraging a separate software library which took care of rendering fonts. In contrast, R2RW has at least 30k words of English text contained in various blocks which require translation. Further, the game encodes its own fonts (9 of them) which stubbornly refuse to be useful for rendering text in nearly any other language.

  Thus, the immediate 2 challenges are :
  

  1. Translating volumes of text to Spanish
  2. Expanding the fonts to represent Spanish characters

  Normally, “figuring out the file format data structures involved” is on the list as well. Thankfully, understanding the formats is not a huge challenge since the folks at the ScummVM project already did all the heavy lifting of reverse engineering the file formats.

  The Pitch
  Here was the plan :

  • Create a tool that can dump out the interesting data from the game’s master resource file.
  • Create a tool that can perform the elaborate file copy described in the previous post. The new file should be bit for bit compatible with the original file.
  • Modify the rewriting tool to repack some modified strings into the new resource file.
  • Unpack the fonts and figure out a way to add new characters.
  • Repack the new fonts into the resource file.
  • Repack message strings with Spanish characters.

  Showing The Work : Modifying Strings
  First, I created the tool to unpack blocks of message string resources. I elected to dump the strings to disk as JSON data since it’s easy to write and read JSON using Python, and it’s quick to check if any mistakes have crept in.

  The next step is to find a string to focus on. So I started the game and looked for the first string I could trigger :

  
  
  

  This shows up in the JSON string dump as :

    
      "Spanish" : " !0205Your quarters on the Lance of Truth are spartan, in accord with your mercenary lifestyle.",
      "English" : " !0205Your quarters on the Lance of Truth are spartan, in accord with your mercenary lifestyle."
    ,
  

  As you can see, many of the strings are encoded with an ID key as part of the string which should probably be left unmodified. I changed the Spanish string :

    
      "Spanish" : " !0205Hey, is this thing on ?",
      "English" : " !0205Your quarters on the Lance of Truth are spartan, in accord with your mercenary lifestyle."
    ,
  

  And then I wrote the repacking tool to substitute this message block for the original one. Look ! The engine liked it !

  
  
  

  Little steps, little steps.

  Showing The Work : Modifying Fonts
  The next little step is to find a place to put the new characters. First, a problem definition : The immediate goal is to translate the game into Spanish. The current fonts encoded in the game resource only support 128 characters, corresponding to 7-bit ASCII. In order to properly express Spanish, 16 new characters are required : á, é, í, ó, ú, ü, ñ (each in upper and lower case for a total of 14 characters) as well as the inverted punctuation symbols : ¿, ¡.

  Again, ScummVM already documents (via code) the font coding format. So I quickly determined that each of the 9 fonts is comprised of 128 individual bitmaps with either 1 or 2 bits per pixel. I wrote a tool to unpack each character into an individual portable grey map (PGM) image. These can be edited with graphics editors or with text editors since they are just text files.

  Where to put the 16 new Spanish characters ? ASCII characters 1-31 are non-printable, so my first theory was that these characters would be empty and could be repurposed. However, after dumping and inspecting, I learned that they represent the same set of characters as seen in DOS Code Page 437. So that’s a no-go (so I assumed ; I didn’t check if any existing strings leveraged those characters).

  My next plan was hope that I could extend the font beyond index 127 and use positions 128-143. This worked superbly. This is the new example string :

    
      "Spanish" : " !0205¿Ves esto ? ¡La puntuacion se hace girar !",
      "English" : " !0205Your quarters on the Lance of Truth are spartan, in accord with your mercenary lifestyle."
    ,
  

  Fortunately, JSON understands UTF-8 and after mapping the 16 necessary characters down to the numeric range of 128-143, I repacked the new fonts and the new string :

  
  
  Translation : “See this ? The punctuation is rotated !”
  

  Another victory. Notice that there are no diacritics in this string. None are required for this translation (according to Google Translate). But adding the diacritics to the 14 characters isn’t my department. My tool does help by prepopulating [aeiounAEIOUN] into the right positions to make editing easier for the Translator. But the tool does make the effort to rotate the punctuation since that is easy to automate.

  Next Steps and Residual Weirdness
  There is another method for storing ASCII text inside the R2RW resource called strip resources. These store conversation scripts. There are plenty of fields in the data structures that I don’t fully understand. So, following the lessons I learned from my previous translation outing, I was determined to modify as little as possible. This means copying over most of the original data structures intact, but changing the field representing the relative offset that points to the corresponding string. This works well since the strings are invariably stored NULL-terminated in a concatenated manner.

  I wanted to document for the record that the format that R2RW uses has some weirdness in they way it handles residual bytes in a resource. The variant of the resource format that R2RW uses requires every block to be aligned on a 16-byte boundary. If there is space between the logical end of the resource and the start of the next resource, there are random bytes in that space. This leads me to believe that these bytes were originally recorded from stale/uninitialized memory. This frustrates me because when I write the initial file copy tool which unpacks and repacks each block, I want the new file to be identical to the original. However, these apparent nonsense bytes at the end thwart that effort.

  But leaving those bytes as 0 produces an acceptable resource file.

  Text On Static Images
  There is one last resource type we are working on translating. There are various bits of text that are rendered as images. For example, from the intro :

  
  
  

  It’s possible to locate and extract the exact image that is overlaid on this scene, though without the colors :

  
  
  

  The palettes are stored in a separate resource type. So it seems the challenge is to figure out the palette in use for these frames and render a transparent image that uses the same palette, then repack the new text-image into the new resource file.

  The post Translating Return To Ringworld first appeared on Breaking Eggs And Making Omelettes.