Recherche avancée

Sur d’autres sites (10660)

• Dreamcast Serial Extractor

  31 décembre 2017, par Multimedia Mike — Sega Dreamcast

  It has not been a very productive year for blogging. But I started the year by describing an unfinished project that I developed for the Sega Dreamcast, so I may as well end the year the same way. The previous project was a media player. That initiative actually met with some amount of success and could have developed into something interesting if I had kept at it.

  By contrast, this post describes an effort that was ultimately a fool’s errand that I spent way too much time trying to make work.

  Problem Statement
  In my neverending quest to analyze the structure of video games while also hoarding a massive collection of them (though I’m proud to report that I did play at least a few of them this past year), I wanted to be able to extract the data from my many Dreamcast titles, both games and demo discs. I had a tool called the DC Coder’s Cable, a serial cable that enables communication between a Dreamcast and a PC. With the right software, you could dump an entire Dreamcast GD-ROM, which contained a gigabyte worth of sectors.

  Problem : The dumping software (named ‘dreamrip’ and written by noted game hacker BERO) operated in a very basic mode, methodically dumping sector after sector and sending it down the serial cable. This meant that it took about 28 hours to extract all the data on a single disc by running at the maximum speed of 115,200 bits/second, or about 11 kilobytes/second. I wanted to create a faster method.

  The Pitch
  I formed a mental model of dreamrip’s operation that looked like this :

  
  
  

  As an improvement, I envisioned this beautiful architecture :
  

  
  
  

  Architectural Assumptions
  My proposed architecture was predicated on the assumption that the disc reading and serial output functions were both I/O-bound operations and that the CPU would be idle much of the time. My big idea was to use that presumably idle CPU time to compress the sectors before sending them over the wire. As long as the CPU can compress the data faster than 11 kbytes/sec, it should be a win. In order to achieve this, I broke the main program into 3 threads :

  1. The first thread reads the sectors ; more specifically, it asks the drive firmware to please read the sectors and make the data available in system RAM
  2. The second thread waits for sector data to appear in memory and then compresses it
  3. The third thread takes the compressed data when it is ready and shuffles it out through the serial cable

  Simple and elegant, right ?

  For data track compression, I wanted to start with zlib in order to prove the architecture, but then also try bzip2 or lzma. As long as they could compress data faster than the serial port could write it, then it should be a win. For audio track compression, I wanted to use the Flake FLAC encoder. According to my notes, I did get both bzip2 compression and the Flake compressor working on the Dreamcast. I recall choosing Flake over the official FLAC encoder because it was much simpler and had fewer dependencies, always an important consideration for platforms such as this.

  Problems
  I worked for quite awhile on this project. I have a lot of notes recorded but a lot of the problems I had remain a bit vague in my memory. However, there was one problem I discovered that eventually sunk the entire initiative :

  The serial output operation is CPU-bound.

  My initial mental model was that the a buffer could be “handed off” to the serial subsystem and the CPU could go back to doing other work. Nope. Turns out that the CPU was participating at every step of the serial transfer.

  Further, I eventually dug into the serial driver code and learned that there was already some compression taking place via the miniLZO library.

  Lessons Learned

  • Recognize the assumptions that you’re making up front at the start of the project.
  • Prototype in order to ensure plausibility
  • Profile to make sure you’re optimizing the right thing (this is something I have learned again and again).

  Another interesting tidbit from my notes : it doesn’t matter how many sectors you read at a time, the overall speed is roughly the same. I endeavored to read 1000 2048-byte data sectors, 1 or 10 or 100 at a time, or all 1000 at once. My results :

  • 1 : 19442 ms
  • 10 : 19207 ms
  • 100 : 19194 ms
  • 1000 : 19320 ms

  No difference. That surprised me.

  Side Benefits
  At one point, I needed to understand how BERO’s dreamrip software was operating. I knew I used to have the source code but I could no longer find it. Instead, I decided to try to reverse engineer what I needed from the SH-4 binary image that I had. It wasn’t an ELF image ; rather, it was a raw binary meant to be loaded at a particular memory location which makes it extra challenging for ‘objdump’. This led to me asking my most viewed and upvoted question on Stack Overflow : “Disassembling A Flat Binary File Using objdump”. The next day, it also led me to post one of my most upvoted answers when I found the solution elsewhere.

  Strangely, I have since tried out the command line shown in my answer and have been unable to make it work. But people keep upvoting both the question and the answer.

  Eventually this all became moot when I discovered a misplaced copy of the source code on one of my computers.

  I strongly recall binging through the Alias TV show while I was slogging away on this project, so I guess that’s a positive association since I got so many fun screenshots out of it.

  The Final Resolution
  Strangely, I was still determined to make this project work even though the Dreamcast SD adapter arrived for me about halfway through the effort. Part of this was just stubbornness, but part of it was my assumptions about serial port speeds, in particular, my assumption that there was a certain speed-of-light type of limitation on serial port speeds so that the SD adapter, operating over the DC’s serial port, would not be appreciably faster than the serial cable.

  This turned out to be very incorrect. In fact, the SD adapter is capable of extracting an entire gigabyte disc image in 35-40 minutes. This is the method I have since been using to extract Dreamcast disc images.

  The post Dreamcast Serial Extractor first appeared on Breaking Eggs And Making Omelettes.

• Why when converting avi video file to another format the first 2-3 seconds are blurry ?

  13 juin 2016, par Sharon Gabriel

  The source file is avi. The target new file is mp4.
  The first 2-3 seconds are blurry. Then after 2-3 second the whole video until the end is smooth and sharp.

  Another sub question is how come that 2.16 GB avi file after conversion using ffmpeg is only 1.34 MB ? It’s not part of a movie or something it’s collection of screenshots images i did in c# and then used AviFile Lib to create from them a avi video file. and yet from 2.16 GB to 1.34 MB and it keep the quality i think almost the same quality like the original avi file and the same duration 2:20 minutes.

  About the blurry problem this is my code where i set the ffmpeg arguments and set the process :

  private void Convert()
  
          {
  
              try
  
              {
  
                  Control.CheckForIllegalCrossThreadCalls = false;
  
                  if (ComboBox1.SelectedIndex == 3) 
  
                  {
  
                      strFFCMD = " -i " + (char)34 + InputFile + (char)34 + " -c:v libx264 -s 1920x1080 -pix_fmt yuv420p -qp 18 -profile high444 -c:a libvo_aacenc -b:a 128k -ar 44100 -ac 2 -y " + OutputFile;
  
                  }    
  
                  psiProcInfo.FileName = exepath;
  
                  psiProcInfo.Arguments = strFFCMD;        
  
                  psiProcInfo.UseShellExecute = false;      
  
                  psiProcInfo.WindowStyle = ProcessWindowStyle.Hidden;    
  
                  psiProcInfo.RedirectStandardError = true;             
  
                  psiProcInfo.RedirectStandardOutput = true;         
  
                  psiProcInfo.CreateNoWindow = true;                 
  
                  prcFFMPEG.StartInfo = psiProcInfo;           
  
                  prcFFMPEG.Start();
  
                  ffReader = prcFFMPEG.StandardError;
  
  
  
                  do
  
                  {
  
                      if (Bgw1.CancellationPending)
  
                      {
  
                          return;
  
                      }
  
                      Button5.Enabled = true;
  
                      Button3.Enabled = false;
  
                      strFFOUT = ffReader.ReadLine();                    
  
                      RichTextBox1.Text = strFFOUT;
  
                      if (strFFOUT != null)
  
                      {
  
                          if (strFFOUT.Contains("frame="))
  
                          {
  
                              currentFramestr = strFFOUT.Substring(7, 6).Trim();
  
                              Regex rx = new Regex(@"^\d+");
  
                              Match m = rx.Match(currentFramestr);
  
                              if (m.Success)
  
                              {
  
                                  currentFrameInt = System.Convert.ToInt32(m.Value);
  
                              }
  
                          }
  
                      }
  
                      string percentage = ((double)ProgressBar1.Value / (double)ProgressBar1.Maximum * 100.0).ToString();
  
                      textBox3.Text = ProgressBar1.Value.ToString();                    
  
                      ProgressBar1.Maximum = FCount + 1;
  
                      ProgressBar1.Value = (currentFrameInt);
  
                      Label12.Text = "Current Encoded Frame: " + currentFrameInt;
  
                      Label11.Text = percentage;
  
                  } while (!(prcFFMPEG.HasExited || string.IsNullOrEmpty(strFFOUT)));
  
              }
  
              catch(Exception err)
  
              {
  
                  string errors = err.ToString();
  
              }
  
          }

  psiProcInfo is ProcessStartInfo

  prcFFMPEG is Process

  And this is how it looks like when i play the target the new created converted video file the mp4 the first seconds :

  Duration : 00:02:20

  Width : 1920 Height : 1080

  Data Rate and Total Rate both : 80kbps

  Frame rate : 2 frames/second

  

  This is the output of the ffmpeg console while converting the file.

    ffmpeg version 2.8.git Copyright (c) 2000-2015 the FFmpeg developers
  
    built with gcc 5.2.0 (GCC)
  
    configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libdcadec --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  
    libavutil      55. 11.100 / 55. 11.100
  
    libavcodec     57. 17.100 / 57. 17.100
  
    libavformat    57. 20.100 / 57. 20.100
  
    libavdevice    57.  0.100 / 57.  0.100
  
    libavfilter     6. 21.100 /  6. 21.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  [avi @ 00000147a882b660] Stream #0: not enough frames to estimate rate; consider increasing probesize
  
  Input #0, avi, from 'C:\temp\video\new.avi':
  
    Duration: 00:02:20.50, start: 0.000000, bitrate: 132710 kb/s
  
      Stream #0:0: Video: rawvideo, bgra, 1920x1080, 2 fps, 2 tbr, 2 tbn, 2 tbc
  
  Please use -profile:a or -profile:v, -profile is ambiguous
  
  Codec AVOption b (set bitrate (in bits/s)) specified for output file #0 (C:\temp\video\5.mp4) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
  
  [libx264 @ 00000147a882c820] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
  
  [libx264 @ 00000147a882c820] profile High, level 4.0
  
  [libx264 @ 00000147a882c820] 264 - core 148 r2638 7599210 - H.264/MPEG-4 AVC codec - Copyleft 2003-2015 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=2 scenecut=40 intra_refresh=0 rc=cqp mbtree=0 qp=18 ip_ratio=1.40 pb_ratio=1.30 aq=0
  
  Output #0, mp4, to 'C:\temp\video\5.mp4':
  
    Metadata:
  
      encoder         : Lavf57.20.100
  
      Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 1920x1080, q=-1--1, 2 fps, 16384 tbn, 2 tbc
  
      Metadata:
  
        encoder         : Lavc57.17.100 libx264
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
  
  Press [q] to stop, [?] for help
  
  frame=    8 fps=0.0 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   15 fps= 14 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   21 fps= 13 q=18.0 size=      92kB time=00:00:00.00 bitrate=N/A speed=   0x    
  
  frame=   30 fps= 14 q=18.0 size=     141kB time=00:00:04.50 bitrate= 257.3kbits/s speed=2.03x    
  
  frame=   37 fps= 13 q=20.0 size=     164kB time=00:00:08.00 bitrate= 167.6kbits/s speed=2.82x    
  
  frame=   46 fps= 14 q=18.0 size=     185kB time=00:00:12.50 bitrate= 121.0kbits/s speed= 3.7x    
  
  frame=   51 fps= 13 q=19.0 size=     194kB time=00:00:15.00 bitrate= 106.1kbits/s speed=3.87x    
  
  frame=   58 fps= 13 q=18.0 size=     210kB time=00:00:18.50 bitrate=  93.2kbits/s speed=4.19x    
  
  frame=   65 fps= 13 q=20.0 size=     224kB time=00:00:22.00 bitrate=  83.6kbits/s speed=4.46x    
  
  frame=   71 fps= 13 q=19.0 size=     238kB time=00:00:25.00 bitrate=  78.1kbits/s speed=4.56x    
  
  frame=   78 fps= 13 q=18.0 size=     253kB time=00:00:28.50 bitrate=  72.6kbits/s speed=4.75x    
  
  frame=   83 fps= 13 q=19.0 size=     265kB time=00:00:31.00 bitrate=  70.0kbits/s speed= 4.7x    
  
  frame=   89 fps= 12 q=20.0 size=     280kB time=00:00:34.00 bitrate=  67.4kbits/s speed=4.73x    
  
  frame=   95 fps= 12 q=19.0 size=     291kB time=00:00:37.00 bitrate=  64.5kbits/s speed=4.73x    
  
  frame=  102 fps= 12 q=18.0 size=     308kB time=00:00:40.50 bitrate=  62.3kbits/s speed=4.84x    
  
  frame=  107 fps= 12 q=19.0 size=     317kB time=00:00:43.00 bitrate=  60.4kbits/s speed=4.82x    
  
  frame=  115 fps= 12 q=19.0 size=     336kB time=00:00:47.00 bitrate=  58.6kbits/s speed=4.96x    
  
  frame=  123 fps= 12 q=20.0 size=     354kB time=00:00:51.00 bitrate=  56.8kbits/s speed=5.09x    
  
  frame=  132 fps= 12 q=20.0 size=     371kB time=00:00:55.50 bitrate=  54.8kbits/s speed=5.25x    
  
  frame=  139 fps= 13 q=20.0 size=     392kB time=00:00:59.00 bitrate=  54.5kbits/s speed=5.32x    
  
  frame=  146 fps= 13 q=19.0 size=     408kB time=00:01:02.50 bitrate=  53.5kbits/s speed=5.37x    
  
  frame=  150 fps= 12 q=20.0 size=     417kB time=00:01:04.50 bitrate=  52.9kbits/s speed=5.28x    
  
  frame=  155 fps= 12 q=18.0 size=     428kB time=00:01:07.00 bitrate=  52.4kbits/s speed=5.25x    
  
  frame=  161 fps= 12 q=20.0 size=     441kB time=00:01:10.00 bitrate=  51.6kbits/s speed=5.26x    
  
  frame=  167 fps= 12 q=19.0 size=     462kB time=00:01:13.00 bitrate=  51.9kbits/s speed=5.29x    
  
  frame=  174 fps= 12 q=20.0 size=     483kB time=00:01:16.50 bitrate=  51.7kbits/s speed=5.33x    
  
  frame=  181 fps= 12 q=18.0 size=     614kB time=00:01:20.00 bitrate=  62.8kbits/s speed=5.36x    
  
  frame=  187 fps= 12 q=20.0 size=     763kB time=00:01:23.00 bitrate=  75.3kbits/s speed=5.35x    
  
  frame=  193 fps= 12 q=19.0 size=     852kB time=00:01:26.00 bitrate=  81.2kbits/s speed=5.36x    
  
  frame=  199 fps= 12 q=18.0 size=     865kB time=00:01:29.00 bitrate=  79.6kbits/s speed=5.37x    
  
  frame=  206 fps= 12 q=20.0 size=     932kB time=00:01:32.50 bitrate=  82.6kbits/s speed=5.39x    
  
  frame=  211 fps= 12 q=20.0 size=     943kB time=00:01:35.00 bitrate=  81.3kbits/s speed=5.38x    
  
  frame=  217 fps= 12 q=18.0 size=    1007kB time=00:01:38.00 bitrate=  84.1kbits/s speed=5.38x    
  
  frame=  223 fps= 12 q=20.0 size=    1175kB time=00:01:41.00 bitrate=  95.3kbits/s speed=5.38x    
  
  frame=  230 fps= 12 q=20.0 size=    1195kB time=00:01:44.50 bitrate=  93.7kbits/s speed=5.42x    
  
  frame=  235 fps= 12 q=18.0 size=    1205kB time=00:01:47.00 bitrate=  92.3kbits/s speed= 5.4x    
  
  frame=  241 fps= 12 q=20.0 size=    1222kB time=00:01:50.00 bitrate=  91.0kbits/s speed= 5.4x    
  
  frame=  247 fps= 12 q=19.0 size=    1232kB time=00:01:53.00 bitrate=  89.3kbits/s speed=5.39x    
  
  frame=  255 fps= 12 q=19.0 size=    1252kB time=00:01:57.00 bitrate=  87.7kbits/s speed=5.45x    
  
  frame=  260 fps= 12 q=20.0 size=    1274kB time=00:01:59.50 bitrate=  87.3kbits/s speed=5.44x    
  
  frame=  267 fps= 12 q=20.0 size=    1287kB time=00:02:03.00 bitrate=  85.7kbits/s speed=5.45x    
  
  frame=  272 fps= 12 q=18.0 size=    1304kB time=00:02:05.50 bitrate=  85.1kbits/s speed=5.43x    
  
  frame=  278 fps= 12 q=20.0 size=    1314kB time=00:02:08.50 bitrate=  83.8kbits/s speed=5.41x    
  
  frame=  281 fps= 12 q=-1.0 Lsize=    1376kB time=00:02:19.50 bitrate=  80.8kbits/s speed=5.76x    
  
  video:1372kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.299861%
  
  [libx264 @ 00000147a882c820] frame I:2     Avg QP:15.00  size: 98930
  
  [libx264 @ 00000147a882c820] frame P:80    Avg QP:18.00  size:  7068

• Announcing the first free software Blu-ray encoder

  25 avril 2010, par Dark Shikari — blu-ray, x264

  For many years it has been possible to make your own DVDs with free software tools. Over the course of the past decade, DVD creation evolved from the exclusive domain of the media publishing companies to something basically anyone could do on their home computer.

  But Blu-ray has yet to get that treatment. Despite the “format war” between Blu-ray and HD DVD ending over two years ago, free software has lagged behind. “Professional” tools for Blu-ray video encoding can cost as much as $100,000 and are often utter garbage. Here are two actual screenshots from real Blu-rays : I wish I was making this up.

  But today, things change. Today we take the first step towards a free software Blu-ray creation toolkit.

  Thanks to tireless work by Kieran Kunyha, Alex Giladi, Lamont Alston, and the Doom9 crowd, x264 can now produce Blu-ray-compliant video. Extra special thanks to The Criterion Collection for sponsoring the final compliance test to confirm x264′s Blu-ray compliance.

  With x264′s powerful compression, as demonstrated by the incredibly popular BD-Rebuilder Blu-ray backup software, it’s quite possible to author Blu-ray disks on DVD9s (dual-layer DVDs) or even DVD5s (single-layer DVDs) with a reasonable level of quality. With a free software encoder and less need for an expensive Blu-ray burner, we are one step closer to putting HD optical media creation in the hands of the everyday user.

  To celebrate this achievement, we are making available for download a demo Blu-ray encoded with x264, containing entirely free content !

  On this Blu-ray are the Open Movie Project films Big Buck Bunny and Elephant’s Dream, available under a Creative Commons license. Additionally, Microsoft has graciously provided about 6 minutes of lossless HD video and audio (from part of a documentary project) under a very liberal license. This footage rounds out the Blu-ray by adding some difficult live-action content in addition to the relatively compressible CGI footage from the Open Movie Project. Finally, we used this sound sample, available under a Creative Commons license.

  You may notice that the Blu-ray image is only just over 2GB. This is intentional ; we have encoded all the content on the disk at appropriate bitrates to be playable from an ordinary 4.7GB DVD. This should make it far easier to burn a copy of the Blu-ray, since Blu-ray burners and writable media are still relatively rare. Most Blu-ray players will treat a DVD containing Blu-ray data as a normal Blu-ray disc. A few, such as the Playstation 3, will not, but you can still play it as a data disc.

  Finally, note that (in accordance with the Blu-ray spec) the disc image file uses the UDF 2.5 filesystem, which may be incompatible with some older virtual drive and DVD burning applications. You’ll also need to play it on an actual Blu-ray player if you want to get the menus and such working correctly. If you’re looking to play it on a PC, a free trial of Arcsoft TMT is available here.

  What are you waiting for ? Grab a copy today !

  UPDATE : Here is an AVCHD-compliant version of the above, which should work better when burned on a DVD-5 instead of a BD-R. (mirror)
  

  What’s left before we have a fully free software Blu-ray creation toolkit ? Audio is already dealt with ; AC3 audio (aka Dolby Digital), the format used in DVD, is still supported by Blu-ray, and there are many free software AC3 encoders. The primary missing application is a free software Blu-ray authoring tool, to combine the video and audio streams to create a Blu-ray file structure with the menus, chapters, and so forth that we have all come to expect. But the hardest part is dealt with : we can now create compatible video and audio streams.

  In the meantime, x264 can be used to create streams to be authored using Blu-Print, Scenarist, Encore or other commercial authoring tools.

  More detailed documentation on the new Blu-ray support and how to use it can be found in the official commit message. Do keep in mind that you have to export to raw H.264 (not MKV or MP4) or else the buffering information will be slightly incorrect. Finally, also note that the encoding settings given as an example are not a good choice for general-purpose encoding : they are intentionally crippled by Blu-ray restrictions, which will significantly reduce compression for ordinary non-Blu-ray encoding.

  In addition to Blu-ray support, the aforementioned commit comes with a lot of fun extras :

  x264 now has native variable-framerate ratecontrol, which makes sure your encodes get a correct target bitrate and proper limiting of maximum bitrate even if the duration of every frame is different and the “framerate” is completely unknown. This helps a lot when encoding from variable-framerate container formats such as FLV and WMV, along with variable-framerate content such as anime.

  x264 now supports pulldown (telecine) in much the same fashion as it is handled in MPEG-2. The calling application can pass in flags representing how to display a frame, allowing easy transcoding from MPEG-2 sources with pulldown, such as broadcast television. The x264 commandline app contains some examples of these (such as the common 3:2 pulldown pattern).

  x264 now also exports HRD timing information, which is critical for compliant transport stream muxing. There is currently an active project to write a fully DVB-compatible free software TS muxer that will be able to interface with x264 for a seamless free software broadcast system. It will likely also be possible to repurpose this muxer as part of a free software Blu-ray authoring package.

  All of this is now available in the latest x264.

  http://mirror05.x264.nl/Dark/bluray%20fail.png