Recherche avancée

Sur d’autres sites (7134)

• Xbox Sphinx Protocol

  21 octobre 2013, par Multimedia Mike — DRM, xbox

  I’ve gone down the rabbit hole of trying to read the Xbox DVD drive from Linux. Honestly, I’m trying to remember why I even care at this point. Perhaps it’s just my metagame of trying to understand how games and related technologies operate. In my last post of the matter, I determined that it is possible to hook an Xbox drive up to a PC using a standard 40-pin IDE interface and read data sectors. However, I learned that just because the Xbox optical drive is reading an Xbox disc, that doesn’t mean it’s just going to read the sectors in response to a host request.

  Oh goodness, no. The drive is going to make the host work for those sectors.

  To help understand the concept of locked/unlocked sectors on an Xbox disc, I offer this simplistic diagram :

  
  
  

  Any DVD drive (including the Xbox drive) is free to read those first 6992 sectors (about 14 MB of data) which just contain a short DVD video asking the user to insert the disc into a proper Xbox console. Reading the remaining sectors involves performing a sequence of SCSI commands that I have taken to calling the “Sphinx Protocol” for reasons I will explain later in this post.

  References
  Doing a little Googling after my last post on the matter produced this site hosting deep, technical Xbox information. It even has a page about exactly what I am trying to achieve : Use an Xbox DVD Drive in Your PC. The page provides a tool named dvdunlocker written by “The Specialist” to perform the necessary unlocking. The archive includes a compiled Windows binary as well as its source code. The source code is written in Delphi Pascal and leverages Windows SCSI APIs. Still, it is well commented and provides a roadmap, which I will try to describe in this post.

  Sphinx Protocol
  Here is a rough flowchart of the steps that are (probably) involved in the unlocking of those remaining sectors. I reverse engineered this based on the Pascal tool described in the previous section. Disclaimer : at the time of this writing, I haven’t tested all of the steps due to some Linux kernel problems, described later.

  
  
  

  Concerning the challenge/response table that the drive sends back, it’s large (0×664 / 1636 bytes), and not all of the bytes’ meanings are known. However, these are the bytes that seem to be necessary (all multi-byte numbers are big endian) :

   bytes 0-1        Size of mode page payload data (should be 0x0662)
   bytes 2-771      Unknown
   byte  772        Should be 1
   byte  773        Number of entries in challenge/response table
   bytes 774-1026   Encrypted challenge/response table
   bytes 1027-1186  Unknown
   bytes 1187-1230  Key basis (44 bytes)
   bytes 1231-1635  Unknown
  

  The challenge/response table is the interesting part, but it’s encrypted with RC4 a.k.a. ARCFOUR. The key is derived from the 44 bytes I have labeled “key basis”– cryptographic literature probably has a better term for it ; chime in if you know what that might be. An SHA-1 hash is computed over the 44 bytes.

  The resulting SHA-1 hash — the first part of it, to be exact — is fed as the key into the RC4 decryption. The output of SHA-1 contains 160 bits of information. 160 / 8 = 20 bytes of information. To express this as a printable hex digest requires 40 characters. The SHA-1 hash is converted to a hex digest and then the first 7 of the characters are fed into the RC4 initialization function as the key. Then, the RC4 decrypter does its work on the 253 bytes of the challenge/response table.

  So that’s why I took to calling this the “Sphinx Protocol” — I felt like I was being challenged with a bizarre riddle. Perhaps that describes a lot of cryptosystems, though You have to admit it sounds kind of cool.

  The challenge/response table contains 23 11-byte records. The format of this table is (again, multi-byte numbers are big-endian) :

   byte  0     This is 1 if this challenge/response pair is valid
   byte  1     Challenge ID
   bytes 2-5   Challenge
   byte  6     Response ID
   bytes 7-10  Response
  

  Example
  It’s useful to note that the challenge/response table and associated key is different for every disc (at least all the ones I have looked at). So this might be data that comes from the disc, since the values will always be the same for a given disc.

  Let’s examine Official Xbox Magazine disc #16 (Indiana Jones and The Emperor’s Tomb) :

  
  
  

  Before I decrypt the challenge/response table, it looks like this :

     0 : 180, 172 : 0xEB100059 ;  66 : 0xD56AFB56
     1 :  34,  71 : 0x8F9BF03A ; 192 : 0xC32CBDF8
     2 : 226, 216 : 0xA29B77F2 ;  12 : 0x4474A6F1
     3 :  72, 122 : 0x9F5ABF33 ; 255 : 0xC5E3C304
     4 :   1, 103 : 0x76142ADA ; 233 : 0xDE145D42 ****
     5 :  49, 193 : 0xA1CD6192 ; 189 : 0x2169DBA5
     6 : 182, 250 : 0x9977894F ;  96 : 0x5A929E2B
     7 : 148,  71 : 0x6DD10A54 ; 115 : 0xF0BDAC4F
     8 :  12,  45 : 0x5D5EB6FD ; 148 : 0x84E60A00
     9 :  99, 121 : 0xFEAED372 ; 201 : 0xDA9986F9
    10 : 172, 230 : 0xE6C0D0B4 ; 214 : 0x9050C250
    11 :  84,  65 : 0x95CB8775 ; 104 : 0x550886C6
    12 : 210,  65 : 0x1ED23619 ; 171 : 0x6DF4A35B
    13 :   2, 155 : 0xD0AAE1E0 ; 130 : 0x00D1FFCF
    14 :  40,   2 : 0x172EFEB8 ; 159 : 0x37E03E50
    15 :  49,  15 : 0x43E5E378 ; 223 : 0x267F9C9A
    16 : 240, 173 : 0x357D5D1C ; 250 : 0x24965D67
    17 :  80, 184 : 0x5E7AF1A3 ;  81 : 0x3A8F69A7
    18 : 154, 186 : 0x6626BEAC ; 245 : 0xE639540A
    19 : 231, 249 : 0xFABAAFB7 ; 227 : 0x4C686A07
    20 : 150, 186 : 0x9A6D7AA3 ; 133 : 0x25971CF0
    21 : 236, 192 : 0x5CD97DD4 ; 247 : 0x26655EFB
    22 :  68, 173 : 0xE2D372E4 ; 207 : 0x103FBF94
  there are 1 valid pairs in the list : 4
  

  My best clue that it’s not right is that there is only 1 valid entry (denoted by my tool using ****). The source I reverse engineered for this data indicates that there needs to be at least 2 valid pairs. After running the RC4 decryption on the table, it looks like this and I get far more valid pairs :

     0 :   1, 174 : 0xBD628255 ;   0 : 0x9F0A31AF ****
     1 :   2, 176 : 0x3151B341 ;   2 : 0x9C87C180
     2 :   3, 105 : 0x018879E5 ;   1 : 0xFF068B5C
     3 :   2,   7 : 0x1F316AAF ;   3 : 0xF420D3ED
     4 :   3,  73 : 0xC2EBFBE9 ;   0 : 0x17062B5B
     5 : 252, 163 : 0xFF14B5CB ; 236 : 0xAF813FBC
     6 :   2, 233 : 0x5EE95C49 ;   1 : 0x37AA5511
     7 :   1, 126 : 0xBD628255 ;   0 : 0x5BA3FBD4 ****
     8 :   3,   4 : 0xB68BFEE6 ;   3 : 0xA8F3B918
     9 :   3,  32 : 0xEA614943 ;   2 : 0xA678D715
    10 :   2, 248 : 0x1BDD374E ;   0 : 0x8D2AC2C7
    11 :   3,  17 : 0x0EABCE81 ;   2 : 0xC90A7242
    12 :   1, 186 : 0xBD628255 ;   0 : 0xC4820242 ****
    13 :   3, 145 : 0xB178F942 ;   3 : 0x4D78AD62
    14 :   3,  37 : 0x4A6CE5E2 ;   2 : 0xBF94E1C6
    15 :   1, 102 : 0xBD628255 ;   0 : 0xFFB83D8D ****
    16 :   3, 122 : 0xF97B0905 ;   1 : 0x38533125
    17 :   3, 197 : 0x57A6865D ;   2 : 0xA61D31EF
    18 :   3,  27 : 0xC7227D7C ;   2 : 0xA3F9BA1E
    19 :   1,  16 : 0xBD628255 ;   0 : 0x8557CCC8 ****
    20 :   2,  53 : 0x1DA9D156 ;   3 : 0xC9051754
    21 :   2,  90 : 0x3CD66BEE ;   3 : 0xFD851D3E
    22 :   1, 252 : 0xBD628255 ;   0 : 0xB3F22701 ****
  there are 6 valid pairs in the list : 0 7 12 15 19 22
  

  So, hopefully, I have the decryption correct.

  Also of note is that you only get one chance to get this unlocking correct– fail, and the drive won’t return a valid DVD structure block again. You will either need to reboot the Xbox or eject & close the tray before you get to try again.

  Problems Making It Work In Linux
  There are a couple of ways to play with SCSI protocols under Linux. In more recent kernels, block devices are named /dev/sda, /dev/sdb, etc. Each of these block devices has a corresponding character device named /dev/sg0, /dev/sg1, etc. ‘sg’ stands for SCSI generic. This character devices can be opened as readable and/or writable and SCSI commands can be freely written with write() and data retrieved with read(). Pretty powerful.

  Except that the one machine I still possess which supports 40-pin IDE/ATAPI devices is running Linux kernel 2.6.24 which dates back to early 2008 and it still enumerates the IDE block devices as /dev/hda, /dev/hdb, etc. There are no corresponding /dev/sgX character devices. What to do ? It seems that a program can still issue SCSI commands using an ioctl() facility named SG_IO.

  I was able to make the SG_IO ioctl() work for the most part (except for the discovery that the Xbox drive doesn’t respond to a basic SCSI Inquiry command). However, I ran into a serious limitation– a program can only open a /dev/hdX block device in read-only mode if the device corresponds to a read-only drive like, for example, a DVD-ROM drive. This means that a program can’t issue SCSI mode select commands to the drive, which counts as writing. This means that my tool can’t unlock the drive.

  Current Status
  So this is where my experiment is blocked right now. I have been trying to compile various Linux kernels to remedy the situation. But I always seem to find myself stuck in one of 2 situations, depending on the configuration options I choose : Either the drives are enumerated with the /dev/hdX convention and I am stuck in read-only mode (with no mode select) ; or the drives are enumerated with /dev/sdX along with corresponding /dev/sgN character devices, in which case the kernel does not recognize the Xbox DVD-ROM drive.

  This makes me wonder if there’s a discrepancy between the legacy ATA/ATAPI drivers (which sees the drive) and the newer SATA/PATA subsystem (which doesn’t see the drive). I also wonder about hacking the kernel logic to allow SCSI mode select logic to proceed to the device for a read-only file handle.

• Using FFmpeg with URL input causes SIGSEGV in AWS Lambda (Python runtime)

  26 mars, par Dave94

  I'm trying to implement a video converting solution on AWS Lambda following their article named Processing user-generated content using AWS Lambda and FFmpeg.
However when I run my command with subprocess.Popen() it returns -11 which translates to SIGSEGV (segmentation fault).
I've tried to process the video with the newest (4.3.1) static build from John Van Sickle's site as with the "official" ffmpeg-lambda-layer but it seems like it doesn't matter which one I use, the result is the same.

  


  If I download the video to the Lambda's /tmp directory and add this downloaded file as an input to FFmpeg it works correctly (with the same parameters). However I'm trying to prevent this as the /tmp directory's max. size is only 512 MB which is not quite enough for me.

  


  The relevant code which returns SIGSEGV :

  


  ffmpeg_cmd = '/opt/bin/ffmpeg -stream_loop -1 -i "' + s3_source_signed_url + '" -i /opt/bin/audio.mp3 -i /opt/bin/watermark.png -shortest -y -deinterlace -vcodec libx264 -pix_fmt yuv420p -preset veryfast -r 30 -g 60 -b:v 4500k -c:a copy -map 0:v:0 -map 1:a:0 -filter_complex scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,setsar=1,overlay=(W-w)/2:(H-h)/2,format=yuv420p -loglevel verbose -f flv -'
command1 = shlex.split(ffmpeg_cmd)
p1 = subprocess.Popen(command1, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p1.communicate()
print(p1.returncode) #prints -11


  


  stderr of FFmpeg :

  


  ffmpeg version 4.1.3-static https://johnvansickle.com/ffmpeg/  Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516
  configuration: --enable-gpl --enable-version3 --enable-static --disable-debug --disable-ffplay --disable-indev=sndio --disable-outdev=sndio --cc=gcc-6 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-gmp --enable-gray --enable-libaom --enable-libfribidi --enable-libass --enable-libvmaf --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librubberband --enable-libsoxr --enable-libspeex --enable-libvorbis --enable-libopus --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzvbi --enable-libzimg
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
[tcp @ 0x728cc00] Starting connection attempt to 52.219.74.177 port 443
[tcp @ 0x728cc00] Successfully connected to 52.219.74.177 port 443
[h264 @ 0x729b780] Reinit context to 1280x720, pix_fmt: yuv420p
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'https://bucket.s3.amazonaws.com --> presigned url with 15 min expiration time':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41isomavc1
    creation_time   : 2015-09-02T07:42:42.000000Z
  Duration: 00:00:15.64, start: 0.000000, bitrate: 2640 kb/s
    Stream #0:0(und): Video: h264 (High), 1 reference frame (avc1 / 0x31637661), yuv420p(tv, bt709, left), 1280x720 [SAR 1:1 DAR 16:9], 2475 kb/s, 25 fps, 25 tbr, 25 tbn, 50 tbc (default)
    Metadata:
      creation_time   : 2015-09-02T07:42:42.000000Z
      handler_name    : L-SMASH Video Handler
      encoder         : AVC Coding
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 160 kb/s (default)
    Metadata:
      creation_time   : 2015-09-02T07:42:42.000000Z
      handler_name    : L-SMASH Audio Handler
[mp3 @ 0x733f340] Skipping 0 bytes of junk at 1344.
Input #1, mp3, from '/opt/bin/audio.mp3':
  Metadata:
    encoded_by      : Logic Pro X
    date            : 2021-01-03
    coding_history  : 
    time_reference  : 158760000
    umid            : 0x0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000004500F9E4
    encoder         : Lavf58.49.100
  Duration: 00:04:01.21, start: 0.025057, bitrate: 320 kb/s
    Stream #1:0: Audio: mp3, 44100 Hz, stereo, fltp, 320 kb/s
    Metadata:
      encoder         : Lavc58.97
Input #2, png_pipe, from '/opt/bin/watermark.png':
  Duration: N/A, bitrate: N/A
    Stream #2:0: Video: png, 1 reference frame, rgba(pc), 701x190 [SAR 1521:1521 DAR 701:190], 25 tbr, 25 tbn, 25 tbc
[Parsed_scale_0 @ 0x7341140] w:1920 h:1080 flags:'bilinear' interl:0
Stream mapping:
  Stream #0:0 (h264) -> scale
  Stream #2:0 (png) -> overlay:overlay
  format -> Stream #0:0 (libx264)
  Stream #1:0 -> #0:1 (copy)
Press [q] to stop, [?] for help
[h264 @ 0x72d8600] Reinit context to 1280x720, pix_fmt: yuv420p
[Parsed_scale_0 @ 0x733c1c0] w:1920 h:1080 flags:'bilinear' interl:0
[graph 0 input from stream 0:0 @ 0x7669200] w:1280 h:720 pixfmt:yuv420p tb:1/25 fr:25/1 sar:1/1 sws_param:flags=2
[graph 0 input from stream 2:0 @ 0x766a980] w:701 h:190 pixfmt:rgba tb:1/25 fr:25/1 sar:1521/1521 sws_param:flags=2
[auto_scaler_0 @ 0x7670240] w:iw h:ih flags:'bilinear' interl:0
[deinterlace_in_2_0 @ 0x766b680] auto-inserting filter 'auto_scaler_0' between the filter 'graph 0 input from stream 2:0' and the filter 'deinterlace_in_2_0'
[Parsed_scale_0 @ 0x733c1c0] w:1280 h:720 fmt:yuv420p sar:1/1 -> w:1920 h:1080 fmt:yuv420p sar:1/1 flags:0x2
[Parsed_pad_1 @ 0x733ce00] w:1920 h:1080 -> w:1920 h:1080 x:0 y:0 color:0x000000FF
[Parsed_setsar_2 @ 0x733da00] w:1920 h:1080 sar:1/1 dar:16/9 -> sar:1/1 dar:16/9
[auto_scaler_0 @ 0x7670240] w:701 h:190 fmt:rgba sar:1521/1521 -> w:701 h:190 fmt:yuva420p sar:1/1 flags:0x2
[Parsed_overlay_3 @ 0x733e440] main w:1920 h:1080 fmt:yuv420p overlay w:701 h:190 fmt:yuva420p
[Parsed_overlay_3 @ 0x733e440] [framesync @ 0x733e5a8] Selected 1/50 time base
[Parsed_overlay_3 @ 0x733e440] [framesync @ 0x733e5a8] Sync level 2
[libx264 @ 0x72c1c00] using SAR=1/1
[libx264 @ 0x72c1c00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x72c1c00] profile Progressive High, level 4.0, 4:2:0, 8-bit
[libx264 @ 0x72c1c00] 264 - core 157 r2969 d4099dd - H.264/MPEG-4 AVC codec - Copyleft 2003-2019 - http://www.videolan.org/x264.html - options: cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=2 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=9 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=60 keyint_min=6 scenecut=40 intra_refresh=0 rc_lookahead=10 rc=abr mbtree=1 bitrate=4500 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, flv, to 'pipe:':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41isomavc1
    encoder         : Lavf58.20.100
    Stream #0:0: Video: h264 (libx264), 1 reference frame ([7][0][0][0] / 0x0007), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 4500 kb/s, 30 fps, 1k tbn, 30 tbc (default)
    Metadata:
      encoder         : Lavc58.35.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/4500000 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: mp3 ([2][0][0][0] / 0x0002), 44100 Hz, stereo, fltp, 320 kb/s
    Metadata:
      encoder         : Lavc58.97
frame=   27 fps=0.0 q=32.0 size=     247kB time=00:00:00.03 bitrate=59500.0kbits/s speed=0.0672x
frame=   77 fps= 77 q=27.0 size=    1115kB time=00:00:02.03 bitrate=4478.0kbits/s speed=2.03x
frame=  126 fps= 83 q=25.0 size=    2302kB time=00:00:04.00 bitrate=4712.4kbits/s speed=2.64x
frame=  177 fps= 87 q=26.0 size=    3576kB time=00:00:06.03 bitrate=4854.4kbits/s speed=2.97x
frame=  225 fps= 88 q=25.0 size=    4910kB time=00:00:07.96 bitrate=5047.8kbits/s speed=3.13x
frame=  272 fps= 89 q=27.0 size=    6189kB time=00:00:09.84 bitrate=5147.9kbits/s speed=3.22x
frame=  320 fps= 90 q=27.0 size=    7058kB time=00:00:11.78 bitrate=4907.5kbits/s speed=3.31x
frame=  372 fps= 91 q=26.0 size=    8098kB time=00:00:13.84 bitrate=4791.0kbits/s speed=3.4x


  


  And that's the end of it. It should continue to do the processing until 00:04:02 as that's my audio's length but it stops here every time (approximately this is my video length).

  


  The relevant code which works correctly :

  


  ffmpeg_cmd = '/opt/bin/ffmpeg -stream_loop -1 -i "' + '/tmp/' + s3_source_key + '" -i /opt/bin/audio.mp3 -i /opt/bin/watermark.png -shortest -y -deinterlace -vcodec libx264 -pix_fmt yuv420p -preset veryfast -r 30 -g 60 -b:v 4500k -c:a copy -map 0:v:0 -map 1:a:0 -filter_complex scale=1920:1080:force_original_aspect_ratio=decrease,pad=1920:1080:(ow-iw)/2:(oh-ih)/2,setsar=1,overlay=(W-w)/2:(H-h)/2,format=yuv420p -loglevel verbose -f flv -'
command1 = shlex.split(ffmpeg_cmd)
p1 = subprocess.Popen(command1, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p1.communicate()
print(p1.returncode) #prints 0


  


  With this code it repeats the video as many times as it has to do to be as long as the audio.

  


  Both versions work correctly on my computer.

  


  This question is almost the same but in my case FFmpeg is able to access the signed URL.

  


• Muxing Android MediaCodec encoded H264 packets into RTMP

  31 décembre 2015, par Vadym

  I am coming from a thread Encoding H.264 from camera with Android MediaCodec. My setup is very similar. However, I attempt to write mux the encoded frames and with javacv and broadcast them via rtmp.

  RtmpClient.java

  ...
  
  private volatile BlockingQueue mFrameQueue = new LinkedBlockingQueue(MAXIMUM_VIDEO_FRAME_BACKLOG);
  
  ...
  
  private void startStream() throws FrameRecorder.Exception, IOException {
  
      if (TextUtils.isEmpty(mDestination)) {
  
          throw new IllegalStateException("Cannot start RtmpClient without destination");
  
      }
  
  
  
      if (mCamera == null) {
  
          throw new IllegalStateException("Cannot start RtmpClient without camera.");
  
      }
  
  
  
      Camera.Parameters cameraParams = mCamera.getParameters();
  
  
  
      mRecorder = new FFmpegFrameRecorder(
  
              mDestination,
  
              mVideoQuality.resX,
  
              mVideoQuality.resY,
  
              (mAudioQuality.channelType.equals(AudioQuality.CHANNEL_TYPE_STEREO) ? 2 : 1));
  
  
  
      mRecorder.setFormat("flv");
  
  
  
      mRecorder.setFrameRate(mVideoQuality.frameRate);
  
      mRecorder.setVideoBitrate(mVideoQuality.bitRate);
  
      mRecorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);
  
  
  
      mRecorder.setSampleRate(mAudioQuality.samplingRate);
  
      mRecorder.setAudioBitrate(mAudioQuality.bitRate);
  
      mRecorder.setAudioCodec(avcodec.AV_CODEC_ID_AAC);
  
  
  
      mVideoStream = new VideoStream(mRecorder, mVideoQuality, mFrameQueue, mCamera);
  
      mAudioStream = new AudioStream(mRecorder, mAudioQuality);
  
  
  
      mRecorder.start();
  
  
  
      // Setup a bufferred preview callback
  
      setupCameraCallback(mCamera, mRtmpClient, DEFAULT_PREVIEW_CALLBACK_BUFFERS,
  
              mVideoQuality.resX * mVideoQuality.resY * ImageFormat.getBitsPerPixel(
  
                      cameraParams.getPreviewFormat())/8);
  
  
  
      try {
  
          mVideoStream.start();
  
          mAudioStream.start();
  
      }
  
      catch(Exception e) {
  
          e.printStackTrace();
  
          stopStream();
  
      }
  
  }
  
  ...
  
  @Override
  
  public void onPreviewFrame(byte[] data, Camera camera) {
  
      boolean frameQueued = false;
  
  
  
      if (mRecorder == null || data == null) {
  
          return;
  
      }
  
  
  
      frameQueued = mFrameQueue.offer(data);
  
  
  
      // return the buffer to be reused - done in videostream
  
      //camera.addCallbackBuffer(data);
  
  }
  
  ...

  VideoStream.java

  ...
  
  @Override
  
  public void run() {
  
      try {
  
          mMediaCodec = MediaCodec.createEncoderByType("video/avc");
  
          MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc", mVideoQuality.resX, mVideoQuality.resY);
  
          mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, mVideoQuality.bitRate);
  
          mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, mVideoQuality.frameRate);
  
          mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420SemiPlanar);
  
          mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1);
  
          mMediaCodec.configure(mediaFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
  
          mMediaCodec.start();
  
      }
  
      catch(IOException e) {
  
          e.printStackTrace();
  
      }
  
  
  
      long startTimestamp = System.currentTimeMillis();
  
      long frameTimestamp = 0;
  
      byte[] rawFrame = null;
  
  
  
      try {
  
          while (!Thread.interrupted()) {
  
              rawFrame = mFrameQueue.take();
  
  
  
              frameTimestamp = 1000 * (System.currentTimeMillis() - startTimestamp);
  
  
  
              encodeFrame(rawFrame, frameTimestamp);
  
  
  
              // return the buffer to be reused
  
              mCamera.addCallbackBuffer(rawFrame);
  
          }
  
      }
  
      catch (InterruptedException ignore) {
  
          // ignore interrup while waiting
  
      }
  
  
  
      // Clean up video stream allocations
  
      try {
  
          mMediaCodec.stop();
  
          mMediaCodec.release();
  
          mOutputStream.flush();
  
          mOutputStream.close();
  
      } catch (Exception e){
  
          e.printStackTrace();
  
      }
  
  }
  
  ...
  
  private void encodeFrame(byte[] input, long timestamp) {
  
      try {
  
          ByteBuffer[] inputBuffers = mMediaCodec.getInputBuffers();
  
          ByteBuffer[] outputBuffers = mMediaCodec.getOutputBuffers();
  
  
  
          int inputBufferIndex = mMediaCodec.dequeueInputBuffer(0);
  
  
  
          if (inputBufferIndex >= 0) {
  
              ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
  
              inputBuffer.clear();
  
              inputBuffer.put(input);
  
              mMediaCodec.queueInputBuffer(inputBufferIndex, 0, input.length, timestamp, 0);
  
          }
  
  
  
          MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
  
  
  
          int outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, 0);
  
  
  
          if (outputBufferIndex >= 0) {
  
              while (outputBufferIndex >= 0) {
  
                  ByteBuffer outputBuffer = outputBuffers[outputBufferIndex];
  
  
  
                  // Should this be a direct byte buffer?
  
                  byte[] outData = new byte[bufferInfo.size - bufferInfo.offset];
  
                  outputBuffer.get(outData);
  
  
  
                  mFrameRecorder.record(outData, bufferInfo.offset, outData.length, timestamp);
  
  
  
                  mMediaCodec.releaseOutputBuffer(outputBufferIndex, false);
  
                  outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, 0);
  
              }
  
          }
  
          else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
  
              outputBuffers = mMediaCodec.getOutputBuffers();
  
          } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
  
              // ignore for now
  
          }
  
      } catch (Throwable t) {
  
          t.printStackTrace();
  
      }
  
  
  
  }
  
  ...

  FFmpegFrameRecorder.java

  ...
  
  // Hackish codec copy frame recording function
  
  public boolean record(byte[] encodedData, int offset, int length, long frameCount) throws Exception {
  
      int ret;
  
  
  
      if (encodedData == null) {
  
          return false;
  
      }
  
  
  
      av_init_packet(video_pkt);
  
  
  
      // this is why i wondered whether I should get outputbuffer data into direct byte buffer
  
      video_outbuf.put(encodedData, 0, encodedData.length);
  
  
  
      video_pkt.data(video_outbuf);
  
      video_pkt.size(video_outbuf_size);
  
  
  
      video_pkt.pts(frameCount);
  
      video_pkt.dts(frameCount);
  
  
  
      video_pkt.stream_index(video_st.index());
  
  
  
      synchronized (oc) {
  
          /* write the compressed frame in the media file */
  
          if (interleaved && audio_st != null) {
  
              if ((ret = av_interleaved_write_frame(oc, video_pkt)) < 0) {
  
                  throw new Exception("av_interleaved_write_frame() error " + ret + " while writing interleaved video frame.");
  
              }
  
          } else {
  
              if ((ret = av_write_frame(oc, video_pkt)) < 0) {
  
                  throw new Exception("av_write_frame() error " + ret + " while writing video frame.");
  
              }
  
          }
  
      }
  
      return (video_pkt.flags() & AV_PKT_FLAG_KEY) == 1;
  
  }
  
  ...

  When I try to stream the video and run ffprobe on it, I get the following output :

  ffprobe version 2.5.3 Copyright (c) 2007-2015 the FFmpeg developers
  
    built on Jan 19 2015 12:56:57 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-55)
  
    configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --enable-bzlib --disable-crystalhd --enable-libass --enable-libdc1394 --enable-libfaac --enable-nonfree --disable-indev=jack --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid --enable-x11grab --enable-avfilter --enable-avresample --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --enable-libcaca --shlibdir=/usr/lib64 --enable-runtime-cpudetect
  
    libavutil      54. 15.100 / 54. 15.100
  
    libavcodec     56. 13.100 / 56. 13.100
  
    libavformat    56. 15.102 / 56. 15.102
  
    libavdevice    56.  3.100 / 56.  3.100
  
    libavfilter     5.  2.103 /  5.  2.103
  
    libavresample   2.  1.  0 /  2.  1.  0
  
    libswscale      3.  1.101 /  3.  1.101
  
    libswresample   1.  1.100 /  1.  1.100
  
    libpostproc    53.  3.100 / 53.  3.100
  
  Metadata:
  
    Server                NGINX RTMP (github.com/arut/nginx-rtmp-module)
  
    width                 320.00
  
    height                240.00
  
    displayWidth          320.00
  
    displayHeight         240.00
  
    duration              0.00
  
    framerate             0.00
  
    fps                   0.00
  
    videodatarate         261.00
  
    videocodecid          7.00
  
    audiodatarate         62.00
  
    audiocodecid          10.00
  
    profile
  
    level
  
  [live_flv @ 0x1edb0820] Could not find codec parameters for stream 0 (Video: none, none, 267 kb/s): unknown codec
  
  Consider increasing the value for the 'analyzeduration' and 'probesize' options
  
  Input #0, live_flv, from 'rtmp://<server>/input/<stream>':
  
    Metadata:
  
      Server          : NGINX RTMP (github.com/arut/nginx-rtmp-module)
  
      displayWidth    : 320
  
      displayHeight   : 240
  
      fps             : 0
  
      profile         :
  
      level           :
  
    Duration: 00:00:00.00, start: 16.768000, bitrate: N/A
  
      Stream #0:0: Video: none, none, 267 kb/s, 1k tbr, 1k tbn, 1k tbc
  
      Stream #0:1: Audio: aac (LC), 16000 Hz, mono, fltp, 63 kb/s
  
  Unsupported codec with id 0 for input stream 0
  
  </stream></server>

  I am not, by any means, an expert in H264 or video encoding. I know that the encoded frames that come out from MediaCodec contain SPS NAL, PPS NAL, and frame NAL units. I’ve also written the MediaCodec output into a file and was able to play it back (I did have to specify the format and framerate as otherwise it would play too fast).

  My assumption is that things should work (see how little I know :)). Knowing that SPS and PPS are written out, decoder should know enough. Yet, ffprobe fails to recognize codec, fps, and other video information. Do I need to pass packet flag information to FFmpegFrameRecorder.java:record() function ? Or should I use direct buffer ? Any suggestion will be appreciated ! I should figure things out with a hint.

  PS : I know that some codecs use Planar and other SemiPlanar color formats. That distinction will come later if I get past this. Also, I didn’t go the Surface to MediaCodec way because I need to support API 17 and it requires more changes than this route, which I think helps me understand the more basic flow. Agan, I appreciate any suggestions. Please let me know if something needs to be clarified.

  Update #1

  So having done more testing, I see that my encoder outputs the following frames :

  000000016742800DDA0507E806D0A1350000000168CE06E2
  
  0000000165B840A6F1E7F0EA24000AE73BEB5F51CC7000233A84240...
  
  0000000141E2031364E387FD4F9BB3D67F51CC7000279B9F9CFE811...
  
  0000000141E40304423FFFFF0B7867F89FAFFFFFFFFFFCBE8EF25E6...
  
  0000000141E602899A3512EF8AEAD1379F0650CC3F905131504F839...
  
  ...

  The very first frame contains SPS and PPS. From what I was able to see, these are transmitted only once. The rest are NAL types 1 and 5. So, my assumption is that, for ffprobe to see stream info not only when the stream starts, I should capture SPS and PPS frames and re-transmit them myself periodically, after a certain number of frames, or perhaps before every I-frame. What do you think ?

  Update #2

  Unable to validate that I’m writing frames successfully. After having tried to read back the written packet, I cannot validate written bytes. As strange, on successful write of IPL image and streaming, I also cannot print out bytes of encoded packet after avcodec_encode_video2. Hit the official dead end.