Recherche avancée

Sur d’autres sites (4186)

• How to convert H264 RTP stream from PCAP to a playable video file

  21 août 2014, par yoosha

  I have captured stream of H264 in PCAP files and trying to create media files from the data. The container is not important (avi,mp4,mkv,…).
  When I’m using videosnarf or rtpbreak (combined with python code that adds 00 00 00 01 before each packet) and then ffmpeg, the result is OK only if the input frame rate is constant (or near constant). However, when the input is vfr, the result plays too fast (and on same rare cases too slow).
  For example :

  videosnarf -i captured.pcap –c
  ffmpeg -i H264-media-1.264 output.avi

  After doing some investigation of the issue I believe now that since the videosnarf (and rtpbreak) are removing the RTP header from the packets, the timestamp is lost and ffmpeg is referring to the input data as cbr.

  1. I would like to know if there is a way to pass (on a separate file ?)
     the timestamps vector or any other information to ffmpeg so the
     result will be created correctly ?
  2. Is there any other way I can take the data out of the PCAP file and play it or convert it and then play it ?
  3. Since all work is done in Python, any suggestion of libraries/modules that can help with the work (even if requires some codding) is welcome as well.

  Note : All work is done offline, no limitations on the output. It can be cbr/vbr, any playable container and transcoding. The only "limitation" I have : it should all run on linux…

  Thanks
  Y

  Some additional information :
  Since the nothing provides the FFMPEG with the timestamp data, i decided to try a different approach : skip videosnarf and use Python code to pipe the packets directly to ffmpeg (using the "-f -i -" options) but then it refuses to accept it unless I provide an SDP file...
  How do I provide the SDP file ? is it an additional input file ? ("-i config.sdp")

  The following code is an unsuccessful try doing the above :

  import time  
  
  import sys  
  
  import shutil  
  
  import subprocess  
  
  import os  
  
  import dpkt  
  
  
  
  if len(sys.argv) < 2:  
  
      print "argument required!"  
  
      print "txpcap <pcap file="file">"  
  
      sys.exit(2)  
  
  pcap_full_path = sys.argv[1]  
  
  
  
  ffmp_cmd = ['ffmpeg','-loglevel','debug','-y','-i','109c.sdp','-f','rtp','-i','-','-na','-vcodec','copy','p.mp4']  
  
  
  
  ffmpeg_proc = subprocess.Popen(ffmp_cmd,stdout = subprocess.PIPE,stdin = subprocess.PIPE)  
  
  
  
  with open(pcap_full_path, "rb") as pcap_file:  
  
      pcapReader = dpkt.pcap.Reader(pcap_file)  
  
      for ts, data in pcapReader:  
  
          if len(data) &lt; 49:  
  
              continue  
  
          ffmpeg_proc.stdin.write(data[42:])
  
  
  
  sout, err = ffmpeg_proc.communicate()  
  
  print "stdout ---------------------------------------"  
  
  print sout  
  
  print "stderr ---------------------------------------"  
  
  print err  
  
  </pcap>

  In general this will pipe the packets from the PCAP file to the following command :

  ffmpeg -loglevel debug -y -i 109c.sdp -f rtp -i - -na -vcodec copy p.mp4

  SDP file : [RTP includes dynamic payload type # 109, H264]

  v=0
  o=- 0 0 IN IP4 ::1
  s=No Name
  c=IN IP4 ::1
  t=0 0
  a=tool:libavformat 53.32.100
  m=video 0 RTP/AVP 109
  a=rtpmap:109 H264/90000
  a=fmtp:109
  packetization-mode=1 ;profile-level-id=64000c ;sprop-parameter-sets=Z2QADKwkpAeCP6wEQAAAAwBAAAAFI8UKkg==,aMvMsiw= ;
  b=AS:200

  Results :

  ffmpeg version 0.10.2 Copyright (c) 2000-2012 the FFmpeg developers
  built on Mar 20 2012 04:34:50 with gcc 4.4.6 20110731 (Red Hat
  4.4.6-3) configuration : —prefix=/usr —libdir=/usr/lib64 —shlibdir=/usr/lib64 —mandir=/usr/share/man —enable-shared —enable-runtime-cpudetect —enable-gpl —enable-version3 —enable-postproc —enable-avfilter —enable-pthreads —enable-x11grab —enable-vdpau —disable-avisynth —enable-frei0r —enable-libopencv —enable-libdc1394 —enable-libdirac —enable-libgsm —enable-libmp3lame —enable-libnut —enable-libopencore-amrnb —enable-libopencore-amrwb —enable-libopenjpeg —enable-librtmp —enable-libschroedinger —enable-libspeex —enable-libtheora —enable-libvorbis —enable-libvpx —enable-libx264 —enable-libxavs —enable-libxvid —extra-cflags=’-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector —param=ssp-buffer-size=4 -m64 -mtune=generic -fPIC’ —disable-stripping libavutil 51. 35.100 / 51. 35.100 libavcodec 53. 61.100 / 53. 61.100 libavformat 53. 32.100
  / 53. 32.100 libavdevice 53. 4.100 / 53. 4.100
  libavfilter 2. 61.100 / 2. 61.100 libswscale 2. 1.100
  / 2. 1.100 libswresample 0. 6.100 / 0. 6.100
  libpostproc 52. 0.100 / 52. 0.100 [sdp @ 0x15c0c00] Format sdp
  probed with size=2048 and score=50 [sdp @ 0x15c0c00] video codec set
  to : h264 [NULL @ 0x15c7240] RTP Packetization Mode : 1 [NULL @
  0x15c7240] RTP Profile IDC : 64 Profile IOP : 0 Level : c [NULL @
  0x15c7240] Extradata set to 0x15c78e0 (size : 36) !error,_recognition
  separate : 1 ; 1 [h264 @ 0x15c7240] error,_recognition combined : 1 ;
  10001 [sdp @ 0x15c0c00] decoding for stream 0 failed [sdp @
  0x15c0c00] Could not find codec parameters (Video : h264) [sdp @
  0x15c0c00] Estimating duration from bitrate, this may be inaccurate
  109c.sdp : could not find codec parameters Traceback (most recent
  call last) : File "./ffpipe.py", line 26, in
  ffmpeg_proc.stdin.write(data[42 :]) IOError : [Errno 32] Broken pipe

  (forgive the mass above, the editor keep on complaining about code that is not indented OK ??)

  I’m working on this issue for days... any help/suggestion/hint will be appreciated.

• ffmpeg and timecode from movie metadata

  17 décembre 2015, par T4ng10r

  I want to add to movie made by digital camcorder timecode. This timecodes are visible during playback in camera, some tool to extract and visualize EXIF metadata also display them.

  I made few attempts to configure ffmpeg, but only succeed in adding CURRENT PC time, not time from file.

  ffmpeg -y -i S1480002.MP4 -vf "drawtext=fontfile=arial.ttf :expansion=normal: text=%{metadata\\:creation_time}: \ x=(w-tw)/2: y=h-(2*lh): fontcolor=white@0.8" output.mp4

  I need to extract creation time from input file metadata. ffprobe display this time, but ffmpeg don’t.

• Reverse Engineering Italian Literature

  1er juillet 2014, par Multimedia Mike — Reverse Engineering

  Some time ago, Diego “Flameeyes” Pettenò tried his hand at reverse engineering a set of really old CD-ROMs containing even older Italian literature. The goal of this RE endeavor would be to extract the useful literature along with any structural metadata (chapters, etc.) and convert it to a more open format suitable for publication at, e.g., Project Gutenberg or Archive.org.

  Unfortunately, the structure of the data thwarted the more simplistic analysis attempts (like inspecting for blocks of textual data). This will require deeper RE techniques. Further frustrating the effort, however, is the fact that the binaries that implement the reading program are written for the now-archaic Windows 3.1 operating system.

  In pursuit of this RE goal, I recently thought of a way to glean more intelligence using DOSBox.

  Prior Work
  There are 6 discs in the full set (distributed along with 6 sequential issues of a print magazine named L’Espresso). Analysis of the contents of the various discs reveals that many of the files are the same on each disc. It was straightforward to identify the set of files which are unique on each disc. This set of files all end with the extension “LZn”, where n = 1..6 depending on the disc number. Further, the root directory of each disc has a file indicating the sequence number (1..6) of the CD. Obviously, these are the interesting targets.

  The LZ file extensions stand out to an individual skilled in the art of compression– could it be a variation of the venerable LZ compression ? That’s actually unlikely because LZ — also seen as LIZ — stands for Letteratura Italiana Zanichelli (Zanichelli’s Italian Literature).

  The Unix ‘file’ command was of limited utility, unable to plausibly identify any of the files.

  Progress was stalled.

  Saying Hello To An Old Frenemy
  I have been showing this screenshot to younger coworkers to see if any of them recognize it :

  
  
  

  Not a single one has seen it before. Senior computer citizen status : Confirmed.

  I recently watched an Ancient DOS Games video about Windows 3.1 games. This episode showed Windows 3.1 running under DOSBox. I had heard this was possible but that it took a little work to get running. I had a hunch that someone else had probably already done the hard stuff so I took to the BitTorrent networks and quickly found a download that had the goods ready to go– a directory of Windows 3.1 files that just had to be dropped into a DOSBox directory and they would be ready to run.

  Aside : Running OS software procured from a BitTorrent network ? Isn’t that an insane security nightmare ? I’m not too worried since it effectively runs under a sandboxed virtual machine, courtesy of DOSBox. I suppose there’s the risk of trojan’d OS software infecting binaries that eventually leave the sandbox.

  Using DOSBox Like ‘strace’
  strace is a tool available on some Unix systems, including Linux, which is able to monitor the system calls that a program makes. In reverse engineering contexts, it can be useful to monitor an opaque, binary program to see the names of the files it opens and how many bytes it reads, and from which locations. I have written examples of this before (wow, almost 10 years ago to the day ; now I feel old for the second time in this post).

  Here’s the pitch : Make DOSBox perform as strace in order to serve as a platform for reverse engineering Windows 3.1 applications. I formed a mental model about how DOSBox operates — abstracted file system classes with methods for opening and reading files — and then jumped into the source code. Sure enough, the code was exactly as I suspected and a few strategic print statements gave me the data I was looking for.

  Eventually, I even took to running DOSBox under the GNU Debugger (GDB). This hasn’t proven especially useful yet, but it has led to an absurd level of nesting :

  
  
  

  The target application runs under Windows 3.1, which is running under DOSBox, which is running under GDB. This led to a crazy situation in which DOSBox had the mouse focus when a GDB breakpoint was triggered. At this point, DOSBox had all desktop input focus and couldn’t surrender it because it wasn’t running. I had no way to interact with the Linux desktop and had to reboot the computer. The next time, I took care to only use the keyboard to navigate the application and trigger the breakpoint and not allow DOSBox to consume the mouse focus.

  New Intelligence
  
  By instrumenting the local file class (virtual HD files) and the ISO file class (CD-ROM files), I was able to watch which programs and dynamic libraries are loaded and which data files the code cares about. I was able to narrow down the fact that the most interesting programs are called LEGGENDO.EXE (‘reading’) and LEGGENDA.EXE (‘legend’ ; this has been a great Italian lesson as well as RE puzzle). The first calls the latter, which displays this view of the data we are trying to get at :

  
  
  

  When first run, the program takes an interest in a file called DBBIBLIO (‘database library’, I suspect) :

  === Read(’LIZ98\DBBIBLIO.LZ1’) : req 337 bytes ; read 337 bytes from pos 0x0
  === Read(’LIZ98\DBBIBLIO.LZ1’) : req 337 bytes ; read 337 bytes from pos 0x151
  === Read(’LIZ98\DBBIBLIO.LZ1’) : req 337 bytes ; read 337 bytes from pos 0x2A2
  [...]
  

  While we were unable to sort out all of the data files in our cursory investigation, a few things were obvious. The structure of this file looked to contain 336-byte records. Turns out I was off by 1– the records are actually 337 bytes each. The count of records read from disc is equal to the number of items shown in the UI.

  Next, the program is interested in a few more files :

  *** isoFile() : ’DEPOSITO\BLOKCTC.LZ1’, offset 0x27D6000, 2911488 bytes large
  === Read(’DEPOSITO\BLOKCTC.LZ1’) : req 96 bytes ; read 96 bytes from pos 0x0
  *** isoFile() : ’DEPOSITO\BLOKCTX0.LZ1’, offset 0x2A9D000, 17152 bytes large
  === Read(’DEPOSITO\BLOKCTX0.LZ1’) : req 128 bytes ; read 128 bytes from pos 0x0
  === Seek(’DEPOSITO\BLOKCTX0.LZ1’) : seek 384 (0x180) bytes, type 0
  === Read(’DEPOSITO\BLOKCTX0.LZ1’) : req 256 bytes ; read 256 bytes from pos 0x180
  === Seek(’DEPOSITO\BLOKCTC.LZ1’) : seek 1152 (0x480) bytes, type 0
  === Read(’DEPOSITO\BLOKCTC.LZ1’) : req 32 bytes ; read 32 bytes from pos 0x480
  === Read(’DEPOSITO\BLOKCTC.LZ1’) : req 1504 bytes ; read 1504 bytes from pos 0x4A0
  [...]

  Eventually, it becomes obvious that BLOKCTC has the juicy meat. There are 32-byte records followed by variable-length encoded text sections. Since there is no text to be found in these files, the text is either compressed, encrypted, or both. Some rough counting (the program seems to disable copy/paste, which thwarts more precise counting), indicates that the text size is larger than the data chunks being read from disc, so compression seems likely. Encryption isn’t out of the question (especially since the program deems it necessary to disable copy and pasting of this public domain literary data), and if it’s in use, that means the key is being read from one of these files.

  Blocked On Disassembly
  So I’m a bit blocked right now. I know exactly where the data lives, but it’s clear that I need to reverse engineer some binary code. The big problem is that I have no idea how to disassemble Windows 3.1 binaries. These are NE-type executable files. Disassemblers abound for MZ files (MS-DOS executables) and PE files (executables for Windows 95 and beyond). NE files get no respect. It’s difficult (but not impossible) to even find data about the format anymore, and details are incomplete. It should be noted, however, the DOSBox-as-strace method described here lends insight into how Windows 3.1 processes NE-type EXEs. You can’t get any more authoritative than that.

  So far, I have tried the freeware version of IDA Pro. Unfortunately, I haven’t been able to get the program to work on my Windows machine for a long time. Even if I could, I can’t find any evidence that it actually supports NE files (the free version specifically mentions MZ and PE, but does not mention NE or LE).

  I found an old copy of Borland’s beloved Turbo Assembler and Debugger package. It has Turbo Debugger for Windows, both regular and 32-bit versions. Unfortunately, the normal version just hangs Windows 3.1 in DOSBox. The 32-bit Turbo Debugger loads just fine but can’t load the NE file.

  I’ve also wondered if DOSBox contains any advanced features for trapping program execution and disassembling. I haven’t looked too deeply into this yet.

  Future Work
  NE files seem to be the executable format that time forgot. I have a crazy brainstorm about repacking NE files as MZ executables so that they could be taken apart with an MZ disassembler. But this will take some experimenting.

  If anyone else has any ideas about ripping open these binaries, I would appreciate hearing them.

  And I guess I shouldn’t be too surprised to learn that all the literature in this corpus is already freely available and easily downloadable anyway. But you shouldn’t be too surprised if that doesn’t discourage me from trying to crack the format that’s keeping this particular copy of the data locked up.