Recherche avancée

Sur d’autres sites (5022)

• An introduction to reverse engineering

  22 janvier 2011

  (This blog is still in hibernation, but I needed somewhere to post this)

  Reverse engineering is one of those wonderful topics, covering everything from simple "guess how this program works" problem solving, to poking at silicon with scanning electron microscopes. I’m always hugely fascinated by articles that walk through the steps involved in these types of activities, so I thought I’d contribute one back to the world.

  In this case, I’m going to be looking at the export bundle format created by the Tandberg Content Server, a device for recording video conferences. The bundle is intended for moving recordings between Tandberg devices, but it’s also the easiest way to get all of the related assets for a recorded conference. Unfortunately, there’s no parser available to take the bundle files (.tcb) and output the component pieces. Well, that just won’t do.

  For this type of reverse engineering, I basically want to learn enough about the TCB format to be able to parse out the individual files within it. The only tools I’ll need in this process are a hex editor, a notepad, and a way to convert between hex and decimal (the OS X calculator will do fine if you’re not the type to do it in your head).

  Step 1 : Basic Research
  After Googling around to see if this was a solved issue, I decided to dive into the format. I brought a sample bundle into my trusty hex editor (in this case Hex Fiend).

  

  A few things are immediately obvious. First, we see the first four bytes are the letters TCSB. Another quick visit to Google confirms this header type isn’t found elsewhere, and there’s essentially no discussion of it. Going a few bytes further, we see "contents.xml." And a few bytes after that, we see what looks like plaintext XML. This is a pretty good clue that the TCB file consists of a . Let’s scan a bit further and see if we can confirm that.
  
  In this segment, we see the end of the XML, and something that could be another filename - "dbtransfer" - followed by what looks like gibberish. That doesn’t help too much. Let’s keep looking.
  
  Great - a .jpg ! Looking a bit further, we see the letters "JFIF," which is recognizable as part of a JPEG header. If you weren’t already familiar with that, a quick google for "jpg hex header" would clear up any confusion. So, we’ve got the basics of the file format down, but we’ll need a little bit more information if we’re going to write a parser.

  Step 2 : Finding the pattern
  We can make an educated guess that a file like this has to provide a few hints to a decoder. We would either expect a table of contents, describing where in the bundle each individual file was located, some sort of stop bit marking the boundary between files, byte offsets describing the locations of files, or a listing of file lengths.

  There isn’t any sign of a table of contents. Let’s start looking for a stop bit, as that would make writing our parser really easy. Want I’m going to do is pull out all of the data between two prospective files, and I want two sets to compare.
  I’ve placed asterisks to flag the bytes corresponding to the filenames, since those are known.

  1E D1 70 4C 25 06 36 4D 42 E9 65 6A 9F 5D 88 38 0A 00 *64 62 74 72 61 6E 73 66 65 72* 42 06 ED 48 0B 50 0A C4 14 D6 63 42 F2 BF E3 9D 20 29 00 00 00 00 00 00 DE E5 FD

  01 0C 00 *63 6F 6E 74 65 6E 74 73 2E 78 6D 6C* 9E 0E FE D3 C9 3A 3A 85 F4 E4 22 FE D0 21 DC D7 53 03 00 00 00 00 00 00

  The first line corresponds to the "dbtransfer" entry, the second to the "contents.xml" entry. Let’s trim the first entry to match the second.

  38 0A 00 *64 62 74 72 61 6E 73 66 65 72* 42 06 ED 48 0B 50 0A C4 14 D6 63 42 F2 BF E3 9D 20 29 00 00 00 00 00 00

  01 0C 00 *63 6F 6E 74 65 6E 74 73 2E 78 6D 6C* 9E 0E FE D3 C9 3A 3A 85 F4 E4 22 FE D0 21 DC D7 53 03 00 00 00 00 00 00

  It looks like we’ve got three bytes before the filename, followed by 18 bytes, followed by six bytes of zero. Unfortunately, there’s no obvious pattern of bits which would correspond to a "break" between segments. However, looking at those first three bytes, we see a 0x0A, and a 0x0C, two small values in the same place. 10 and 12. Interesting - the 12 entry corresponds with "contents.xml" and the 10 entry corresponds with "dbtransfer". Could that byte describe the length of the filename ? Let’s look at our much longer JPG entry to be sure.

  70 4A 00 *77 77 77 5C 73 6C 69 64 65 73 5C 64 37 30 64 35 34 63 66 2D 32 39 35 62 2D 34 31 34 63 2D 61 38 64 66 2D 32 66 37 32 64 66 33 30 31 31 35 65 5C 74 68 75 6D 62 6E 61 69 6C 73 5C 74 68 75 6D 62 6E 61 69 6C 30 30 2E 6A 70 67*

  0x4A - 74, corresponding to a 74 character filename. Looks like we’re in business.

  At this point, it’s worth an aside to talk about endianness. I happen to know that the Tandberg Content Server runs Windows on Intel, so I went into this with the assumption that the format was little-endian. However, if you’re not sure, it’s always worth looking at words backwards and forwards, just in case.

  So we know how to find our filename. Now how do we find our file data ? Let’s go back to our JPEG. We know that JPEGs start with 0xFFD8FFE0, and a quick trip to Google also tells us that they end with 0xFFD9. We can use that to pull a sample jpeg out of our TCB, save it to disk, and confirm that we’re on the right track.
  

  This is one of those great steps in reverse engineering - concrete proof that you’re on the right track. Everything seems to go quicker from this point on.

  So, we know we’ve got a JPEG file in a continuous 2177 byte segment. We know that the format used byte lengths to describe filenames - maybe it also uses byte lengths to describe file lengths. Let’s look for 2177, or 0x8108, near our JPEG.

  

  Well, that’s a good sign. But, it could be coincidental, so at this point we’d want to check a few other files to be sure. In fact, looking further in some file, we find some larger .mp4 files which don’t quite match our guess. It turns out that file length is a 32bit value, not a 16bit value - with our two jpegs, the larger bytes just happened to be zeros.

  Step 3 : Writing a parser

  "Bbbbbut...", I hear you say ! "You have all these chunks of data you don’t understand !"

  True enough, but all I care about is getting the files out, with the proper names. I don’t care about creation dates, file permissions, or any of the other crud that this file format likely contains.

  

  Let’s look at the first two files in this bundle. A little bit of byte counting shows us the pattern that we can follow. We’ll treat the first file as a special case. After that, we seek 16 bytes from the end of file data to find the filename length (2 bytes), then we’re at the filename, then we seek 16 bytes to find the file length (4 bytes) and seek another 4 bytes to find the start of the file data. Rinse, repeat.

  I wrote a quick parser in PHP, since the eventual use for this information is part of a larger PHP-based application, but any language with basic raw file handling would work just as well.

  tcsParser.txt
  This was about the simplest possible type of reverse engineering - we had known data in an unknown format, without any compression or encryption. It only gets harder from here...

• ffmpeg stream rejected by youtube because it's too slow

  15 septembre 2019, par DeadlyBacon

  I have an app that sends WebM video to a socket in my server, the socket then executes ffmpeg to transform the video to flv and send it to a youtube rtmp ingester.

  Thing is, youtube rejects the stream, saying that the broadcast status is incorrect, no matter what I do.

  In my naivette I initially tried to stream 1080p and 720p, that failed, so I went down, assuming that that might help me with the bitrate issue... It did not.

  The error that Youtube gives me is in spanish but it basically says that the stream is too slow and I should lower resolution or bitrate(I’m already @ 240p and it’s not working).

  Edit : Here’s the error, translated :

  Main broadcast :
  YouTube is not receiving enough video to guarantee a fluid broadcast. Buffering will occur.

  Main Broadcast : We are not receiving video data at a fast enough speed. Your audience may experience buffering. Make sure that your connection is fast enough o consider using a lower bitrate.

  Here is my call to ffmpeg thus far (language is node.js if that matters.) :

  const ffmpeg = child_process.spawn('ffmpeg', [
  
  
  
      '-f', 'lavfi', '-i', 'anullsrc',
  
  
  
      //    '-re', // I was told i shouldnt use this parameter. i dont know, honestly.
  
  
  
  
  
  
  
      // FFmpeg will read input video from STDIN
  
      '-i', '-',
  
  
  
      // Because we're using a generated audio source which never ends,
  
      // specify that we'll stop at end of other input.  Remove this line if you
  
      // send audio from the browser.
  
      '-shortest',
  
  
  
      '-vcodec', 'libx264',
  
  
  
      '-acodec', 'aac',
  
  
  
      //ffmpeg
  
      //-re -loop 1
  
      //-framerate 2 -i test1.jpg -i https://xxxxxxxxxxx:8443/live.ogg
  
      //-c:a aac
  
      //-s 2560x1440
  
      //-ab 128k -vcodec libx264 -pix_fmt yuv420p -maxrate 2048k -bufsize 2048k
  
      //'-framerate', '30',
  
      '-r', '24',
  
      //'-s', '2560x1440',
  
  
  
  
  
      '-force_key_frames','expr:gte(t\,n_forced/2)',
  
  
  
      '-preset', 'ultrafast',
  
      '-pix_fmt', 'yuv420p',
  
      '-s', '426x240',
  
      '-crf', '23',
  
      '-bf', '2',
  
      '-q:a', '1',
  
      '-ac', '2',
  
      '-ar','48000',
  
      '-use_editlist','0',
  
      '-movflags','+faststart',
  
      //  '-ab', '128k',
  
      '-g', '48',
  
  
  
      //'-minrate', '1500k',
  
      '-minrate', '1000k',
  
      '-maxrate', '2000k',
  
  
  
      '-bufsize', '2000k',/*
  
      '-g', '30',*/
  
      //'-keyint_min', '30',
  
      //'-t', '30',
  
  
  
      ////
  
      '-deadline', 'realtime',
  
      '-cpu-used','-16',
  
  
  
      '-tune', 'zerolatency',
  
  
  
      '-threads', '4',
  
  
  
      //-g 2 -strict experimental -f flv rtmp://a.rtmp.youtube.com/live2/xxxxxxxxxxxxx
  
  
  
  
  
      // FLV is the container format used in conjunction with RTMP
  
      '-f', 'flv',
  
  
  
  
  
  
  
      // The output RTMP URL.
  
      // For debugging, you could set this to a filename like 'test.flv', and play
  
      // the resulting file with VLC.  Please also read the security considerations
  
      // later on in this tutorial.
  
      rtmpUrl
  
  ]);

  Edit : input is fed via stdin whenever the socket receives data.

  In my completely uneducated opinion, the "-deadline realtime" & the "-tune zerolatency" parts seemed to help somewhat but not enough to get me streamin’.

  Here is some of the output of ffmpeg :

  FFmpeg STDERR: frame=   35 fps=3.1 q=23.0 size=     104kB time=00:00:08.44 bitrate= 100.4kbits/s dup=0 drop=3 speed=0.76x    
  
  FFmpeg STDERR: frame=   35 fps=3.0 q=23.0 size=     104kB time=00:00:08.44 bitrate= 100.4kbits/s dup=0 drop=3 speed=0.726x    
  
  FFmpeg STDERR: frame=   35 fps=2.9 q=23.0 size=     104kB time=00:00:08.44 bitrate= 100.4kbits/s dup=0 drop=3 speed=0.696x    
  
  FFmpeg STDERR: frame=   36 fps=2.8 q=23.0 size=     109kB time=00:00:09.45 bitrate=  94.6kbits/s dup=0 drop=3 speed=0.747x    
  
  FFmpeg STDERR: frame=   36 fps=2.7 q=23.0 size=     109kB time=00:00:09.45 bitrate=  94.6kbits/s dup=0 drop=3 speed=0.719x    
  
  FFmpeg STDERR: frame=   38 fps=2.8 q=23.0 size=     121kB time=00:00:11.45 bitrate=  86.7kbits/s dup=0 drop=3 speed=0.839x   

  This is extremely frustrating.

  Please, guys, any guidance is good. what is it that I should be maximizing or minimizing ? how do I do that ? I honestly do not care about everything else going to hell (for example the video freezing or being severely pixelated at times) I just care about being able to stream

  Thank you in advance for anything that might help me !

  UPDATE : I created a new broadcast, with variable bitrate, I broadcasted to it, it was at "status:optimal" for a few seconds then fps and speed went down and the stream went back to "status : incorrect"

• FFmpeg drawtext filter error : "Cannot find a valid font for the family Sans"

  7 septembre 2024, par fatdrogen

  I'm trying to add text overlay to a video using FFmpeg's drawtext filter in my iOS app, but I'm encountering the following error :

  


  error : Cannot load default config file

  


  ERROR : Cannot find a valid font for the family Sans

  


  ERROR : Error initializing filter 'drawtext'

  


  ERROR : with args 'text=write some thing :'

  


  Here's the relevant part of my FFmpeg command :

  


  Copydrawtext=text='write some thing '

  


  Questions :

  


  What's causing this error, and how can I resolve it ?
Is there a way to specify a font that's guaranteed to be available on iOS devices ?
Are there any alternative approaches to adding text overlay in FFmpeg that might avoid this issue ?

  


  Environment :

  


  Platform : iOS
FFmpeg : Using MobileFFmpeg library
Swift

  


  Any help or guidance would be greatly appreciated !

  


  func makeGIFData(asset: AVAsset,  startTime: Double, endTime: Double, rotation: Double, overlayText: String) async throws -> Data {
    let fileName = "\(ProcessInfo.processInfo.globallyUniqueString)_input.mp4"
    let fileURL = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent(fileName)
    
    // Export the asset to a temporary file
    guard let exporter = try? await AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetHighestQuality) else {
        throw NSError(domain: "GIFConversionError", code: -1, userInfo: [NSLocalizedDescriptionKey: "Failed to create AVAssetExportSession"])
    }
    
    exporter.outputURL = fileURL
    exporter.outputFileType = .mp4
    exporter.timeRange = CMTimeRange(start: CMTime(seconds: startTime, preferredTimescale: 600), end: CMTime(seconds: endTime, preferredTimescale: 600))
    
    do {
        try await exporter.export()
    } catch {
        throw NSError(domain: "GIFConversionError", code: -2, userInfo: [NSLocalizedDescriptionKey: "Failed to export video: \(error.localizedDescription)"])
    }
    
    let outfileName = "\(ProcessInfo.processInfo.globallyUniqueString)_outfile.gif"
    let outfileURL = URL(fileURLWithPath: NSTemporaryDirectory()).appendingPathComponent(outfileName)
    
    // Get video dimensions
    guard let track = try? await asset.loadTracks(withMediaType: .video).first,
          let videoSize = try? await track.load(.naturalSize) else {
        throw NSError(domain: "GIFConversionError", code: -4, userInfo: [NSLocalizedDescriptionKey: "Failed to get video dimensions"])
    }
    
    // Prepare FFmpeg command
    var command = "-i \(fileURL.path) -vf "
    
    
    // Add text overlay
    let escapedText = overlayText.replacingOccurrences(of: ":", with: "\\:").replacingOccurrences(of: "'", with: "\\'")
    let fontColor = textColor.toHexString()
    
    // Use a more universally available font, or provide multiple options
    filters += ",drawtext=text='write some thing ':"
    
    // Add palette generation and application
    filters += ",split[s0][s1];[s0]palettegen[p];[s1][p]paletteuse"
    
    // Add filters to command
    command += "\"\(filters)\""
    
    // Finalize command
    command += " -loop 0 \(outfileURL.path)"
    
    print("FFmpeg command: \(command)")
    
    let startTime = CFAbsoluteTimeGetCurrent()
    
    let result = MobileFFmpeg.execute(command)
    
    if result != RETURN_CODE_SUCCESS {
        throw NSError(domain: "GIFConversionError", code: Int(result), userInfo: [NSLocalizedDescriptionKey: "FFmpeg conversion failed"])
    }
    
    let timeElapsed = CFAbsoluteTimeGetCurrent() - startTime
    print("Time elapsed: \(timeElapsed) s.")
    
    // Read the generated GIF file
    guard let gifData = try? Data(contentsOf: outfileURL) else {
        throw NSError(domain: "GIFConversionError", code: -3, userInfo: [NSLocalizedDescriptionKey: "Failed to read generated GIF file"])
    }
    
    print("Animated GIF data created successfully. Size: \(gifData.count) bytes")
    
    // Clean up temporary files
    try? FileManager.default.removeItem(at: fileURL)
    try? FileManager.default.removeItem(at: outfileURL)
    
    return gifData
}
'''