Advanced search

On other websites (2471)

• A Better Process Runner

  1 January 2011, by Multimedia Mike — Python

  I was recently processing a huge corpus of data. It went like this: For each file in a large set, run 'cmdline-tool <file>', capture the output and log results to a database, including whether the tool crashed. I wrote it in Python. I have done this exact type of the thing enough times in Python that I’m starting to notice a pattern.

  Every time I start writing such a program, I always begin with using Python’s commands module because it’s the easiest thing to do. Then I always have to abandon the module when I remember the hard way that whatever ’cmdline-tool’ is, it might run errant and try to execute forever. That’s when I import (rather, copy over) my process runner from FATE, the one that is able to kill a process after it has been running too long. I have used this module enough times that I wonder if I should spin it off into a new Python module.

  Or maybe I’m going about this the wrong way. Perhaps when the data set reaches a certain size, I’m really supposed to throw it on some kind of distributed cluster rather than task it to a Python script (a multithreaded one, to be sure, but one that runs on a single machine). Running the job on a distributed architecture wouldn’t obviate the need for such early termination. But hopefully, such architectures already have that functionality built in. It’s something to research in the new year.

  I guess there are also process limits, enforced by the shell. I don’t think I have ever gotten those to work correctly, though.

• Why does compressed buffer needs to be bigger than input buffer in LZW compression?

  13 April 2016, by Sir DrinksCoffeeALot

  I’m currently working on implementing LZW compression and decompression methods from FFmpeg source code to my project. What i stumbled upon is that the size of output buffer (where compressed data will be stored) needs to be bigger than size of input buffer that we want to compress. Isn’t that contradictionary to the compression itself?

  Next part of the code is located in ff_lzw_encode() function which is part of lzwenc.c source file.

  if (insize * 3 > (s->bufsize - s->output_bytes) * 2)
  
  {
  
      printf("Size of output buffer is too small!\n");
  
      return -1;
  
  }

  For my particular example, i’m trying to compress raw video frames before sending them locally. But if i allocate memory for a buffer that is size of (insize * 3) / 2 (where compressed data will be stored), wouldn’t that take more time to send using send() function than sending raw buffer which is size of insize?

• FFmpeg: Read RTCP lost packets info

  8 June 2018, by Vitor Vanacor

  It seems to be something trivial, but I couldn’t find a way to get the information of the packets lost, usually sent by RTCP in RTP streams.

  I’ve seen this question about reading RTCP packets, but I have two issues with it:

  1. It uses RTSP structures. In my case, the RTP stream is not necessarily RTSP, so I imagine it wouldn’t work.
  2. Even if it was (or worked without RTSP), it uses the rtsp.h header, and as far as I understand this file functions are not exposed (the only exposed headers in /libavformat when compiling FFmpeg are avio.h, libavformat.h and version.h. Maybe I’m wrong here?)

  I also found this function in the rtpdec.h that calculates the packet loss using the RTPStatistics read from RTPDemuxContext. But I don’t understand how I would access this RTPDemuxContext (the function I found that returns this structure is in rtsp.h, so I’d be back to the previously mentioned problem).

  So, any help regarding these doubts that I have or another different way to extract the packet loss information would be appreciated!