Recherche avancée

Médias (1)

Mot : - Tags -/musée

Autres articles (40)

  • Les formats acceptés

    28 janvier 2010, par

    Les commandes suivantes permettent d’avoir des informations sur les formats et codecs gérés par l’installation local de ffmpeg :
    ffmpeg -codecs ffmpeg -formats
    Les format videos acceptés en entrée
    Cette liste est non exhaustive, elle met en exergue les principaux formats utilisés : h264 : H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 m4v : raw MPEG-4 video format flv : Flash Video (FLV) / Sorenson Spark / Sorenson H.263 Theora wmv :
    Les formats vidéos de sortie possibles
    Dans un premier temps on (...)

  • List of compatible distributions

    26 avril 2011, par

    The table below is the list of Linux distributions compatible with the automated installation script of MediaSPIP. Distribution nameVersion nameVersion number Debian Squeeze 6.x.x Debian Weezy 7.x.x Debian Jessie 8.x.x Ubuntu The Precise Pangolin 12.04 LTS Ubuntu The Trusty Tahr 14.04
    If you want to help us improve this list, you can provide us access to a machine whose distribution is not mentioned above or send the necessary fixes to add (...)

  • Selection of projects using MediaSPIP

    2 mai 2011, par

    The examples below are representative elements of MediaSPIP specific uses for specific projects.
    MediaSPIP farm @ Infini
    The non profit organizationInfini develops hospitality activities, internet access point, training, realizing innovative projects in the field of information and communication technologies and Communication, and hosting of websites. It plays a unique and prominent role in the Brest (France) area, at the national level, among the half-dozen such association. Its members (...)

Sur d’autres sites (7368)

  • avconv with subprocess

    17 juillet 2012, par Niclas Nilsson

    It's seems like I never really manage to wrap my head around subprocess. This commandline works in bash

    avconv -i "concat:/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment0.ts|/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment1.ts" -vcodec copy -acodec copy /tmp/test1.ts

    Bu if I try this :

    cmdline = ['avconv', '-i "concat:/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment0.ts|/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment1.ts"', '-vcodec copy', '-acodec copy', '/tmp/test1.ts']
    
subprocess.call(cmdline)

    It exit avconv with the following error :

    avconv version 0.8.3-4:0.8.3-0ubuntu0.12.04.1, Copyright (c) 2000-2012 the Libav developers
    
  built on Jun 12 2012 16:52:09 with gcc 4.6.3
    
Unrecognized option 'i "concat:/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment0.ts|/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment1.ts"'
    
Failed to set value '-vcodec copy' for option 'i "concat:/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment0.ts|/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment1.ts"'
    
1

    I've tried some variants (including shell=True) but I can't figure out the problem.

    Update

    Cjb made an answer that really worked. But my real code is more complicated. I believe it's hard to get something out of it. But I throw it in here just in case there is an obvoius problem that I missed.

    def run_avconv(output_dir, fnames):
    
    full_fnames = [os.path.join(output_dir, fname.replace('\n', ''))
    
                   for fname in fnames]
    
    concatted_files = '|'.join(full_fnames)
    
    cmd_line = [AVCONV_CMD,
    
                '-i',
    
                '"concat:' + concatted_files + '"',
    
                '-vcodec',
    
                'copy',
    
                '-acodec',
    
                'copy',
    
                os.path.join(output_dir, 'out.ts')]
    
    subprocess.Popen(cmd_line)

    the full_fnames variable is :

    ['/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment0.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment1.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment2.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment3.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment4.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment5.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment6.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment7.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment8.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment9.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment10.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment11.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment12.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment13.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment14.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment15.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment16.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment17.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment18.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment19.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment20.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment21.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment22.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment23.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment24.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment25.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment26.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment27.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment28.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment29.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment30.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment31.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment32.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment33.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment34.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment35.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment36.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment37.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment38.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment39.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment40.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment41.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment42.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment43.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment44.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment45.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment46.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment47.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment48.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment49.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment50.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment51.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment52.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment53.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment54.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment55.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment56.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment57.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment58.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment59.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment60.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment61.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment62.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment63.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment64.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment65.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment66.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment67.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment68.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment69.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment70.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment71.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment72.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment73.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment74.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment75.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment76.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment77.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment78.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment79.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment80.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment81.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment82.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment83.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment84.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment85.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment86.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment87.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment88.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment89.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment90.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment91.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment92.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment93.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment94.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment95.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment96.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment97.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment98.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment99.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment100.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment101.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment102.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment103.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment104.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment105.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment106.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment107.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment108.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment109.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment110.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment111.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment112.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment113.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment114.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment115.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment116.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment117.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment118.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment119.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment120.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment121.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment122.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment123.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment124.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment125.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment126.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment127.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment128.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment129.ts',
    
 '/tmp/BABAR_AND_THE_A-011A-hts-a-v1-cc3651d01841d748_Layer6/6148_Period1/segment130.ts']

  • Emulate an IP Camera

    23 décembre 2013, par Goro

    I am using software (DVR) that is meant to talk directly to an IP camera. I am trying to pass a h264 stream directly into it, but it does not work if I just throw a stream onto it.

    Can you recommend any "spoofing" software to emulate an IP camera ? For the sake of argument, we can say that the camera we want to emulate is Axis P3301.

    Given that a lot of software out there can talk to IP cameras, is there a way to use something like vlc/ffmpeg to look like its an IP camera ?

    Thanks.

  • A Better Process Runner

    1er janvier 2011, par Multimedia Mike — Python

    I was recently processing a huge corpus of data. It went like this : For each file in a large set, run 'cmdline-tool <file>', capture the output and log results to a database, including whether the tool crashed. I wrote it in Python. I have done this exact type of the thing enough times in Python that I’m starting to notice a pattern.

    Every time I start writing such a program, I always begin with using Python’s commands module because it’s the easiest thing to do. Then I always have to abandon the module when I remember the hard way that whatever ’cmdline-tool’ is, it might run errant and try to execute forever. That’s when I import (rather, copy over) my process runner from FATE, the one that is able to kill a process after it has been running too long. I have used this module enough times that I wonder if I should spin it off into a new Python module.

    Or maybe I’m going about this the wrong way. Perhaps when the data set reaches a certain size, I’m really supposed to throw it on some kind of distributed cluster rather than task it to a Python script (a multithreaded one, to be sure, but one that runs on a single machine). Running the job on a distributed architecture wouldn’t obviate the need for such early termination. But hopefully, such architectures already have that functionality built in. It’s something to research in the new year.

    I guess there are also process limits, enforced by the shell. I don’t think I have ever gotten those to work correctly, though.

Navigation