Recherche avancée
Autres articles (60)
-
Contribute to documentation
13 avril 2011Documentation is vital to the development of improved technical capabilities.
MediaSPIP welcomes documentation by users as well as developers - including : critique of existing features and functions articles contributed by developers, administrators, content producers and editors screenshots to illustrate the above translations of existing documentation into other languages
To contribute, register to the project users’ mailing (...) -
Submit bugs and patches
13 avril 2011Unfortunately a software is never perfect.
If you think you have found a bug, report it using our ticket system. Please to help us to fix it by providing the following information : the browser you are using, including the exact version as precise an explanation as possible of the problem if possible, the steps taken resulting in the problem a link to the site / page in question
If you think you have solved the bug, fill in a ticket and attach to it a corrective patch.
You may also (...) -
MediaSPIP 0.1 Beta version
25 avril 2011, parMediaSPIP 0.1 beta is the first version of MediaSPIP proclaimed as "usable".
The zip file provided here only contains the sources of MediaSPIP in its standalone version.
To get a working installation, you must manually install all-software dependencies on the server.
If you want to use this archive for an installation in "farm mode", you will also need to proceed to other manual (...)
Sur d’autres sites (4643)
-
ffmpeg takes a while to start
17 octobre 2020, par SuspendedI have this command in python script, in a loop :


ffmpeg -i somefile.mp4 -ss 00:03:12 -t 00:00:35 piece.mp4 -loglevel error -stats


It cuts out pieces of input file (-i). Input filename, as well as start time (-ss) and length of the piece I cut out (-t) varies, so it reads number of mp4 files and cuts out number of pieces from each one. During execution of the script it might be called around 100 times. My problem is that each time before it starts, there is a delay of few seconds and it adds up to significant time. How can I get it to start immediately ?


The script (process_videos.py) :


import subprocess
import sys
import math
import time

class TF:
 """TimeFormatter class (TF).
This class' reason for being is to convert time in short
form, e.g. 1:33, 0:32, or 23 into long form accepted by
mp4cut function in bash, e.g. 00:01:22, 00:00:32, etc"""

def toLong(self, shrt):
 """Converts time to its long form"""
 sx = '00:00:00'
 ladd = 8 - len(shrt)
 n = sx[:ladd] + shrt
 return n

def toShort(self, lng):
 """Converts time to short form"""
 if lng[0] == '0' or lng[0] == ':':
 return self.toShort(lng[1:])
 else:
 return lng

def toSeconds(self, any_time):
 """Converts time to seconds"""
 if len(any_time) < 3:
 return int(any_time)
 tt = any_time.split(':')
 if len(any_time) < 6: 
 return int(tt[0])*60 + int(tt[1])
 return int(tt[0])*3600 + int(tt[1])*60 + int(tt[2])

def toTime(self, secsInt):
 """"""
 tStr = ''
 hrs, mins, secs = 0, 0, 0
 if secsInt >= 3600:
 hrs = math.floor(secsInt / 3600)
 secsInt = secsInt % 3600
 if secsInt >= 60:
 mins = math.floor(secsInt / 60)
 secsInt = secsInt % 60
 secs = secsInt
 return str(hrs).zfill(2) + ':' + str(mins).zfill(2) + ':' + str(secs).zfill(2)

def minus(self, t_start, t_end):
 """"""
 t_e = self.toSeconds(t_end)
 t_s = self.toSeconds(t_start)
 t_r = t_e - t_s
 hrs, mins, secs = 0, 0, 0
 if t_r >= 3600:
 hrs = math.floor(t_r / 3600)
 t_r = t_r - (hrs * 3600)
 if t_r >= 60:
 mins = math.floor(t_r / 60)
 t_r = t_r - (mins * 60)
 secs = t_r
 hrsf = str(hrs).zfill(2)
 minsf = str(mins).zfill(2)
 secsf = str(secs).zfill(2)
 t_fnl = hrsf + ':' + minsf + ':' + secsf
 return t_fnl

def go_main():
 tf = TF()
 vid_n = 0
 arglen = len(sys.argv)
 if arglen == 2:
 with open(sys.argv[1], 'r') as f_in:
 lines = f_in.readlines()
 start = None
 end = None
 cnt = 0
 for line in lines:
 if line[:5] == 'BEGIN':
 start = cnt
 if line[:3] == 'END':
 end = cnt
 cnt += 1
 if start == None or end == None:
 print('Invalid file format. start = {}, end = {}'.format(start,end))
 return
 else:
 lines_r = lines[start+1:end]
 del lines
 print('videos to process: {}'.format(len(lines_r)))
 f_out_prefix = ""
 for vid in lines_r:
 vid_n += 1
 print('\nProcessing video {}/{}'.format(vid_n, len(lines_r)))
 f_out_prefix = 'v' + str(vid_n) + '-'
 dat = vid.split('!')[1:3]
 title = dat[0]
 dat_t = dat[1].split(',')
 v_pieces = len(dat_t)
 piece_n = 0
 video_pieces = []
 cmd1 = "echo -n \"\" > tmpfile"
 subprocess.run(cmd1, shell=True) 
 print(' new tmpfile created')
 for v_times in dat_t:
 piece_n += 1
 f_out = f_out_prefix + str(piece_n) + '.mp4'
 video_pieces.append(f_out)
 print(' piece filename {} added to video_pieces list'.format(f_out))
 v_times_spl = v_times.split('-')
 v_times_start = v_times_spl[0]
 v_times_end = v_times_spl[1]
 t_st = tf.toLong(v_times_start)
 t_dur = tf.toTime(tf.toSeconds(v_times_end) - tf.toSeconds(v_times_start))
 cmd3 = ["ffmpeg", "-i", title, "-ss", t_st, "-t", t_dur, f_out, "-loglevel", "error", "-stats"]
 print(' cutting out piece {}/{} - {}'.format(piece_n, len(dat_t), t_dur))
 subprocess.run(cmd3)
 for video_piece_name in video_pieces:
 cmd4 = "echo \"file " + video_piece_name + "\" >> tmpfile"
 subprocess.run(cmd4, shell=True)
 print(' filename {} added to tmpfile'.format(video_piece_name))
 vname = f_out_prefix[:-1] + ".mp4"
 print(' name of joined file: {}'.format(vname))
 cmd5 = "ffmpeg -f concat -safe 0 -i tmpfile -c copy joined.mp4 -loglevel error -stats"
 to_be_joined = " ".join(video_pieces)
 print(' joining...')
 join_cmd = subprocess.Popen(cmd5, shell=True)
 join_cmd.wait()
 print(' joined!')
 cmd6 = "mv joined.mp4 " + vname
 rename_cmd = subprocess.Popen(cmd6, shell=True)
 rename_cmd.wait()
 print(' File joined.mp4 renamed to {}'.format(vname))
 cmd7 = "rm " + to_be_joined
 rm_cmd = subprocess.Popen(cmd7, shell=True)
 rm_cmd.wait()
 print('rm command completed - pieces removed')
 cmd8 = "rm tmpfile"
 subprocess.run(cmd8, shell=True)
 print('tmpfile removed')
 print('All done')
 else:
 print('Incorrect number of arguments')

############################
if __name__ == '__main__':
 go_main()


process_videos.py is called from bash terminal like this :


$ python process_videos.py video_data 


video_data file has the following format :


BEGIN
!first_video.mp4!3-23,55-1:34,2:01-3:15,3:34-3:44!
!second_video.mp4!2-7,12-44,1:03-1:33!
END


My system details :


System: Host: snowflake Kernel: 5.4.0-52-generic x86_64 bits: 64 Desktop: Gnome 3.28.4
 Distro: Ubuntu 18.04.5 LTS
Machine: Device: desktop System: Gigabyte product: N/A serial: N/A
Mobo: Gigabyte model: Z77-D3H v: x.x serial: N/A BIOS: American Megatrends v: F14 date: 05/31/2012
CPU: Quad core Intel Core i5-3570 (-MCP-) cache: 6144 KB 
 clock speeds: max: 3800 MHz 1: 1601 MHz 2: 1601 MHz 3: 1601 MHz 4: 1602 MHz
Drives: HDD Total Size: 1060.2GB (55.2% used)
 ID-1: /dev/sda model: ST31000524AS size: 1000.2GB
 ID-2: /dev/sdb model: Corsair_Force_GT size: 60.0GB
Partition: ID-1: / size: 366G used: 282G (82%) fs: ext4 dev: /dev/sda1
 ID-2: swap-1 size: 0.70GB used: 0.00GB (0%) fs: swap dev: /dev/sda5
Info: Processes: 313 Uptime: 16:37 Memory: 3421.4/15906.9MB Client: Shell (bash) inxi: 2.3.56


-
Decoding VP8 On A Sega Dreamcast
20 février 2011, par Multimedia Mike — Sega Dreamcast, VP8I got Google’s libvpx VP8 codec library to compile and run on the Sega Dreamcast with its Hitachi/Renesas SH-4 200 MHz CPU. So give Google/On2 their due credit for writing portable software. I’m not sure how best to illustrate this so please accept this still photo depicting my testbench Dreamcast console driving video to my monitor :
Why ? Because I wanted to try my hand at porting some existing software to this console and because I tend to be most comfortable working with assorted multimedia software components. This seemed like it would be a good exercise.
You may have observed that the video is blue. Shortest, simplest answer : Pure laziness. Short, technical answer : Path of least resistance for getting through this exercise. Longer answer follows.
Update : I did eventually realize that the Dreamcast can work with YUV textures. Read more in my followup post.
Process and Pitfalls
libvpx comes with a number of little utilities includingdecode_to_md5.c. The first order of business was porting over enough source files to make the VP8 decoder compile along with the MD5 testbench utility.Again, I used the KallistiOS (KOS) console RTOS (aside : I’m still working to get modern Linux kernels compiled for the Dreamcast). I started by configuring and compiling libvpx on a regular desktop Linux system. From there, I was able to modify a number of configuration options to make the build more amenable to the embedded RTOS.
I had to create a few shim header files that mapped various functions related to threading and synchronization to their KOS equivalents. For example, KOS has a threading library cleverly named kthreads which is mostly compatible with the more common pthread library functions. KOS apparently also predates stdint.h, so I had to contrive a file with those basic types.So I got everything compiled and then uploaded the binary along with a small VP8 IVF test vector. Imagine my surprise when an MD5 sum came out of the serial console. Further, visualize my utter speechlessness when I noticed that the MD5 sum matched what my desktop platform produced. It worked !
Almost. When I tried to decode all frames in a test vector, the program would invariably crash. The problem was that the file that manages motion compensation (reconinter.c) needs to define MUST_BE_ALIGNED which compiles byte-wise block copy functions. This is necessary for CPUs like the SH-4 which can’t load unaligned data. Apparently, even ARM CPUs these days can handle unaligned memory accesses which is why this isn’t a configure-time option.
Showing The Work
I completed the first testbench application which ran the MD5 test on all 17 official IVF test vectors. The SH-4/Dreamcast version aces the whole suite.However, this is a video game console, so I had better be able to show the decoded video. The Dreamcast is strictly RGB— forget about displaying YUV data directly. I could take the performance hit to convert YUV -> RGB. Or, I could just display the intensity information (Y plane) rendered on a random color scale (I chose blue) on an RGB565 texture (the DC’s graphics hardware can also do paletted textures but those need to be rearranged/twiddled/swizzled).
Results
So, can the Dreamcast decode VP8 video in realtime ? Sure ! Well, I really need to qualify. In the test depicted in the picture, it seems to be realtime (though I wasn’t enforcing proper frame timings, just decoding and displaying as quickly as possible). Obviously, I wasn’t bothering to properly convert YUV -> RGB. Plus, that Big Buck Bunny test vector clip is only 176x144. Obviously, no audio decoding either.So, realtime playback, with a little fine print.
On the plus side, it’s trivial to get the Dreamcast video hardware to upscale that little blue image to fullscreen.
I was able to tally the total milliseconds’ worth of wall clock time required to decode the 17 VP8 test vectors. As you can probably work out from this list, when I try to play a 320x240 video, things start to break down.
- Processed 29 176x144 frames in 987 milliseconds.
- Processed 49 176x144 frames in 1809 milliseconds.
- Processed 49 176x144 frames in 704 milliseconds.
- Processed 29 176x144 frames in 255 milliseconds.
- Processed 49 176x144 frames in 339 milliseconds.
- Processed 48 175x143 frames in 2446 milliseconds.
- Processed 29 176x144 frames in 432 milliseconds.
- Processed 2 1432x888 frames in 2060 milliseconds.
- Processed 49 176x144 frames in 1884 milliseconds.
- Processed 57 320x240 frames in 5792 milliseconds.
- Processed 29 176x144 frames in 989 milliseconds.
- Processed 29 176x144 frames in 740 milliseconds.
- Processed 29 176x144 frames in 839 milliseconds.
- Processed 49 175x143 frames in 2849 milliseconds.
- Processed 260 320x240 frames in 29719 milliseconds.
- Processed 29 176x144 frames in 962 milliseconds.
- Processed 29 176x144 frames in 933 milliseconds.
-
ffmpeg takes too long to start
17 octobre 2020, par SuspendedI have this command in python script, in a loop :


ffmpeg -i somefile.mp4 -ss 00:03:12 -t 00:00:35 piece.mp4 -loglevel error -stats


It cuts out pieces of input file (-i). Input filename, as well as start time (-ss) and length of the piece I cut out (-t) varies, so it reads number of mp4 files and cuts out number of pieces from each one. During execution of the script it might be called around 100 times. My problem is that each time before it starts, there is a delay of 6-15 seconds and it adds up to significant time. How can I get it to start immediately ?


Initially I thought it was process priority problem, but I noticed that even during the "pause", all processors work at 100%, so apparently some work is being done.


The script (process_videos.py) :


import subprocess
import sys
import math
import time

class TF:
 """TimeFormatter class (TF).
This class' reason for being is to convert time in short
form, e.g. 1:33, 0:32, or 23 into long form accepted by
mp4cut function in bash, e.g. 00:01:22, 00:00:32, etc"""

def toLong(self, shrt):
 """Converts time to its long form"""
 sx = '00:00:00'
 ladd = 8 - len(shrt)
 n = sx[:ladd] + shrt
 return n

def toShort(self, lng):
 """Converts time to short form"""
 if lng[0] == '0' or lng[0] == ':':
 return self.toShort(lng[1:])
 else:
 return lng

def toSeconds(self, any_time):
 """Converts time to seconds"""
 if len(any_time) < 3:
 return int(any_time)
 tt = any_time.split(':')
 if len(any_time) < 6: 
 return int(tt[0])*60 + int(tt[1])
 return int(tt[0])*3600 + int(tt[1])*60 + int(tt[2])

def toTime(self, secsInt):
 """"""
 tStr = ''
 hrs, mins, secs = 0, 0, 0
 if secsInt >= 3600:
 hrs = math.floor(secsInt / 3600)
 secsInt = secsInt % 3600
 if secsInt >= 60:
 mins = math.floor(secsInt / 60)
 secsInt = secsInt % 60
 secs = secsInt
 return str(hrs).zfill(2) + ':' + str(mins).zfill(2) + ':' + str(secs).zfill(2)

def minus(self, t_start, t_end):
 """"""
 t_e = self.toSeconds(t_end)
 t_s = self.toSeconds(t_start)
 t_r = t_e - t_s
 hrs, mins, secs = 0, 0, 0
 if t_r >= 3600:
 hrs = math.floor(t_r / 3600)
 t_r = t_r - (hrs * 3600)
 if t_r >= 60:
 mins = math.floor(t_r / 60)
 t_r = t_r - (mins * 60)
 secs = t_r
 hrsf = str(hrs).zfill(2)
 minsf = str(mins).zfill(2)
 secsf = str(secs).zfill(2)
 t_fnl = hrsf + ':' + minsf + ':' + secsf
 return t_fnl

def go_main():
 tf = TF()
 vid_n = 0
 arglen = len(sys.argv)
 if arglen == 2:
 with open(sys.argv[1], 'r') as f_in:
 lines = f_in.readlines()
 start = None
 end = None
 cnt = 0
 for line in lines:
 if line[:5] == 'BEGIN':
 start = cnt
 if line[:3] == 'END':
 end = cnt
 cnt += 1
 if start == None or end == None:
 print('Invalid file format. start = {}, end = {}'.format(start,end))
 return
 else:
 lines_r = lines[start+1:end]
 del lines
 print('videos to process: {}'.format(len(lines_r)))
 f_out_prefix = ""
 for vid in lines_r:
 vid_n += 1
 print('\nProcessing video {}/{}'.format(vid_n, len(lines_r)))
 f_out_prefix = 'v' + str(vid_n) + '-'
 dat = vid.split('!')[1:3]
 title = dat[0]
 dat_t = dat[1].split(',')
 v_pieces = len(dat_t)
 piece_n = 0
 video_pieces = []
 cmd1 = "echo -n \"\" > tmpfile"
 subprocess.run(cmd1, shell=True) 
 print(' new tmpfile created')
 for v_times in dat_t:
 piece_n += 1
 f_out = f_out_prefix + str(piece_n) + '.mp4'
 video_pieces.append(f_out)
 print(' piece filename {} added to video_pieces list'.format(f_out))
 v_times_spl = v_times.split('-')
 v_times_start = v_times_spl[0]
 v_times_end = v_times_spl[1]
 t_st = tf.toLong(v_times_start)
 t_dur = tf.toTime(tf.toSeconds(v_times_end) - tf.toSeconds(v_times_start))
 cmd3 = ["ffmpeg", "-i", title, "-ss", t_st, "-t", t_dur, f_out, "-loglevel", "error", "-stats"]
 print(' cutting out piece {}/{} - {}'.format(piece_n, len(dat_t), t_dur))
 subprocess.run(cmd3)
 for video_piece_name in video_pieces:
 cmd4 = "echo \"file " + video_piece_name + "\" >> tmpfile"
 subprocess.run(cmd4, shell=True)
 print(' filename {} added to tmpfile'.format(video_piece_name))
 vname = f_out_prefix[:-1] + ".mp4"
 print(' name of joined file: {}'.format(vname))
 cmd5 = "ffmpeg -f concat -safe 0 -i tmpfile -c copy joined.mp4 -loglevel error -stats"
 to_be_joined = " ".join(video_pieces)
 print(' joining...')
 join_cmd = subprocess.Popen(cmd5, shell=True)
 join_cmd.wait()
 print(' joined!')
 cmd6 = "mv joined.mp4 " + vname
 rename_cmd = subprocess.Popen(cmd6, shell=True)
 rename_cmd.wait()
 print(' File joined.mp4 renamed to {}'.format(vname))
 cmd7 = "rm " + to_be_joined
 rm_cmd = subprocess.Popen(cmd7, shell=True)
 rm_cmd.wait()
 print('rm command completed - pieces removed')
 cmd8 = "rm tmpfile"
 subprocess.run(cmd8, shell=True)
 print('tmpfile removed')
 print('All done')
 else:
 print('Incorrect number of arguments')

############################
if __name__ == '__main__':
 go_main()


process_videos.py is called from bash terminal like this :


$ python process_videos.py video_data 


video_data file has the following format :


BEGIN
!first_video.mp4!3-23,55-1:34,2:01-3:15,3:34-3:44!
!second_video.mp4!2-7,12-44,1:03-1:33!
END


My system details :


System: Host: snowflake Kernel: 5.4.0-52-generic x86_64 bits: 64 Desktop: Gnome 3.28.4
 Distro: Ubuntu 18.04.5 LTS
Machine: Device: desktop System: Gigabyte product: N/A serial: N/A
Mobo: Gigabyte model: Z77-D3H v: x.x serial: N/A BIOS: American Megatrends v: F14 date: 05/31/2012
CPU: Quad core Intel Core i5-3570 (-MCP-) cache: 6144 KB 
 clock speeds: max: 3800 MHz 1: 1601 MHz 2: 1601 MHz 3: 1601 MHz 4: 1602 MHz
Drives: HDD Total Size: 1060.2GB (55.2% used)
 ID-1: /dev/sda model: ST31000524AS size: 1000.2GB
 ID-2: /dev/sdb model: Corsair_Force_GT size: 60.0GB
Partition: ID-1: / size: 366G used: 282G (82%) fs: ext4 dev: /dev/sda1
 ID-2: swap-1 size: 0.70GB used: 0.00GB (0%) fs: swap dev: /dev/sda5
Info: Processes: 313 Uptime: 16:37 Memory: 3421.4/15906.9MB Client: Shell (bash) inxi: 2.3.56



UPDATE :


Following Charles' advice, I used performance sampling :


# perf record -a -g sleep 180


...and here's the report :


Samples: 74K of event 'cycles', Event count (approx.): 1043554519767
 Children Self Command Shared Object
- 50.56% 45.86% ffmpeg libavcodec.so.57.107.100 
 - 3.10% 0x4489480000002825 
 0.64% 0x7ffaf24b92f0 
 - 2.12% 0x5f7369007265646f 
 av_default_item_name 
 1.39% 0 
- 44.48% 40.59% ffmpeg libx264.so.152 
 5.78% x264_add8x8_idct_avx2.skip_prologue 
 3.13% x264_add8x8_idct_avx2.skip_prologue 
 2.91% x264_add8x8_idct_avx2.skip_prologue 
 2.31% x264_add8x8_idct_avx.skip_prologue 
 2.03% 0 
 1.78% 0x1 
 1.26% x264_add8x8_idct_avx2.skip_prologue 
 1.09% x264_add8x8_idct_avx.skip_prologue 
 1.06% x264_me_search_ref 
 0.97% x264_add8x8_idct_avx.skip_prologue 
 0.60% x264_me_search_ref 
- 38.01% 0.00% ffmpeg [unknown] 
 4.10% 0 
 - 3.49% 0x4489480000002825 
 0.70% 0x7ffaf24b92f0 
 0.56% 0x7f273ae822f0 
 0.50% 0x7f0c4768b2f0 
 - 2.29% 0x5f7369007265646f 
 av_default_item_name 
 1.99% 0x1 
 10.13% 10.12% ffmpeg [kernel.kallsyms] 
- 3.14% 0.73% ffmpeg libavutil.so.55.78.100 
 2.34% av_default_item_name 
- 1.73% 0.21% ffmpeg libpthread-2.27.so 
 - 0.70% pthread_cond_wait@@GLIBC_2.3.2 
 - 0.62% entry_SYSCALL_64_after_hwframe 
 - 0.62% do_syscall_64 
 - 0.57% __x64_sys_futex 
 0.52% do_futex 
 0.93% 0.89% ffmpeg libc-2.27.so 
- 0.64% 0.64% swapper [kernel.kallsyms] 
 0.63% secondary_startup_64 
 0.21% 0.18% ffmpeg libavfilter.so.6.107.100 
 0.20% 0.11% ffmpeg libavformat.so.57.83.100 
 0.12% 0.11% ffmpeg ffmpeg 
 0.11% 0.00% gnome-terminal- [unknown] 
 0.09% 0.07% ffmpeg libm-2.27.so 
 0.08% 0.07% ffmpeg ld-2.27.so 
 0.04% 0.04% gnome-terminal- libglib-2.0.so.0.5600.4




Syndiquer tout le site
