Recherche avancée
Médias (1)
-
The Great Big Beautiful Tomorrow
28 octobre 2011, par
Mis à jour : Octobre 2011
Langue : English
Type : Texte
Autres articles (33)
-
Keeping control of your media in your hands
13 avril 2011, parThe vocabulary used on this site and around MediaSPIP in general, aims to avoid reference to Web 2.0 and the companies that profit from media-sharing.
While using MediaSPIP, you are invited to avoid using words like "Brand", "Cloud" and "Market".
MediaSPIP is designed to facilitate the sharing of creative media online, while allowing authors to retain complete control of their work.
MediaSPIP aims to be accessible to as many people as possible and development is based on expanding the (...) -
Création définitive du canal
12 mars 2010, parLorsque votre demande est validée, vous pouvez alors procéder à la création proprement dite du canal. Chaque canal est un site à part entière placé sous votre responsabilité. Les administrateurs de la plateforme n’y ont aucun accès.
A la validation, vous recevez un email vous invitant donc à créer votre canal.
Pour ce faire il vous suffit de vous rendre à son adresse, dans notre exemple "http://votre_sous_domaine.mediaspip.net".
A ce moment là un mot de passe vous est demandé, il vous suffit d’y (...) -
Emballe médias : à quoi cela sert ?
4 février 2011, parCe plugin vise à gérer des sites de mise en ligne de documents de tous types.
Il crée des "médias", à savoir : un "média" est un article au sens SPIP créé automatiquement lors du téléversement d’un document qu’il soit audio, vidéo, image ou textuel ; un seul document ne peut être lié à un article dit "média" ;
Sur d’autres sites (4651)
-
ffmpeg takes too long to start
17 octobre 2020, par SuspendedI have this command in python script, in a loop :


ffmpeg -i somefile.mp4 -ss 00:03:12 -t 00:00:35 piece.mp4 -loglevel error -stats


It cuts out pieces of input file (-i). Input filename, as well as start time (-ss) and length of the piece I cut out (-t) varies, so it reads number of mp4 files and cuts out number of pieces from each one. During execution of the script it might be called around 100 times. My problem is that each time before it starts, there is a delay of 6-15 seconds and it adds up to significant time. How can I get it to start immediately ?


Initially I thought it was process priority problem, but I noticed that even during the "pause", all processors work at 100%, so apparently some work is being done.


The script (process_videos.py) :


import subprocess
import sys
import math
import time

class TF:
 """TimeFormatter class (TF).
This class' reason for being is to convert time in short
form, e.g. 1:33, 0:32, or 23 into long form accepted by
mp4cut function in bash, e.g. 00:01:22, 00:00:32, etc"""

def toLong(self, shrt):
 """Converts time to its long form"""
 sx = '00:00:00'
 ladd = 8 - len(shrt)
 n = sx[:ladd] + shrt
 return n

def toShort(self, lng):
 """Converts time to short form"""
 if lng[0] == '0' or lng[0] == ':':
 return self.toShort(lng[1:])
 else:
 return lng

def toSeconds(self, any_time):
 """Converts time to seconds"""
 if len(any_time) < 3:
 return int(any_time)
 tt = any_time.split(':')
 if len(any_time) < 6: 
 return int(tt[0])*60 + int(tt[1])
 return int(tt[0])*3600 + int(tt[1])*60 + int(tt[2])

def toTime(self, secsInt):
 """"""
 tStr = ''
 hrs, mins, secs = 0, 0, 0
 if secsInt >= 3600:
 hrs = math.floor(secsInt / 3600)
 secsInt = secsInt % 3600
 if secsInt >= 60:
 mins = math.floor(secsInt / 60)
 secsInt = secsInt % 60
 secs = secsInt
 return str(hrs).zfill(2) + ':' + str(mins).zfill(2) + ':' + str(secs).zfill(2)

def minus(self, t_start, t_end):
 """"""
 t_e = self.toSeconds(t_end)
 t_s = self.toSeconds(t_start)
 t_r = t_e - t_s
 hrs, mins, secs = 0, 0, 0
 if t_r >= 3600:
 hrs = math.floor(t_r / 3600)
 t_r = t_r - (hrs * 3600)
 if t_r >= 60:
 mins = math.floor(t_r / 60)
 t_r = t_r - (mins * 60)
 secs = t_r
 hrsf = str(hrs).zfill(2)
 minsf = str(mins).zfill(2)
 secsf = str(secs).zfill(2)
 t_fnl = hrsf + ':' + minsf + ':' + secsf
 return t_fnl

def go_main():
 tf = TF()
 vid_n = 0
 arglen = len(sys.argv)
 if arglen == 2:
 with open(sys.argv[1], 'r') as f_in:
 lines = f_in.readlines()
 start = None
 end = None
 cnt = 0
 for line in lines:
 if line[:5] == 'BEGIN':
 start = cnt
 if line[:3] == 'END':
 end = cnt
 cnt += 1
 if start == None or end == None:
 print('Invalid file format. start = {}, end = {}'.format(start,end))
 return
 else:
 lines_r = lines[start+1:end]
 del lines
 print('videos to process: {}'.format(len(lines_r)))
 f_out_prefix = ""
 for vid in lines_r:
 vid_n += 1
 print('\nProcessing video {}/{}'.format(vid_n, len(lines_r)))
 f_out_prefix = 'v' + str(vid_n) + '-'
 dat = vid.split('!')[1:3]
 title = dat[0]
 dat_t = dat[1].split(',')
 v_pieces = len(dat_t)
 piece_n = 0
 video_pieces = []
 cmd1 = "echo -n \"\" > tmpfile"
 subprocess.run(cmd1, shell=True) 
 print(' new tmpfile created')
 for v_times in dat_t:
 piece_n += 1
 f_out = f_out_prefix + str(piece_n) + '.mp4'
 video_pieces.append(f_out)
 print(' piece filename {} added to video_pieces list'.format(f_out))
 v_times_spl = v_times.split('-')
 v_times_start = v_times_spl[0]
 v_times_end = v_times_spl[1]
 t_st = tf.toLong(v_times_start)
 t_dur = tf.toTime(tf.toSeconds(v_times_end) - tf.toSeconds(v_times_start))
 cmd3 = ["ffmpeg", "-i", title, "-ss", t_st, "-t", t_dur, f_out, "-loglevel", "error", "-stats"]
 print(' cutting out piece {}/{} - {}'.format(piece_n, len(dat_t), t_dur))
 subprocess.run(cmd3)
 for video_piece_name in video_pieces:
 cmd4 = "echo \"file " + video_piece_name + "\" >> tmpfile"
 subprocess.run(cmd4, shell=True)
 print(' filename {} added to tmpfile'.format(video_piece_name))
 vname = f_out_prefix[:-1] + ".mp4"
 print(' name of joined file: {}'.format(vname))
 cmd5 = "ffmpeg -f concat -safe 0 -i tmpfile -c copy joined.mp4 -loglevel error -stats"
 to_be_joined = " ".join(video_pieces)
 print(' joining...')
 join_cmd = subprocess.Popen(cmd5, shell=True)
 join_cmd.wait()
 print(' joined!')
 cmd6 = "mv joined.mp4 " + vname
 rename_cmd = subprocess.Popen(cmd6, shell=True)
 rename_cmd.wait()
 print(' File joined.mp4 renamed to {}'.format(vname))
 cmd7 = "rm " + to_be_joined
 rm_cmd = subprocess.Popen(cmd7, shell=True)
 rm_cmd.wait()
 print('rm command completed - pieces removed')
 cmd8 = "rm tmpfile"
 subprocess.run(cmd8, shell=True)
 print('tmpfile removed')
 print('All done')
 else:
 print('Incorrect number of arguments')

############################
if __name__ == '__main__':
 go_main()


process_videos.py is called from bash terminal like this :


$ python process_videos.py video_data 


video_data file has the following format :


BEGIN
!first_video.mp4!3-23,55-1:34,2:01-3:15,3:34-3:44!
!second_video.mp4!2-7,12-44,1:03-1:33!
END


My system details :


System: Host: snowflake Kernel: 5.4.0-52-generic x86_64 bits: 64 Desktop: Gnome 3.28.4
 Distro: Ubuntu 18.04.5 LTS
Machine: Device: desktop System: Gigabyte product: N/A serial: N/A
Mobo: Gigabyte model: Z77-D3H v: x.x serial: N/A BIOS: American Megatrends v: F14 date: 05/31/2012
CPU: Quad core Intel Core i5-3570 (-MCP-) cache: 6144 KB 
 clock speeds: max: 3800 MHz 1: 1601 MHz 2: 1601 MHz 3: 1601 MHz 4: 1602 MHz
Drives: HDD Total Size: 1060.2GB (55.2% used)
 ID-1: /dev/sda model: ST31000524AS size: 1000.2GB
 ID-2: /dev/sdb model: Corsair_Force_GT size: 60.0GB
Partition: ID-1: / size: 366G used: 282G (82%) fs: ext4 dev: /dev/sda1
 ID-2: swap-1 size: 0.70GB used: 0.00GB (0%) fs: swap dev: /dev/sda5
Info: Processes: 313 Uptime: 16:37 Memory: 3421.4/15906.9MB Client: Shell (bash) inxi: 2.3.56



UPDATE :


Following Charles' advice, I used performance sampling :


# perf record -a -g sleep 180


...and here's the report :


Samples: 74K of event 'cycles', Event count (approx.): 1043554519767
 Children Self Command Shared Object
- 50.56% 45.86% ffmpeg libavcodec.so.57.107.100 
 - 3.10% 0x4489480000002825 
 0.64% 0x7ffaf24b92f0 
 - 2.12% 0x5f7369007265646f 
 av_default_item_name 
 1.39% 0 
- 44.48% 40.59% ffmpeg libx264.so.152 
 5.78% x264_add8x8_idct_avx2.skip_prologue 
 3.13% x264_add8x8_idct_avx2.skip_prologue 
 2.91% x264_add8x8_idct_avx2.skip_prologue 
 2.31% x264_add8x8_idct_avx.skip_prologue 
 2.03% 0 
 1.78% 0x1 
 1.26% x264_add8x8_idct_avx2.skip_prologue 
 1.09% x264_add8x8_idct_avx.skip_prologue 
 1.06% x264_me_search_ref 
 0.97% x264_add8x8_idct_avx.skip_prologue 
 0.60% x264_me_search_ref 
- 38.01% 0.00% ffmpeg [unknown] 
 4.10% 0 
 - 3.49% 0x4489480000002825 
 0.70% 0x7ffaf24b92f0 
 0.56% 0x7f273ae822f0 
 0.50% 0x7f0c4768b2f0 
 - 2.29% 0x5f7369007265646f 
 av_default_item_name 
 1.99% 0x1 
 10.13% 10.12% ffmpeg [kernel.kallsyms] 
- 3.14% 0.73% ffmpeg libavutil.so.55.78.100 
 2.34% av_default_item_name 
- 1.73% 0.21% ffmpeg libpthread-2.27.so 
 - 0.70% pthread_cond_wait@@GLIBC_2.3.2 
 - 0.62% entry_SYSCALL_64_after_hwframe 
 - 0.62% do_syscall_64 
 - 0.57% __x64_sys_futex 
 0.52% do_futex 
 0.93% 0.89% ffmpeg libc-2.27.so 
- 0.64% 0.64% swapper [kernel.kallsyms] 
 0.63% secondary_startup_64 
 0.21% 0.18% ffmpeg libavfilter.so.6.107.100 
 0.20% 0.11% ffmpeg libavformat.so.57.83.100 
 0.12% 0.11% ffmpeg ffmpeg 
 0.11% 0.00% gnome-terminal- [unknown] 
 0.09% 0.07% ffmpeg libm-2.27.so 
 0.08% 0.07% ffmpeg ld-2.27.so 
 0.04% 0.04% gnome-terminal- libglib-2.0.so.0.5600.4




-
How to set crontab in order to run multiple python and a shell scripts ?
5 janvier 2021, par Alexander MitsouI need to start three python3 scripts and a shell script using crontab. These scripts should run at the same time without any delay. Each script runs exactly for one minute. For instance I have scheduled crontab to run these scripts every 5 minutes.


My problem is that, if I attempt to run each script individually from terminal it executes with no further errors, but using crontab nothing happens.


DISCLAIMER : If I set up the Python3 scripts individually in crontab, they work fine !


Here's my crontab set up :


*/5 * * * * cd /home/user/Desktop/ && /usr/bin/python3 script1.py >> report1.log

*/5 * * * * cd /home/user/Desktop/ && /usr/bin/python3 script2.py >> report2.log

*/5 * * * * cd /home/user/Desktop/ && /usr/bin/python3 script3.py >> report3.log

*/5 * * * * cd /home/user/Desktop/ && /usr/bin/sh script4.sh >> report4.log 


In addition I need to mention that the shell script contains this command (FFMPEG) :


#!/bin/bash

parent_dir=`dirname \`pwd\`` 
folder_name="/Data/Webcam" 
new_path=$parent_dir$folder_name 


if [ -d "$new_path" ]; then
 echo "video_audio folder exists..."
else
 echo "Creating video_audio folder in the current directory..."
 mkdir -p -- "$new_path"
 sudo chmod 777 "$new_path"
 echo "Folder created"
 echo
fi

now=$(date +%F) 
now="$( echo -e "$now" | tr '-' '_' )"
sub_dir=$new_path'/'$now 

if [ -d "$sub_dir" ]; then
 echo "Date Sub-directory exists..."
 echo
else
 echo "Error: ${sub_dir} not found..."
 echo "Creating date sub-directory..."
 mkdir -p -- "$sub_dir"
 sudo chmod 777 "$sub_dir"
 echo "Date sub-directory created..."
 echo
fi

fname=$(date +%H_%M_%S)".avi"
video_dir=$sub_dir'/'$fname
ffmpeg -f pulse -ac 1 -i default -f v4l2 -i /dev/video0 -vcodec libx264 -t 00:01:00 $video_dir 


The log file of that script contain the following :


video_audio folder exists...
Date Sub-directory exists...

Package ffmpeg is already installed...
Package v4l-utils is already installed...

Package: ffmpeg
Status: install ok installed
Priority: optional
Section: video
Installed-Size: 2010
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: amd64
Multi-Arch: foreign
Version: 7:4.2.4-1ubuntu0.1
Replaces: libav-tools (<< 6:12~~), qt-faststart (<< 7:2.7.1-3~)
Depends: libavcodec58 (= 7:4.2.4-1ubuntu0.1), libavdevice58 (= 7:4.2.4-1ubuntu0.1), libavfilter7 (= 7:4.2.4-1ubuntu0.1), libavformat58 (= 7:4.2.4-1ubuntu0.1), libavresample4 (= 7:4.2.4-1ubuntu0.1), libavutil56 (= 7:4.2.4-1ubuntu0.1), libc6 (>= 2.29), libpostproc55 (= 7:4.2.4-1ubuntu0.1), libsdl2-2.0-0 (>= 2.0.10), libswresample3 (= 7:4.2.4-1ubuntu0.1), libswscale5 (= 7:4.2.4-1ubuntu0.1)
Suggests: ffmpeg-doc
Breaks: libav-tools (<< 6:12~~), qt-faststart (<< 7:2.7.1-3~), winff (<< 1.5.5-5~)
Description: Tools for transcoding, streaming and playing of multimedia files
 FFmpeg is the leading multimedia framework, able to decode, encode, transcode,
 mux, demux, stream, filter and play pretty much anything that humans and
 machines have created. It supports the most obscure ancient formats up to the
 cutting edge.
 .
 This package contains:
 * ffmpeg: a command line tool to convert multimedia files between formats
 * ffplay: a simple media player based on SDL and the FFmpeg libraries
 * ffprobe: a simple multimedia stream analyzer
 * qt-faststart: a utility to rearrange Quicktime files
Homepage: https://ffmpeg.org/
Original-Maintainer: Debian Multimedia Maintainers <debian-multimedia@lists.debian.org>
Package: v4l-utils
Status: install ok installed
Priority: optional
Section: utils
Installed-Size: 2104
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: amd64
Version: 1.18.0-2build1
Replaces: ivtv-utils (<< 1.4.1-2), media-ctl
Depends: libv4l-0 (= 1.18.0-2build1), libv4l2rds0 (= 1.18.0-2build1), libc6 (>= 2.17), libgcc-s1 (>= 3.0), libstdc++6 (>= 5.2), libudev1 (>= 183)
Breaks: ivtv-utils (<< 1.4.1-2), media-ctl
Description: Collection of command line video4linux utilities
 v4l-utils contains the following video4linux command line utilities:
 .
 decode_tm6000: decodes tm6000 proprietary format streams
 rds-ctl: tool to receive and decode Radio Data System (RDS) streams
 v4l2-compliance: tool to test v4l2 API compliance of drivers
 v4l2-ctl, cx18-ctl, ivtv-ctl: tools to control v4l2 controls from the cmdline
 v4l2-dbg: tool to directly get and set registers of v4l2 devices
 v4l2-sysfs-path: sysfs helper tool
Original-Maintainer: Gregor Jasny <gjasny@googlemail.com>
Homepage: https://linuxtv.org/downloads/v4l-utils/


Due to the reason that the python files are of the same structure I'm uploading a sample file here :


# -*- coding: utf-8 -*-
from threading import Timer
from pynput.mouse import Listener
import logging
import sys
import os
sys.path.insert(0, os.path.join(os.path.dirname(
 os.path.realpath(__file__)), "../"))

from Functions import utils as ut

if __name__=='__main__':

 ut.initialize_dirs()
 rec_file = ''.join(('mouse_',ut.get_date(),'.txt'))
 raw_data = ut.get_name('Mouse')
 rec_file = os.path.join(raw_data,rec_file)
 logging.basicConfig(filename=rec_file,level=logging.DEBUG,format="%(asctime)s %(message)s")

 try:
 with Listener(on_move=ut.on_move, on_click=ut.on_click,on_scroll=ut.on_scroll) as listener:
 Timer(60, listener.stop).start()
 listener.join()
 except KeyboardInterrupt as err:
 print(err)
 sys.exit(0)

 print('Exiting logger...')



I'm also uploading the functions that I use :


# -*- coding: utf-8 -*-
from serial import Serial
from datetime import datetime, timedelta
import pandas as pd
import collections
import logging
import shutil
import serial
import time
import sys
import os

click_held = False
button = None


def on_move(x,y):
 """The callback to call when mouse move events occur

 Args:
 x (float): The new pointer position
 y (float): The new pointer poisition
 """
 if click_held:
 logging.info("MV {0:>8} {1:>8} {2:>8}:".format(x,y,str(None)))
 else:
 logging.info("MV {0:>8} {1:>8} {2:>8}:".format(x,y,str(None)))


def on_click(x,y,button,pressed):
 """The callback to call when a mouse button is clicked

 Args:
 x (float): Mouse coordinates on screen
 y (float): Mouse coordinates on screen
 button (str): one of the Button values
 pressed (bool): Pressed is whether the button was pressed
 """
 global click_held
 if pressed:
 click_held = True
 logging.info("CLK {0:>7} {1:>6} {2:>13}".format(x,y,button))
 else:
 click_held = False
 logging.info("RLS {0:>7} {1:>6} {2:>13}".format(x,y,button))


def on_scroll(x,y,dx,dy):
 """The callback to call when mouse scroll events occur

 Args:
 x (float): The new pointer position on screen
 y (float): The new pointer position on screen
 dx (int): The horizontal scroll. The units of scrolling is undefined
 dy (int): The vertical scroll. The units of scrolling is undefined
 """
 if dy == -1:
 logging.info("SCRD {0:>6} {1:>6} {2:>6}".format(x,y,str(None)))
 elif dy == 1:
 logging.info("SCRU {0:>6} {1:>6} {2:>6}".format(x,y,str(None)))
 else:
 pass


def on_press_keys(key):
 """The callback to call when a button is pressed.

 Args:
 key (str): A KeyCode,a Key or None if the key is unknown
 """
 subkeys = [
 'Key.alt','Key.alt_gr','Key.alt_r','Key.backspace',
 'Key.space','Key.ctrl','Key.ctrl_r','Key.down',
 'Key.up','Key.left','Key.right','Key.page_down',
 'Key.page_up','Key.enter','Key.shift','Key.shift_r'
 ]

 key = str(key).strip('\'')
 if(key in subkeys):
 #print(key)
 logging.info(key)
 else:
 pass


def record_chair(output_file):
 """Read the data stream coming from the serial monitor
 in order to get the sensor readings

 Args:
 output_file (str): The file name, where the data stream will be stored
 """
 serial_port = "/dev/ttyACM0"
 baud_rate = 9600
 ser = serial.Serial(serial_port,baud_rate)
 logging.basicConfig(filename=output_file,level=logging.DEBUG,format="%(asctime)s %(message)s")
 flag = False
 start = time.time()
 while time.time() - start < 60.0:
 try:
 serial_data = str(ser.readline().decode().strip('\r\n'))
 time.sleep(0.2)
 tmp = serial_data.split(' ')[0] #Getting Sensor Id
 if(tmp == 'A0'):
 flag = True
 if (flag and tmp != 'A4'):
 #print(serial_data)
 logging.info(serial_data)
 if(flag and tmp == 'A4'):
 flag = False
 #print(serial_data)
 logging.info(serial_data)
 except (UnicodeDecodeError, KeyboardInterrupt) as err:
 print(err)
 print(err.args)
 sys.exit(0)


def initialize_dirs():
 """Create the appropriate directories in order to save
 and process the collected data
 """
 current_path = os.path.abspath(os.getcwd())
 os.chdir('..')
 current_path = (os.path.abspath(os.curdir)) #/Multodal_User_Monitoring
 current_path = os.path.join(current_path,'Data')
 create_subdirs([current_path])

 #Create mouse log folder
 mouse = os.path.join(current_path,'Mouse')
 create_subdirs([mouse])
 #Create mouse subfolders
 names = concat_names(mouse)
 create_subdirs(names)

 #Create keyboard log folder
 keyboard = os.path.join(current_path,'Keyboard')
 create_subdirs([keyboard])
 #Create keyboard subfolders
 names = concat_names(keyboard)
 create_subdirs(names)

 #Create the chair log folder
 chair = os.path.join(current_path,'Chair')
 create_subdirs([chair])
 #Create chair subfolders
 names = concat_names(chair)
 create_subdirs(names)

 #Create webcam log folder
 webcam = os.path.join(current_path,'Webcam')
 create_subdirs([webcam])

def concat_names(dir) -> str:
 """Concatenate the given folder names
 with the appropriate path

 Args:
 dir (str): The directory to create the subfolders

 Returns:
 list: The new absolute paths
 """
 raw_data = os.path.join(dir,'Raw')
 edited_data = os.path.join(dir,'Edited_logs')
 csv_data = os.path.join(dir,'CSV')
 features = os.path.join(dir,'Features')
 dirs = [raw_data,edited_data,csv_data,features]
 return dirs


def create_subdirs(paths):
 """Create sub directories given some absolute paths

 Args:
 paths (list): A list containing the paths to be created
 """
 for index,path in enumerate(paths):
 if(os.path.isdir(paths[index])):
 pass
 else:
 os.mkdir(paths[index])


def round_down(num,divisor) -> int:
 """Round the number of lines contained into the recording file,
 down to the nearest multiple of the given divisor

 Args:
 num (int): The number of lines contained into the given log file
 divisor (int): The divisor in order to get tuples of divisor

 Returns:
 int: The nearest multiple of five
 """
 return num-(num%divisor)


def get_date() -> str:
 """Get the current date in order to properly name
 the recored log files
 Returns:
 str: The current date in: YY_MM_DD format
 """
 return datetime.now().strftime('%Y_%m_%d')


def get_name(modality) -> str:
 """Save the recorded log into /Data//Raw

 Args:
 modality (str): The log data source

 Returns:
 str: The absolute path where each recording is saved
 """
 current_path = os.path.abspath(os.getcwd())
 current_path = os.path.join(current_path,'Data')

 if modality == 'Chair':
 chair_path = os.path.join(current_path,modality,'Raw')
 return chair_path

 elif modality == 'Mouse':
 mouse_path = os.path.join(current_path,modality,'Raw')
 return mouse_path

 elif modality == 'Keyboard':
 keyboard_path = os.path.join(current_path,modality,'Raw')
 return keyboard_path


def crawl_dir(target,folder) -> str:
 """Enumerate all the given files in a directory
 based on the given file extension

 Args:
 target (str): The file to search for
 folder (str): The folder to search

 Returns:
 [type]: A list containing the file names
 """
 current_path = os.path.abspath(os.getcwd())
 path = os.path.join(current_path,folder)
 file_names =[]
 for f in os.listdir(path):
 if(f.endswith(target)):
 fname=os.path.join(path,f)
 file_names.append(fname)
 return file_names


def convert_keys2_csv(input_file,output_file):
 """Convert the data stream file(keylogger recording) from .txt to .csv format

 Args:
 input_file (str): The data stream file in .txt format
 output_file (str): The csv extension file name
 """
 df = pd.read_fwf(input_file)
 col_names = ['Date','Time','Key']
 df.to_csv(output_file,header=col_names,encoding='utf-8',index=False)


def convert_mouse2_csv(input_file,output_file):
 """Convert the data stream file(mouselogger recording) from .txt to .csv format

 Args:
 input_file (str): The data stream file in .txt format
 output_file (str): The csv extension file name
 """
 df = pd.read_fwf(input_file)
 col_names = ['Date','Time','Action','PosX','PosY','Button']
 df.to_csv(output_file,header=col_names,encoding='utf-8',index=False)


def convert_chair_2_csv(input_file,output_file):
 """Convert the data stream file(chair recording)
 from .txt to .csv format

 Args:
 input_file (str): The data stream file in .txt format
 output_file (str): The csv extension file name
 """
 if(os.path.isfile(input_file)):
 pass
 else:
 print('Invalid working directory...')
 print('Aborting...')
 sys.exit(0)

 tmp0,tmp1,tmp2,tmp3,tmp4 = 0,1,2,3,4

 line_number = 0
 for line in open(input_file).readlines():
 line_number += 1

 rounded_line = round_down(line_number,5)
 d = collections.defaultdict(list)

 with open(input_file,'r') as f1:
 lines = f1.readlines()
 for i in range(rounded_line // 5):
 #Sensor:Analog input 0 values
 Sid0 = lines[i+tmp0]
 temp = Sid0.split()
 d['Sid0'].append([temp[0],temp[1],temp[2],temp[3]])
 #Sensor:Analog input 1 values
 Sid1 = lines[i+tmp1]
 temp = Sid1.split()
 d['Sid1'].append([temp[0],temp[1],temp[2],temp[3]])
 #Sensor:Analog input 2 values
 Sid2 = lines[i+tmp2]
 temp = Sid2.split()
 d['Sid2'].append([temp[0],temp[1],temp[2],temp[3]])
 #Sensor:Analog input 3 values
 Sid3 = lines[i+tmp3]
 temp = Sid3.split()
 d['Sid3'].append([temp[0],temp[1],temp[2],temp[3]])
 #Sensor:Analog input 4 values
 Sid4 = lines[i+tmp4]
 temp = Sid4.split()
 d['Sid4'].append([temp[0],temp[1],temp[2],temp[3]])

 tmp0 += 4
 tmp1 += 4
 tmp2 += 4
 tmp3 += 4
 tmp4 += 4

 l = []
 for i in range(rounded_line // 5):
 date = d['Sid0'][i][0]
 time = d['Sid0'][i][1]
 A0_val = d['Sid0'][i][3]
 A1_val = d['Sid1'][i][3]
 A2_val = d['Sid2'][i][3]
 A3_val = d['Sid3'][i][3]
 A4_val = d['Sid4'][i][3]
 l.append([date,time,A0_val,A1_val,A2_val,A3_val,A4_val])

 sensor_readings_df = pd.DataFrame.from_records(l)
 sensor_readings_df.columns = ['Date','Time','A0','A1','A2','A3','A4']
 sensor_readings_df.to_csv(output_file, encoding='utf-8', index=False)
 del l


def parse_raw_data(modality):
 """Convert each modality's raw data into csv format and move
 the edited raw data into the appropriate Edited_logs folder

 Args:
 modality (str): The data source
 """
 #Change directories
 current_path = os.path.abspath(os.getcwd()) #/Functions
 os.chdir('..')
 current_path = (os.path.abspath(os.curdir)) #/Multimodal_User_Monitoring
 os.chdir('./Data')#/Multimodal_User_Monitoring/Data
 current_path = (os.path.abspath(os.curdir)) #/Multimodal_User_Monitoring/Data
 current_path = os.path.join(current_path,modality) #example: /Multimodal_User_Monitoring/Data/<modality>
 raw_data_path = os.path.join(current_path,'Raw')
 csv_data_path = os.path.join(current_path,'CSV')
 edited_logs_path = os.path.join(current_path,'Edited_logs')

 txt_names = crawl_dir('.txt',raw_data_path)
 csv_names = []
 for elem in txt_names:
 name = elem.split('/')[-1].split('.')[0]
 csv_name = name+'.csv'
 tmp = os.path.join(csv_data_path,csv_name)
 csv_names.append(tmp)

 if modality == 'Mouse':
 if len(txt_names) == len(csv_names):
 for i, elem in enumerate(txt_names):
 #for i in range(len(txt_names)):
 convert_mouse2_csv(txt_names[i],csv_names[i])
 shutil.move(txt_names[i],edited_logs_path)

 elif modality == 'Keyboard':
 if len(txt_names) == len(csv_names):
 for i, elem in enumerate(txt_names):
 #for i in range(len(txt_names)):
 convert_keys2_csv(txt_names[i],csv_names[i])
 shutil.move(txt_names[i],edited_logs_path)

 elif modality == 'Chair':
 if len(txt_names) == len(csv_names):
 for i, elem in enumerate(txt_names):
 #for i in range(len(txt_names)):
 convert_chair_2_csv(txt_names[i],csv_names[i])
 shutil.move(txt_names[i],edited_logs_path)

</modality>

I need to mention that the logs of the python3 scripts are empty


-
fftools/ffplay : get rid of flush_pkt
7 février 2021, par Marton Balint
Syndiquer tout le site
