Recherche avancée

Sur d’autres sites (11223)

• Long-running AWS Linux process suddenly freezes once in a while

  12 décembre 2022, par Incredi Blame

  I am running a non-stop process on Ubuntu LTS AWS t2.micro instance. Process is ffmpeg downloading network stream from a CCTV. It crashes every 10 min cause of network errors, so I run it in a cycle script like this :

  


  while true
do
    ~command~
done


  


  The cycler-script itself never crashes.

  


  Everything works fine for hours (I check with ssh from time to time). But then sometimes ffmpeg process just freezes. I checked logs, it is writing the stream and it literally freezes mid-way through the stream, no error messages, nothing, as if it is still waiting for the next packet. Just stops working and never wakes up. Usually if the stream goes down ffmpeg complains, spits errors out and crashes. But on this occasion there is nothing and no crash either.

  


  Another mystery is that I run several of such ffmpeg processes and they all freeze at the same time.

  


  The issue with that freezing is the cycler can't restart ffmpegs, as they haven't crashed. When I check frozen processes with ps -aux they are shown as SL, which doesn't tell much, as they are always marked this way. top -i doesn't display them at all as active. When they work as normal it does. If I kill those frozen processes the cycler restarts them and everything works again as it should.

  


  I can always kill those ffmpegs once in a while on some schedule and let cycler restart them. Or check if they froze by timestamps they output and kill them then, so theoretically the problem is solvable. But I just want to know what causes it.

  


  I haven't figured out yet if this freezing happens at the same time of the day, trying to track that now. I don't think it could be related to free space shortage, as I have a cleaner cron job and free space is always available.

  


  So I have the following questions :

  


  

  1. Is there anything on AWS itself which could cause this freezing ? Like some kind of regular maintenance job that messes with running processes ?

  


  2. Could it be caused by Linux freezing long-running processes ? Probably not, as they all freeze at the same time while current running time is different for each.

  


  3. Last one, my cron job that cleans space also cleans out logs that ffmpeg is appending to. The script goes like this :

  


  


  tail "ffmpeg.log" -n 10000 > "tmp.log" && \
    cat "tmp.log" > "ffmpeg.log" && \
    rm "tmp.log"


  


  The logs now are already full and are never fully cleared. Cron job just preserves the last 10000 lines. It runs every 2 min and runs fine for hours. Could it be responsible for this sudden freezing due to some coincidental timing ?

  


  If none of the above is responsible for freezing it must be some streaming issue on the server side, which I have no access to. In that case I would have to rely on detecting freezing and restarting ffmpegs.

  


• Best way to stream FFMPEG conversion to HTML5 video element ?

  4 août 2021, par SamTheFam

  Context

  


  Me and my friend are currently working on a media player desktop application using Electron (which implements website code into a desktop application). For the video element we are using the HTML5 video element, and loading a video from the user's local machine. Most browsers only support MP4, OGG, and WebM (Mozilla Specification), but we would like to support more formats to make it applicable to more users. For this reason, we decided to use the FFMPEG video converter. As FFMPEG takes some time to convert videos, we would like to have a streaming implementation that converts the video while playing the HTML5 video.

  


  The Problem

  


  The problem is that we cannot figure out the best way to stream our video whilst it is converting to another format. The primary issue is that we are unsure how we can implement skipping in the video, where we convert that specfic point if skipped to.

  


  For further clarification of the issue, if a user provided a .mkv file, we would start the FFMPEG conversion to .mp4, and would convert enough for the video to start playing. While the video is being watched, FFMPEG would continue converting before the user has reached that point in the video. If a user wanted to skip through the video, we would need a way to fast-forward the conversion to a specific point in the video, thus allowing the user to watch that section of the video, if it has not already been converted.

  


  Ideally we would like the conversion to be faster than the playback of the video, but if this has a cost in quality, we would prefer to make it optional to the end user.

  


  The first solution that came to our heads was to convert small sections of the video, and produce many small (3-5 second) videos, that would be loaded into the video element respectively. The issue here is that the video element takes quite a lot of time to load new source files and as a result make an awful user experience, with the video going black every x seconds.

  


  Another possible solution we stumbled accross was mentioned here. If we were to implement this, we would start a HTTP server on the user's local machine. The issue we had with this is that it could be a rather bloated solution and could be tedious to maintain. Along with this, it would likely slow down the start time for the application, and the runtime as well.

  


  Thank you in advance, we look forward to your responses.

  


• Add logo on concatenated video

  5 septembre 2020, par ottpeter

  I'm trying to add a logo on top of a concatenated video. The video concatenation and the adding of the logo should be in the same filter complex. My filter complex currently looks like this (it is generated by Bash) :

  


  [0:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v0];[1:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v1];[2:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v2];[3:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v3];[4:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v4];[5:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v5];[6:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v6];[7:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v7];[8:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v8];[9:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v9];[10:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v10];[11:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v11];[12:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v12];[13:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v13];[14:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v14];[15:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v15];[16:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v16];[17:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v17];[18:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v18];[19:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v19];[20:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v20];[21:v]scale=854:480:force_original_aspect_ratio=1,pad=width=854:height=480:x='if(lt(in_w,854),(854-in_w)/2,0)':0,setsar=1[v21];movie=/home/logo.png, scale=180:-1 [logo];
[v0][v1][v2][v3][v4][v5][v6][v7][v8][v9][v10][v11][v12][v13][v14][v15][v16][v17][v18][v19][v20][v21] concat=unsafe=1:n=22:v=1:a=0[conc]; [conc][logo] overlay=50:50 [outv];


  


  On 0bin, it is easier to read. So, there is a really long video concatenation part, and on the end, I would try to add the logo like this :

  


  


  [conc][logo] overlay=50:50 [outv] ;

  


  


  This is the error I'm getting :

  


  [AVFilterGraph @ 0x5640862fc520] No such filter: '' 
Error initializing complex filters.
Invalid argument


  


  How to do this correctly ?

  


  EDIT :
This is the Bash script :

  


      echo "Creating the video list..."
    # Number of videos
    VID_NUMBER=$(find $VIDS_PATH -maxdepth 1 -type f -printf . | wc -c)
    # File names can not contain spaces. INPUT_LIST will contain the list of input files, all files in VIDS folder.
    INPUT_LIST=""
    for file in $VIDS_PATH*
    do
        if [ ${file: -4} == ".jpg" ]
        then
            INPUT_LIST="${INPUT_LIST} -loop 1 -t $IMAGE_DURATION -i $file"
        else
            INPUT_LIST="${INPUT_LIST} -i $file"
        fi
    done

    echo "Creating filter complex..."
    # This is the beginning of the filter that is used for concatenation. Has to be an entry for all videos.
    FILTER_COMPLEX_LIST=""
    for ((i=0; i<$VID_NUMBER; i++))
    do
        FILTER_COMPLEX_LIST="${FILTER_COMPLEX_LIST}[$i:v]scale=$RES_X:$RES_Y:force_original_aspect_ratio=1,pad=width=$RES_X:height=$RES_Y:x='if(lt(in_w,$RES_X),($RES_X-in_w)/2,0)':0,setsar=1[v$i];"
    done

    # Insert logo
    FILTER_COMPLEX_LIST="${FILTER_COMPLEX_LIST}movie=$LOGO, scale=180:-1, setpts=PTS-STARTPTS [logo];"

    # Filter final line. This is also generated in a loop. Tells ffmpeg which videos to concat and specifies output [outv]. No sound.
    FILTER_FINAL_LINE=""
    for ((i=0; i<$VID_NUMBER; i++))
    do
        FILTER_FINAL_LINE="${FILTER_FINAL_LINE}[v$i]"
    done
    FILTER_FINAL_LINE="${FILTER_FINAL_LINE} concat=unsafe=1:n=22:v=1:a=0[conc]; [conc][logo] overlay=50:50 [outv];"

    # Run ffmpeg
    date +"%Y %b %d %H:%M:%S Starting ffmpeg to stream videos ..." >> /home/streamer.log
    ffmpeg \
        -y \
        -fflags +genpts \
        $INPUT_LIST \
        -filter_complex "\
        $FILTER_COMPLEX_LIST \
        $FILTER_FINAL_LINE" \
        -map "[outv]" \
        $OUTPUT_SETTINGS \
        $OUTPUT

        sleep 1
done