Recherche avancée

Médias (1)

Mot : - Tags -/epub

Autres articles (56)

  • Mise à disposition des fichiers

    14 avril 2011, par

    Par défaut, lors de son initialisation, MediaSPIP ne permet pas aux visiteurs de télécharger les fichiers qu’ils soient originaux ou le résultat de leur transformation ou encodage. Il permet uniquement de les visualiser.
    Cependant, il est possible et facile d’autoriser les visiteurs à avoir accès à ces documents et ce sous différentes formes.
    Tout cela se passe dans la page de configuration du squelette. Il vous faut aller dans l’espace d’administration du canal, et choisir dans la navigation (...)

  • Gestion générale des documents

    13 mai 2011, par

    MédiaSPIP ne modifie jamais le document original mis en ligne.
    Pour chaque document mis en ligne il effectue deux opérations successives : la création d’une version supplémentaire qui peut être facilement consultée en ligne tout en laissant l’original téléchargeable dans le cas où le document original ne peut être lu dans un navigateur Internet ; la récupération des métadonnées du document original pour illustrer textuellement le fichier ;
    Les tableaux ci-dessous expliquent ce que peut faire MédiaSPIP (...)

  • Mediabox : ouvrir les images dans l’espace maximal pour l’utilisateur

    8 février 2011, par

    La visualisation des images est restreinte par la largeur accordée par le design du site (dépendant du thème utilisé). Elles sont donc visibles sous un format réduit. Afin de profiter de l’ensemble de la place disponible sur l’écran de l’utilisateur, il est possible d’ajouter une fonctionnalité d’affichage de l’image dans une boite multimedia apparaissant au dessus du reste du contenu.
    Pour ce faire il est nécessaire d’installer le plugin "Mediabox".
    Configuration de la boite multimédia
    Dès (...)

Sur d’autres sites (6977)

  • How to render two videos with alpha channel in real time in pygame ?

    21 décembre 2024, par Francesco Calderone

    I need to play two videos with synched sound in real-time with Pygame.
Pygame does not currently support video streams, so I am using a ffmpeg subprocess.
The first video is a prores422_hq. This is a background video with no alpha channel.
The second video is a prores4444 overlay video with an alpha channel, and it needs to be played in real-tim on top of the first video (with transparency).
All of this needs synched sound from the first base video only.

    


    I have tried many libraries, including pymovie pyav and opencv. The best result so far is to use a subprocess with ffmpeg.

    


    ffmpeg -i testing/stefano_prores422_hq.mov -stream_loop -1 -i testing/key_prores4444.mov -filter_complex "[1:v]format=rgba,colorchannelmixer=aa=1.0[overlay];[0:v][overlay]overlay" -f nut pipe:1 | ffplay -

    


    When running this in the terminal and playing with ffplay, everything is perfect, the overlay looks good, no dropped frames, and the sound is in synch.

    


    However, trying to feed that to pygame via a subprocess creates either video delays and drop frames or audio not in synch.

    


    EXAMPLE ONE :

    


    # SOUND IS NOT SYNCHED - sound is played via ffplay
import pygame
import subprocess
import numpy as np
import sys

def main():
    pygame.init()
    screen_width, screen_height = 1920, 1080
    screen = pygame.display.set_mode((screen_width, screen_height))
    pygame.display.set_caption("PyGame + FFmpeg Overlay with Audio")
    clock = pygame.time.Clock()

    # LAUNCH AUDIO-ONLY SUBPROCESS
    audio_cmd = [
        "ffplay",
        "-nodisp",          # no video window
        "-autoexit",        # exit when video ends
        "-loglevel", "quiet",
        "testing/stefano_prores422_hq.mov"
    ]
    audio_process = subprocess.Popen(audio_cmd)

    # LAUNCH VIDEO-OVERLAY SUBPROCESS
    ffmpeg_command = [
        "ffmpeg",
        "-re",                        # read at native frame rate
        "-i", "testing/stefano_prores422_hq.mov",
        "-stream_loop", "-1",         # loop alpha video
        "-i", "testing/key_prores4444.mov",
        "-filter_complex",
        "[1:v]format=rgba,colorchannelmixer=aa=1.0[overlay];"  # ensure alpha channel
        "[0:v][overlay]overlay",      # overlay second input onto first
        "-f", "rawvideo",             # output raw video
        "-pix_fmt", "rgba",           # RGBA format
        "pipe:1"                      # write to STDOUT
    ]
    video_process = subprocess.Popen(
        ffmpeg_command,
        stdout=subprocess.PIPE,
        stderr=subprocess.DEVNULL
    )
    frame_size = screen_width * screen_height * 4  # RGBA = 4 bytes/pixel
    running = True
    while running:
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                running = False
                break

        raw_frame = video_process.stdout.read(frame_size)

        if len(raw_frame) < frame_size:
            running = False
            break
        # Convert raw bytes -> NumPy array -> PyGame surface
        frame_array = np.frombuffer(raw_frame, dtype=np.uint8)
        frame_array = frame_array.reshape((screen_height, screen_width, 4))
        frame_surface = pygame.image.frombuffer(frame_array.tobytes(), 
                                                (screen_width, screen_height), 
                                                "RGBA")
        screen.blit(frame_surface, (0, 0))
        pygame.display.flip()
        clock.tick(25)
    video_process.terminate()
    video_process.wait()
    audio_process.terminate()
    audio_process.wait()
    pygame.quit()
    sys.exit()

if __name__ == "__main__":
    main()



    


    EXAMPLE TWO

    


    # NO VIDEO OVERLAY - SOUND SYNCHED
import ffmpeg
import pygame
import sys
import numpy as np
import tempfile
import os

def extract_audio(input_file, output_file):
    """Extract audio from video file to temporary WAV file"""
    (
        ffmpeg
        .input(input_file)
        .output(output_file, acodec='pcm_s16le', ac=2, ar='44100')
        .overwrite_output()
        .run(capture_stdout=True, capture_stderr=True)
    )

def get_video_fps(input_file):
    probe = ffmpeg.probe(input_file)
    video_info = next(s for s in probe['streams'] if s['codec_type'] == 'video')
    fps_str = video_info.get('r_frame_rate', '25/1')
    num, den = map(int, fps_str.split('/'))
    return num / den

input_file = "testing/stefano_prores422_hq.mov"

# Create temporary WAV file
temp_audio = tempfile.NamedTemporaryFile(suffix='.wav', delete=False)
temp_audio.close()
extract_audio(input_file, temp_audio.name)

probe = ffmpeg.probe(input_file)
video_info = next(s for s in probe['streams'] if s['codec_type'] == 'video')
width = int(video_info['width'])
height = int(video_info['height'])
fps = get_video_fps(input_file)

process = (
    ffmpeg
    .input(input_file)
    .output('pipe:', format='rawvideo', pix_fmt='rgb24')
    .run_async(pipe_stdout=True)
)

pygame.init()
pygame.mixer.init(frequency=44100, size=-16, channels=2, buffer=4096)
clock = pygame.time.Clock()
screen = pygame.display.set_mode((width, height))

pygame.mixer.music.load(temp_audio.name)
pygame.mixer.music.play()

frame_count = 0
start_time = pygame.time.get_ticks()

while True:
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            pygame.mixer.music.stop()
            os.unlink(temp_audio.name)
            sys.exit()

    in_bytes = process.stdout.read(width * height * 3)
    if not in_bytes:
        break

    # Calculate timing for synchronization
    expected_frame_time = frame_count * (1000 / fps)
    actual_time = pygame.time.get_ticks() - start_time
    
    if actual_time < expected_frame_time:
        pygame.time.wait(int(expected_frame_time - actual_time))
    
    in_frame = (
        np.frombuffer(in_bytes, dtype="uint8")
        .reshape([height, width, 3])
    )
    out_frame = pygame.surfarray.make_surface(np.transpose(in_frame, (1, 0, 2)))
    screen.blit(out_frame, (0, 0))
    pygame.display.flip()
    
    frame_count += 1

pygame.mixer.music.stop()
process.wait()
pygame.quit()
os.unlink(temp_audio.name)


    


    I also tried using pygame mixer and a separate mp3 audio file, but that didn't work either. Any help on how to synch the sound while keeping the playback of both videos to 25 FPS would be greatly appreciated !!!

    


  • How to render two videos with alpha channel in real time in pygame with synched audio ?

    21 décembre 2024, par Francesco Calderone

    I need to play two videos with synched sound in real-time with Pygame.
Pygame does not currently support video streams, so I am using a ffmpeg subprocess.
The first video is a prores422_hq. This is a background video with no alpha channel.
The second video is a prores4444 overlay video with an alpha channel, and it needs to be played in real-tim on top of the first video (with transparency).
All of this needs synched sound from the first base video only.

    


    I have tried many libraries, including pymovie pyav and opencv. The best result so far is to use a subprocess with ffmpeg.

    


    ffmpeg -i testing/stefano_prores422_hq.mov -stream_loop -1 -i testing/key_prores4444.mov -filter_complex "[1:v]format=rgba,colorchannelmixer=aa=1.0[overlay];[0:v][overlay]overlay" -f nut pipe:1 | ffplay -

    


    When running this in the terminal and playing with ffplay, everything is perfect, the overlay looks good, no dropped frames, and the sound is in synch.

    


    However, trying to feed that to pygame via a subprocess creates either video delays and drop frames or audio not in synch.

    


    EXAMPLE ONE :

    


    # SOUND IS NOT SYNCHED - sound is played via ffplay
import pygame
import subprocess
import numpy as np
import sys

def main():
    pygame.init()
    screen_width, screen_height = 1920, 1080
    screen = pygame.display.set_mode((screen_width, screen_height))
    pygame.display.set_caption("PyGame + FFmpeg Overlay with Audio")
    clock = pygame.time.Clock()

    # LAUNCH AUDIO-ONLY SUBPROCESS
    audio_cmd = [
        "ffplay",
        "-nodisp",          # no video window
        "-autoexit",        # exit when video ends
        "-loglevel", "quiet",
        "testing/stefano_prores422_hq.mov"
    ]
    audio_process = subprocess.Popen(audio_cmd)

    # LAUNCH VIDEO-OVERLAY SUBPROCESS
    ffmpeg_command = [
        "ffmpeg",
        "-re",                        # read at native frame rate
        "-i", "testing/stefano_prores422_hq.mov",
        "-stream_loop", "-1",         # loop alpha video
        "-i", "testing/key_prores4444.mov",
        "-filter_complex",
        "[1:v]format=rgba,colorchannelmixer=aa=1.0[overlay];"  # ensure alpha channel
        "[0:v][overlay]overlay",      # overlay second input onto first
        "-f", "rawvideo",             # output raw video
        "-pix_fmt", "rgba",           # RGBA format
        "pipe:1"                      # write to STDOUT
    ]
    video_process = subprocess.Popen(
        ffmpeg_command,
        stdout=subprocess.PIPE,
        stderr=subprocess.DEVNULL
    )
    frame_size = screen_width * screen_height * 4  # RGBA = 4 bytes/pixel
    running = True
    while running:
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                running = False
                break

        raw_frame = video_process.stdout.read(frame_size)

        if len(raw_frame) < frame_size:
            running = False
            break
        # Convert raw bytes -> NumPy array -> PyGame surface
        frame_array = np.frombuffer(raw_frame, dtype=np.uint8)
        frame_array = frame_array.reshape((screen_height, screen_width, 4))
        frame_surface = pygame.image.frombuffer(frame_array.tobytes(), 
                                                (screen_width, screen_height), 
                                                "RGBA")
        screen.blit(frame_surface, (0, 0))
        pygame.display.flip()
        clock.tick(25)
    video_process.terminate()
    video_process.wait()
    audio_process.terminate()
    audio_process.wait()
    pygame.quit()
    sys.exit()

if __name__ == "__main__":
    main()



    


    EXAMPLE TWO

    


    # NO VIDEO OVERLAY - SOUND SYNCHED
import ffmpeg
import pygame
import sys
import numpy as np
import tempfile
import os

def extract_audio(input_file, output_file):
    """Extract audio from video file to temporary WAV file"""
    (
        ffmpeg
        .input(input_file)
        .output(output_file, acodec='pcm_s16le', ac=2, ar='44100')
        .overwrite_output()
        .run(capture_stdout=True, capture_stderr=True)
    )

def get_video_fps(input_file):
    probe = ffmpeg.probe(input_file)
    video_info = next(s for s in probe['streams'] if s['codec_type'] == 'video')
    fps_str = video_info.get('r_frame_rate', '25/1')
    num, den = map(int, fps_str.split('/'))
    return num / den

input_file = "testing/stefano_prores422_hq.mov"

# Create temporary WAV file
temp_audio = tempfile.NamedTemporaryFile(suffix='.wav', delete=False)
temp_audio.close()
extract_audio(input_file, temp_audio.name)

probe = ffmpeg.probe(input_file)
video_info = next(s for s in probe['streams'] if s['codec_type'] == 'video')
width = int(video_info['width'])
height = int(video_info['height'])
fps = get_video_fps(input_file)

process = (
    ffmpeg
    .input(input_file)
    .output('pipe:', format='rawvideo', pix_fmt='rgb24')
    .run_async(pipe_stdout=True)
)

pygame.init()
pygame.mixer.init(frequency=44100, size=-16, channels=2, buffer=4096)
clock = pygame.time.Clock()
screen = pygame.display.set_mode((width, height))

pygame.mixer.music.load(temp_audio.name)
pygame.mixer.music.play()

frame_count = 0
start_time = pygame.time.get_ticks()

while True:
    for event in pygame.event.get():
        if event.type == pygame.QUIT:
            pygame.mixer.music.stop()
            os.unlink(temp_audio.name)
            sys.exit()

    in_bytes = process.stdout.read(width * height * 3)
    if not in_bytes:
        break

    # Calculate timing for synchronization
    expected_frame_time = frame_count * (1000 / fps)
    actual_time = pygame.time.get_ticks() - start_time
    
    if actual_time < expected_frame_time:
        pygame.time.wait(int(expected_frame_time - actual_time))
    
    in_frame = (
        np.frombuffer(in_bytes, dtype="uint8")
        .reshape([height, width, 3])
    )
    out_frame = pygame.surfarray.make_surface(np.transpose(in_frame, (1, 0, 2)))
    screen.blit(out_frame, (0, 0))
    pygame.display.flip()
    
    frame_count += 1

pygame.mixer.music.stop()
process.wait()
pygame.quit()
os.unlink(temp_audio.name)


    


    I also tried using pygame mixer and a separate mp3 audio file, but that didn't work either. Any help on how to synch the sound while keeping the playback of both videos to 25 FPS would be greatly appreciated !!!

    


  • How to catch connection lost in avcodec_decode_video2 ?

    3 juillet 2014, par 4ntoine

    I’m using android ffmpeg port and i have native crash while playing and connection lost :

    07-03 16:06:45.036      831-831/? D/CrashAnrDetector﹕ Build: samsung/hltexx/hlte:4.4.2/KOT49H/N9005XXUENA7:user/release-keys
    
    Hardware: MSM8974
    
    Revision: 8
    
    Bootloader: N9005XXUENA7
    
    Radio: unknown
    
    Kernel: Linux version 3.4.0-348029 (dpi@SWDC3316) (gcc version 4.7 (GCC) ) #1 SMP PREEMPT Sat Jan 11 09:45:51 KST 2014
    
    *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
    
    Build fingerprint: 'samsung/hltexx/hlte:4.4.2/KOT49H/N9005XXUENA7:user/release-keys'
    
    Revision: '8'
    
    pid: 21599, tid: 25027, name: AsyncTask #4  >>> com.company.project <<<
    
    signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0037000f
    
    r0 8ef40b10  r1 00000002  r2 8ef40b0c  r3 00000000
    
    r4 825d0270  r5 79b51e20  r6 00370007  r7 002a001f
    
    r8 8ef40b0c  r9 79b10160  sl 8ef40bd8  fp 79c62cf8
    
    ip 00000000  sp 8ef40ae4  lr 7c0ed1db  pc 7be277fc  cpsr 280f0030
    
    d0  0000000000000000  d1  0000000000000000
    
    d2  0000000000000000  d3  0000000000000000
    
    d4  3fd34413509f79fb  d5  40c3880000000000
    
    d6  426d1a94a2000000  d7  42a0c6f7a0b5ed8d
    
    d8  3f13bcd3441bd127  d9  461c40000000168b
    
    d10 00000000441bd127  d11 0000000000000000
    
    d12 0000000000000000  d13 0000000000000000
    
    d14 0000000000000000  d15 0000000000000000
    
    d16 00240014003e000e  d17 0028001c00340004
    
    d18 0021001100390009  d19 002d001900310001
    
    d20 00270017003d000d  d21 002a001f00370007
    
    d22 00220012003a000a  d23 002e001a00320002
    
    d24 0202020202020202  d25 fefefefefefefefe
    
    d26 40e8fd171d2bf551  d27 402305681ecd4aa1
    
    d28 3ff0000000000000  d29 007f007f007f007f
    
    d30 ffffffffffffffff  d31 4000000000000000
    
    scr 88000012
    
    backtrace:
    
    #00  pc 001217fc  /data/app-lib/com.company.project-178/libffmpeg-neon.so (av_packet_get_side_data+11)
    
    #01  pc 003e71d7  /data/app-lib/com.company.project-178/libffmpeg-neon.so (avcodec_decode_video2+118)
    
    stack:
    
    8ef40aa4  00000000
    
    8ef40aa8  00000001
    
    8ef40aac  00000000
    
    8ef40ab0  0000a09f
    
    8ef40ab4  00000000
    
    8ef40ab8  00000001
    
    8ef40abc  4029abd9  /system/lib/libgui.so (android::Surface::queueBuffer(ANativeWindowBuffer*, int)+300)
    
    8ef40ac0  79afa958
    
    8ef40ac4  4002bf80
    
    8ef40ac8  00000002
    
    8ef40acc  00000000
    
    8ef40ad0  00000000
    
    8ef40ad4  00000000
    
    8ef40ad8  79b51fd8
    
    8ef40adc  825d0270
    
    8ef40ae0  79b51e20
    
    #00  8ef40ae4  825d0270
    
    ........  ........
    
    #01  8ef40ae4  825d0270
    
    8ef40ae8  79b51e20
    
    8ef40aec  8ef40b10  [stack:25027]
    
    8ef40af0  8192cfc0
    
    8ef40af4  8ef40b0c  [stack:25027]
    
    8ef40af8  00000001
    
    8ef40afc  00000000
    
    8ef40b00  00000000
    
    8ef40b04  00000000
    
    8ef40b08  00000000
    
    8ef40b0c  00000000
    
    8ef40b10  003f000f
    
    8ef40b14  00250015
    
    8ef40b18  00350005
    
    8ef40b1c  0029001d
    
    8ef40b20  00390009
    
    memory near r0:
    
    8ef40af0 8192cfc0 8ef40b0c 00000001 00000000
    
    8ef40b00 00000000 00000000 00000000 00000000
    
    8ef40b10 003f000f 00250015 00350005 0029001d
    
    8ef40b20 00390009 00210011 00310001 002d0019
    
    8ef40b30 003d000d 00270017 00370007 002a001f
    
    8ef40b40 003a000a 00220012 00320002 002e001a
    
    8ef40b50 003e000e 00240014 00340004 0028001c
    
    8ef40b60 4029a577 825d0270 79b51e20 7a049803
    
    8ef40b70 8192cfc0 787bcf70 7b409008 79ded900
    
    8ef40b80 7b409008 7a02c783 1d2bf551 40e8fd17
    
    8ef40b90 7f62b6ae 3ff07dd9 6f177a70 40e8fc9f
    
    8ef40ba0 1d2bf551 40e8fd17 7f62b6ae 3ff07dd9
    
    8ef40bb0 ea5ecf87 0000000b 7b4c48f4 00000000
    
    8ef40bc0 0000001c 79afa598 798ed350 0000000b
    
    8ef40bd0 000fba45 00000000 00000000 00000001
    
    8ef40be0 000f4240 000002c0 53b52b34 9adb02b9
    
    memory near r2:
    
    8ef40aec 8ef40b10 8192cfc0 8ef40b0c 00000001
    
    8ef40afc 00000000 00000000 00000000 00000000
    
    8ef40b0c 00000000 003f000f 00250015 00350005
    
    8ef40b1c 0029001d 00390009 00210011 00310001
    
    8ef40b2c 002d0019 003d000d 00270017 00370007
    
    8ef40b3c 002a001f 003a000a 002200
    
07-03 16:06:45.036      831-831/? D/CrashAnrDetector﹕ processName:com.company.project
    
07-03 16:06:45.036      831-831/? D/CrashAnrDetector﹕ broadcastEvent : com.company.project SYSTEM_TOMBSTONE
    
07-03 16:06:45.036     831-1281/? I/ActivityManager﹕ Killing 24245:com.google.android.apps.magazines/u0a131 (adj 15): empty #31
    
07-03 16:06:45.046      831-831/? W/ContextImpl﹕ Calling a method in the system process without a qualified user: android.app.ContextImpl.sendBroadcast:1468 com.android.server.analytics.data.collection.application.CrashAnrDetector.broadcastEvent:296 com.android.server.analytics.data.collection.application.CrashAnrDetector.processDropBoxEntry:254 com.android.server.analytics.data.collection.application.CrashAnrDetector.access$100:60 com.android.server.analytics.data.collection.application.CrashAnrDetector$1.onReceive:102
    
07-03 16:06:45.066  25171-25171/? I/SELinux﹕ Function: selinux_android_load_priority [0], There is no sepolicy file.
    
07-03 16:06:45.066  25171-25171/? I/SELinux﹕ Function: selinux_android_load_priority , loading version is VE=SEPF_SM-N9005_4.4.2_0013
    
07-03 16:06:45.066  25171-25171/? I/SELinux﹕ selinux_android_seapp_context_reload: seapp_contexts file is loaded from /data/security/spota/seapp_contexts
    
07-03 16:06:45.116   1049-31569/? D/KeyguardViewMediator﹕ setHidden false
    
07-03 16:06:45.116   1049-31569/? D/KeyguardUpdateMonitor﹕ sendKeyguardVisibilityChanged(true)
    
07-03 16:06:45.116    1049-1049/? D/KeyguardUpdateMonitor﹕ handleKeyguardVisibilityChanged(1)
    
07-03 16:06:45.166     831-1281/? I/WindowState﹕ WIN DEATH: Window{44ddb5e8 u0 SurfaceView}
    
07-03 16:06:45.166   1049-11393/? D/KeyguardViewMediator﹕ setHidden false
    
07-03 16:06:45.166   1049-11393/? D/KeyguardUpdateMonitor﹕ sendKeyguardVisibilityChanged(true)
    
07-03 16:06:45.166    1049-1049/? D/KeyguardUpdateMonitor﹕ handleKeyguardVisibilityChanged(1)
    
07-03 16:06:45.166      831-841/? I/ActivityManager﹕ Process com.company.project (pid 21599) (adj 0) has died.

    How can i fix it - catch exceptions or any other solution ?

Navigation