
Recherche avancée
Autres articles (91)
-
Gestion de la ferme
2 mars 2010, parLa ferme est gérée dans son ensemble par des "super admins".
Certains réglages peuvent être fais afin de réguler les besoins des différents canaux.
Dans un premier temps il utilise le plugin "Gestion de mutualisation" -
La gestion des forums
3 novembre 2011, parSi les forums sont activés sur le site, les administrateurs ont la possibilité de les gérer depuis l’interface d’administration ou depuis l’article même dans le bloc de modification de l’article qui se trouve dans la navigation de la page.
Accès à l’interface de modération des messages
Lorsqu’il est identifié sur le site, l’administrateur peut procéder de deux manières pour gérer les forums.
S’il souhaite modifier (modérer, déclarer comme SPAM un message) les forums d’un article particulier, il a à sa (...) -
Gestion des droits de création et d’édition des objets
8 février 2011, parPar défaut, beaucoup de fonctionnalités sont limitées aux administrateurs mais restent configurables indépendamment pour modifier leur statut minimal d’utilisation notamment : la rédaction de contenus sur le site modifiables dans la gestion des templates de formulaires ; l’ajout de notes aux articles ; l’ajout de légendes et d’annotations sur les images ;
Sur d’autres sites (8410)
-
output stream video produced by ffmpeg gets divided into sections on screen upon streaming
22 avril 2024, par Kaustubh SonawaneI'm trying to combine cv2 frames with numpy audio array to produce a hls playlist. I'm giving input using pipe. The problem is that while streaming the output hls playlist produced by ffmpeg, the video screen is divided into 4 parts and the sizes of the 4 parts keep getting changed slowly.
Also the audio is extremely distorted.

Attaching images for reference :
at timestamp 00:00
at timestamp 00:03

Here is the script I have written until now :


import cv2
import subprocess
import numpy as np
import librosa

def create_hls_playlist(width, height, frames, audio_data, output_playlist, fps=30, sample_rate=44100):
 print(fps)
 print(height, width)
 print(audio_np.dtype)
 video_args = [
 '-y', 
 "-fflags", "+discardcorrupt",
 '-f', 'rawvideo',
 '-vcodec', 'rawvideo',
 '-s', '{}x{}'.format(width, height),
 '-pix_fmt', 'bgr24',
 '-r', str(fps),
 '-i', 'pipe:', 
 ]

 audio_args = [
 '-f', 's16le', 
 '-ar', str(sample_rate), 
 '-ac', '1', 
 '-i', 'pipe:', 
 ]


 output_args = [
 "-vf", "scale=1920x1080,setdar=16/9",
 '-an',
 '-c:v', 'libx264',
 '-preset', 'ultrafast',
 '-f', 'hls',
 '-hls_time', '2',
 '-hls_list_size', '0',
 #'-loglevel', 'debug',
 output_playlist,
 ]

 ffmpeg_command = ['ffmpeg'] + video_args + output_args

 ffmpeg_process = subprocess.Popen(ffmpeg_command, stdin=subprocess.PIPE)

 try:
 for frame in frames[:10]:
 print(f"Frame shape: {frame.shape}, Frame data type: {frame.dtype}")
 # Check for any NaN or infinite values in the frame
 if np.isnan(frame).any() or np.isinf(frame).any():
 print("Frame contains NaN or infinite values!")
 ffmpeg_process.stdin.write(frame.to_string())

 except Exception as e:
 print(f"Error occurred while writing frames: {e}")
 
 # Write audio data to ffmpeg process
 audio_bytes = audio_data
 ffmpeg_process.stdin.write(audio_bytes)

 for frame in frames[10:]:
 ffmpeg_process.stdin.write(frame)
 
 ffmpeg_process.stdin.flush()

 ffmpeg_process.stdin.close()

 ffmpeg_process.wait()

 ffmpeg_process.stdin.close()


def capture_frames(video_path):
 frames = []
 cap = cv2.VideoCapture(video_path)
 width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
 height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

 while cap.isOpened():
 ret, frame = cap.read()
 if not ret:
 break
 frames.append(frame)

 cap.release()
 return frames, width, height

def load_audio(file_path, duration=None, sr=44100):
 audio_data, _ = librosa.load(file_path, sr=sr, duration=duration)
 print("finished audio-----------------------------------")
 print(audio_data)
 audio_duration = librosa.get_duration(y=audio_data, sr=sr)

 if duration is not None and audio_duration > duration:
 start_sample = 0
 end_sample = int(duration * sr)
 audio_data = audio_data[start_sample:end_sample]
 if np.isnan(audio_data).any() or np.isinf(audio_data).any():
 raise ValueError("Audio data contains NaN or infinite values")

 # Normalize audio data
 # audio_data /= np.max(np.abs(audio_data))
 audio_data = (audio_data * 32767).astype(np.int16)
 print(f"Audio data shape: {audio_data.shape}")
 print(f"Audio data type: {audio_data.dtype}")
 return audio_data

def get_video_frame_rate(video_path):
 cap = cv2.VideoCapture(video_path)
 if not cap.isOpened():
 raise ValueError(f"Unable to open the video file: {video_path}")

 frame_rate = cap.get(cv2.CAP_PROP_FPS)
 print(frame_rate)
 cap.release()
 return frame_rate

video_path = 'sample.mp4'

output_playlist = 'output.m3u8'
mp3_file_path = 'audio.mp3'
duration = 11 # seconds
sample_rate = 44100 # Hz


if __name__ == "__main__":
 frames, width, height = capture_frames(video_path)
 audio_np = load_audio(mp3_file_path, duration=duration, sr=sample_rate)
 create_hls_playlist(width, height, frames, audio_np, output_playlist)



And the output :


brain@brainywire:~/k/ffmpeg-e$ /bin/python3 /home//ffmpeg-e/stream_f.py
finished audio-----------------------------------
[1.7182100e-06 1.8538897e-06 1.9645518e-06 ... 5.4211184e-02 5.2549735e-02
 5.0875921e-02]
Audio data shape: (485100,)
Audio data type: int16
30
720 1280
int16
Frame shape: (720, 1280, 3), Frame data type: uint8
Error occurred while writing frames: 'numpy.ndarray' object has no attribute 'to_string'
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
 built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
 configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
 libavutil 56. 70.100 / 56. 70.100
 libavcodec 58.134.100 / 58.134.100
 libavformat 58. 76.100 / 58. 76.100
 libavdevice 58. 13.100 / 58. 13.100
 libavfilter 7.110.100 / 7.110.100
 libswscale 5. 9.100 / 5. 9.100
 libswresample 3. 9.100 / 3. 9.100
 libpostproc 55. 9.100 / 55. 9.100
Input #0, rawvideo, from 'pipe:':
 Duration: N/A, start: 0.000000, bitrate: 663552 kb/s
 Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 1280x720, 663552 kb/s, 30 tbr, 30 tbn, 30 tbc
Guessed Channel Layout for Input Stream #1.0 : mono
Input #1, s16le, from 'pipe:':
 Duration: N/A, bitrate: 705 kb/s
 Stream #1:0: Audio: pcm_s16le, 44100 Hz, mono, s16, 705 kb/s
Stream mapping:
 Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
 Stream #1:0 -> #0:1 (pcm_s16le (native) -> aac (native))
[libx264 @ 0x5557bf4369c0] using SAR=1/1
[libx264 @ 0x5557bf4369c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x5557bf4369c0] profile High 4:4:4 Predictive, level 4.0, 4:4:4, 8-bit
[libx264 @ 0x5557bf4369c0] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=6 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[s16le @ 0x5557bf3fcf80] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
Output #0, hls, to 'output.m3u8':
 Metadata:
 encoder : Lavf58.76.100
 Stream #0:0: Video: h264, yuv444p(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 90k tbn
 Metadata:
 encoder : Lavc58.134.100 libx264
 Side data:
 cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
 Stream #0:1: Audio: aac (LC), 44100 Hz, mono, fltp, 69 kb/s
 Metadata:
 encoder : Lavc58.134.100 aac
[rawvideo @ 0x5557bf3e8640] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
[hls @ 0x5557bf4335c0] Opening 'output0.ts' for writingtrate=N/A speed=5.06x 
[hls @ 0x5557bf4335c0] Opening 'output.m3u8.tmp' for writing
[rawvideo @ 0x5557bf3e8640] Packet corrupt (stream = 0, dts = 345), dropping it.
[hls @ 0x5557bf4335c0] Opening 'output1.ts' for writingtrate=N/A speed=5.31x 
[hls @ 0x5557bf4335c0] Opening 'output.m3u8.tmp' for writing
frame= 345 fps=166 q=-1.0 Lsize=N/A time=00:00:11.50 bitrate=N/A speed=5.52x 
video:21389kB audio:95kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x5557bf4369c0] frame I:2 Avg QP:21.00 size:666528
[libx264 @ 0x5557bf4369c0] frame P:343 Avg QP:24.78 size: 59968
[libx264 @ 0x5557bf4369c0] mb I I16..4: 100.0% 0.0% 0.0%
[libx264 @ 0x5557bf4369c0] mb P I16..4: 22.8% 0.0% 0.0% P16..4: 51.1% 0.0% 0.0% 0.0% 0.0% skip:26.1%
[libx264 @ 0x5557bf4369c0] coded y,u,v intra: 24.0% 3.7% 3.3% inter: 30.6% 4.9% 4.8%
[libx264 @ 0x5557bf4369c0] i16 v,h,dc,p: 43% 38% 10% 9%
[libx264 @ 0x5557bf4369c0] kb/s:15236.21
[aac @ 0x5557bf438400] Qavg: 118.713



I noticed this packet corrupt error and after some digging on stackoverflow found a flag to drop corrupt packets. But that didn't solve the problem.


If I do not drop the corrupted packets, I get this error, if it helps to give any more info :


[rawvideo @ 0x61a72571ab80] PACKET SIZE: 2764800, STRIDE: 3840
[libx264 @ 0x61a725754c40] frame= 323 QP=24.00 NAL=2 Slice:P Poc:146 I:2340 P:4474 SKIP:1346 size=62313 bytes
[rawvideo @ 0x61a72571ab80] PACKET SIZE: 2764800, STRIDE: 3840
[libx264 @ 0x61a725754c40] frame= 324 QP=24.00 NAL=2 Slice:P Poc:148 I:1538 P:4449 SKIP:2173 size=56970 bytes
[rawvideo @ 0x61a72571ab80] PACKET SIZE: 2764800, STRIDE: 3840
[libx264 @ 0x61a725754c40] frame= 325 QP=24.00 NAL=2 Slice:P Poc:150 I:2553 P:4310 SKIP:1297 size=64782 bytes
pipe:: corrupt input packet in stream 0
[rawvideo @ 0x61a72571ab80] PACKET SIZE: 2751960, STRIDE: 3822
[rawvideo @ 0x61a72571ab80] Invalid buffer size, packet size 2751960 < expected frame_size 2764800
Error while decoding stream #0:0: Invalid argument
[out_0_0 @ 0x61a72575f400] EOF on sink link out_0_0:default.
[out_0_1 @ 0x61a726577f80] EOF on sink link out_0_1:default.
No more output streams to write to, finishing.
[libx264 @ 0x61a725754c40] frame= 326 QP=24.00 NAL=2 Slice:P Poc:152 I:1076 P:4002 SKIP:3082 size=44541 bytes
[libx264 @ 0x61a725754c40] frame= 327 QP=24.00 NAL=2 Slice:P Poc:154 I:2493 P:4545 SKIP:1122 size=62617 bytes
[libx264 @ 0x61a725754c40] frame= 328 QP=24.00 NAL=2 Slice:P Poc:156 I:1383 P:4328 SKIP:2449 size=50782 bytes
[libx264 @ 0x61a725754c40] frame= 329 QP=24.00 NAL=2 Slice:P Poc:158 I:2179 P:4376 SKIP:1605 size=57972 bytes
[libx264 @ 0x61a725754c40] frame= 330 QP=24.00 NAL=2 Slice:P Poc:160 I:925 P:3772 SKIP:3463 size=37181 bytes



I tried to give bufsize flags too, but even that wouldnt help.


I'm a total beginner to ffmpeg. I tried to give the correct resolution, tried all pix_fmt from the documentation, and also experimented with different values of fps. But the problem with split screen and audio distortion continues.


-
6 Crucial Benefits of Conversion Rate Optimisation
26 février 2024, par Erin -
How to Measure Marketing Effectiveness : A Step-by-Step Guide
22 février 2024, par Erin