Recherche avancée

Sur d’autres sites (11850)

• memory leak on FFMPEG RTSP streams ?

  1er août 2024, par user3260912

  I have been having an issue recently where I have several IP cameras set up and running. Every so often, one of the streams will lock the entire virtual machine it is running in up. Looking at top, Ffmpeg is using 86% of 64 GB of RAM when the machine locks up. Yes, I realize my IP camera password


  


  As you can see from the screenshot, the RTSP stream process is using 51 GB of RAM to pull the images to a jpeg file.

  


  Any ideas if this is a known issue ?

  


  ffmpeg version 4.4.2-0ubuntu0.22.04.1+esm4 Copyright (c) 2000-2021 the FFmpeg developers

  


• output stream video produced by ffmpeg gets divided into sections on screen upon streaming

  22 avril 2024, par Kaustubh Sonawane

  I'm trying to combine cv2 frames with numpy audio array to produce a hls playlist. I'm giving input using pipe. The problem is that while streaming the output hls playlist produced by ffmpeg, the video screen is divided into 4 parts and the sizes of the 4 parts keep getting changed slowly.
Also the audio is extremely distorted.
  
Attaching images for reference :
at timestamp 00:00
at timestamp 00:03

  


  Here is the script I have written until now :

  


  import cv2
import subprocess
import numpy as np
import librosa

def create_hls_playlist(width, height, frames, audio_data, output_playlist, fps=30, sample_rate=44100):
    print(fps)
    print(height, width)
    print(audio_np.dtype)
    video_args = [
        '-y', 
        "-fflags", "+discardcorrupt",
        '-f', 'rawvideo',
        '-vcodec', 'rawvideo',
        '-s', '{}x{}'.format(width, height),
        '-pix_fmt', 'bgr24',
        '-r', str(fps),
        '-i', 'pipe:', 
    ]

    audio_args = [
        '-f', 's16le',  
        '-ar', str(sample_rate),  
        '-ac', '1',  
        '-i', 'pipe:', 
    ]


    output_args = [
        "-vf", "scale=1920x1080,setdar=16/9",
        '-an',
        '-c:v', 'libx264',
        '-preset', 'ultrafast',
        '-f', 'hls',
        '-hls_time', '2',
        '-hls_list_size', '0',
        #'-loglevel', 'debug',
        output_playlist,
    ]

    ffmpeg_command = ['ffmpeg'] + video_args + output_args

    ffmpeg_process = subprocess.Popen(ffmpeg_command, stdin=subprocess.PIPE)

    try:
        for frame in frames[:10]:
            print(f"Frame shape: {frame.shape}, Frame data type: {frame.dtype}")
            # Check for any NaN or infinite values in the frame
            if np.isnan(frame).any() or np.isinf(frame).any():
                print("Frame contains NaN or infinite values!")
            ffmpeg_process.stdin.write(frame.to_string())

    except Exception as e:
        print(f"Error occurred while writing frames: {e}")
    
    # Write audio data to ffmpeg process
    audio_bytes = audio_data
    ffmpeg_process.stdin.write(audio_bytes)

    for frame in frames[10:]:
      ffmpeg_process.stdin.write(frame)
    
    ffmpeg_process.stdin.flush()

    ffmpeg_process.stdin.close()

    ffmpeg_process.wait()

    ffmpeg_process.stdin.close()


def capture_frames(video_path):
    frames = []
    cap = cv2.VideoCapture(video_path)
    width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
    height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

    while cap.isOpened():
        ret, frame = cap.read()
        if not ret:
            break
        frames.append(frame)

    cap.release()
    return frames, width, height

def load_audio(file_path, duration=None, sr=44100):
    audio_data, _ = librosa.load(file_path, sr=sr, duration=duration)
    print("finished audio-----------------------------------")
    print(audio_data)
    audio_duration = librosa.get_duration(y=audio_data, sr=sr)

    if duration is not None and audio_duration > duration:
        start_sample = 0
        end_sample = int(duration * sr)
        audio_data = audio_data[start_sample:end_sample]
    if np.isnan(audio_data).any() or np.isinf(audio_data).any():
        raise ValueError("Audio data contains NaN or infinite values")

    # Normalize audio data
    # audio_data /= np.max(np.abs(audio_data))
    audio_data = (audio_data * 32767).astype(np.int16)
    print(f"Audio data shape: {audio_data.shape}")
    print(f"Audio data type: {audio_data.dtype}")
    return audio_data

def get_video_frame_rate(video_path):
    cap = cv2.VideoCapture(video_path)
    if not cap.isOpened():
        raise ValueError(f"Unable to open the video file: {video_path}")

    frame_rate = cap.get(cv2.CAP_PROP_FPS)
    print(frame_rate)
    cap.release()
    return frame_rate

video_path = 'sample.mp4'

output_playlist = 'output.m3u8'
mp3_file_path = 'audio.mp3'
duration = 11  # seconds
sample_rate = 44100  # Hz


if __name__ == "__main__":
    frames, width, height = capture_frames(video_path)
    audio_np = load_audio(mp3_file_path, duration=duration, sr=sample_rate)
    create_hls_playlist(width, height, frames, audio_np, output_playlist)


  


  And the output :

  


  brain@brainywire:~/k/ffmpeg-e$ /bin/python3 /home//ffmpeg-e/stream_f.py
finished audio-----------------------------------
[1.7182100e-06 1.8538897e-06 1.9645518e-06 ... 5.4211184e-02 5.2549735e-02
 5.0875921e-02]
Audio data shape: (485100,)
Audio data type: int16
30
720 1280
int16
Frame shape: (720, 1280, 3), Frame data type: uint8
Error occurred while writing frames: 'numpy.ndarray' object has no attribute 'to_string'
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, rawvideo, from 'pipe:':
  Duration: N/A, start: 0.000000, bitrate: 663552 kb/s
  Stream #0:0: Video: rawvideo (BGR[24] / 0x18524742), bgr24, 1280x720, 663552 kb/s, 30 tbr, 30 tbn, 30 tbc
Guessed Channel Layout for Input Stream #1.0 : mono
Input #1, s16le, from 'pipe:':
  Duration: N/A, bitrate: 705 kb/s
  Stream #1:0: Audio: pcm_s16le, 44100 Hz, mono, s16, 705 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> h264 (libx264))
  Stream #1:0 -> #0:1 (pcm_s16le (native) -> aac (native))
[libx264 @ 0x5557bf4369c0] using SAR=1/1
[libx264 @ 0x5557bf4369c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x5557bf4369c0] profile High 4:4:4 Predictive, level 4.0, 4:4:4, 8-bit
[libx264 @ 0x5557bf4369c0] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=6 threads=18 lookahead_threads=3 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=25 scenecut=0 intra_refresh=0 rc=crf mbtree=0 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
[s16le @ 0x5557bf3fcf80] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
Output #0, hls, to 'output.m3u8':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264, yuv444p(tv, progressive), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 90k tbn
    Metadata:
      encoder         : Lavc58.134.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  Stream #0:1: Audio: aac (LC), 44100 Hz, mono, fltp, 69 kb/s
    Metadata:
      encoder         : Lavc58.134.100 aac
[rawvideo @ 0x5557bf3e8640] Thread message queue blocking; consider raising the thread_queue_size option (current value: 8)
[hls @ 0x5557bf4335c0] Opening 'output0.ts' for writingtrate=N/A speed=5.06x    
[hls @ 0x5557bf4335c0] Opening 'output.m3u8.tmp' for writing
[rawvideo @ 0x5557bf3e8640] Packet corrupt (stream = 0, dts = 345), dropping it.
[hls @ 0x5557bf4335c0] Opening 'output1.ts' for writingtrate=N/A speed=5.31x    
[hls @ 0x5557bf4335c0] Opening 'output.m3u8.tmp' for writing
frame=  345 fps=166 q=-1.0 Lsize=N/A time=00:00:11.50 bitrate=N/A speed=5.52x    
video:21389kB audio:95kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[libx264 @ 0x5557bf4369c0] frame I:2     Avg QP:21.00  size:666528
[libx264 @ 0x5557bf4369c0] frame P:343   Avg QP:24.78  size: 59968
[libx264 @ 0x5557bf4369c0] mb I  I16..4: 100.0%  0.0%  0.0%
[libx264 @ 0x5557bf4369c0] mb P  I16..4: 22.8%  0.0%  0.0%  P16..4: 51.1%  0.0%  0.0%  0.0%  0.0%    skip:26.1%
[libx264 @ 0x5557bf4369c0] coded y,u,v intra: 24.0% 3.7% 3.3% inter: 30.6% 4.9% 4.8%
[libx264 @ 0x5557bf4369c0] i16 v,h,dc,p: 43% 38% 10%  9%
[libx264 @ 0x5557bf4369c0] kb/s:15236.21
[aac @ 0x5557bf438400] Qavg: 118.713


  


  I noticed this packet corrupt error and after some digging on stackoverflow found a flag to drop corrupt packets. But that didn't solve the problem.

  


  If I do not drop the corrupted packets, I get this error, if it helps to give any more info :

  


  [rawvideo @ 0x61a72571ab80] PACKET SIZE: 2764800, STRIDE: 3840
[libx264 @ 0x61a725754c40] frame= 323 QP=24.00 NAL=2 Slice:P Poc:146 I:2340 P:4474 SKIP:1346 size=62313 bytes
[rawvideo @ 0x61a72571ab80] PACKET SIZE: 2764800, STRIDE: 3840
[libx264 @ 0x61a725754c40] frame= 324 QP=24.00 NAL=2 Slice:P Poc:148 I:1538 P:4449 SKIP:2173 size=56970 bytes
[rawvideo @ 0x61a72571ab80] PACKET SIZE: 2764800, STRIDE: 3840
[libx264 @ 0x61a725754c40] frame= 325 QP=24.00 NAL=2 Slice:P Poc:150 I:2553 P:4310 SKIP:1297 size=64782 bytes
pipe:: corrupt input packet in stream 0
[rawvideo @ 0x61a72571ab80] PACKET SIZE: 2751960, STRIDE: 3822
[rawvideo @ 0x61a72571ab80] Invalid buffer size, packet size 2751960 < expected frame_size 2764800
Error while decoding stream #0:0: Invalid argument
[out_0_0 @ 0x61a72575f400] EOF on sink link out_0_0:default.
[out_0_1 @ 0x61a726577f80] EOF on sink link out_0_1:default.
No more output streams to write to, finishing.
[libx264 @ 0x61a725754c40] frame= 326 QP=24.00 NAL=2 Slice:P Poc:152 I:1076 P:4002 SKIP:3082 size=44541 bytes
[libx264 @ 0x61a725754c40] frame= 327 QP=24.00 NAL=2 Slice:P Poc:154 I:2493 P:4545 SKIP:1122 size=62617 bytes
[libx264 @ 0x61a725754c40] frame= 328 QP=24.00 NAL=2 Slice:P Poc:156 I:1383 P:4328 SKIP:2449 size=50782 bytes
[libx264 @ 0x61a725754c40] frame= 329 QP=24.00 NAL=2 Slice:P Poc:158 I:2179 P:4376 SKIP:1605 size=57972 bytes
[libx264 @ 0x61a725754c40] frame= 330 QP=24.00 NAL=2 Slice:P Poc:160 I:925  P:3772 SKIP:3463 size=37181 bytes


  


  I tried to give bufsize flags too, but even that wouldnt help.

  


  I'm a total beginner to ffmpeg. I tried to give the correct resolution, tried all pix_fmt from the documentation, and also experimented with different values of fps. But the problem with split screen and audio distortion continues.

  


• FFMPEG decode from RTP dump file into mp3 file

  5 février 2021, par pingvincible

  I'm trying to save RTP stream into mp3 file. I use this command :

  


  ffmpeg -loglevel debug -protocol_whitelist file -f rtp -i microphone.rtpdump -f mp3 microphone.mp3


  


  I get this result :

  


  user@pc:~/$ ffmpeg-amrnb -loglevel debug -protocol_whitelist file -f rtp -i microphone.rtpdump -f mp3 microphone.mp3
ffmpeg version N-100958-g4f3d8cb554 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-17ubuntu1~20.04)
  configuration: --enable-gpl --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-librtmp --enable-libtheora --enable-libvorbis --enable-nonfree --enable-version3
  libavutil      56. 64.100 / 56. 64.100
  libavcodec     58.120.100 / 58.120.100
  libavformat    58. 65.101 / 58. 65.101
  libavdevice    58. 11.103 / 58. 11.103
  libavfilter     7.102.100 /  7.102.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100
  libpostproc    55.  8.100 / 55.  8.100
Splitting the commandline.
Reading option '-loglevel' ... matched as option 'loglevel' (set logging level) with argument 'debug'.
Reading option '-protocol_whitelist' ... matched as AVOption 'protocol_whitelist' with argument 'file'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'rtp'.
Reading option '-i' ... matched as input url with argument 'microphone.rtpdump'.
Reading option '-f' ... matched as option 'f' (force format) with argument 'mp3'.
Reading option 'microphone.mp3' ... matched as output url.
Finished splitting the commandline.
Parsing a group of options: global .
Applying option loglevel (set logging level) with argument debug.
Successfully parsed a group of options.
Parsing a group of options: input url microphone.rtpdump.
Applying option f (force format) with argument rtp.
Successfully parsed a group of options.
Opening an input file: microphone.rtpdump.
[rtp @ 0x556947200580] Unable to receive RTP payload type 97 without an SDP file describing it
microphone.rtpdump: Invalid data found when processing input


  


  It looks like that microphone.rtpdump file format is correct as ffmpeg can find RTP payload type 97. The problem is that I don't understand how to use SDP file in this situation.

  


  I have an SDP file for this payload type which I use, when I send data over network. It looks like this :

  


  v=0
o=- 0 0 IN IP4 127.0.0.1
s=No Name
c=IN IP4 127.0.0.1
t=0 0
a=tool:libavformat 58.65.101
m=audio 1234 RTP/AVP 97
b=AS:12
a=rtpmap:97 AMR/8000/1
a=fmtp:97 octet-align=1


  


  And now I want to decode RTP stream from file, not by receiving it from network.

  


  How to adapt my SDP file to read RTP stream from file ?

  


  UPDATE : My rtpdump file is not a real rtpdump file format. It is just payloads from UDP packets written into file without any additional headers.