Advanced search

On other websites (4558)

• Efficient real-time video stream processing and forwarding with RTMP servers

  19 May 2023, by dumbQuestions

  I have a scenario where I need to retrieve a video stream from an RTMP server, apply image processing (specifically, adding blur to frames), and then forward the processed stream to another RTMP server (in this case, Twitch).

  


  Currently, I'm using ffmpeg in conjunction with cv2 to retrieve and process the stream. However, this approach introduces significant lag when applying the blur. I'm seeking an alternative method that can achieve the desired result more efficiently. I did attempt to solely rely on ffmpeg for the entire process, but I couldn't find a way to selectively process frames based on a given condition and subsequently transmit only those processed frames.

  


  Is there a more efficient approach or alternative solution that can address this issue and enable real-time video stream processing with minimal lag?

  


  Thanks in advance!

  


  def forward_stream(server_url, stream_key, twitch_stream_key):
    get_ffmpeg_command = [...]

    send_ffmpeg_command [...]

    # Start get FFmpeg process
    read_process = subprocess.Popen(get_ffmpeg_command, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)

    # Start send FFmpeg process
    send_process = send_process = subprocess.Popen(send_ffmpeg_command, stdin=subprocess.PIPE, stderr=subprocess.DEVNULL)

    # Open video capture
    cap = cv2.VideoCapture(f'{server_url}')

    while True:
        # Read the frame
        ret, frame = cap.read()
        if ret:
            # Apply machine learning algorithm
            should_blur = machine_learning_algorithm(frame)

            # Apply blur if necessary
            if machine_learning_algorithm(frame):
                frame = cv2.blur(frame, (25, 25))

            # Write the frame to FFmpeg process
            send_process.stdin.write(frame.tobytes())
        else:
            break

    # Release resources
    cap.release()
    read_process.stdin.close()
    read_process.wait()



  


• Transcoding WAV audio to AAC in an MP4 container using FFmpeg C API

  7 November 2022, by vstrom coder

  I'm trying to compose source video and audio into a final MP4 video.
I have a problem with the WAV audio. After decoding and filtering, I'm getting an error from the output encoder: [aac @ 0x145e04c40] more samples than frame size

  


  I initially used the following filter graph (minimal reproducible example):

  


      abuffer -> aformat -> abuffersink


  


  At this point I was getting the error mentioned above.

  


  Then, I tried to insert a aresample filter to the graph:

  


      abuffer -> aresample -> aformat -> abuffersink


  


  But still getting the same error.
This was based on the fact that the ffmpeg CLI uses this filter when converting WAV to MP4:

  


  Command:

  


      ffmpeg -i source.wav output.mp4 -loglevel debug


  


  Output contains:

  


      [graph_0_in_0_0 @ 0x138f06200] Setting 'time_base' to value '1/44100'
    [graph_0_in_0_0 @ 0x138f06200] Setting 'sample_rate' to value '44100'
    [graph_0_in_0_0 @ 0x138f06200] Setting 'sample_fmt' to value 's16'
    [graph_0_in_0_0 @ 0x138f06200] Setting 'channel_layout' to value 'mono'
    [graph_0_in_0_0 @ 0x138f06200] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:mono
    [format_out_0_0 @ 0x138f06620] Setting 'sample_fmts' to value 'fltp'
    [format_out_0_0 @ 0x138f06620] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
    [format_out_0_0 @ 0x138f06620] auto-inserting filter 'auto_aresample_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_0'
    [AVFilterGraph @ 0x138f060f0] query_formats: 4 queried, 6 merged, 3 already done, 0 delayed
    [auto_aresample_0 @ 0x138f06c30] [SWR @ 0x120098000] Using s16p internally between filters
    [auto_aresample_0 @ 0x138f06c30] ch:1 chl:mono fmt:s16 r:44100Hz -> ch:1 chl:mono fmt:fltp r:44100Hz
    Output #0, mp4, to 'output.mp4':
      Metadata:
        encoder         : Lavf59.27.100
      Stream #0:0, 0, 1/44100: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, delay 1024, 69 kb/s
        Metadata:
          encoder         : Lavc59.37.100 aac


  


  I'm trying to figure out whether I should use the SWR library directly as exemplified in the transcode_aac example.

  


• ClassX installation "codec not found" due to dependencies [migrated]

  25 March 2014, by khateeb

  ClassX is an interactive lecture streaming system developed in the Electrical Engineering Department at Stanford University.
  Unlike conventional lecture capturing systems, ClassX requires very simple consumer-grade equipment and minimal human operation.

  I faced problems during installing it, I hope you have a solution.

  BTW: I successfully installed it 2 years ago, but now I think the problem as the dependencies and Ubuntu versions are different than the versions we used two years ago.

  Detailed description of the problem:

  • I’m using Ubuntu 12.04
  • I followed the instructions @ ClassX installation guide, and all steps till step 4 are successfully done (the encoder bin file generated).
  • When trying to encode the video using the classX web system, it shows the encoding completed after few seconds.However, there are no tiles generated.

  • I tried to execute the command at CX_log.txt, and the following error appears.

    mahmoud@Mahmoud-HP-Pavilion-dv5-Notebook-PC: $ sudo perl /var/www/ClassXWebSystem/system/publishers/web/actions/encode.pl "/var/www/ClassXWebSystem/content/encoding/FALL_2013_2014/CS106A_FALL_2013_2014/lecSEven" "/var/www/ClassXWebSystem/content/encoding/FALL_2013_2014/CS106A_FALL_2013_2014/.encoding_1372706251" "/var/www/ClassXWebSystem/content/streaming/FALL_2013_2014/CS106A_FALL_2013_2014/lecSEven" "/var/www/ClassXWebSystem/system/publishers/bin" classx y n n
    [sudo] password for mahmoud:
    00068.jpg
    ..
    .
    00068.mp4
    Input #0, mov,mp4,m4a,3gp,3g2,mj2, from ’/var/www/ClassXWebSystem/content/encoding/FALL_2013_2014/CS106A_FALL_2013_2014/.encoding_1372706251/00068.mp4’:
    Metadata:
    major_brand : isom
    minor_version : 512
    compatible_brands: isomiso2avc1mp41
    encoder : Lavf52.39.0
    Duration: 00:02:30.05, start: 0.000000, bitrate: 8141 kb/s
    Stream #0.0(und): Video: h264, yuv420p, 1920x1080 [PAR 1:1 DAR 16:9], 8007 kb/s, 29.95 fps, 29.97 tbr, 2997 tbn, 59.94 tbc
    Stream #0.1(und): Audio: aac, 44100 Hz, stereo, s16, 128 kb/s
    Output #0, mp4, to ’/var/www/ClassXWebSystem/content/encoding/FALL_2013_2014/CS106A_FALL_2013_2014/.encoding_1372706251/stream0.mp4’:
    Stream #0.0: Video: [0][0][0][0] / 0x0000, yuv420p, 640x360, q=32-36, 64 kb/s, 90k tbn, 14.99 tbc
    codec not found