Recherche avancée

Sur d’autres sites (11982)

• Decoding the h.264 stream from a serial port

  18 mars, par Peter

  I would like to know if there is a reliable way to decode an H.264 NAL stream coming through a serial port using software.

  


  So far, I have managed to decode a single frame using a python script. In this script, I first write the incoming data to a file, and when the end-of-frame marker 00_00_00_01 appears, I display the frame using ffplay.

  


  import serial
import subprocess
import os
import time

ser = serial.Serial('COM3', 115200, timeout=1)
output_file = "output.264"

# Variable to store the ffplay process
ffplay_process = None

# Open the file for writing in binary mode
with open(output_file, "wb") as file:

    print("Writing bytes to output.264. Waiting for the end-of-frame marker 0x00000001.")

    buffer = bytearray()
    marker = b'\x00\x00\x00\x01'

    try:
        while True:
            if ser.in_waiting:  # If there is data in the buffer
                data = ser.read(ser.in_waiting)  # Read all available bytes
                buffer.extend(data)

                # Check if the end-of-frame marker is in the buffer
                while marker in buffer:
                    index = buffer.index(marker) + len(marker)  # Position after the marker
                    frame = buffer[:index]  # Extract the frame
                    buffer = buffer[index:]  # Keep the remaining data

                    print(f"Frame recorded: {len(frame)} bytes")
                    file.write(frame)  # Write the frame to the file
                    file.flush()  # Force writing to disk

                    # Close the ffplay window if it is already open
                    if ffplay_process and ffplay_process.poll() is None:
                        ffplay_process.terminate()
                        ffplay_process.wait()  # Wait for the process to terminate

                    # Play the recorded frame, reopening the window
                    ffplay_process = subprocess.Popen(["ffplay", "-f", "h264", "-i", output_file])

    except KeyboardInterrupt:
        print("\nRecording stopped.")
    finally:
        # Close the serial port and the ffplay process
        ser.close()


  


  However, each time a new end-of-frame marker is detected, the ffplay window closes and reopens to show the next frame. It will flicker when transferring the video. Is there a way to display the frames in the same window for seamless playback when streaming video ?

  


  Or is there a better approach or software that is more suited for this task ? I do not know where to start, so I will be glad for any hints.

  


• Error audio loading when runing Whisper Open AI model

  9 juin 2024, par John mick

  The problem I'm trying to solve is that I can't run Whisper model for some audio, it says something related to audio decoding.

  


  payload.wav: Invalid data found when processing input.
raise RuntimeError(f"Failed to load audio: {e.stderr.decode()}") from e


  


  I tried using the micro-machines.wav and it works fine but when i used other audio it gives me an error.

  


  import whisper

model = whisper.load_model("base")
text=model.transcribe('micro-machines.wav',fp16=False)
print(text)
text=model.transcribe('payload.wav',fp16=False)
print(text)


  


  Error I'm getting for payload :

  


  d:\...\venv\lib\site-packages\whisper\transcribe.py:79: UserWarning: FP16 is not supported on CPU; using FP32 instead&#xA;  warnings.warn("FP16 is not supported on CPU; using FP32 instead")                                                                                        &#xA;Traceback (most recent call last):&#xA;  File "d:\...\venv\lib\site-packages\whisper\audio.py", line 42, in load_audio&#xA;    ffmpeg.input(file, threads=0)                                                                                    &#xA;  File "d:\...\venv\lib\site-packages\ffmpeg\_run.py", line 325, in run        &#xA;    raise Error(&#x27;ffmpeg&#x27;, out, err)                                                                                  &#xA;ffmpeg._run.Error: ffmpeg error (see stderr output for detail)                                                       &#xA;&#xA;The above exception was the direct cause of the following exception:&#xA;&#xA;Traceback (most recent call last):&#xA;  File "C:\....\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main&#xA;    return _run_code(code, main_globals, None,&#xA;  File "C:\.....\Python\Python39\lib\runpy.py", line 87, in _run_code&#xA;    exec(code, run_globals)&#xA;  File "D:\...\venv\Scripts\whisper.exe\__main__.py", line 7, in <module>&#xA;  File "d:\...\venv\lib\site-packages\whisper\transcribe.py", line 314, in cli&#xA;    result = transcribe(model, audio_path, temperature=temperature, **args)&#xA;  File "d:\...\venv\lib\site-packages\whisper\transcribe.py", line 85, in transcribe&#xA;    mel = log_mel_spectrogram(audio)&#xA;  File "d:\...\venv\lib\site-packages\whisper\audio.py", line 111, in log_mel_spectrogram&#xA;    audio = load_audio(audio)&#xA;  File "d:\...\venv\lib\site-packages\whisper\audio.py", line 47, in load_audio&#xA;    raise RuntimeError(f"Failed to load audio: {e.stderr.decode()}") from e&#xA;RuntimeError: Failed to load audio: ffmpeg version 6.0-essentials_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers&#xA;  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)&#xA;  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enab&#xA;le-gmp --enable-lzma --enable-zlib --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-sdl2 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxv&#xA;id --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-libass --enable-libfreetype --enable-libfribidi --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf &#xA;--enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libgme --enable-libopenmpt --enable-libo&#xA;pencore-amrwb --enable-libmp3lame --enable-libtheora --enable-libvo-amrwbenc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enab&#xA;le-librubberband&#xA;  libavutil      58.  2.100 / 58.  2.100&#xA;  libavcodec     60.  3.100 / 60.  3.100&#xA;  libavformat    60.  3.100 / 60.  3.100&#xA;  libavdevice    60.  1.100 / 60.  1.100&#xA;  libavfilter     9.  3.100 /  9.  3.100&#xA;  libswscale      7.  1.100 /  7.  1.100&#xA;  libswresample   4. 10.100 /  4. 10.100&#xA;  libpostproc    57.  1.100 / 57.  1.100&#xA;payload.wav: Invalid data found when processing input&#xA;</module>

  


  I tried searching for solutions and I found one which says It appears that the code failed to load the audio file for some reason and even failed to display that error because e.stderr did not contain a valid UTF-8 string

  


• configure : Separate subsystem for Immersive Audio Model

  3 avril 2024, par Eugene Zemtsov

  configure : Separate subsystem for Immersive Audio Model
  This change allows users to build libavfomat without support
  
  for Immersive Audio Model by specifying —disable-iamf.
  
  It helps to save on binary size in cases where it's important.
  Co-authored-by : James Almer <jamrial@gmail.com>
  
  Signed-off-by : Eugene Zemtsov <eugene@chromium.org>
  
  Signed-off-by : James Almer <jamrial@gmail.com>
  

  • [DH] configure
  • [DH] libavformat/mov.c
  • [DH] libavformat/movenc.c