Recherche avancée

Sur d’autres sites (8156)

• Transcoding WAV audio to AAC in an MP4 container using FFmpeg C API

  7 novembre 2022, par vstrom coder

  I'm trying to compose source video and audio into a final MP4 video.
I have a problem with the WAV audio. After decoding and filtering, I'm getting an error from the output encoder : [aac @ 0x145e04c40] more samples than frame size

  


  I initially used the following filter graph (minimal reproducible example) :

  


      abuffer -> aformat -> abuffersink


  


  At this point I was getting the error mentioned above.

  


  Then, I tried to insert a aresample filter to the graph :

  


      abuffer -> aresample -> aformat -> abuffersink


  


  But still getting the same error.
This was based on the fact that the ffmpeg CLI uses this filter when converting WAV to MP4 :

  


  Command :

  


      ffmpeg -i source.wav output.mp4 -loglevel debug


  


  Output contains :

  


      [graph_0_in_0_0 @ 0x138f06200] Setting 'time_base' to value '1/44100'
    [graph_0_in_0_0 @ 0x138f06200] Setting 'sample_rate' to value '44100'
    [graph_0_in_0_0 @ 0x138f06200] Setting 'sample_fmt' to value 's16'
    [graph_0_in_0_0 @ 0x138f06200] Setting 'channel_layout' to value 'mono'
    [graph_0_in_0_0 @ 0x138f06200] tb:1/44100 samplefmt:s16 samplerate:44100 chlayout:mono
    [format_out_0_0 @ 0x138f06620] Setting 'sample_fmts' to value 'fltp'
    [format_out_0_0 @ 0x138f06620] Setting 'sample_rates' to value '96000|88200|64000|48000|44100|32000|24000|22050|16000|12000|11025|8000|7350'
    [format_out_0_0 @ 0x138f06620] auto-inserting filter 'auto_aresample_0' between the filter 'Parsed_anull_0' and the filter 'format_out_0_0'
    [AVFilterGraph @ 0x138f060f0] query_formats: 4 queried, 6 merged, 3 already done, 0 delayed
    [auto_aresample_0 @ 0x138f06c30] [SWR @ 0x120098000] Using s16p internally between filters
    [auto_aresample_0 @ 0x138f06c30] ch:1 chl:mono fmt:s16 r:44100Hz -> ch:1 chl:mono fmt:fltp r:44100Hz
    Output #0, mp4, to 'output.mp4':
      Metadata:
        encoder         : Lavf59.27.100
      Stream #0:0, 0, 1/44100: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, delay 1024, 69 kb/s
        Metadata:
          encoder         : Lavc59.37.100 aac


  


  I'm trying to figure out whether I should use the SWR library directly as exemplified in the transcode_aac example.

  


• Documented ffmpeg commands not recognized by ffmpeg

  2 avril 2020, par agconti

  I'm trying to use options like, ldash and http_opts, as the dash muxer docs describe but FFmpeg doesn't recognize them. I'm on the latest released version of ffmpeg, v4.2.2. I see the changes in the ffmpeg master branch but not in the v4.2 release branch. Does ffmpeg not recognize them because they haven't been released yet ?

  



  Here's the dash muxer docs for reference : https://ffmpeg.org/ffmpeg-all.html#dash-2

  



  Here's a minimal example command with uncut output :

  



  Andrews-MacBook-Pro :: dev/test ‹master› » ffmpeg -re -i test.mp4 \                                   
-map 0 -map 0 -c:a libfdk_aac -c:v libx264 \
-b:v:0 800k -b:v:1 300k -s:v:1 320x170 -profile:v:1 baseline \
-profile:v:0 main -bf 1 \
-b_strategy 0 -ar:a:1 22050 \
-adaptation_sets "id=0,streams=v id=1,streams=a" \
-ldash 1 \
-f dash ./output/out.mpd

ffmpeg version 4.2.2 Copyright (c) 2000-2019 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Unrecognized option 'ldash'.
Error splitting the argument list: Option not found


  


• How to efficiently store variable frame rate video stream in a pyqt application ?

  1er octobre 2024, par Jeroen De Geeter

  I am developing a PyQT (PySide6) application that needs to display and store multiple camera streams at the same time. The display of the camera streams goes well, however, storing these streams seems to slow down the application significantly up to a point where the GUI doesn't work fluently anymore.

  


  I have a minimal working example using a stub to demonstrate how my code currently works. However, given that it is a minimal working example, it will not visibly slow down.

  


  import sys
from time import sleep

import av
import numpy as np
import pyqtgraph as pg
from PySide6.QtCore import QThread, Signal, Slot, Qt
from PySide6.QtWidgets import QApplication, QHBoxLayout, QWidget, QVBoxLayout, QPushButton, QGroupBox


class RGBCameraStub(QThread):

    newFrame = Signal(np.ndarray)

    def __init__(self):
        super().__init__()
        self.killSwitch = True

    def stop(self):
        self.killSwitch = False
        self.quit()
        self.wait()

    def run(self):
        self.killSwitch = True
        while self.killSwitch:
            self.newFrame.emit((np.random.rand(1456, 1080, 3) * 255).astype(np.uint8))
            sleep((20 + int(np.random.rand() * 30))/ 1000)


class VideoWriter(QThread):

    def __init__(self):
        super().__init__()
        self.output_container = av.open('output_video.mkv', mode='w')
        self.stream = self.output_container.add_stream('ffv1', rate=None)
        self.stream.width = 1456
        self.stream.height = 1080
        self.stream.pix_fmt = 'yuv420p'

    @Slot(np.ndarray)
    def addFrame(self, frame: np.ndarray):
        av_frame = av.VideoFrame.from_ndarray(frame, format='rgb24')
        av_frame.pts = None # Leave emtpy for auto-handling - variable framerate?
        for packet in self.stream.encode(av_frame):
            self.output_container.mux(packet)

    def stop(self):
        self.output_container.close()
        self.quit()
        self.wait()

    def run(self):
        self.exec()


class VideoBox(QGroupBox):

    def __init__(self, title):
        super().__init__(title=title)
        self.createLayout()
        self.videoWidget.setImage((np.random.rand(1456, 1080, 3) * 255).astype(np.uint8))

    def createLayout(self):
        layout = QVBoxLayout()
        self.videoWidget = pg.RawImageWidget()
        layout.addWidget(self.videoWidget)
        self.setLayout(layout)
        self.setStyleSheet("""QGroupBox {
            border: 1px solid #494B4F;
            margin-top: 8px;
            min-width: 180px;
            min-height: 180px;
            padding: 2px 0px 0px 0px;
            }
        QGroupBox::title {
            color: #aeb0b8;
            subcontrol-origin: margin;
            subcontrol-position: top left;
            left: 20px;
            padding: 0 8px;
        }""")

    def setImage(self, data: np.ndarray):
        self.videoWidget.setImage(data)

class MainWindow(QWidget):

    closeSignal = Signal()

    def __init__(self):
        super().__init__()
        self.setGeometry(0, 0, 900, 720)
        self.createLayout()

    def createLayout(self):
        self.vimbaImage = VideoBox("RGB")
        self.info = self.infoLayout()

        layout = QVBoxLayout()
        layout.addWidget(self.vimbaImage)
        layout.addWidget(self.info)
        self.setLayout(layout)

        self.setAttribute(Qt.WA_StyledBackground, True)
        self.setStyleSheet("MainWindow { background-color: #1e1f22; }")

    def infoLayout(self):
        widget = QWidget()
        layout = QVBoxLayout()

        rgbButtonWidget = QWidget()
        buttonLayout = QHBoxLayout()
        self.connectButton = QPushButton('Connect', parent=self)
        self.disconnectButton = QPushButton('Disconnect', parent=self)
        buttonLayout.addWidget(self.connectButton)
        buttonLayout.addWidget(self.disconnectButton)
        buttonLayout.addStretch()
        rgbButtonWidget.setLayout(buttonLayout)
        layout.addWidget(rgbButtonWidget)

        widget.setLayout(layout)
        return widget

    def closeEvent(self, event):
        self.closeSignal.emit()
        event.accept()



if __name__ == "__main__":
    app = QApplication(sys.argv)

    rgbCamera = RGBCameraStub()
    videoWriter = VideoWriter()
    videoWriter.start()

    main_window = MainWindow()

    # Button connections
    main_window.connectButton.clicked.connect(rgbCamera.start)
    main_window.disconnectButton.clicked.connect(rgbCamera.stop)
    # main_window.disconnectButton.clicked.connect(videoWriter.stop)

    # Display frames
    rgbCamera.newFrame.connect(main_window.vimbaImage.setImage)

    # Write frame to file
    rgbCamera.newFrame.connect(videoWriter.addFrame)

    # Close application
    main_window.closeSignal.connect(rgbCamera.stop)
    main_window.closeSignal.connect(videoWriter.stop)

    main_window.show()
    sys.exit(app.exec())



  


  My question(s) therefore are :

  


  

  • How can I increase the performance of the VideoWriter ? I am currently adding frame by frame as soon as the camera thread provides a new frame. Maybe this is not the best approach ?

  


  • The frame rate of the camera is not completely stable, I therefore set av_frame.pts = None but maybe this is also not the approach to take ?

  


  • With code as is, the resulting media file quickly blows up in size, is there a way of dealing with this without quality loss ?

  


  


  As a side not, I currently use the PyAV wrapper for the FFmpeg libraries, however I am open to other suggestions.