Newest 'ffmpeg' Questions - Stack Overflow

Les articles publiés sur le site

• Why does my yuva444p video always lose alpha channel after transcoding to vp9 ?

  1er avril, par Ian Ye

  I'm trying to convert a video with alpha channel using FFmpeg, but the transparency information gets lost in the output.

  Here is the input video (prores, yuva444p) detail with ffprobe:

   Stream #0:1[0x2](und): Video: prores (4444) (ap4h / 0x68347061), yuva444p12le(bt709), 360x480, 73314 kb/s, 60.13 fps, 60 tbr, 600 tbn (default)
        Metadata:
          creation_time   : 2019-05-21T21:23:03.000000Z
          handler_name    : Core Media Video
          vendor_id       : appl
          encoder         : Apple ProRes 4444
  

  And my command is below:

  ffmpeg -i alpha_prores.mov -c:v libvpx-vp9 -pix_fmt yuva420p -auto-alt-ref 0 out.webm
  

  There is the output info:

  Output #0, webm, to 'out.webm':
    Metadata:
      major_brand     : qt
      minor_version   : 0
      compatible_brands: qt
      com.apple.quicktime.creationdate: 2019-05-14T13:47:17-0700
      com.apple.quicktime.location.ISO6709: +37.3367-122.0094/
      com.apple.quicktime.make: Apple
      com.apple.quicktime.model: iPhone X
      com.apple.quicktime.software: 12.1.2
      encoder         : Lavf61.7.100
    Stream #0:0(und): Video: vp9, yuva420p(tv, bt709, progressive), 360x480, q=2-31, 60 fps, 1k tbn (default)
        Metadata:
          creation_time   : 2019-05-21T21:23:03.000000Z
          handler_name    : Core Media Video
          vendor_id       : appl
          encoder         : Lavc61.19.101 libvpx-vp9
        Side data:
          cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
  

  We can see pix_fmt is still yuva420p, but after finished, I use ffprobe to check it, and the result changed!

    Stream #0:0: Video: vp9 (Profile 0), yuv420p(tv, bt709, progressive), 360x480, SAR 1:1 DAR 3:4, 60 fps, 60 tbr, 1k tbn (default)
        Metadata:
          alpha_mode      : 1
          HANDLER_NAME    : Core Media Video
          VENDOR_ID       : appl
          ENCODER         : Lavc61.19.101 libvpx-vp9
          DURATION        : 00:00:05.950000000
  

  The pix_fmt turn to yuv420p, which means lose alpha channel. Also we can see there is an extra info in metadata alpha_mode: 1.

  By the way, the result webm video can display its transparent pixel correctly in browser.

  When I try to overlay it on a mp4 video, transparent pixels turn to black pixels:

  ffmpeg -i background.mp4 -i out.webm -filter_complex "[0][1]overlay=x=10:y=10" output2.mp4
  

  And when I overlay the prores video on the same mp4, result is correct.

  ffmpeg -i background.mp4 -i alpha_prores.mov -filter_complex "[0][1]overlay=x=10:y=10" output2.mp4
  

  So I wonder the difference between yuva and alpha_mode:1.

  And the reason why vp9 would lose alpha channel when overlay on a mp4 video.

  It's better if you can provide other pixel format for my purpose(I want it can display correctly when overlay on mp4)!

  Thanks!

• How can I build a custom version of opencv while enabling CUDA and opengl ?

  1er avril, par Josh

  I have a hard requirement of python3.7 for certain libraries (aeneas & afaligner). I've been using the regular opencv-python and ffmpeg libraries in my program and they've been working find.

  Recently I wanted to adjust my program to use h264 instead of mpeg4 and ran down a licensing rabbit hole of how opencv-python uses a build of ffmpeg with opengl codecs off to avoid licensing issues. x264 is apparently opengl, and is disabled in the opencv-python library.

  In order to solve this issue, I built a custom build of opencv using another custom build of ffmpeg both with opengl enabled. This allowed me to use the x264 encoder with the VideoWriter in my python program.

  Here's the dockerfile of how I've been running it:

  FROM python:3.7-slim
  
  # Set optimization flags and number of cores globally
  ENV CFLAGS="-O3 -march=native -ffast-math -flto -fno-fat-lto-objects -ffunction-sections -fdata-sections" \
      CXXFLAGS="-O3 -march=native -ffast-math -flto -fno-fat-lto-objects -ffunction-sections -fdata-sections" \
      LDFLAGS="-flto -fno-fat-lto-objects -Wl,--gc-sections" \
      MAKEFLAGS="-j\$(nproc)"
  
  # Combine all system dependencies in a single layer
  RUN apt-get update && apt-get install -y --no-install-recommends \
      build-essential \
      cmake \
      git \
      wget \
      unzip \
      yasm \
      pkg-config \
      libsm6 \
      libxext6 \
      libxrender-dev \
      libglib2.0-0 \
      libavcodec-dev \
      libavformat-dev \
      libswscale-dev \
      libavutil-dev \
      libswresample-dev \
      nasm \
      mercurial \
      libnuma-dev \
      espeak \
      libespeak-dev \
      libtiff5-dev \
      libjpeg62-turbo-dev \
      libopenjp2-7-dev \
      zlib1g-dev \
      libfreetype6-dev \
      liblcms2-dev \
      libwebp-dev \
      tcl8.6-dev \
      tk8.6-dev \
      python3-tk \
      libharfbuzz-dev \
      libfribidi-dev \
      libxcb1-dev \
      python3-dev \
      python3-setuptools \
      libsndfile1 \
      libavdevice-dev \
      libavfilter-dev \
      libpostproc-dev \
      && apt-get clean \
      && rm -rf /var/lib/apt/lists/*
  
  # Build x264 with optimizations
  RUN cd /tmp && \
      wget https://code.videolan.org/videolan/x264/-/archive/master/x264-master.tar.bz2 && \
      tar xjf x264-master.tar.bz2 && \
      cd x264-master && \
      ./configure \
          --enable-shared \
          --enable-pic \
          --enable-asm \
          --enable-lto \
          --enable-strip \
          --enable-optimizations \
          --bit-depth=8 \
          --disable-avs \
          --disable-swscale \
          --disable-lavf \
          --disable-ffms \
          --disable-gpac \
          --disable-lsmash \
          --extra-cflags="-O3 -march=native -ffast-math -fomit-frame-pointer -flto -fno-fat-lto-objects" \
          --extra-ldflags="-O3 -flto -fno-fat-lto-objects" && \
      make && \
      make install && \
      cd /tmp && \
      # Build FFmpeg with optimizations
      wget https://ffmpeg.org/releases/ffmpeg-7.1.tar.bz2 && \
      tar xjf ffmpeg-7.1.tar.bz2 && \
      cd ffmpeg-7.1 && \
      ./configure \
          --enable-gpl \
          --enable-libx264 \
          --enable-shared \
          --enable-nonfree \
          --enable-pic \
          --enable-asm \
          --enable-optimizations \
          --enable-lto \
          --enable-pthreads \
          --disable-debug \
          --disable-static \
          --disable-doc \
          --disable-ffplay \
          --disable-ffprobe \
          --disable-filters \
          --disable-programs \
          --disable-postproc \
          --extra-cflags="-O3 -march=native -ffast-math -fomit-frame-pointer -flto -fno-fat-lto-objects -ffunction-sections -fdata-sections" \
          --extra-ldflags="-O3 -flto -fno-fat-lto-objects -Wl,--gc-sections" \
          --prefix=/usr/local && \
      make && \
      make install && \
      ldconfig && \
      rm -rf /tmp/*
  
  # Install Python dependencies first
  RUN pip install --no-cache-dir --upgrade pip setuptools wheel && \
      pip install --no-cache-dir numpy py-spy
  
  # Build OpenCV with optimized configuration
  RUN cd /tmp && \
      # Download specific OpenCV version archives
      wget -O opencv.zip https://github.com/opencv/opencv/archive/4.8.0.zip && \
      wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.8.0.zip && \
      unzip opencv.zip && \
      unzip opencv_contrib.zip && \
      mv opencv-4.8.0 opencv && \
      mv opencv_contrib-4.8.0 opencv_contrib && \
      rm opencv.zip opencv_contrib.zip && \
      cd opencv && \
      mkdir build && cd build && \
      cmake \
          -D CMAKE_BUILD_TYPE=RELEASE \
          -D CMAKE_C_FLAGS="-O3 -march=native -ffast-math -flto -fno-fat-lto-objects -ffunction-sections -fdata-sections" \
          -D CMAKE_CXX_FLAGS="-O3 -march=native -ffast-math -flto -fno-fat-lto-objects -ffunction-sections -fdata-sections -Wno-deprecated" \
          -D CMAKE_EXE_LINKER_FLAGS="-flto -fno-fat-lto-objects -Wl,--gc-sections" \
          -D CMAKE_SHARED_LINKER_FLAGS="-flto -fno-fat-lto-objects -Wl,--gc-sections" \
          -D CMAKE_INSTALL_PREFIX=/usr/local \
          -D ENABLE_FAST_MATH=ON \
          -D CPU_BASELINE_DETECT=ON \
          -D CPU_BASELINE=SSE3 \
          -D CPU_DISPATCH=SSE4_1,SSE4_2,AVX,AVX2,AVX512_SKX,FP16 \
          -D WITH_OPENMP=ON \
          -D OPENCV_ENABLE_NONFREE=ON \
          -D WITH_FFMPEG=ON \
          -D FFMPEG_ROOT=/usr/local \
          -D OPENCV_EXTRA_MODULES_PATH=/tmp/opencv_contrib/modules \
          -D PYTHON_EXECUTABLE=/usr/local/bin/python3.7 \
          -D PYTHON3_EXECUTABLE=/usr/local/bin/python3.7 \
          -D PYTHON3_INCLUDE_DIR=/usr/local/include/python3.7m \
          -D PYTHON3_LIBRARY=/usr/local/lib/libpython3.7m.so \
          -D PYTHON3_PACKAGES_PATH=/usr/local/lib/python3.7/site-packages \
          -D PYTHON3_NUMPY_INCLUDE_DIRS=/usr/local/lib/python3.7/site-packages/numpy/core/include \
          -D BUILD_opencv_python3=ON \
          -D INSTALL_PYTHON_EXAMPLES=OFF \
          -D BUILD_TESTS=OFF \
          -D BUILD_PERF_TESTS=OFF \
          -D BUILD_EXAMPLES=OFF \
          -D BUILD_DOCS=OFF \
          -D BUILD_opencv_apps=OFF \
          -D WITH_OPENCL=OFF \
          -D WITH_CUDA=OFF \
          -D WITH_IPP=OFF \
          -D WITH_TBB=OFF \
          -D WITH_V4L=OFF \
          -D WITH_QT=OFF \
          -D WITH_GTK=OFF \
          -D BUILD_LIST=core,imgproc,imgcodecs,videoio,python3 \
          .. && \
      make && \
      make install && \
      ldconfig && \
      rm -rf /tmp/*
  
  # Set working directory and copy application code
  WORKDIR /app
  
  COPY requirements.txt .
  
  RUN apt-get update && apt-get install -y --no-install-recommends ffmpeg
  
  RUN pip install --no-cache-dir aeneas afaligner && \
      pip install --no-cache-dir -r requirements.txt
  
  COPY . .
  
  # Make entrypoint executable
  RUN chmod +x entrypoint.sh
  ENTRYPOINT ["./entrypoint.sh"]
  

  My trouble now, is I've been considering running parts of my program on my GPU, it's creating graphics for a video after all. I have no idea how to edit my Dockerfile to make the opencv build run with CUDA enabled, every combination I try leads to issues.

  How can I tell which version of CUDA, opencv and ffmpeg are compatible with python 3.7?

• FFMPEG demuxer seek error in Chrome range slider with AWS audio file

  31 mars, par Tania Rascia

  I'm encountering an issue where if you click or slide enough on a range slider in Chrome, it will eventually stop working and give this error, an error 2 (network error):

  PIPELINE_ERROR_READ: FFmpegDemuxer: demuxer seek failed
  

  If I google this error, I only find the Chrome source code:

  Line 1749 of Chrome source code

  First, this issue only happens in Chrome, not Firefox. Second, I can only get it to happen with an encrypted file from AWS, which I can't get into a sandbox, so the sandbox I made will never encounter that error with the random audio file I used.

  Here's the sandbox.

  The only difference I can find between this code and the failing code is the source of the audio file (AWS).

• How to fix av_interleaved_write_frame() broken pipe error in php

  31 mars, par Adekunle Adeyeye

  I have an issue using ffmpeg to stream audio and parse to google cloud speech to text in PHP.

  It returns this output. I have tried delaying some part of the script, that did not solve it. I have also checked for similar questions. however, they are mostly in python and none of the solutions actually work for this.

    built with gcc 8 (GCC)
    cpudetect
    libavutil      56. 31.100 / 56. 31.100
    libavcodec     58. 54.100 / 58. 54.100
    libavformat    58. 29.100 / 58. 29.100
    libavdevice    58.  8.100 / 58.  8.100
    libavfilter     7. 57.100 /  7. 57.100
    libavresample   4.  0.  0 /  4.  0.  0
    libswscale      5.  5.100 /  5.  5.100
    libswresample   3.  5.100 /  3.  5.100
    libpostproc    55.  5.100 / 55.  5.100
  Input #0, mp3, from 'https://npr-ice.streamguys1.com/live.mp3':
    Metadata:
      icy-br          : 96
      icy-description : NPR Program Stream
      icy-genre       : News and Talk
      icy-name        : NPR Program Stream
      icy-pub         : 0
      StreamTitle     :
    Duration: N/A, start: 0.000000, bitrate: 96 kb/s
      Stream #0:0: Audio: mp3, 32000 Hz, stereo, fltp, 96 kb/s
  Stream mapping:
    Stream #0:0 -> #0:0 (mp3 (mp3float) -> pcm_s16le (native))
  Press [q] to stop, [?] for help
  Output #0, s16le, to 'pipe:':
    Metadata:
      icy-br          : 96
      icy-description : NPR Program Stream
      icy-genre       : News and Talk
      icy-name        : NPR Program Stream
      icy-pub         : 0
      StreamTitle     :
      encoder         : Lavf58.29.100
      Stream #0:0: Audio: pcm_s16le, 16000 Hz, mono, s16, 256 kb/s
      Metadata:
        encoder         : Lavc58.54.100 pcm_s16le
  **av_interleaved_write_frame(): Broken pipe** 256.0kbits/s speed=1.02x
  **Error writing trailer of pipe:: Broken pipe**
  size=      54kB time=00:00:01.76 bitrate= 250.8kbits/s speed=0.465x
  video:0kB audio:55kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
  Conversion failed!
  

  this is my PHP code

  require_once 'vendor/autoload.php';
      
      $projectId = "xxx-45512";
      putenv('GOOGLE_APPLICATION_CREDENTIALS=' . __DIR__ . '/xxx-45512-be3eb805f1d7.json');
      
      // Database connection
      $pdo = new PDO('mysql:host=localhost;dbname=', '', '');
      $pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
      
      $url = "https://npr-ice.streamguys1.com/live.mp3";
      
      $ffmpegCmd = "ffmpeg -re -i $url -acodec pcm_s16le -ac 1 -ar 16000 -f s16le -";
      
      $fp = popen($ffmpegCmd, "r");
      if (!$fp) {
          die("Failed to open FFmpeg stream.");
      }
      sleep(5);
  
      try {
          $client = new SpeechClient(['transport' => 'grpc', 'credentials' => json_decode(file_get_contents(getenv('GOOGLE_APPLICATION_CREDENTIALS')), true)]);
      } catch (Exception $e) {
          echo 'Error: ' . $e->getMessage(); 
          exit;
      }
      
      $recognitionConfig = new RecognitionConfig([
          'auto_decoding_config' => new AutoDetectDecodingConfig(),
          'language_codes' => ['en-US'],
          'model' => 'long',
      ]);
      
      $streamingConfig = new StreamingRecognitionConfig([
          'config' => $recognitionConfig,
      ]);
      
      $configRequest = new StreamingRecognizeRequest([
          'recognizer' => "projects/$projectId/locations/global/recognizers/_",
          'streaming_config' => $streamingConfig,
      ]);
      
      
      function streamAudio($fp)
      {
          while (!feof($fp)) {
              yield fread($fp, 4096);
          }
      }
      
      $responses = $client->streamingRecognize([
      'requests' => (function () use ($configRequest, $fp) {
              yield $configRequest; // Send initial config
              foreach (streamAudio($fp) as $audioChunk) {
                  yield new StreamingRecognizeRequest(['audio' => $audioChunk]);
              }
          })()]
      );
      
      // $responses = $speechClient->streamingRecognize();
      // $responses->writeAll([$request,]);
      
      foreach ($responses as $response) {
          foreach ($response->getResults() as $result) {
              $transcript = $result->getAlternatives()[0]->getTranscript();
              // echo "Transcript: $transcript\n";
      
              // Insert into the database
              $stmt = $pdo->prepare("INSERT INTO transcriptions (transcript) VALUES (:transcript)");
              $stmt->execute(['transcript' => $transcript]);
          }
      }
      
      
      pclose($fp);
      $client->close();
  

  I'm not sure what the issue is at this time.

  UPDATE

  I've done some more debugging and i have gotten the error to clear and to stream actually starts. However, I expect the audio to transcribe and update my database but instead I get this error when i close the stream

  

  this is my updated code

      $handle = popen($ffmpegCommand, "r");
  
      try {
          $client = new SpeechClient(['transport' => 'grpc', 'credentials' => json_decode(file_get_contents(getenv('GOOGLE_APPLICATION_CREDENTIALS')), true)]);
      } catch (Exception $e) {
          echo 'Error: ' . $e->getMessage(); 
          exit;
      }
      
      try {
      $recognitionConfig = (new RecognitionConfig())
          ->setAutoDecodingConfig(new AutoDetectDecodingConfig())
          ->setLanguageCodes(['en-US'], ['en-UK'])
          ->setModel('long');
      } catch (Exception $e) {
          echo 'Error: ' . $e->getMessage(); 
          exit;
      }
      
      try {
          $streamConfig = (new StreamingRecognitionConfig())
          ->setConfig($recognitionConfig);
      } catch (Exception $e) {
          echo 'Error: ' . $e->getMessage();
          exit;
      }
      try {
          $configRequest = (new StreamingRecognizeRequest())
          ->setRecognizer("projects/$projectId/locations/global/recognizers/_")
          ->setStreamingConfig($streamConfig);
      } catch (Exception $e) {
          echo 'Error: ' . $e->getMessage(); 
          exit;
      }
      
      $stream = $client->streamingRecognize();
      $stream->write($configRequest);
      
      mysqli_query($conn, "INSERT INTO transcriptions (transcript) VALUES ('bef')");
      
      while (!feof($handle)) {
          $chunk = fread($handle, 25600);
          // printf('chunk: ' . $chunk);
          if ($chunk !== false) {
              try {
                  $request = (new StreamingRecognizeRequest())
                          ->setAudio($chunk);
                      $stream->write($request);
              } catch (Exception $e) {
                  printf('Errorc: ' . $e->getMessage());
              }
          }
      }
      
      
      $insr = json_encode($stream);
      mysqli_query($conn, "INSERT INTO transcriptions (transcript) VALUES ('$insr')");
      
      foreach ($stream->read() as $response) {
          mysqli_query($conn, "INSERT INTO transcriptions (transcript) VALUES ('loop1')");
          foreach ($response->getResults() as $result) {
              mysqli_query($conn, "INSERT INTO transcriptions (transcript) VALUES ('loop2')");
              foreach ($result->getAlternatives() as $alternative) {
                  $trans = $alternative->getTranscript();
                  mysqli_query($conn, "INSERT INTO transcriptions (transcript) VALUES ('$trans')");
              }
          }
      }
      
      pclose($handle);
      $stream->close();
      $client->close();```
  

• RTSP Frame Grabbing creates smeared , pixeled and corrupted images

  31 mars, par Robob

  I am trying to capture a single frame per second from a RTSP stream with following command ffmpeg -i rtsp://XXX -q:v 1 -vf fps=fps=1 -strftime 1 ZZZZ\%H_%M_%S.jpg

  But some of the frames are smeared ,pixeled and corrupted - this effect is drastically increases if rtsp resolution is increased (if the resolution is decreased for example to 720P most of the frames are OK)

  I have to say that playing same rtsp stream in VLC or FFPLAY is flowless.

  How I can fix it to grab better quality

  Thanks in advance.