Recherche avancée

Sur d’autres sites (11577)

• How to use audio frame after decode mp3 file using pyav, ffmpeg, python

  2 janvier 2021, par Long Tran Dai

  I am using using python with pyav, ffmpeg to decode mp3 in the memory. I know there are some other way to do it, like pipe ffmpeg command. However, I would like to explore pyav and ffmpeg API. So I have the following code. It works but the sound is very noisy, although hearable :

  


  import numpy as np&#xA;import av # to convert mp3 to wav using ffmpeg&#xA;import pyaudio # to play music&#xA;&#xA;mp3_path = &#x27;D:/MyProg/python/SauTimThiepHong.mp3&#x27;&#xA;&#xA;def decodeStream(mp3_path):&#xA;  # Run NOT OK&#xA;  &#xA;  container = av.open(mp3_path)&#xA;  stream = next(s for s in container.streams if s.type == &#x27;audio&#x27;)&#xA;  frame_count = 0&#xA;  data = bytearray()&#xA;  for packet in container.demux(stream):&#xA;    # <class>&#xA;    # We need to skip the "flushing" packets that `demux` generates.&#xA;    #if frame_count == 5000 : break         &#xA;    if packet.dts is None:&#xA;        continue&#xA;    for frame in packet.decode():   &#xA;        #&#xA;        # type(frame) : <class>&#xA;        #frame.samples = 1152 : 1152 diem du lieu : Number of audio samples (per channel)&#xA;        # moi frame co size = 1152 (diem) * 2 (channels) * 4 (bytes / diem) = 9216 bytes&#xA;        # 11021 frames&#xA;        #arr = frame.to_ndarray() # arr.nbytes = 9216&#xA;&#xA;        #channels = []  &#xA;        channels = frame.to_ndarray().astype("float16")&#xA;        #for plane in frame.planes:&#xA;            #channels.append(plane.to_bytes()) #plane has 4 bytes / sample, but audio has only 2 bytes&#xA;        #    channels.append(np.frombuffer(plane, dtype=np.single).astype("float16"))&#xA;            #channels.append(np.frombuffer(plane, dtype=np.single)) # kieu np.single co 4 bytes&#xA;        if not frame.is_corrupt:&#xA;            #data.extend(np.frombuffer(frame.planes[0], dtype=np.single).astype("float16")) # 1 channel: noisy&#xA;            # type(planes) : <class>&#xA;            frame_count &#x2B;= 1&#xA;            #print( &#x27;>>>> %04d&#x27; % frame_count, frame)   &#xA;            #if frame_count == 5000 : break     &#xA;            # mix channels:&#xA;            for i in range(frame.samples):                &#xA;                for ch in channels: # dec_ctx->channels&#xA;                    data.extend(ch[i]) #noisy&#xA;                    #fwrite(frame->data[ch] &#x2B; data_size*i, 1, data_size, outfile)&#xA;  return bytes(data)&#xA;</class></class></class>

  


  I use pipe ffmpeg to get decoded data to compare and find they are different :

  


  def RunFFMPEG(mp3_path, target_fs = "44100"):
    # Run OK
    import subprocess
    # init command
    ffmpeg_command = ["ffmpeg", "-i", mp3_path,
                   "-ab", "128k", "-acodec", "pcm_s16le", "-ac", "0", "-ar", target_fs, "-map",
                   "0:a", "-map_metadata", "-1", "-sn", "-vn", "-y",
                   "-f", "wav", "pipe:1"]
    # excute ffmpeg command
    pipe = subprocess.run(ffmpeg_command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, bufsize= 10**8)
    # debug
    #print(pipe.stdout, pipe.stderr)
    # read signal as numpy array and assign sampling rate
    #audio_np = np.frombuffer(buffer=pipe.stdout, dtype=np.uint16, offset=44)
    #audio_np = np.frombuffer(buffer=pipe.stdout, dtype=np.uint16)
    #sig, fs  = audio_np, target_fs
    #return audio_np
    return pipe.stdout[78:]     


  


  Then I use pyaudio to play data and find it very noisy

  


  p = pyaudio.PyAudio()
streamOut = p.open(format=pyaudio.paInt16, channels=2, rate= 44100, output=True)
#streamOut = p.open(format=pyaudio.paInt16, channels=1, rate= 44100, output=True)

mydata = decodeStream(mp3_path)
print("bytes of mydata = ", len(mydata))
#print("bytes of mydata = ", mydata.nbytes)

ffMpegdata = RunFFMPEG(mp3_path)
print("bytes of ffMpegdata = ", len(ffMpegdata)) 
#print("bytes of ffMpegdata = ", ffMpegdata.nbytes)

minlen = min(len(mydata), len(ffMpegdata))
print("mydata == ffMpegdata", mydata[:minlen] == ffMpegdata[:minlen]) # ffMpegdata.tobytes()[:minlen] )

#bytes of mydata =  50784768
#bytes of ffMpegdata =  50784768
#mydata == ffMpegdata False

streamOut.write(mydata)
streamOut.write(ffMpegdata)
streamOut.stop_stream()
streamOut.close()
p.terminate()


  


  Please help me to understand decoded frame of pyav api (after for frame in packet.decode() :). Should it be processed more ? or I have some error ?

  


  It makes me crazy for 3 days. I could not guess where to go.

  


  Thank you very much.

  


• Precise method of segmenting & transcoding video+audio (via ffmpeg), into an on-demand HLS stream ?

  17 novembre 2019, par Felix

  recently I’ve been messing around with FFMPEG and streams through Nodejs. My ultimate goal is to serve a transcoded video stream - from any input filetype - via HTTP, generated in real-time as it’s needed in segments.

  I’m currently attempting to handle this using HLS. I pre-generate a dummy m3u8 manifest using the known duration of the input video. It contains a bunch of URLs that point to individual constant-duration segments. Then, once the client player starts requesting the individual URLs, I use the requested path to determine which time range of video the client needs. Then I transcode the video and stream that segment back to them.

  Now for the problem : This approach mostly works, but has a small audio bug. Currently, with most test input files, my code produces a video that - while playable - seems to have a very small (< .25 second) audio skip at the start of each segment.

  I think this may be an issue with splitting using time in ffmpeg, where possibly the audio stream cannot be accurately sliced at the exact frame the video is. So far, I’ve been unable to figure out a solution to this problem.

  If anybody has any direction they can steer me - or even a prexisting library/server that solves this use-case - I appreciate the guidance. My knowledge of video encoding is fairly limited.

  I’ll include an example of my relevant current code below, so others can see where I’m stuck. You should be able to run this as a Nodejs Express server, then point any HLS player at localhost:8080/master to load the manifest and begin playback. See the transcode.get('/segment/:seg.ts' line at the end, for the relevant transcoding bit.

  'use strict';
  
  const express = require('express');
  
  const ffmpeg = require('fluent-ffmpeg');
  
  let PORT = 8080;
  
  let HOST = 'localhost';
  
  const transcode = express();
  
  
  
  
  
  /*
  
   * This file demonstrates an Express-based server, which transcodes & streams a video file.
  
   * All transcoding is handled in memory, in chunks, as needed by the player.
  
   *
  
   * It works by generating a fake manifest file for an HLS stream, at the endpoint "/m3u8".
  
   * This manifest contains links to each "segment" video clip, which browser-side HLS players will load as-needed.
  
   *
  
   * The "/segment/:seg.ts" endpoint is the request destination for each clip,
  
   * and uses FFMpeg to generate each segment on-the-fly, based off which segment is requested.
  
   */
  
  
  
  
  
  const pathToMovie = 'C:\\input-file.mp4';  // The input file to stream as HLS.
  
  const segmentDur = 5; //  Controls the duration (in seconds) that the file will be chopped into.
  
  
  
  
  
  const getMetadata = async(file) => {
  
      return new Promise( resolve => {
  
          ffmpeg.ffprobe(file, function(err, metadata) {
  
              console.log(metadata);
  
              resolve(metadata);
  
          });
  
      });
  
  };
  
  
  
  
  
  
  
  // Generate a "master" m3u8 file, which the player should point to:
  
  transcode.get('/master', async(req, res) => {
  
      res.set({"Content-Disposition":"attachment; filename=\"m3u8.m3u8\""});
  
      res.send(`#EXTM3U
  
  #EXT-X-STREAM-INF:BANDWIDTH=150000
  
  /m3u8?num=1
  
  #EXT-X-STREAM-INF:BANDWIDTH=240000
  
  /m3u8?num=2`)
  
  });
  
  
  
  // Generate an m3u8 file to emulate a premade video manifest. Guesses segments based off duration.
  
  transcode.get('/m3u8', async(req, res) => {
  
      let met = await getMetadata(pathToMovie);
  
      let duration = met.format.duration;
  
  
  
      let out = '#EXTM3U\n' +
  
          '#EXT-X-VERSION:3\n' +
  
          `#EXT-X-TARGETDURATION:${segmentDur}\n` +
  
          '#EXT-X-MEDIA-SEQUENCE:0\n' +
  
          '#EXT-X-PLAYLIST-TYPE:VOD\n';
  
  
  
      let splits = Math.max(duration / segmentDur);
  
      for(let i=0; i< splits; i++){
  
          out += `#EXTINF:${segmentDur},\n/segment/${i}.ts\n`;
  
      }
  
      out+='#EXT-X-ENDLIST\n';
  
  
  
      res.set({"Content-Disposition":"attachment; filename=\"m3u8.m3u8\""});
  
      res.send(out);
  
  });
  
  
  
  // Transcode the input video file into segments, using the given segment number as time offset:
  
  transcode.get('/segment/:seg.ts', async(req, res) => {
  
      const segment = req.params.seg;
  
      const time = segment * segmentDur;
  
  
  
      let proc = new ffmpeg({source: pathToMovie})
  
          .seekInput(time)
  
          .duration(segmentDur)
  
          .outputOptions('-preset faster')
  
          .outputOptions('-g 50')
  
          .outputOptions('-profile:v main')
  
          .withAudioCodec('aac')
  
          .outputOptions('-ar 48000')
  
          .withAudioBitrate('155k')
  
          .withVideoBitrate('1000k')
  
          .outputOptions('-c:v h264')
  
          .outputOptions(`-output_ts_offset ${time}`)
  
          .format('mpegts')
  
          .on('error', function(err, st, ste) {
  
              console.log('an error happened:', err, st, ste);
  
          }).on('progress', function(progress) {
  
              console.log(progress);
  
          })
  
          .pipe(res, {end: true});
  
  });
  
  
  
  transcode.listen(PORT, HOST);
  
  console.log(`Running on http://${HOST}:${PORT}`);

• How do I download the image with metadata in Python ? [closed]

  19 octobre 2024, par Temp Account

  I am downloading some images in Python from the Airtable API and am trying to make a slideshow with them using ffmpeg. I download the images :

  &#xA;

  urllib2.urlretrieve(img['url'], "output/images/image_"+str(i)+".jpeg")


  &#xA;

  However, when I run the following ffmpeg command

  &#xA;

  ffmpeg -framerate 4/60 -i output/images/image_%d.jpeg output/out.mp4


  &#xA;

  I get the following error :

  &#xA;

  ffmpeg version 6.1.1-3ubuntu5 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 13 (Ubuntu 13.2.0-23ubuntu3)
  configuration: --prefix=/usr --extra-version=3ubuntu5 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-l
inux-gnu --arch=amd64 --enable-gpl --disable-stripping --disable-omx --enable-gnutls --enable-libaom --enable-libass --enable-libbs2b --enable
-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribi
di --enable-libglslang --enable-libgme --enable-libgsm --enable-libharfbuzz --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enab
le-libopenmpt --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheo
ra --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libx
vid --enable-libzimg --enable-openal --enable-opencl --enable-opengl --disable-sndio --enable-libvpl --disable-libmfx --enable-libdc1394 --ena
ble-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-ladspa --enable-libbluray --enable-libjack --enable-libpulse --enable-librabbitmq --enable-librist --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libx264 --enable-libzmq --enable-libzvbi --enab
le-lv2 --enable-sdl2 --enable-libplacebo --enable-librav1e --enable-pocketsphinx --enable-librsvg --enable-libjxl --enable-shared
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
[mjpeg @ 0x593283e5e3c0] bits 150 is invalid
[mjpeg @ 0x593283e5e3c0] bits 28 is invalid
[image2 @ 0x593283e5d380] Could not find codec parameters for stream 0 (Video: mjpeg (Lossless), none(bt470bg/unknown/unknown), lossless): uns
pecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, image2, from 'output/images/image_%d.jpeg':
  Duration: 00:01:00.00, start: 0.000000, bitrate: N/A
  Stream #0:0: Video: mjpeg (Lossless), none(bt470bg/unknown/unknown), lossless, 0.07 fps, 0.07 tbr, 0.07 tbn
File 'output/out.mp4' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[mjpeg @ 0x593283e5f180] mjpeg: unsupported coding type (cf)
[mjpeg @ 0x593283e5f180] mjpeg: unsupported coding type (c8)
[mjpeg @ 0x593283e5f180] bits 150 is invalid
[vist#0:0/mjpeg @ 0x593283e5f000] Error submitting packet to decoder: Invalid data found when processing input
[mjpeg @ 0x593283e5f180] bits 28 is invalid
[vist#0:0/mjpeg @ 0x593283e5f000] Error submitting packet to decoder: Invalid data found when processing input
[mjpeg @ 0x593283e5f180] mjpeg: unsupported coding type (ce)
[mjpeg @ 0x593283e5f180] mjpeg: unsupported coding type (c6)
[mjpeg @ 0x593283e5f180] unable to decode APP fields: Invalid data found when processing input
    Last message repeated 1 times
[vist#0:0/mjpeg @ 0x593283e5f000] Error submitting packet to decoder: Invalid data found when processing input
[mjpeg @ 0x593283e5f180] unable to decode APP fields: Invalid data found when processing input
[mjpeg @ 0x593283e5f180] invalid id 255
[vist#0:0/mjpeg @ 0x593283e5f000] Error submitting packet to decoder: Invalid data found when processing input
Cannot determine format of input stream 0:0 after EOF
Error marking filters as finished
Error while filtering: Invalid data found when processing input
[vist#0:0/mjpeg @ 0x593283e5f000] Decode error rate 1 exceeds maximum 0.666667
[out#0/mp4 @ 0x593283e603c0] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0kB time=N/A bitrate=N/A speed=N/A    
Conversion failed!



  &#xA;

  However, downloading the images in Chrome then creating the slideshow is successful. The images from Chrome have metadata of the filetype (JPEG), width and height. The images downloaded with Python have no metadata. How do I download that information so that my ffmpeg command will succeed ?

  &#xA;

  Thanks !

  &#xA;