Recherche avancée

Sur d’autres sites (9896)

• How to setup a virtual mic and pipe audio to it from node.js

  28 octobre 2018, par Niellles

  Summary of what I am trying to achieve :

  I’m currently doing some work on a Discord bot. I’m trying to join a voice channel, which is the easy part, and then use the combined audio of the speakers in that voice channel as input for a webpage in a web browser. It doesn’t really matter which browser it is as long as it can be controlled with Selenium.

  ---

  What I’ve tried/looked into so far

  My bot so far is written up in Python using the discord.py API wrapper. Unfortunately listening to, as opposed to putting in, audio hasn’t been exactly implemented great − let alone documented − with discord.py. This made me decide to switch to node.js (i.e. discord.js) for the voice channel stuff of my bot.

  After switching to discord.js it was pretty easy to determine who’s talking and create an audio stream (PCM stream) for that user. For the next part I though I’d just pipe the audio stream to a virtual microphone and select that as the audio input on the browser. You can even use FFMPEG from within node.js ¹, to get something that looks like this :

  const Discord = require("discord.js");
  
  const client = new Discord.Client();
  
  
  
  client.on('ready', () => {
  
    voiceChannel = client.channels.get('SOME_CHANNEL_ID');
  
    voiceChannel.join()
  
      .then(conn => {
  
        console.log('Connected')
  
  
  
        const receiver = conn.createReceiver();
  
  
  
        conn.on('speaking', (user, speaking) => {
  
          if (speaking) {
  
            const audioStream = receiver.createPCMStream(user);
  
  
  
            ffmpeg(stream)
  
                .inputFormat('s32le')
  
                .audioFrequency(16000)
  
                .audioChannels(1)
  
                .audioCodec('pcm_s16le')
  
                .format('s16le')
  
                .pipe(someVirtualMic);          
  
          }
  
        });
  
      })
  
      .catch(console.log);
  
    }); 
  
  
  
  client.login('SOME_TOKEN');

  This last part, creating and streaming to a virtual microphone, has proven to be rather complicated. I’ve read a ton of SO posts and documentation on both The Advanced Linux Sound Architecture (ALSA) and the JACK Audio Connection Kit, but I simply can’t figure out how to setup a virtual microphone that will show up as a mic in my browser, or how to pipe audio to it.

  Any help or pointers to a solution would be greatly appreciated !

  ---

  Addendum

  For the past couple of days I’ve kept on looking into to this issue. I’ve now learned about ALSA loopback devices and feel that the solution must be there.

  I’ve pretty much followed a post that talks about loopback devices and aims to achieve the following :

  Simply imagine that you have a physical link between one OUT and one
  IN of the same device.

  I’ve set up the devices as described in the post and now two new audio devices show up when selecting a microphone in Firefox. I’d expect one, but I that may be because I don’t entirely understand the loopback devices (yet).

  The loop back devices are created and I think that they’re linked (if I understood the aforementioned article correctly). Assuming that’s the case the only problem I have to tackle is streaming the audio via FFMPEG from within node.js.

  

• video parse : ValueError : Attempt to convert a value (None) with an unsupported type () to a Tensor

  16 février 2024, par John Siddarth

  I am trying to learn how to use classifications on video data and i am replicating the code from this website

  


  And the code i am trying to use is :

  


  # The way this tutorial uses the `TimeDistributed` layer requires TF>=2.10
pip install -U "tensorflow>=2.10.0"

pip install remotezip tqdm opencv-python
pip install -q git+https://github.com/tensorflow/docs

import tqdm
import random
import pathlib
import itertools
import collections

import os
import cv2
import numpy as np
import remotezip as rz

import tensorflow as tf

# Some modules to display an animation using imageio.
import imageio
from IPython import display
from urllib import request
from tensorflow_docs.vis import embed

def format_frames(frame, output_size):
  """
    Pad and resize an image from a video.

    Args:
      frame: Image that needs to resized and padded. 
      output_size: Pixel size of the output frame image.

    Return:
      Formatted frame with padding of specified output size.
  """
  frame = tf.image.convert_image_dtype(frame, tf.float32)
  frame = tf.image.resize_with_pad(frame, *output_size)
  return frame

def frames_from_video_file(video_path, n_frames, output_size = (224,224), frame_step = 15):
  """
    Creates frames from each video file present for each category.

    Args:
      video_path: File path to the video.
      n_frames: Number of frames to be created per video file.
      output_size: Pixel size of the output frame image.

    Return:
      An NumPy array of frames in the shape of (n_frames, height, width, channels).
  """
  # Read each video frame by frame
  result = []
  src = cv2.VideoCapture(str(video_path))  

  video_length = src.get(cv2.CAP_PROP_FRAME_COUNT)

  need_length = 1 + (n_frames - 1) * frame_step

  if need_length > video_length:
    start = 0
  else:
    max_start = video_length - need_length
    start = random.randint(0, max_start + 1)

  src.set(cv2.CAP_PROP_POS_FRAMES, start)
  # ret is a boolean indicating whether read was successful, frame is the image itself
  ret, frame = src.read()
  result.append(format_frames(frame, output_size))

  for _ in range(n_frames - 1):
    for _ in range(frame_step):
      ret, frame = src.read()
    if ret:
      frame = format_frames(frame, output_size)
      result.append(frame)
    else:
      result.append(np.zeros_like(result[0]))
  src.release()
  result = np.array(result)[..., [2, 1, 0]]

  return result

curl -O https://upload.wikimedia.org/wikipedia/commons/8/86/End_of_a_jam.ogv

video_path = "End_of_a_jam.ogv"

sample_video = frames_from_video_file(video_path, n_frames = 10)
sample_video.shape


  


  Here's a summary of the troubleshooting steps I attempted :

  


  


  • Check File Path : I verified that the file path to the video was correct and accessible from the Python environment.

    


  


  • Verify File Permissions : I ensured that the video file had the necessary permissions to be read by the Python code.

    


  


  • Test with Absolute Path : I attempted to use an absolute file path to access the video file to eliminate any ambiguity in the file's location.

    


  


  • Check File Format and Encoding : I examined the video file to ensure it was in a supported format and encoded properly for reading by OpenCV.

    


  


  


  Despite attempting these steps, I encountered challenges in accessing and processing the video file.

  


• ffmpeg error : Could not write header for output file #0 (incorrect codec parameters ?) : Muxer not found

  30 janvier 2016, par Benjamin Russell

  I am trying to segment a mp4 video file for HLS playback and I am getting the following error :

  "Could not write header for output file #0 (incorrect codec parameters
   ?) : Muxer not found"

  Does anyone know what I am doing wrong ?

  input command :

  sudo ffmpeg -y -i adwd.mp4 -vcodec copy -acodec copy -force_key_frames 30 -map 0  -f segment segment_list adwd.m3u8 -segment_time 10 -segment_format mpeg_ts -segment_list_type m3u8 segment%d.ts
  
  
  
  
  
      ffmpeg version 2.8.5 Copyright (c) 2000-2016 the FFmpeg developers
  
        built with Apple LLVM version 7.0.2 (clang-700.1.81)
  
        configuration: --prefix=/usr/local/Cellar/ffmpeg/2.8.5 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-opencl --enable-libx264 --enable-libmp3lame --enable-libvo-aacenc --enable-libxvid --enable-vda
  
        libavutil      54. 31.100 / 54. 31.100
  
        libavcodec     56. 60.100 / 56. 60.100
  
        libavformat    56. 40.101 / 56. 40.101
  
        libavdevice    56.  4.100 / 56.  4.100
  
        libavfilter     5. 40.101 /  5. 40.101
  
        libavresample   2.  1.  0 /  2.  1.  0
  
        libswscale      3.  1.101 /  3.  1.101
  
        libswresample   1.  2.101 /  1.  2.101
  
        libpostproc    53.  3.100 / 53.  3.100

  Video information :

  General / Container Stream #1
  
      Total Video Streams for this File.................1
  
      Total Audio Streams for this File.................1
  
      Video Codecs Used.................................AVC
  
      Audio Codecs Used.................................AAC LC
  
      File Format.......................................MPEG-4
  
      Play Time.........................................9h 1mn
  
      Total File Size...................................588 MiB
  
      Total Stream BitRate..............................152 Kbps
  
      Encoded with......................................Lavf56.40.101
  
  Video Stream #1
  
      Codec (Human Name)................................AVC
  
      Codec (FourCC)....................................avc1
  
      Codec Profile.....................................High@L2.1
  
      Frame Width.......................................426 pixels
  
      Frame Height......................................240 pixels
  
      Frame Rate........................................29.970 fps
  
      Total Frames......................................973039
  
      Display Aspect Ratio..............................16:9
  
      Scan Type.........................................Progressive
  
      Color Space.......................................YUV
  
      Codec Settings (Summary)..........................CABAC / 4 Ref Frames
  
      QF (like Gordian Knot)............................0.005
  
      Codec Settings (CABAC)............................Yes
  
      Codec Settings (Reference Frames).................4
  
      Video Stream Length...............................9h 1mn 7s 68ms
  
      Video Stream BitRate..............................15.2 Kbps
  
      Video Stream BitRate Mode.........................VBR
  
      Bit Depth.........................................8 bits
  
      Video Stream Size.................................58.9 MiB (10%)
  
      Video Encoder.....................................x264 - core 148 r2601 a0cd7d3
  
      Video Encoder (Settings)..........................cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00
  
  Audio Stream #1
  
      Codec.............................................AAC
  
      Codec (FourCC)....................................40
  
      Audio Stream Length...............................9h 1mn 7s 104ms
  
      Audio Stream BitRate..............................128 Kbps
  
      Audio Stream BitRate Mode.........................CBR
  
      Number of Audio Channels..........................2
  
      Audio Channel's Positions.........................Front: L R
  
      Sampling Rate.....................................44.1 KHz
  
      Audio Stream Size.................................495 MiB (84%)