Recherche avancée
Autres articles (38)
-
Publier sur MédiaSpip
13 juin 2013Puis-je poster des contenus à partir d’une tablette Ipad ?
Oui, si votre Médiaspip installé est à la version 0.2 ou supérieure. Contacter au besoin l’administrateur de votre MédiaSpip pour le savoir -
HTML5 audio and video support
13 avril 2011, parMediaSPIP uses HTML5 video and audio tags to play multimedia files, taking advantage of the latest W3C innovations supported by modern browsers.
The MediaSPIP player used has been created specifically for MediaSPIP and can be easily adapted to fit in with a specific theme.
For older browsers the Flowplayer flash fallback is used.
MediaSPIP allows for media playback on major mobile platforms with the above (...) -
De l’upload à la vidéo finale [version standalone]
31 janvier 2010, parLe chemin d’un document audio ou vidéo dans SPIPMotion est divisé en trois étapes distinctes.
Upload et récupération d’informations de la vidéo source
Dans un premier temps, il est nécessaire de créer un article SPIP et de lui joindre le document vidéo "source".
Au moment où ce document est joint à l’article, deux actions supplémentaires au comportement normal sont exécutées : La récupération des informations techniques des flux audio et video du fichier ; La génération d’une vignette : extraction d’une (...)
Sur d’autres sites (5847)
-
How to improve web camera streaming latency to v4l2loopback device with ffmpeg ?
11 mars, par Made by MosesI'm trying to stream my iPhone camera to my PC on LAN.


What I've done :


- 

-
HTTP server with html page and streaming script :


I use WebSockets here and maybe WebRTC is better choice but it seems like network latency is good enough






async function beginCameraStream() {
 const mediaStream = await navigator.mediaDevices.getUserMedia({
 video: { facingMode: "user" },
 });

 websocket = new WebSocket(SERVER_URL);

 websocket.onopen = () => {
 console.log("WS connected");

 const options = { mimeType: "video/mp4", videoBitsPerSecond: 1_000_000 };
 mediaRecorder = new MediaRecorder(mediaStream, options);

 mediaRecorder.ondataavailable = async (event) => {
 // to measure latency I prepend timestamp to the actual video bytes chunk
 const timestamp = Date.now();
 const timestampBuffer = new ArrayBuffer(8);
 const dataView = new DataView(timestampBuffer);
 dataView.setBigUint64(0, BigInt(timestamp), true);
 const data = await event.data.bytes();

 const result = new Uint8Array(data.byteLength + 8);
 result.set(new Uint8Array(timestampBuffer), 0);
 result.set(data, 8);

 websocket.send(result);
 };

 mediaRecorder.start(100); // Collect 100ms chunks
 };
}


- 

-
Server to process video chunks






import { serve } from "bun";
import { Readable } from "stream";

const V4L2LOOPBACK_DEVICE = "/dev/video10";

export const setupFFmpeg = (v4l2device) => {
 // prettier-ignore
 return spawn("ffmpeg", [
 '-i', 'pipe:0', // Read from stdin
 '-pix_fmt', 'yuv420p', // Pixel format
 '-r', '30', // Target 30 fps
 '-f', 'v4l2', // Output format
 v4l2device, // Output to v4l2loopback device
 ]);
};

export class FfmpegStream extends Readable {
 _read() {
 // This is called when the stream wants more data
 // We push data when we get chunks
 }
}

function main() {
 const ffmpeg = setupFFmpeg(V4L2LOOPBACK_DEVICE);
 serve({
 port: 8000,
 fetch(req, server) {
 if (server.upgrade(req)) {
 return; // Upgraded to WebSocket
 }
 },
 websocket: {
 open(ws) {
 console.log("Client connected");
 const stream = new FfmpegStream();
 stream.pipe(ffmpeg?.stdin);

 ws.data = {
 stream,
 received: 0,
 };
 },
 async message(ws, message) {
 const view = new DataView(message.buffer, 0, 8);
 const ts = Number(view.getBigUint64(0, true));
 ws.data.received += message.byteLength;
 const chunk = new Uint8Array(message.buffer, 8, message.byteLength - 8);

 ws.data.stream.push(chunk);

 console.log(
 [
 `latency: ${Date.now() - ts} ms`,
 `chunk: ${message.byteLength}`,
 `total: ${ws.data.received}`,
 ].join(" | "),
 );
 },
 },
 });
}

main();


After I try to open the v4l2loopback device


cvlc v4l2:///dev/video10


picture is delayed for at least 1.5 sec which is unacceptable for my project.


Thoughts :


- 

- Problem doesn't seems to be with network latency




latency: 140 ms | chunk: 661 Bytes | total: 661 Bytes
latency: 206 ms | chunk: 16.76 KB | total: 17.41 KB
latency: 141 ms | chunk: 11.28 KB | total: 28.68 KB
latency: 141 ms | chunk: 13.05 KB | total: 41.74 KB
latency: 199 ms | chunk: 11.39 KB | total: 53.13 KB
latency: 141 ms | chunk: 16.94 KB | total: 70.07 KB
latency: 139 ms | chunk: 12.67 KB | total: 82.74 KB
latency: 142 ms | chunk: 13.14 KB | total: 95.88 KB


150ms is actually too much for 15KB on LAN but there can some issue with my router


- 

- As far as I can tell it neither ties to ffmpeg throughput :




Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'pipe:0':
 Metadata:
 major_brand : iso5
 minor_version : 1
 compatible_brands: isomiso5hlsf
 creation_time : 2025-03-09T17:16:49.000000Z
 Duration: 00:00:01.38, start:
0.000000, bitrate: N/A
 Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuvj420p(pc), 1280x720, 4012 kb/s, 57.14 fps, 29.83 tbr, 600 tbn, 1200 tbc (default)
 Metadata:
 rotate : 90
 creation_time : 2025-03-09T17:16:49.000000Z
 handler_name : Core Media Video
 Side data:
 displaymatrix: rotation of -90.00 degrees

Stream mapping:
 Stream #0:0 -> #0:0 (h264 (native) -> rawvideo (native))

[swscaler @ 0x55d8d0b83100] deprecated pixel format used, make sure you did set range correctly

Output #0, video4linux2,v4l2, to '/dev/video10':
 Metadata:
 major_brand : iso5
 minor_version : 1
 compatible_brands: isomiso5hlsf
 encoder : Lavf58.45.100

Stream #0:0(und): Video: rawvideo (I420 / 0x30323449), yuv420p, 720x1280, q=2-31, 663552 kb/s, 60 fps, 60 tbn, 60 tbc (default)
 Metadata:
 encoder : Lavc58.91.100 rawvideo
 creation_time : 2025-03-09T17:16:49.000000Z
 handler_name : Core Media Video
 Side data:
 displaymatrix: rotation of -0.00 degrees

frame= 99 fps=0.0 q=-0.0 size=N/A time=00:00:01.65 bitrate=N/A dup=50 drop=0 speed=2.77x
frame= 137 fps=114 q=-0.0 size=N/A time=00:00:02.28 bitrate=N/A dup=69 drop=0 speed=1.89x
frame= 173 fps= 98 q=-0.0 size=N/A time=00:00:02.88 bitrate=N/A dup=87 drop=0 speed=1.63x
frame= 210 fps= 86 q=-0.0 size=N/A time=00:00:03.50 bitrate=N/A dup=105 drop=0 speed=1.44x
frame= 249 fps= 81 q=-0.0 size=N/A time=00:00:04.15 bitrate=N/A dup=125 drop=0 speed=1.36
frame= 279 fps= 78 q=-0.0 size=N/A time=00:00:04.65 bitrate=N/A dup=139 drop=0 speed=1.31x


- 

-
I also tried to write the video stream directly to
video.mp4file and immediately open it withvlcbut it only can be successfully opened after 1.5 sec.

-
I've tried to use OBS v4l2 input source instead of vlc but the latency is the same








Update №1


When i try to stream actual
.mp4file toffmpegit works almost immediately with 0.2sec delay to spin up the ffmpeg itself :

cat video.mp4 | ffmpeg -re -i pipe:0 -pix_fmt yuv420p -f v4l2 /dev/video10 & ; sleep 0.2 && cvlc v4l2:///dev/video10


So the problem is apparently with streaming process


-
-
how to add audio using ffmpeg when recording video from browser and streaming to Youtube/Twitch ?
26 juillet 2021, par Tosh VelagaI have a web application I am working on that allows the user to stream video from their browser and simultaneously livestream to both Youtube and Twitch using ffmpeg. The application works fine when I don't need to send any of the audio. Currently I am getting the error below when I try to record video and audio. I am new to using ffmpeg and so any help would be greatly appreciated. Here is also my repo if needed : https://github.com/toshvelaga/livestream


Here is my node.js server with ffmpeg


const child_process = require('child_process') // To be used later for running FFmpeg
const express = require('express')
const http = require('http')
const WebSocketServer = require('ws').Server
const NodeMediaServer = require('node-media-server')
const app = express()
const cors = require('cors')
const path = require('path')
const logger = require('morgan')
require('dotenv').config()

app.use(logger('dev'))
app.use(cors())

app.use(express.json({ limit: '200mb', extended: true }))
app.use(
 express.urlencoded({ limit: '200mb', extended: true, parameterLimit: 50000 })
)

var authRouter = require('./routes/auth')
var compareCodeRouter = require('./routes/compareCode')

app.use('/', authRouter)
app.use('/', compareCodeRouter)

if (process.env.NODE_ENV === 'production') {
 // serve static content
 // npm run build
 app.use(express.static(path.join(__dirname, 'client/build')))

 app.get('*', (req, res) => {
 res.sendFile(path.join(__dirname, 'client/build', 'index.html'))
 })
}

const PORT = process.env.PORT || 8080

app.listen(PORT, () => {
 console.log(`Server is starting on port ${PORT}`)
})

const server = http.createServer(app).listen(3000, () => {
 console.log('Listening on PORT 3000...')
})


const wss = new WebSocketServer({
 server: server,
})

wss.on('connection', (ws, req) => {
 const ffmpeg = child_process.spawn('ffmpeg', [
 // works fine when I use this but when I need audio problems arise
 // '-f',
 // 'lavfi',
 // '-i',
 // 'anullsrc',

 '-i',
 '-',

 '-f',
 'flv',
 '-c',
 'copy',
 `${process.env.TWITCH_STREAM_ADDRESS}`,
 '-f',
 'flv',
 '-c',
 'copy',
 `${process.env.YOUTUBE_STREAM_ADDRESS}`,
 // '-f',
 // 'flv',
 // '-c',
 // 'copy',
 // `${process.env.FACEBOOK_STREAM_ADDRESS}`,
 ])

 ffmpeg.on('close', (code, signal) => {
 console.log(
 'FFmpeg child process closed, code ' + code + ', signal ' + signal
 )
 ws.terminate()
 })

 ffmpeg.stdin.on('error', (e) => {
 console.log('FFmpeg STDIN Error', e)
 })

 ffmpeg.stderr.on('data', (data) => {
 console.log('FFmpeg STDERR:', data.toString())
 })

 ws.on('message', (msg) => {
 console.log('DATA', msg)
 ffmpeg.stdin.write(msg)
 })

 ws.on('close', (e) => {
 console.log('kill: SIGINT')
 ffmpeg.kill('SIGINT')
 })
})

const config = {
 rtmp: {
 port: 1935,
 chunk_size: 60000,
 gop_cache: true,
 ping: 30,
 ping_timeout: 60,
 },
 http: {
 port: 8000,
 allow_origin: '*',
 },
}

var nms = new NodeMediaServer(config)
nms.run()


Here is my frontend code that records the video/audio and sends to server :


import React, { useState, useEffect, useRef } from 'react'
import Navbar from '../../components/Navbar/Navbar'
import './Dashboard.css'

const CAPTURE_OPTIONS = {
 audio: true,
 video: true,
}

function Dashboard() {
 const [mute, setMute] = useState(false)
 const videoRef = useRef()
 const ws = useRef()
 const mediaStream = useUserMedia(CAPTURE_OPTIONS)

 let liveStream
 let liveStreamRecorder

 if (mediaStream && videoRef.current && !videoRef.current.srcObject) {
 videoRef.current.srcObject = mediaStream
 }

 const handleCanPlay = () => {
 videoRef.current.play()
 }

 useEffect(() => {
 ws.current = new WebSocket(
 window.location.protocol.replace('http', 'ws') +
 '//' + // http: -> ws:, https: -> wss:
 'localhost:3000'
 )

 ws.current.onopen = () => {
 console.log('WebSocket Open')
 }

 return () => {
 ws.current.close()
 }
 }, [])

 const startStream = () => {
 liveStream = videoRef.current.captureStream(30) // 30 FPS
 liveStreamRecorder = new MediaRecorder(liveStream, {
 mimeType: 'video/webm;codecs=h264',
 videoBitsPerSecond: 3 * 1024 * 1024,
 })
 liveStreamRecorder.ondataavailable = (e) => {
 ws.current.send(e.data)
 console.log('send data', e.data)
 }
 // Start recording, and dump data every second
 liveStreamRecorder.start(1000)
 }

 const stopStream = () => {
 liveStreamRecorder.stop()
 ws.current.close()
 }

 const toggleMute = () => {
 setMute(!mute)
 }

 return (
 <>
 <navbar></navbar>
 <div style="{{" classname="'main'">
 <div>
 
 </div>
 <div classname="'button-container'">
 <button>Go Live</button>
 <button>Stop Recording</button>
 <button>Share Screen</button>
 <button>Mute</button>
 </div>
 </div>
 >
 )
}

const useUserMedia = (requestedMedia) => {
 const [mediaStream, setMediaStream] = useState(null)

 useEffect(() => {
 async function enableStream() {
 try {
 const stream = await navigator.mediaDevices.getUserMedia(requestedMedia)
 setMediaStream(stream)
 } catch (err) {
 console.log(err)
 }
 }

 if (!mediaStream) {
 enableStream()
 } else {
 return function cleanup() {
 mediaStream.getVideoTracks().forEach((track) => {
 track.stop()
 })
 }
 }
 }, [mediaStream, requestedMedia])

 return mediaStream
}

export default Dashboard


-
python subprocess ffmpeg return code = 69
13 juin 2023, par Tim ChenI try to call ffmpeg through the
subprocess.run(['ffmpeg', '-i', file_name, output_file_name], capture_output=True, text=True)command in python to convert the audio file incoming from the front end to wav format file. The backend code is as follows, using python+fastapi :

@app.post("/api/upload/convert")
async def convert_upload_file(request: Request, file: UploadFile = File(...)):
 token = uuid.uuid4().hex
 tmpFileName = os.path.join(os.path.dirname(__file__), token)
 with open(tmpFileName, "wb") as buffer:
 buffer.write(await file.read())
 await file.seek(0)
 output_path = tmpFileName + '-output.wav'
 command = ['ffmpeg', '-i', tmpFileName, output_path]
 result = subprocess.run(command, capture_output=True, text=True)


This code usually works, but there are some scenarios where it doesn't work. The audio file is recorded by js code (specifically
navigator.mediaDevices.getUserMedia({audio: true})).
The code of the audio recorded in windows chrome can run normally and get the converted wav file, but the audio recorded from ios15 safari for more than 3 seconds cannot be converted, promptingreturncode=69. The error message is as follows :

CompletedProcess(args=['ffmpeg', '-i', '5cfb52c503a646bda0f422b517c8014a', '5cfb52c503a646bda0f422b517c8014a-output.wav'], returncode=69, stdout='', stderr="
ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 70.100 / 56. 70.100
libavcodec 58.134.100 / 58.134.100
libavformat 58. 76.100 / 58. 76.100
libavdevice 58. 13.100 / 58. 13.100
libavfilter 7.110.100 / 7.110.100
libswscale 5. 9.100 / 5. 9.100
libswresample 3. 9.100 / 3. 9.100
libpostproc 55. 9.100 / 55. 9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '5cfb52c503a646bda0f422b517c8014a':
 Metadata:
 major_brand : iso5
 minor_version : 1
 compatible_brands: isomiso5hlsf
 creation_time : 2023-06-11T16:36:53.000000Z
 Duration: 00:00:07.06, start: 0.000000, bitrate: 187 kb/s
 Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 184 kb/s (default)
 Metadata:
 creation_time : 2023-06-11T16:36:53.000000Z
 handler_name : Core Media Audio
 vendor_id : [0][0][0][0]
Stream mapping:
 Stream #0:0 -> #0:0 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Output #0, wav, to '5cfb52c503a646bda0f422b517c8014a-output.wav':
 Metadata:
 major_brand : iso5
 minor_version : 1
 compatible_brands: isomiso5hlsf
 ISFT : Lavf58.76.100
 Stream #0:0(und): Audio: pcm_s16le ([1][0][0][0] / 0x0001), 48000 Hz, mono, s16, 768 kb/s (default)
 Metadata:
 creation_time : 2023-06-11T16:36:53.000000Z
 handler_name : Core Media Audio
 vendor_id : [0][0][0][0]
 encoder : Lavc58.134.100 pcm_s16le
size= 2kB time=00:00:00.00 bitrate=N/A speed=N/A 
[aac @ 0x55f1f8f19fc0] Sample rate index in program config element does not match the sample rate index configured by the container.
[aac @ 0x55f1f8f19fc0] Too large remapped id is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[aac @ 0x55f1f8f19fc0] If you want to help, upload a sample of this file to https://streams.videolan.org/upload/ and contact the ffmpeg-devel mailing list. (ffmpeg-devel@ffmpeg.org)
Error while decoding stream #0:0: Not yet implemented in FFmpeg, patches welcome
[aac @ 0x55f1f8f19fc0] Multiple frames in a packet.
[aac @ 0x55f1f8f19fc0] Reserved bit set.
[aac @ 0x55f1f8f19fc0] Number of bands (18) exceeds limit (13).
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0x55f1f8f19fc0] Reserved bit set.
[aac @ 0x55f1f8f19fc0] Prediction is not allowed in AAC-LC.
Error while decoding stream #0:0: Invalid data found when processing input
[aac @ 0x55f1f8f19fc0] Reserved bit set.


For the abnormal code, I tried to execute
ffmpeg -i input output.wavafter fastapi handle request on the command line andsubprocess.run(['ffmpeg', '-i', file_name, output_path], capture_output =True, text=True), all succeeded, which means that the final file must be normal, otherwise the subsequent verification work will get the same error.

This confuses me, is there some information I'm missing ?


Syndiquer tout le site
