
Recherche avancée
Médias (1)
-
Rennes Emotion Map 2010-11
19 octobre 2011, par
Mis à jour : Juillet 2013
Langue : français
Type : Texte
Autres articles (80)
-
Personnaliser en ajoutant son logo, sa bannière ou son image de fond
5 septembre 2013, parCertains thèmes prennent en compte trois éléments de personnalisation : l’ajout d’un logo ; l’ajout d’une bannière l’ajout d’une image de fond ;
-
Ecrire une actualité
21 juin 2013, parPrésentez les changements dans votre MédiaSPIP ou les actualités de vos projets sur votre MédiaSPIP grâce à la rubrique actualités.
Dans le thème par défaut spipeo de MédiaSPIP, les actualités sont affichées en bas de la page principale sous les éditoriaux.
Vous pouvez personnaliser le formulaire de création d’une actualité.
Formulaire de création d’une actualité Dans le cas d’un document de type actualité, les champs proposés par défaut sont : Date de publication ( personnaliser la date de publication ) (...) -
Le profil des utilisateurs
12 avril 2011, parChaque utilisateur dispose d’une page de profil lui permettant de modifier ses informations personnelle. Dans le menu de haut de page par défaut, un élément de menu est automatiquement créé à l’initialisation de MediaSPIP, visible uniquement si le visiteur est identifié sur le site.
L’utilisateur a accès à la modification de profil depuis sa page auteur, un lien dans la navigation "Modifier votre profil" est (...)
Sur d’autres sites (9578)
-
Node.js readable maximize throughput/performance for compute intense readable - Writable doesn't pull data fast enough
31 décembre 2022, par flohallGeneral setup


I developed an application using AWS Lambda node.js 14.
I use a custom
Readable
implementationFrameCreationStream
that uses node-canvas to draw images, svgs and more on a canvas. This result is then extracted as a raw image buffer in BGRA. A single image buffer contains 1920 * 1080 * 4 Bytes = 8294400 Bytes 8 MB.
This is then piped tostdin
of achild_process
runningffmpeg
.
ThehighWaterMark
of myReadable
inobjectMode:true
is set to 25 so that the internal buffer can use up to 8 MB * 25 = 200 MB.

All this works fine and also doesn't contain too much RAM. But I noticed after some time, that the performance is not ideally.


Performance not optimal


I have an example input that generates a video of 315 frames. If I set
highWaterMark
to a value above 25 the performance increases to the point, when I set to a value of 315 or above.

For some reason
ffmpeg
doesn't start to pull any data untilhighWaterMark
is reached. Obviously thats not what I want.ffmpeg
should always consume data if minimum 1 frame is cached in theReadable
and if it has finished processing the frame before. And theReadable
should produce more frames as longhighWaterMark
isn't reached or the last frame has been reached. So ideally theReadable
and theWriteable
are busy all the time.

I found another way to improve the speed. If I add a timeout in the
_read()
method of theReadable
after let's say every tenth frame for 100 ms. Then theffmpeg
-Writable
will use this timeout to write some frames toffmpeg
.

It seems like frames aren't passed to
ffmpeg
during frame creation because some node.js main thread is busy ?

The fastest result I have if I increase
highWaterMark
above the amount of frames - which doesn't work for longer videos as this would make the AWS Lambda RAM explode. And this makes the whole streaming idea useless. Using timeouts always gives me stomach pain. Also depending on the execution on different environments a good fitting timeout might differ. Any ideas ?

FrameCreationStream


import canvas from 'canvas';
import {Readable} from 'stream';
import {IMAGE_STREAM_BUFFER_SIZE, PerformanceUtil, RenderingLibraryError, VideoRendererInput} from 'vm-rendering-backend-commons';
import {AnimationAssets, BufferType, DrawingService, FullAnimationData} from 'vm-rendering-library';

/**
 * This is a proper back pressure compatible implementation of readable for a having a stream to read single frames from.
 * Whenever read() is called a new frame is created and added to the stream.
 * read() will be called internally until options.highWaterMark has been reached.
 * then calling read will be paused until one frame is read from the stream.
 */
export class FrameCreationStream extends Readable {

 drawingService: DrawingService;
 endFrameIndex: number;
 currentFrameIndex: number = 0;
 startFrameIndex: number;
 frameTimer: [number, number];
 readTimer: [number, number];
 fullAnimationData: FullAnimationData;

 constructor(animationAssets: AnimationAssets, fullAnimationData: FullAnimationData, videoRenderingInput: VideoRendererInput, frameTimer: [number, number]) {
 super({highWaterMark: IMAGE_STREAM_BUFFER_SIZE, objectMode: true});

 this.frameTimer = frameTimer;
 this.readTimer = PerformanceUtil.startTimer();

 this.fullAnimationData = fullAnimationData;

 this.startFrameIndex = Math.floor(videoRenderingInput.startFrameId);
 this.currentFrameIndex = this.startFrameIndex;
 this.endFrameIndex = Math.floor(videoRenderingInput.endFrameId);

 this.drawingService = new DrawingService(animationAssets, fullAnimationData, videoRenderingInput, canvas);
 console.time("read");
 }

 /**
 * this method is only overwritten for debugging
 * @param size
 */
 read(size?: number): string | Buffer {

 console.log("read("+size+")");
 const buffer = super.read(size);
 console.log(buffer);
 console.log(buffer?.length);
 if(buffer) {
 console.timeLog("read");
 }
 return buffer;
 }

 // _read() will be called when the stream wants to pull more data in.
 // _read() will be called again after each call to this.push(dataChunk) once the stream is ready to accept more data. https://nodejs.org/api/stream.html#readable_readsize
 // this way it is ensured, that even though this.createImageBuffer() is async, only one frame is created at a time and the order is kept
 _read(): void {
 // as frame numbers are consecutive and unique, we have to draw each frame number (also the first and the last one)
 if (this.currentFrameIndex <= this.endFrameIndex) {
 PerformanceUtil.logTimer(this.readTimer, 'WAIT -> READ\t');
 this.createImageBuffer()
 .then(buffer => this.optionalTimeout(buffer))
 // push means adding a buffered raw frame to the stream
 .then((buffer: Buffer) => {
 this.readTimer = PerformanceUtil.startTimer();
 // the following two frame numbers start with 1 as first value
 const processedFrameNumberOfScene = 1 + this.currentFrameIndex - this.startFrameIndex;
 const totalFrameNumberOfScene = 1 + this.endFrameIndex - this.startFrameIndex;
 // the overall frameId or frameIndex starts with frameId 0
 const processedFrameIndex = this.currentFrameIndex;
 this.currentFrameIndex++;
 this.push(buffer); // nothing besides logging should happen after calling this.push(buffer)
 console.log(processedFrameNumberOfScene + ' of ' + totalFrameNumberOfScene + ' processed - full video frameId: ' + processedFrameIndex + ' - buffered frames: ' + this.readableLength);
 })
 .catch(err => {
 // errors will be finally handled, when subscribing to frameCreation stream in ffmpeg service
 // this log is just generated for tracing errors and if for some reason the handling in ffmpeg service doesn't work
 console.log("createImageBuffer: ", err);
 this.emit("error", err);
 });
 } else {
 // push(null) makes clear that this stream has ended
 this.push(null);
 PerformanceUtil.logTimer(this.frameTimer, 'FRAME_STREAM');
 }
 }

 private optionalTimeout(buffer: Buffer): Promise<buffer> {
 if(this.currentFrameIndex % 10 === 0) {
 return new Promise(resolve => setTimeout(() => resolve(buffer), 140));
 }
 return Promise.resolve(buffer);
 }

 // prevent memory leaks - without this lambda memory will increase with every call
 _destroy(): void {
 this.drawingService.destroyStage();
 }

 /**
 * This creates a raw pixel buffer that contains a single frame of the video drawn by the rendering library
 *
 */
 public async createImageBuffer(): Promise<buffer> {

 const drawTimer = PerformanceUtil.startTimer();
 try {
 await this.drawingService.drawForFrame(this.currentFrameIndex);
 } catch (err: any) {
 throw new RenderingLibraryError(err);
 }

 PerformanceUtil.logTimer(drawTimer, 'DRAW -> FRAME\t');

 const bufferTimer = PerformanceUtil.startTimer();
 // Creates a raw pixel buffer, containing simple binary data
 // the exact same information (BGRA/screen ratio) has to be provided to ffmpeg, because ffmpeg cannot detect format for raw input
 const buffer = await this.drawingService.toBuffer(BufferType.RAW);
 PerformanceUtil.logTimer(bufferTimer, 'CANVAS -> BUFFER');

 return buffer;
 }
}
</buffer></buffer>


FfmpegService


import {ChildProcess, execFile} from 'child_process';
import {Readable} from 'stream';
import {FPS, StageSize} from 'vm-rendering-library';
import {
 FfmpegError,
 LOCAL_MERGE_VIDEOS_TEXT_FILE, LOCAL_SOUND_FILE_PATH,
 LOCAL_VIDEO_FILE_PATH,
 LOCAL_VIDEO_SOUNDLESS_MERGE_FILE_PATH
} from "vm-rendering-backend-commons";

/**
 * This class bundles all ffmpeg usages for rendering one scene.
 * FFmpeg is a console program which can transcode nearly all types of sounds, images and videos from one to another.
 */
export class FfmpegService {

 ffmpegPath: string = null;


 constructor(ffmpegPath: string) {
 this.ffmpegPath = ffmpegPath;
 }

 /**
 * Convert a stream of raw images into an .mp4 video using the command line program ffmpeg.
 *
 * @param inputStream an input stream containing images in raw format BGRA
 * @param stageSize the size of a single frame in pixels (minimum is 2*2)
 * @param outputPath the filepath to write the resulting video to
 */
 public imageToVideo(inputStream: Readable, stageSize: StageSize, outputPath: string): Promise<void> {
 const args: string[] = [
 '-f',
 'rawvideo',
 '-r',
 `${FPS}`,
 '-pix_fmt',
 'bgra',
 '-s',
 `${stageSize.width}x${stageSize.height}`,
 '-i',
 // input "-" means input will be passed via pipe (streamed)
 '-',
 // codec that also QuickTime player can understand
 '-vcodec',
 'libx264',
 '-pix_fmt',
 'yuv420p',
 /*
 * "-movflags faststart":
 * metadata at beginning of file
 * needs more RAM
 * file will be broken, if not finished properly
 * higher application compatibility
 * better for browser streaming
 */
 '-movflags',
 'faststart',
 // "-preset ultrafast", //use this to speed up compression, but quality/compression ratio gets worse
 // don't overwrite an existing file here,
 // but delete file in the beginning of execution index.ts
 // (this is better for local testing believe me)
 outputPath
 ];

 return this.execFfmpegPromise(args, inputStream);
 }

 private execFfmpegPromise(args: string[], inputStream?: Readable): Promise<void> {
 const ffmpegServiceSelf = this;
 return new Promise(function (resolve, reject) {
 const executionProcess: ChildProcess = execFile(ffmpegServiceSelf.ffmpegPath, args, (err) => {
 if (err) {
 reject(new FfmpegError(err));
 } else {
 console.log('ffmpeg finished');
 resolve();
 }
 });
 if (inputStream) {
 // it's important to listen on errors of input stream before piping it into the write stream
 // if we don't do this here, we get an unhandled promise exception for every issue in the input stream
 inputStream.on("error", err => {
 reject(err);
 });
 // don't reject promise here as the error will also be thrown inside execFile and will contain more debugging info
 // this log is just generated for tracing errors and if for some reason the handling in execFile doesn't work
 inputStream.pipe(executionProcess.stdin).on("error", err => console.log("pipe stream: " , err));
 }
 });
 }
}
</void></void>


-
9 Ways to Customise Your Matomo Like a Pro
5 octobre 2022, par Erin -
WordPress Analytics plugin WP-Piwik reaches version 1.0.0 (and 50,000 active users)
29 mai 2015, par André Bräkling — PluginsAfter six years of development, we are proud to announce the 1.0.0 release of our WP-Piwik WordPress plugin !
Started as a simple plugin to show a selection of statistics within the WordPress dashboard, WP-Piwik has become a full Piwik integration plugin. The plugin automatically adds the Piwik tracking code to your WordPress sites. The plugin displays your analytics reports directly within the WordPress admin panel. WordPress networks (“multisite”), CDN URLs and the Piwik proxy script are also supported.
According to WordPress.org the plugin is being used by more than 50,000 WordPress sites !
This article explains how to install WP-Piwik and how to configure it to work with your Piwik instance.
Install WP-Piwik
You can get WP-Piwik using WordPress’ plugin management. Login to your WordPress admin dashboard and go to « Plugins » → « Add New ». Enter « WP-Piwik » into the search field at the top right, press enter and next to WP-Piwik choose « Install Now ».
If you want to use WP-Piwik in your simple WordPress blog you can just click « Activate Plugin » and WP-Piwik will ask you to configure your Piwik connection.
Running a WordPress network/multisite you can choose to « Network Activate » the plugin after the installation process. In this case, WP-Piwik will be a fully automated feature of your WordPress network automatically tracking your sites in the same Piwik instance in separate Websites.
Alternatively you can download WP-Piwik manually from the WordPress website and upload all files to your `wp-content/plugins` directory.
Configure your Piwik connection
WP-Piwik lets you choose between three connection modes :
- Self-hosted (HTTP API) : This is the default option for a self-hosted Piwik and should work for most configurations. You just have to know your Piwik URL, which is the URL you enter to access Piwik, and your auth token (see below). WP-Piwik will connect to Piwik using http(s)-requests.
- Self-hosted (PHP API) : Choose this, if your self-hosted Piwik and WordPress are running on the same machine and you know the full server path to your Piwik instance. Beside the full server path, you also need to know your auth token (see below).
- Cloud-hosted (Piwik Pro) : If you are using a cloud-hosted Piwik by Piwik Pro, you just need to know your user name and your auth token (see below).
Setting up WP-Piwik
To configure WP-Piwik you will need to specify your Authentication token.
- If the site you want to track in Piwik is already configured in your Piwik, you only need to specify a token_auth for a user with `view` permission.
- If you want WP-Piwik to create the website in Piwik (or if you use WP-Piwik in network mode which requires to be able to configure your sites), you should specify a token_auth which has Super User access (after the setting up phase is completed you can set the authentication token back to the token of a `view` user).
To find your token_auth in Piwik, click on your user name in the right right corner of your Piwik dashboard, then click the « API » in the left menu. The API page displays your auth token in a colored box, just behind the “&token_auth=” string. The screenshot below shows the token_auth anonymous, but your real one will be an alpha numerous random string like a1ec31524a8eabc7a546d71d68b28d17.
That’s it. After you entered your connection data and submitted the form, WP-Piwik will welcome you with some information :
You can now start to configure WP-Piwik and enable the tracking code. Learn about any setting by clicking on the small question mark sign. If you have any problem configuring or using WP-Piwik feel free to use the WordPress support forum related to WP-Piwik.
Translating WP-Piwik
We invite you to join our translation community at Transifex and help to translate WP-Piwik in more languages !
Happy WordPress Analytics !