Recherche avancée

Médias (1)

Mot : - Tags -/punk

Autres articles (99)

  • MediaSPIP 0.1 Beta version

    25 avril 2011, par

    MediaSPIP 0.1 beta is the first version of MediaSPIP proclaimed as "usable".
    The zip file provided here only contains the sources of MediaSPIP in its standalone version.
    To get a working installation, you must manually install all-software dependencies on the server.
    If you want to use this archive for an installation in "farm mode", you will also need to proceed to other manual (...)

  • Personnaliser en ajoutant son logo, sa bannière ou son image de fond

    5 septembre 2013, par

    Certains thèmes prennent en compte trois éléments de personnalisation : l’ajout d’un logo ; l’ajout d’une bannière l’ajout d’une image de fond ;

  • Websites made ​​with MediaSPIP

    2 mai 2011, par

    This page lists some websites based on MediaSPIP.

Sur d’autres sites (9689)

  • VLC dead input for RTP stream

    27 mars, par CaptainCheese

    I'm working on creating an rtp stream that's meant to display live waveform data from Pioneer prolink players. The motivation for sending this video out is to be able to receive it in a flutter frontend. I initially was just sending a base-24 encoding of the raw ARGB packed ints per frame across a Kafka topic to it but processing this data in flutter proved to be untenable and was bogging down the main UI thread. Not sure if this is the most optimal way of going about this but just trying to get anything to work if it means some speedup on the frontend. So the issue the following implementation is experiencing is that when I run vlc --rtsp-timeout=120000 --network-caching=30000 -vvvv stream_1.sdp where

    


    % cat stream_1.sdp
v=0
o=- 0 1 IN IP4 127.0.0.1
s=RTP Stream
c=IN IP4 127.0.0.1
t=0 0
a=tool:libavformat
m=video 5007 RTP/AVP 96
a=rtpmap:96 H264/90000


    


    I see (among other questionable logs) the following :

    


    [0000000144c44d10] live555 demux error: no data received in 10s, aborting
[00000001430ee2f0] main input debug: EOF reached
[0000000144b160c0] main decoder debug: killing decoder fourcc `h264'
[0000000144b160c0] main decoder debug: removing module "videotoolbox"
[0000000144b164a0] main packetizer debug: removing module "h264"
[0000000144c44d10] main demux debug: removing module "live555"
[0000000144c45bb0] main stream debug: removing module "record"
[0000000144a64960] main stream debug: removing module "cache_read"
[0000000144c29c00] main stream debug: removing module "filesystem"
[00000001430ee2f0] main input debug: Program doesn't contain anymore ES
[0000000144806260] main playlist debug: dead input
[0000000144806260] main playlist debug: changing item without a request (current 0/1)
[0000000144806260] main playlist debug: nothing to play
[0000000142e083c0] macosx interface debug: Playback has been ended
[0000000142e083c0] macosx interface debug: Releasing IOKit system sleep blocker (37463)


    


    This is sort of confusing because when I run ffmpeg -protocol_whitelist file,crypto,data,rtp,udp -i stream_1.sdp -vcodec libx264 -f null -
I see a number logs about

    


    [h264 @ 0x139304080] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x139304080] decode_slice_header error
[h264 @ 0x139304080] no frame!


    


    After which I see the stream is received and I start getting telemetry on it :

    


    Input #0, sdp, from 'stream_1.sdp':
  Metadata:
    title           : RTP Stream
  Duration: N/A, start: 0.016667, bitrate: N/A
  Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 1200x200, 60 fps, 60 tbr, 90k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x107f04f40] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x107f04f40] profile High, level 3.1, 4:2:0, 8-bit
Output #0, null, to 'pipe:':
  Metadata:
    title           : RTP Stream
    encoder         : Lavf61.7.100
  Stream #0:0: Video: h264, yuv420p(tv, progressive), 1200x200, q=2-31, 60 fps, 60 tbn
      Metadata:
        encoder         : Lavc61.19.101 libx264
      Side data:
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[out#0/null @ 0x60000069c000] video:144KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
frame= 1404 fps= 49 q=-1.0 Lsize=N/A time=00:00:23.88 bitrate=N/A speed=0.834x


    


    Not sure why VLC is turning me down like some kind of Berghain bouncer that lets nobody in the entire night.

    


    I initially tried just converting the ARGB ints to a YUV420p buffer and used this to create the Frame objects but I couldn't for the life of me figure out how to properly initialize it as the attempts I made kept spitting out garbled junk.

    


    Please go easy on me, I've made an unhealthy habit of resolving nearly all of my coding questions by simply lurking the internet for answers but that's not really helping me solve this issue.

    


    Here's the Java I'm working on (the meat of the rtp comms occurs within updateWaveformForPlayer()) :

    


    package com.bugbytz.prolink;&#xA;&#xA;import org.apache.kafka.clients.producer.KafkaProducer;&#xA;import org.apache.kafka.clients.producer.Producer;&#xA;import org.apache.kafka.clients.producer.ProducerConfig;&#xA;import org.apache.kafka.clients.producer.ProducerRecord;&#xA;import org.bytedeco.ffmpeg.global.avcodec;&#xA;import org.bytedeco.ffmpeg.global.avutil;&#xA;import org.bytedeco.javacv.FFmpegFrameGrabber;&#xA;import org.bytedeco.javacv.FFmpegFrameRecorder;&#xA;import org.bytedeco.javacv.FFmpegLogCallback;&#xA;import org.bytedeco.javacv.Frame;&#xA;import org.bytedeco.javacv.FrameGrabber;&#xA;import org.deepsymmetry.beatlink.CdjStatus;&#xA;import org.deepsymmetry.beatlink.DeviceAnnouncement;&#xA;import org.deepsymmetry.beatlink.DeviceAnnouncementAdapter;&#xA;import org.deepsymmetry.beatlink.DeviceFinder;&#xA;import org.deepsymmetry.beatlink.Util;&#xA;import org.deepsymmetry.beatlink.VirtualCdj;&#xA;import org.deepsymmetry.beatlink.data.BeatGridFinder;&#xA;import org.deepsymmetry.beatlink.data.CrateDigger;&#xA;import org.deepsymmetry.beatlink.data.MetadataFinder;&#xA;import org.deepsymmetry.beatlink.data.TimeFinder;&#xA;import org.deepsymmetry.beatlink.data.WaveformDetail;&#xA;import org.deepsymmetry.beatlink.data.WaveformDetailComponent;&#xA;import org.deepsymmetry.beatlink.data.WaveformFinder;&#xA;&#xA;import java.awt.*;&#xA;import java.awt.image.BufferedImage;&#xA;import java.io.File;&#xA;import java.nio.ByteBuffer;&#xA;import java.text.DecimalFormat;&#xA;import java.util.ArrayList;&#xA;import java.util.HashMap;&#xA;import java.util.HashSet;&#xA;import java.util.Map;&#xA;import java.util.Properties;&#xA;import java.util.Set;&#xA;import java.util.concurrent.ExecutionException;&#xA;import java.util.concurrent.Executors;&#xA;import java.util.concurrent.ScheduledExecutorService;&#xA;import java.util.concurrent.ScheduledFuture;&#xA;import java.util.concurrent.TimeUnit;&#xA;&#xA;import static org.bytedeco.ffmpeg.global.avutil.AV_PIX_FMT_RGB24;&#xA;&#xA;public class App {&#xA;    public static ArrayList<track> tracks = new ArrayList&lt;>();&#xA;    public static boolean dbRead = false;&#xA;    public static Properties props = new Properties();&#xA;    private static Map recorders = new HashMap&lt;>();&#xA;    private static Map frameCount = new HashMap&lt;>();&#xA;&#xA;    private static final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);&#xA;    private static final int FPS = 60;&#xA;    private static final int FRAME_INTERVAL_MS = 1000 / FPS;&#xA;&#xA;    private static Map schedules = new HashMap&lt;>();&#xA;&#xA;    private static Set<integer> streamingPlayers = new HashSet&lt;>();&#xA;&#xA;    public static String byteArrayToMacString(byte[] macBytes) {&#xA;        StringBuilder sb = new StringBuilder();&#xA;        for (int i = 0; i &lt; macBytes.length; i&#x2B;&#x2B;) {&#xA;            sb.append(String.format("%02X%s", macBytes[i], (i &lt; macBytes.length - 1) ? ":" : ""));&#xA;        }&#xA;        return sb.toString();&#xA;    }&#xA;&#xA;    private static void updateWaveformForPlayer(int player) throws Exception {&#xA;        Integer frame_for_player = frameCount.get(player);&#xA;        if (frame_for_player == null) {&#xA;            frame_for_player = 0;&#xA;            frameCount.putIfAbsent(player, frame_for_player);&#xA;        }&#xA;&#xA;        if (!WaveformFinder.getInstance().isRunning()) {&#xA;            WaveformFinder.getInstance().start();&#xA;        }&#xA;        WaveformDetail detail = WaveformFinder.getInstance().getLatestDetailFor(player);&#xA;&#xA;        if (detail != null) {&#xA;            WaveformDetailComponent component = (WaveformDetailComponent) detail.createViewComponent(&#xA;                    MetadataFinder.getInstance().getLatestMetadataFor(player),&#xA;                    BeatGridFinder.getInstance().getLatestBeatGridFor(player)&#xA;            );&#xA;            component.setMonitoredPlayer(player);&#xA;            component.setPlaybackState(player, TimeFinder.getInstance().getTimeFor(player), true);&#xA;            component.setAutoScroll(true);&#xA;            int width = 1200;&#xA;            int height = 200;&#xA;            Dimension dimension = new Dimension(width, height);&#xA;            component.setPreferredSize(dimension);&#xA;            component.setSize(dimension);&#xA;            component.setScale(1);&#xA;            component.doLayout();&#xA;&#xA;            // Create a fresh BufferedImage and clear it before rendering&#xA;            BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);&#xA;            Graphics2D g = image.createGraphics();&#xA;            g.clearRect(0, 0, width, height);  // Clear any old content&#xA;&#xA;            // Draw waveform into the BufferedImage&#xA;            component.paint(g);&#xA;            g.dispose();&#xA;&#xA;            int port = 5004 &#x2B; player;&#xA;            String inputFile = port &#x2B; "_" &#x2B; frame_for_player &#x2B; ".mp4";&#xA;            // Initialize the FFmpegFrameRecorder for YUV420P&#xA;            FFmpegFrameRecorder recorder_file = new FFmpegFrameRecorder(inputFile, width, height);&#xA;            FFmpegLogCallback.set();  // Enable FFmpeg logging for debugging&#xA;            recorder_file.setFormat("mp4");&#xA;            recorder_file.setVideoCodec(avcodec.AV_CODEC_ID_H264);&#xA;            recorder_file.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);  // Use YUV420P format directly&#xA;            recorder_file.setFrameRate(FPS);&#xA;&#xA;            // Set video options&#xA;            recorder_file.setVideoOption("preset", "ultrafast");&#xA;            recorder_file.setVideoOption("tune", "zerolatency");&#xA;            recorder_file.setVideoOption("x264-params", "repeat-headers=1");&#xA;            recorder_file.setGopSize(FPS);&#xA;            try {&#xA;                recorder_file.start();  // Ensure this is called before recording any frames&#xA;                System.out.println("Recorder started successfully for player: " &#x2B; player);&#xA;            } catch (org.bytedeco.javacv.FFmpegFrameRecorder.Exception e) {&#xA;                e.printStackTrace();&#xA;            }&#xA;&#xA;            // Get all pixels in one call&#xA;            int[] pixels = new int[width * height];&#xA;            image.getRGB(0, 0, width, height, pixels, 0, width);&#xA;            recorder_file.recordImage(width,height,Frame.DEPTH_UBYTE,1,3 * width, AV_PIX_FMT_RGB24, ByteBuffer.wrap(argbToByteArray(pixels, width, height)));&#xA;            recorder_file.stop();&#xA;            recorder_file.release();&#xA;            final FFmpegFrameRecorder recorder = recorders.get(player);&#xA;            FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(inputFile);&#xA;&#xA;&#xA;            try {&#xA;                grabber.start();&#xA;            } catch (Exception e) {&#xA;                e.printStackTrace();&#xA;            }&#xA;            if (recorder == null) {&#xA;                try {&#xA;                    String outputStream = "rtp://127.0.0.1:" &#x2B; port;&#xA;                    FFmpegFrameRecorder initial_recorder = new FFmpegFrameRecorder(outputStream, grabber.getImageWidth(), grabber.getImageHeight());&#xA;                    initial_recorder.setFormat("rtp");&#xA;                    initial_recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);&#xA;                    initial_recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);&#xA;                    initial_recorder.setFrameRate(grabber.getFrameRate());&#xA;                    initial_recorder.setGopSize(FPS);&#xA;                    initial_recorder.setVideoOption("x264-params", "keyint=60");&#xA;                    initial_recorder.setVideoOption("rtsp_transport", "tcp");&#xA;                    initial_recorder.start();&#xA;                    recorders.putIfAbsent(player, initial_recorder);&#xA;                    frameCount.putIfAbsent(player, 0);&#xA;                    putToRTP(player, grabber, initial_recorder);&#xA;                }&#xA;                catch (Exception e) {&#xA;                    e.printStackTrace();&#xA;                }&#xA;            }&#xA;            else {&#xA;                putToRTP(player, grabber, recorder);&#xA;            }&#xA;            File file = new File(inputFile);&#xA;            if (file.exists() &amp;&amp; file.delete()) {&#xA;                System.out.println("Successfully deleted file: " &#x2B; inputFile);&#xA;            } else {&#xA;                System.out.println("Failed to delete file: " &#x2B; inputFile);&#xA;            }&#xA;        }&#xA;    }&#xA;&#xA;    public static void putToRTP(int player, FFmpegFrameGrabber grabber, FFmpegFrameRecorder recorder) throws FrameGrabber.Exception {&#xA;        final Frame frame = grabber.grabFrame();&#xA;        int frameCount_local = frameCount.get(player);&#xA;        frame.keyFrame = frameCount_local&#x2B;&#x2B; % FPS == 0;&#xA;        frameCount.put(player, frameCount_local);&#xA;        try {&#xA;            recorder.record(frame);&#xA;        } catch (FFmpegFrameRecorder.Exception e) {&#xA;            throw new RuntimeException(e);&#xA;        }&#xA;    }&#xA;    public static byte[] argbToByteArray(int[] argb, int width, int height) {&#xA;        int totalPixels = width * height;&#xA;        byte[] byteArray = new byte[totalPixels * 3];  // 4 bytes per pixel (ARGB)&#xA;&#xA;        for (int i = 0; i &lt; totalPixels; i&#x2B;&#x2B;) {&#xA;            int argbPixel = argb[i];&#xA;&#xA;            byteArray[i * 3] = (byte) ((argbPixel >> 16) &amp; 0xFF);  // Red&#xA;            byteArray[i * 3 &#x2B; 1] = (byte) ((argbPixel >> 8) &amp; 0xFF);   // Green&#xA;            byteArray[i * 3 &#x2B; 2] = (byte) (argbPixel &amp; 0xFF);  // Blue&#xA;        }&#xA;&#xA;        return byteArray;&#xA;    }&#xA;&#xA;&#xA;    public static void main(String[] args) throws Exception {&#xA;        VirtualCdj.getInstance().setDeviceNumber((byte) 4);&#xA;        CrateDigger.getInstance().addDatabaseListener(new DBService());&#xA;        props.put("bootstrap.servers", "localhost:9092");&#xA;        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");&#xA;        props.put("value.serializer", "com.bugbytz.prolink.CustomSerializer");&#xA;        props.put(ProducerConfig.MAX_REQUEST_SIZE_CONFIG, "20971520");&#xA;&#xA;        VirtualCdj.getInstance().addUpdateListener(update -> {&#xA;            if (update instanceof CdjStatus) {&#xA;                try (Producer producer = new KafkaProducer&lt;>(props)) {&#xA;                    DecimalFormat df_obj = new DecimalFormat("#.##");&#xA;                    DeviceStatus deviceStatus = new DeviceStatus(&#xA;                            update.getDeviceNumber(),&#xA;                            ((CdjStatus) update).isPlaying() || !((CdjStatus) update).isPaused(),&#xA;                            ((CdjStatus) update).getBeatNumber(),&#xA;                            update.getBeatWithinBar(),&#xA;                            Double.parseDouble(df_obj.format(update.getEffectiveTempo())),&#xA;                            Double.parseDouble(df_obj.format(Util.pitchToPercentage(update.getPitch()))),&#xA;                            update.getAddress().getHostAddress(),&#xA;                            byteArrayToMacString(DeviceFinder.getInstance().getLatestAnnouncementFrom(update.getDeviceNumber()).getHardwareAddress()),&#xA;                            ((CdjStatus) update).getRekordboxId(),&#xA;                            update.getDeviceName()&#xA;                    );&#xA;                    ProducerRecord record = new ProducerRecord&lt;>("device-status", "device-" &#x2B; update.getDeviceNumber(), deviceStatus);&#xA;                    try {&#xA;                        producer.send(record).get();&#xA;                    } catch (InterruptedException ex) {&#xA;                        throw new RuntimeException(ex);&#xA;                    } catch (ExecutionException ex) {&#xA;                        throw new RuntimeException(ex);&#xA;                    }&#xA;                    producer.flush();&#xA;                    if (!WaveformFinder.getInstance().isRunning()) {&#xA;                        try {&#xA;                            WaveformFinder.getInstance().start();&#xA;                        } catch (Exception ex) {&#xA;                            throw new RuntimeException(ex);&#xA;                        }&#xA;                    }&#xA;                }&#xA;            }&#xA;        });&#xA;        DeviceFinder.getInstance().addDeviceAnnouncementListener(new DeviceAnnouncementAdapter() {&#xA;            @Override&#xA;            public void deviceFound(DeviceAnnouncement announcement) {&#xA;                if (!streamingPlayers.contains(announcement.getDeviceNumber())) {&#xA;                    streamingPlayers.add(announcement.getDeviceNumber());&#xA;                    schedules.putIfAbsent(announcement.getDeviceNumber(), scheduler.scheduleAtFixedRate(() -> {&#xA;                        try {&#xA;                            Runnable task = () -> {&#xA;                                try {&#xA;                                    updateWaveformForPlayer(announcement.getDeviceNumber());&#xA;                                } catch (InterruptedException e) {&#xA;                                    System.out.println("Thread interrupted");&#xA;                                } catch (Exception e) {&#xA;                                    throw new RuntimeException(e);&#xA;                                }&#xA;                                System.out.println("Lambda thread work completed!");&#xA;                            };&#xA;                            task.run();&#xA;                        } catch (Exception e) {&#xA;                            e.printStackTrace();&#xA;                        }&#xA;                    }, 0, FRAME_INTERVAL_MS, TimeUnit.MILLISECONDS));&#xA;                }&#xA;            }&#xA;&#xA;            @Override&#xA;            public void deviceLost(DeviceAnnouncement announcement) {&#xA;                if (streamingPlayers.contains(announcement.getDeviceNumber())) {&#xA;                    schedules.get(announcement.getDeviceNumber()).cancel(true);&#xA;                    streamingPlayers.remove(announcement.getDeviceNumber());&#xA;                }&#xA;            }&#xA;        });&#xA;        BeatGridFinder.getInstance().start();&#xA;        MetadataFinder.getInstance().start();&#xA;        VirtualCdj.getInstance().start();&#xA;        TimeFinder.getInstance().start();&#xA;        DeviceFinder.getInstance().start();&#xA;        CrateDigger.getInstance().start();&#xA;&#xA;        try {&#xA;            LoadCommandConsumer consumer = new LoadCommandConsumer("localhost:9092", "load-command-group");&#xA;            Thread consumerThread = new Thread(consumer::startConsuming);&#xA;            consumerThread.start();&#xA;&#xA;            Runtime.getRuntime().addShutdownHook(new Thread(() -> {&#xA;                consumer.shutdown();&#xA;                try {&#xA;                    consumerThread.join();&#xA;                } catch (InterruptedException e) {&#xA;                    Thread.currentThread().interrupt();&#xA;                }&#xA;            }));&#xA;            Thread.sleep(60000);&#xA;        } catch (InterruptedException e) {&#xA;            System.out.println("Interrupted, exiting.");&#xA;        }&#xA;    }&#xA;}&#xA;</integer></track>

    &#xA;

  • Generating test data – Introducing the Piwik Platform

    9 octobre 2014, par Thomas Steur — Development

    This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to create a command). This time you’ll learn how to generate test data.

    Developers are developing on their local Piwik instance which usually does not contain useful data compared to a real Piwik installation in production (only a few test visits and a few tests users and websites). The ‘VisitorGenerator’ plugin lets you generate any number of visits, websites, users, goals and more. The generator makes sure there will be data for each report so you can easily test anything.

    Getting started

    In this series of posts, we assume that you have already installed Piwik. If not, visit the Piwik Developer Zone where you’ll find the Installation guide for developers.

    Installing the VisitorGenerator plugin

    The easiest way to install the plugin is by using the Marketplace in Piwik itself. It is accessible via Settings => Marketplace => Get new functionality. There you’ll find the plugin “VisitorGenerator” which you can install and activate in one click.

    If your Piwik instance is not connected to the internet you can download the plugin from the VisitorGenerator page on the Marketplace. Afterwards you can install the plugin by going to Settings => Marketplace => Uploading a plugin and uploading the previously downloaded ZIP file.

    If you have already installed the plugin make sure it is activated by going to Settings => Plugins.

    Generating websites

    After you have installed the plugin you can add as many websites as you need. This is useful for instance when you want to test something that affects many websites such as the ‘All Websites’ dashboard or the Websites manager. To generate any number of websites use the following command :

    ./console visitorgenerator:generate-website --limit=10

    This will generate 10 websites. If you need more websites simply specify a higher limit. In case you are wondering the names and URLs of the websites are randomly generated by the Faker PHP library.

    Generating goals

    In case you want to test anything related to Goals you should execute the following command :

    ./console visitorgenerator:generate-goals --idsite=1

    This will generate a few goals for the specified site. The generated goals are defined in a way to make sure there will be conversions when generating the visits in the next step.

    Generating visits

    To generate visits there are two possibilities. Either via the Piwik UI by going to Settings => Visitor Generator or by using the command line. The UI is a bit limited in generating visits so we recommend to use the command line. There you can generate visits as follows :

    ./console visitorgenerator:generate-visits --idsite=1

    This will generate many different visits for the current day. Don’t worry if it takes a while, it will insert quite a few visits by default.

    In case you want to generate visits for multiple days in the past as well you can specify the --days option.

    ./console visitorgenerator:generate-visits --idsite=1 --days=5

    Providing your own logs

    Half of the generated visits are randomly generated and half of the visits are based on real logs to make sure there is data for each report. If you want to generate visits based on your own logs for a more realistic testing just place your log files in the plugins/VisitorGenerator/data folder and make sure the file name ends with .log. You can find a few examples in the VisitorGenerator data folder.

    To generate visits based only on real log files then use the --no-fake option.

    ./console visitorgenerator:generate-visits --idsite=1 --no-fake

    All generated visits will come from the logs and no random visits nor random fake data will be used.

    Advanced features

    We are regularly adding new commands, tools and runtime checks to make your life as a developer easier. For instance you can also generate users and annotations. In the future we want to extend the plugin to create visits in the background to make sure there will be constantly new actions in the real time report.

    Are you missing any kind of generator or any other feature to make your life as a developer easier ? Let us know by email, we are listening !

    Would you like to know more about the Piwik platform ? Go to our Piwik Developer Zone where you’ll find guides and references on how to develop plugin and themes.

  • Generating test data – Introducing the Piwik Platform

    9 octobre 2014, par Thomas Steur — Development

    This is the next post of our blog series where we introduce the capabilities of the Piwik platform (our previous post was How to create a command). This time you’ll learn how to generate test data.

    Developers are developing on their local Piwik instance which usually does not contain useful data compared to a real Piwik installation in production (only a few test visits and a few tests users and websites). The ‘VisitorGenerator’ plugin lets you generate any number of visits, websites, users, goals and more. The generator makes sure there will be data for each report so you can easily test anything.

    Getting started

    In this series of posts, we assume that you have already installed Piwik. If not, visit the Piwik Developer Zone where you’ll find the Installation guide for developers.

    Installing the VisitorGenerator plugin

    The easiest way to install the plugin is by using the Marketplace in Piwik itself. It is accessible via Settings => Marketplace => Get new functionality. There you’ll find the plugin “VisitorGenerator” which you can install and activate in one click.

    If your Piwik instance is not connected to the internet you can download the plugin from the VisitorGenerator page on the Marketplace. Afterwards you can install the plugin by going to Settings => Marketplace => Uploading a plugin and uploading the previously downloaded ZIP file.

    If you have already installed the plugin make sure it is activated by going to Settings => Plugins.

    Generating websites

    After you have installed the plugin you can add as many websites as you need. This is useful for instance when you want to test something that affects many websites such as the ‘All Websites’ dashboard or the Websites manager. To generate any number of websites use the following command :

    ./console visitorgenerator:generate-website --limit=10

    This will generate 10 websites. If you need more websites simply specify a higher limit. In case you are wondering the names and URLs of the websites are randomly generated by the Faker PHP library.

    Generating goals

    In case you want to test anything related to Goals you should execute the following command :

    ./console visitorgenerator:generate-goals --idsite=1

    This will generate a few goals for the specified site. The generated goals are defined in a way to make sure there will be conversions when generating the visits in the next step.

    Generating visits

    To generate visits there are two possibilities. Either via the Piwik UI by going to Settings => Visitor Generator or by using the command line. The UI is a bit limited in generating visits so we recommend to use the command line. There you can generate visits as follows :

    ./console visitorgenerator:generate-visits --idsite=1

    This will generate many different visits for the current day. Don’t worry if it takes a while, it will insert quite a few visits by default.

    In case you want to generate visits for multiple days in the past as well you can specify the --days option.

    ./console visitorgenerator:generate-visits --idsite=1 --days=5

    Providing your own logs

    Half of the generated visits are randomly generated and half of the visits are based on real logs to make sure there is data for each report. If you want to generate visits based on your own logs for a more realistic testing just place your log files in the plugins/VisitorGenerator/data folder and make sure the file name ends with .log. You can find a few examples in the VisitorGenerator data folder.

    To generate visits based only on real log files then use the --no-fake option.

    ./console visitorgenerator:generate-visits --idsite=1 --no-fake

    All generated visits will come from the logs and no random visits nor random fake data will be used.

    Advanced features

    We are regularly adding new commands, tools and runtime checks to make your life as a developer easier. For instance you can also generate users and annotations. In the future we want to extend the plugin to create visits in the background to make sure there will be constantly new actions in the real time report.

    Are you missing any kind of generator or any other feature to make your life as a developer easier ? Let us know by email, we are listening !

    Would you like to know more about the Piwik platform ? Go to our Piwik Developer Zone where you’ll find guides and references on how to develop plugin and themes.