Recherche avancée

Médias (0)

Mot : - Tags -/api

Aucun média correspondant à vos critères n’est disponible sur le site.

Autres articles (55)

  • Websites made ​​with MediaSPIP

    2 mai 2011, par

    This page lists some websites based on MediaSPIP.

  • Gestion générale des documents

    13 mai 2011, par

    MédiaSPIP ne modifie jamais le document original mis en ligne.
    Pour chaque document mis en ligne il effectue deux opérations successives : la création d’une version supplémentaire qui peut être facilement consultée en ligne tout en laissant l’original téléchargeable dans le cas où le document original ne peut être lu dans un navigateur Internet ; la récupération des métadonnées du document original pour illustrer textuellement le fichier ;
    Les tableaux ci-dessous expliquent ce que peut faire MédiaSPIP (...)

  • Creating farms of unique websites

    13 avril 2011, par

    MediaSPIP platforms can be installed as a farm, with a single "core" hosted on a dedicated server and used by multiple websites.
    This allows (among other things) : implementation costs to be shared between several different projects / individuals rapid deployment of multiple unique sites creation of groups of like-minded sites, making it possible to browse media in a more controlled and selective environment than the major "open" (...)

Sur d’autres sites (5330)

  • FFMpeg : ffmpeg failed to execute command error

    4 mars 2020, par Richard McFriend Oluwamuyiwa

    I am trying to transcode a video file of 1.2mb file uploaded to my website server via php/html upload, but I keep getting the error :

    PHP Fatal error :

    Uncaught exception
    ’Alchemy\BinaryDriver\Exception\ExecutionFailureException’ with
    message ’ffmpeg failed to execute command ’/usr/local/bin/ffmpeg’ ’-y’
    ’-i’
    ’/home/user/public_html/contents/videos/1490719990_MP4_360p_Short_video_clip_nature_mp4.mp4’
    ’-async’ ’1’ ’-metadata:s:v:0’ ’start_time=0’ ’-r’ ’16’ ’-b_strategy’
    ’1’ ’-bf’ ’3’ ’-g’ ’9’ ’-vcodec’ ’libx264’ ’-acodec’ ’libmp3lame’
    ’-b:v’ ’128k’ ’-refs’ ’6’ ’-coder’ ’1’ ’-sc_threshold’ ’40’ ’-flags’
    ’+loop’ ’-me_range’ ’16’ ’-subq’ ’7’ ’-i_qfactor’ ’0.71’ ’-qcomp’
    ’0.6’ ’-qdiff’ ’4’ ’-trellis’ ’1’ ’-b:a’ ’8k’ ’-ac’ ’1’ ’-pass’ ’1’
    ’-passlogfile’
    ’/tmp/ffmpeg-passes58dab05a323b6eknk4/pass-58dab05a32465’
    ’/home/user/public_html/contents/videos/1490719990_MP4_360p_Short_video_clip_nature_mp4_22995.mp4’’
    in
    /home/user/public_html/app/ffmpeg/vendor/alchemy/binary-driver/src/Alchemy/BinaryDriver/ProcessRunner.php:100

    Stack trace :

    0 /home/user/public_html/app/ffmpeg/vendor/alchemy/binary-driver/src/Alchemy/BinaryDriver/ProcessRunner.php(72) :

    Alch in
    /home/user/public_html/app/ffmpeg/vendor/php-ffmpeg/php-ffmpeg/src/FFMpeg/Media/Video.php
    on line 168

    Funny thing is that the same server is extracting frames and getting duration of the same file using ffprobe and ffmpeg.

    Here is the code I am using to transcode :

    $ffmpeg = $ffmpeg = FFMpeg\FFMpeg::create(['timeout'=>3600, 'ffmpeg.thread'=>12, 'ffmpeg.binaries'  => '/usr/local/bin/ffmpeg',
       'ffprobe.binaries' => '/usr/local/bin/ffprobe']);

               $ffprobe_prep = FFMpeg\FFProbe::create(['ffmpeg.binaries'  => '/usr/local/bin/ffmpeg',
       'ffprobe.binaries' => '/usr/local/bin/ffprobe']);
               $ffprobe = $ffprobe_prep->format($video_file);

               $video = $ffmpeg->open($video_file);

               // Get video duration to ensure our videos are never longer than our video limit.
               $duration = $ffprobe->get('duration');

               // Use mp4 format and set the audio bitrate to 56Kbit and Mono channel.
               // TODO: Try stereo later...
               $format = new FFMpeg\Format\Video\X264('libmp3lame', 'libx264');
               $format
                   -> setKiloBitrate(128)
                   -> setAudioChannels(1)
                   -> setAudioKiloBitrate(8);

               $first = $ffprobe_prep
                           ->streams($video_file)
                           ->videos()
                           ->first();

               $width = $first->get('width');

               if($width > VIDEO_WIDTH){
                   // Resize to 558 x 314 and resize to fit width.
                   $video
                       ->filters()
                       ->resize(new FFMpeg\Coordinate\Dimension(VIDEO_WIDTH, ceil(VIDEO_WIDTH / 16 * 9)));
               }

               // Trim to videos longer than three minutes to 3 minutes.
               if($duration > MAX_VIDEO_PLAYTIME){

                   $video
                       ->filters()
                       ->clip(FFMpeg\Coordinate\TimeCode::fromSeconds(0), FFMpeg\Coordinate\TimeCode::fromSeconds(MAX_VIDEO_PLAYTIME));
               }

               // Change the framerate to 16fps and GOP as 9.
               $video
                   ->filters()
                   ->framerate(new FFMpeg\Coordinate\FrameRate(16), 9);

               // Synchronize audio and video
               $video->filters()->synchronize();

               $video->save($format, $video_file_new_2);

    I have contacted my host to no vital assistance. The only useful information they can provide me is that ffmpeg was compiled on the server with libmp3lame support.

    This code works perfect on localhost

    Any help as to why I may be getting the error and how to correct it is appreciated.

  • VLC dead input for RTP stream

    27 mars, par CaptainCheese

    I'm working on creating an rtp stream that's meant to display live waveform data from Pioneer prolink players. The motivation for sending this video out is to be able to receive it in a flutter frontend. I initially was just sending a base-24 encoding of the raw ARGB packed ints per frame across a Kafka topic to it but processing this data in flutter proved to be untenable and was bogging down the main UI thread. Not sure if this is the most optimal way of going about this but just trying to get anything to work if it means some speedup on the frontend. So the issue the following implementation is experiencing is that when I run vlc --rtsp-timeout=120000 --network-caching=30000 -vvvv stream_1.sdp where

    


    % cat stream_1.sdp
v=0
o=- 0 1 IN IP4 127.0.0.1
s=RTP Stream
c=IN IP4 127.0.0.1
t=0 0
a=tool:libavformat
m=video 5007 RTP/AVP 96
a=rtpmap:96 H264/90000


    


    I see (among other questionable logs) the following :

    


    [0000000144c44d10] live555 demux error: no data received in 10s, aborting
[00000001430ee2f0] main input debug: EOF reached
[0000000144b160c0] main decoder debug: killing decoder fourcc `h264'
[0000000144b160c0] main decoder debug: removing module "videotoolbox"
[0000000144b164a0] main packetizer debug: removing module "h264"
[0000000144c44d10] main demux debug: removing module "live555"
[0000000144c45bb0] main stream debug: removing module "record"
[0000000144a64960] main stream debug: removing module "cache_read"
[0000000144c29c00] main stream debug: removing module "filesystem"
[00000001430ee2f0] main input debug: Program doesn't contain anymore ES
[0000000144806260] main playlist debug: dead input
[0000000144806260] main playlist debug: changing item without a request (current 0/1)
[0000000144806260] main playlist debug: nothing to play
[0000000142e083c0] macosx interface debug: Playback has been ended
[0000000142e083c0] macosx interface debug: Releasing IOKit system sleep blocker (37463)


    


    This is sort of confusing because when I run ffmpeg -protocol_whitelist file,crypto,data,rtp,udp -i stream_1.sdp -vcodec libx264 -f null -
I see a number logs about

    


    [h264 @ 0x139304080] non-existing PPS 0 referenced
    Last message repeated 1 times
[h264 @ 0x139304080] decode_slice_header error
[h264 @ 0x139304080] no frame!


    


    After which I see the stream is received and I start getting telemetry on it :

    


    Input #0, sdp, from 'stream_1.sdp':
  Metadata:
    title           : RTP Stream
  Duration: N/A, start: 0.016667, bitrate: N/A
  Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(progressive), 1200x200, 60 fps, 60 tbr, 90k tbn
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0x107f04f40] using cpu capabilities: ARMv8 NEON
[libx264 @ 0x107f04f40] profile High, level 3.1, 4:2:0, 8-bit
Output #0, null, to 'pipe:':
  Metadata:
    title           : RTP Stream
    encoder         : Lavf61.7.100
  Stream #0:0: Video: h264, yuv420p(tv, progressive), 1200x200, q=2-31, 60 fps, 60 tbn
      Metadata:
        encoder         : Lavc61.19.101 libx264
      Side data:
        cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[out#0/null @ 0x60000069c000] video:144KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
frame= 1404 fps= 49 q=-1.0 Lsize=N/A time=00:00:23.88 bitrate=N/A speed=0.834x


    


    Not sure why VLC is turning me down like some kind of Berghain bouncer that lets nobody in the entire night.

    


    I initially tried just converting the ARGB ints to a YUV420p buffer and used this to create the Frame objects but I couldn't for the life of me figure out how to properly initialize it as the attempts I made kept spitting out garbled junk.

    


    Please go easy on me, I've made an unhealthy habit of resolving nearly all of my coding questions by simply lurking the internet for answers but that's not really helping me solve this issue.

    


    Here's the Java I'm working on (the meat of the rtp comms occurs within updateWaveformForPlayer()) :

    


    package com.bugbytz.prolink;&#xA;&#xA;import org.apache.kafka.clients.producer.KafkaProducer;&#xA;import org.apache.kafka.clients.producer.Producer;&#xA;import org.apache.kafka.clients.producer.ProducerConfig;&#xA;import org.apache.kafka.clients.producer.ProducerRecord;&#xA;import org.bytedeco.ffmpeg.global.avcodec;&#xA;import org.bytedeco.ffmpeg.global.avutil;&#xA;import org.bytedeco.javacv.FFmpegFrameGrabber;&#xA;import org.bytedeco.javacv.FFmpegFrameRecorder;&#xA;import org.bytedeco.javacv.FFmpegLogCallback;&#xA;import org.bytedeco.javacv.Frame;&#xA;import org.bytedeco.javacv.FrameGrabber;&#xA;import org.deepsymmetry.beatlink.CdjStatus;&#xA;import org.deepsymmetry.beatlink.DeviceAnnouncement;&#xA;import org.deepsymmetry.beatlink.DeviceAnnouncementAdapter;&#xA;import org.deepsymmetry.beatlink.DeviceFinder;&#xA;import org.deepsymmetry.beatlink.Util;&#xA;import org.deepsymmetry.beatlink.VirtualCdj;&#xA;import org.deepsymmetry.beatlink.data.BeatGridFinder;&#xA;import org.deepsymmetry.beatlink.data.CrateDigger;&#xA;import org.deepsymmetry.beatlink.data.MetadataFinder;&#xA;import org.deepsymmetry.beatlink.data.TimeFinder;&#xA;import org.deepsymmetry.beatlink.data.WaveformDetail;&#xA;import org.deepsymmetry.beatlink.data.WaveformDetailComponent;&#xA;import org.deepsymmetry.beatlink.data.WaveformFinder;&#xA;&#xA;import java.awt.*;&#xA;import java.awt.image.BufferedImage;&#xA;import java.io.File;&#xA;import java.nio.ByteBuffer;&#xA;import java.text.DecimalFormat;&#xA;import java.util.ArrayList;&#xA;import java.util.HashMap;&#xA;import java.util.HashSet;&#xA;import java.util.Map;&#xA;import java.util.Properties;&#xA;import java.util.Set;&#xA;import java.util.concurrent.ExecutionException;&#xA;import java.util.concurrent.Executors;&#xA;import java.util.concurrent.ScheduledExecutorService;&#xA;import java.util.concurrent.ScheduledFuture;&#xA;import java.util.concurrent.TimeUnit;&#xA;&#xA;import static org.bytedeco.ffmpeg.global.avutil.AV_PIX_FMT_RGB24;&#xA;&#xA;public class App {&#xA;    public static ArrayList<track> tracks = new ArrayList&lt;>();&#xA;    public static boolean dbRead = false;&#xA;    public static Properties props = new Properties();&#xA;    private static Map recorders = new HashMap&lt;>();&#xA;    private static Map frameCount = new HashMap&lt;>();&#xA;&#xA;    private static final ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);&#xA;    private static final int FPS = 60;&#xA;    private static final int FRAME_INTERVAL_MS = 1000 / FPS;&#xA;&#xA;    private static Map schedules = new HashMap&lt;>();&#xA;&#xA;    private static Set<integer> streamingPlayers = new HashSet&lt;>();&#xA;&#xA;    public static String byteArrayToMacString(byte[] macBytes) {&#xA;        StringBuilder sb = new StringBuilder();&#xA;        for (int i = 0; i &lt; macBytes.length; i&#x2B;&#x2B;) {&#xA;            sb.append(String.format("%02X%s", macBytes[i], (i &lt; macBytes.length - 1) ? ":" : ""));&#xA;        }&#xA;        return sb.toString();&#xA;    }&#xA;&#xA;    private static void updateWaveformForPlayer(int player) throws Exception {&#xA;        Integer frame_for_player = frameCount.get(player);&#xA;        if (frame_for_player == null) {&#xA;            frame_for_player = 0;&#xA;            frameCount.putIfAbsent(player, frame_for_player);&#xA;        }&#xA;&#xA;        if (!WaveformFinder.getInstance().isRunning()) {&#xA;            WaveformFinder.getInstance().start();&#xA;        }&#xA;        WaveformDetail detail = WaveformFinder.getInstance().getLatestDetailFor(player);&#xA;&#xA;        if (detail != null) {&#xA;            WaveformDetailComponent component = (WaveformDetailComponent) detail.createViewComponent(&#xA;                    MetadataFinder.getInstance().getLatestMetadataFor(player),&#xA;                    BeatGridFinder.getInstance().getLatestBeatGridFor(player)&#xA;            );&#xA;            component.setMonitoredPlayer(player);&#xA;            component.setPlaybackState(player, TimeFinder.getInstance().getTimeFor(player), true);&#xA;            component.setAutoScroll(true);&#xA;            int width = 1200;&#xA;            int height = 200;&#xA;            Dimension dimension = new Dimension(width, height);&#xA;            component.setPreferredSize(dimension);&#xA;            component.setSize(dimension);&#xA;            component.setScale(1);&#xA;            component.doLayout();&#xA;&#xA;            // Create a fresh BufferedImage and clear it before rendering&#xA;            BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);&#xA;            Graphics2D g = image.createGraphics();&#xA;            g.clearRect(0, 0, width, height);  // Clear any old content&#xA;&#xA;            // Draw waveform into the BufferedImage&#xA;            component.paint(g);&#xA;            g.dispose();&#xA;&#xA;            int port = 5004 &#x2B; player;&#xA;            String inputFile = port &#x2B; "_" &#x2B; frame_for_player &#x2B; ".mp4";&#xA;            // Initialize the FFmpegFrameRecorder for YUV420P&#xA;            FFmpegFrameRecorder recorder_file = new FFmpegFrameRecorder(inputFile, width, height);&#xA;            FFmpegLogCallback.set();  // Enable FFmpeg logging for debugging&#xA;            recorder_file.setFormat("mp4");&#xA;            recorder_file.setVideoCodec(avcodec.AV_CODEC_ID_H264);&#xA;            recorder_file.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);  // Use YUV420P format directly&#xA;            recorder_file.setFrameRate(FPS);&#xA;&#xA;            // Set video options&#xA;            recorder_file.setVideoOption("preset", "ultrafast");&#xA;            recorder_file.setVideoOption("tune", "zerolatency");&#xA;            recorder_file.setVideoOption("x264-params", "repeat-headers=1");&#xA;            recorder_file.setGopSize(FPS);&#xA;            try {&#xA;                recorder_file.start();  // Ensure this is called before recording any frames&#xA;                System.out.println("Recorder started successfully for player: " &#x2B; player);&#xA;            } catch (org.bytedeco.javacv.FFmpegFrameRecorder.Exception e) {&#xA;                e.printStackTrace();&#xA;            }&#xA;&#xA;            // Get all pixels in one call&#xA;            int[] pixels = new int[width * height];&#xA;            image.getRGB(0, 0, width, height, pixels, 0, width);&#xA;            recorder_file.recordImage(width,height,Frame.DEPTH_UBYTE,1,3 * width, AV_PIX_FMT_RGB24, ByteBuffer.wrap(argbToByteArray(pixels, width, height)));&#xA;            recorder_file.stop();&#xA;            recorder_file.release();&#xA;            final FFmpegFrameRecorder recorder = recorders.get(player);&#xA;            FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(inputFile);&#xA;&#xA;&#xA;            try {&#xA;                grabber.start();&#xA;            } catch (Exception e) {&#xA;                e.printStackTrace();&#xA;            }&#xA;            if (recorder == null) {&#xA;                try {&#xA;                    String outputStream = "rtp://127.0.0.1:" &#x2B; port;&#xA;                    FFmpegFrameRecorder initial_recorder = new FFmpegFrameRecorder(outputStream, grabber.getImageWidth(), grabber.getImageHeight());&#xA;                    initial_recorder.setFormat("rtp");&#xA;                    initial_recorder.setVideoCodec(avcodec.AV_CODEC_ID_H264);&#xA;                    initial_recorder.setPixelFormat(avutil.AV_PIX_FMT_YUV420P);&#xA;                    initial_recorder.setFrameRate(grabber.getFrameRate());&#xA;                    initial_recorder.setGopSize(FPS);&#xA;                    initial_recorder.setVideoOption("x264-params", "keyint=60");&#xA;                    initial_recorder.setVideoOption("rtsp_transport", "tcp");&#xA;                    initial_recorder.start();&#xA;                    recorders.putIfAbsent(player, initial_recorder);&#xA;                    frameCount.putIfAbsent(player, 0);&#xA;                    putToRTP(player, grabber, initial_recorder);&#xA;                }&#xA;                catch (Exception e) {&#xA;                    e.printStackTrace();&#xA;                }&#xA;            }&#xA;            else {&#xA;                putToRTP(player, grabber, recorder);&#xA;            }&#xA;            File file = new File(inputFile);&#xA;            if (file.exists() &amp;&amp; file.delete()) {&#xA;                System.out.println("Successfully deleted file: " &#x2B; inputFile);&#xA;            } else {&#xA;                System.out.println("Failed to delete file: " &#x2B; inputFile);&#xA;            }&#xA;        }&#xA;    }&#xA;&#xA;    public static void putToRTP(int player, FFmpegFrameGrabber grabber, FFmpegFrameRecorder recorder) throws FrameGrabber.Exception {&#xA;        final Frame frame = grabber.grabFrame();&#xA;        int frameCount_local = frameCount.get(player);&#xA;        frame.keyFrame = frameCount_local&#x2B;&#x2B; % FPS == 0;&#xA;        frameCount.put(player, frameCount_local);&#xA;        try {&#xA;            recorder.record(frame);&#xA;        } catch (FFmpegFrameRecorder.Exception e) {&#xA;            throw new RuntimeException(e);&#xA;        }&#xA;    }&#xA;    public static byte[] argbToByteArray(int[] argb, int width, int height) {&#xA;        int totalPixels = width * height;&#xA;        byte[] byteArray = new byte[totalPixels * 3];  // 4 bytes per pixel (ARGB)&#xA;&#xA;        for (int i = 0; i &lt; totalPixels; i&#x2B;&#x2B;) {&#xA;            int argbPixel = argb[i];&#xA;&#xA;            byteArray[i * 3] = (byte) ((argbPixel >> 16) &amp; 0xFF);  // Red&#xA;            byteArray[i * 3 &#x2B; 1] = (byte) ((argbPixel >> 8) &amp; 0xFF);   // Green&#xA;            byteArray[i * 3 &#x2B; 2] = (byte) (argbPixel &amp; 0xFF);  // Blue&#xA;        }&#xA;&#xA;        return byteArray;&#xA;    }&#xA;&#xA;&#xA;    public static void main(String[] args) throws Exception {&#xA;        VirtualCdj.getInstance().setDeviceNumber((byte) 4);&#xA;        CrateDigger.getInstance().addDatabaseListener(new DBService());&#xA;        props.put("bootstrap.servers", "localhost:9092");&#xA;        props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");&#xA;        props.put("value.serializer", "com.bugbytz.prolink.CustomSerializer");&#xA;        props.put(ProducerConfig.MAX_REQUEST_SIZE_CONFIG, "20971520");&#xA;&#xA;        VirtualCdj.getInstance().addUpdateListener(update -> {&#xA;            if (update instanceof CdjStatus) {&#xA;                try (Producer producer = new KafkaProducer&lt;>(props)) {&#xA;                    DecimalFormat df_obj = new DecimalFormat("#.##");&#xA;                    DeviceStatus deviceStatus = new DeviceStatus(&#xA;                            update.getDeviceNumber(),&#xA;                            ((CdjStatus) update).isPlaying() || !((CdjStatus) update).isPaused(),&#xA;                            ((CdjStatus) update).getBeatNumber(),&#xA;                            update.getBeatWithinBar(),&#xA;                            Double.parseDouble(df_obj.format(update.getEffectiveTempo())),&#xA;                            Double.parseDouble(df_obj.format(Util.pitchToPercentage(update.getPitch()))),&#xA;                            update.getAddress().getHostAddress(),&#xA;                            byteArrayToMacString(DeviceFinder.getInstance().getLatestAnnouncementFrom(update.getDeviceNumber()).getHardwareAddress()),&#xA;                            ((CdjStatus) update).getRekordboxId(),&#xA;                            update.getDeviceName()&#xA;                    );&#xA;                    ProducerRecord record = new ProducerRecord&lt;>("device-status", "device-" &#x2B; update.getDeviceNumber(), deviceStatus);&#xA;                    try {&#xA;                        producer.send(record).get();&#xA;                    } catch (InterruptedException ex) {&#xA;                        throw new RuntimeException(ex);&#xA;                    } catch (ExecutionException ex) {&#xA;                        throw new RuntimeException(ex);&#xA;                    }&#xA;                    producer.flush();&#xA;                    if (!WaveformFinder.getInstance().isRunning()) {&#xA;                        try {&#xA;                            WaveformFinder.getInstance().start();&#xA;                        } catch (Exception ex) {&#xA;                            throw new RuntimeException(ex);&#xA;                        }&#xA;                    }&#xA;                }&#xA;            }&#xA;        });&#xA;        DeviceFinder.getInstance().addDeviceAnnouncementListener(new DeviceAnnouncementAdapter() {&#xA;            @Override&#xA;            public void deviceFound(DeviceAnnouncement announcement) {&#xA;                if (!streamingPlayers.contains(announcement.getDeviceNumber())) {&#xA;                    streamingPlayers.add(announcement.getDeviceNumber());&#xA;                    schedules.putIfAbsent(announcement.getDeviceNumber(), scheduler.scheduleAtFixedRate(() -> {&#xA;                        try {&#xA;                            Runnable task = () -> {&#xA;                                try {&#xA;                                    updateWaveformForPlayer(announcement.getDeviceNumber());&#xA;                                } catch (InterruptedException e) {&#xA;                                    System.out.println("Thread interrupted");&#xA;                                } catch (Exception e) {&#xA;                                    throw new RuntimeException(e);&#xA;                                }&#xA;                                System.out.println("Lambda thread work completed!");&#xA;                            };&#xA;                            task.run();&#xA;                        } catch (Exception e) {&#xA;                            e.printStackTrace();&#xA;                        }&#xA;                    }, 0, FRAME_INTERVAL_MS, TimeUnit.MILLISECONDS));&#xA;                }&#xA;            }&#xA;&#xA;            @Override&#xA;            public void deviceLost(DeviceAnnouncement announcement) {&#xA;                if (streamingPlayers.contains(announcement.getDeviceNumber())) {&#xA;                    schedules.get(announcement.getDeviceNumber()).cancel(true);&#xA;                    streamingPlayers.remove(announcement.getDeviceNumber());&#xA;                }&#xA;            }&#xA;        });&#xA;        BeatGridFinder.getInstance().start();&#xA;        MetadataFinder.getInstance().start();&#xA;        VirtualCdj.getInstance().start();&#xA;        TimeFinder.getInstance().start();&#xA;        DeviceFinder.getInstance().start();&#xA;        CrateDigger.getInstance().start();&#xA;&#xA;        try {&#xA;            LoadCommandConsumer consumer = new LoadCommandConsumer("localhost:9092", "load-command-group");&#xA;            Thread consumerThread = new Thread(consumer::startConsuming);&#xA;            consumerThread.start();&#xA;&#xA;            Runtime.getRuntime().addShutdownHook(new Thread(() -> {&#xA;                consumer.shutdown();&#xA;                try {&#xA;                    consumerThread.join();&#xA;                } catch (InterruptedException e) {&#xA;                    Thread.currentThread().interrupt();&#xA;                }&#xA;            }));&#xA;            Thread.sleep(60000);&#xA;        } catch (InterruptedException e) {&#xA;            System.out.println("Interrupted, exiting.");&#xA;        }&#xA;    }&#xA;}&#xA;</integer></track>

    &#xA;

  • Introducing the BigQuery & Data Warehouse Export feature

    30 janvier, par Matomo Core Team

    Matomo is built on a simple truth : your data belongs to you, and you should have complete control over it. That’s why we’re excited to launch our new BigQuery & Data Warehouse Export feature for Matomo Cloud, giving you even more ways to work with your analytics data. 

    Until now, getting raw data from Matomo Cloud required APIs and custom scripts, or waiting for engineering help.  

    Our new BigQuery & Data Warehouse Export feature removes those barriers. You can now access your raw, unaggregated data and schedule regular exports straight to your data warehouse. 

    The feature works with all major data warehouses including (but not limited to) : 

    • Google BigQuery 
    • Amazon Redshift 
    • Snowflake 
    • Azure Synapse Analytics 
    • Apache Hive 
    • Teradata 

    You can schedule exports, combine your Matomo data with other data sources in your data warehouse, and easily query data with SQL-like queries. 

    Direct raw data access for greater data portability 

    Waiting for engineering support can delay your work. Managing API connections and writing scripts can be time-consuming. This keeps you from focusing on what you do best—analysing data. 

    BigQuery create-table-menu

    With the BigQuery & Data Warehouse Export feature, you get direct access to your raw Matomo data without the technical setup. So, you can spend more time analysing data and finding insights that matter. 

    Bringing your data together 

    Answering business questions often requires data from multiple sources. A single customer interaction might span your CRM, web analytics, sales systems, and more. Piecing this data together manually is time-consuming—what starts as a seemingly simple question from stakeholders can turn into hours of work collecting and comparing data across different tools. 

    This feature lets you combine your Matomo data with data from other business systems in your data warehouse. Instead of switching between tools or manually comparing spreadsheets, you can analyse all your data in one place to better understand how customers interact with your business. 

    Easy, custom analysis with SQL-like queries 

    Standard, pre-built reports often don’t address the specific, detailed questions that analysts need to answer.  

    When you use the BigQuery & Data Warehouse Export feature, you can use SQL-like queries in your data warehouse to do detailed, customised analysis. This flexibility allows you to explore your data in depth and uncover specific insights that aren’t possible with pre-built reports. 

    Here is an example of how you might use SQL-like query to compare the behaviours of paying vs. non-paying users : 

    				
                                            <xmp>SELECT  

    custom_dimension_value AS user_type, -- Assuming 'user_type' is stored in a custom dimension

    COUNT(*) AS total_visits,  

    AVG(visit_total_time) AS avg_duration,

    SUM(conversion.revenue) AS total_spent  

    FROM  

    `your_project.your_dataset.matomo_log_visit` AS visit

    LEFT JOIN  

    `your_project.your_dataset.matomo_log_conversion` AS conversion  

    ON  

    visit.idvisit = conversion.idvisit  

    GROUP BY  

    custom_dimension_value; </xmp>
                                   

    This query helps you compare metrics such as the number of visits, average session duration, and total amount spent between paying and non-paying users. It provides a full view of behavioural differences between these groups. 

    Advanced data manipulation and visualisation 

    When you need to create detailed reports or dive deep into data analysis, working within the constraints of a fixed user interface (UI) can limit your ability to draw insights. 

    Exporting your Matomo data to a data warehouse like BigQuery provides greater flexibility for in-depth manipulation and advanced visualisations, enabling you to uncover deeper insights and tailor your reports more effectively. 

    Getting started 

    To set up data warehouse exports in your Matomo : 

    1. Go to System Admin (cog icon in the top right corner) 
    2. Select ‘Export’ from the left-hand menu 
    3. Choose ‘BigQuery & Data Warehouse’ 

    You’ll find detailed instructions in our data warehouse exports guide 

    Please note, enabling this feature will cost an additional 10% of your current subscription. You can view the exact cost by following the steps above. 

    New to Matomo ? Start your 21-day free trial now (no credit card required), or request a demo.