Sur d’autres sites (13491)

• Capture from multiple streams concurrently, best way to do it and how to reduce CPU usage

  19 juin 2019, par DRONE_6969

  I am currently in the process of writing an application that will capture a lot of RTSP streams(in my case its 12) and display it on the QT widget. The problem arouses when I am going beyond around 6-7 streams, the CPU usage spikes and there is visible stutter.

  The reason why I think that it is not QT draw function is because I have done some checking to measure how much time it takes to draw an incoming image from camera and just sample images I had, it is always a lot less than 33 milliseconds(even if there are 12 widgets being updated).

  I also just ran opencv capture method without drawing and got pretty much the same CPU consumption as if I was drawing the frames (lost like 10% CPU at most and GPU usage went to zero).

  IMPORTANT : I am using RTSP stream which is a h264 stream.

  IF IT MATTERS MY SPECS :

  Intel Core i7-6700 @ 3.40GHZ(8 CPUS)
  Memory : 16gb
  GPU : Intel HD Graphics 530

  (Also I ran my code on a computer with dedicated Graphics card, it did eliminate some stutter but CPU usage is still pretty high)

  I am currently using OPENCV 4.1.0 with GSTREAMER enabled and built, I also have the OPENCV-WORLD version, there is no difference in performance.

  I have created a special class called Camera that holds its frame size constraints and various control functions as well stream function. The stream function is being ran on a separate thread, whenever stream() function is done with current frame it sends ready Mat via onNewFrame event I created which converts to QPixmap and updates widget’s lastImage variable. This way I can update image in a more thread safe way.

  I have tried to manipulate those VideoCapture.set() values, but it didn’t really help.

  This is my stream function (Ignore the bool return, it doesn’t do anything it is a remnant from couple of minutes ago when I was trying to use std::async) :

  bool Camera::stream() {
  
      /* This function is meant to run on a separate thread and fill up the buffer independantly of
  
      main stream thread */
  
      //cv::setNumThreads(100);
  
      /* Rules for these slightly changed! */
  
      Mat pre;  // Grab initial undoctored frame
  
      //pre = Mat::zeros(size, CV_8UC1);
  
      Mat frame; // Final modified frame
  
      frame = Mat::zeros(size, CV_8UC1);
  
      if (!pre.isContinuous()) pre = pre.clone();
  
  
  
      ipCam.open(streamUrl, CAP_FFMPEG);
  
  
  
  
  
      while (ipCam.isOpened() && capture) {
  
          // If camera is opened wel need to capture and process the frame
  
          try {
  
              auto start = std::chrono::system_clock::now();
  
  
  
              ipCam >> pre;
  
  
  
              if (pre.empty()) {
  
                  /* Check for blank frame, return error if there is a blank frame*/
  
                  cerr << id << ": ERROR! blank frame grabbed\n";
  
                  for (FrameListener* i : clients) {
  
                      i->onNotification(1); // Notify clients about this shit
  
                  }
  
                  break;
  
              }
  
  
  
              else {
  
                  // Only continue if frame not empty
  
  
  
                  if (pre.cols != size.width && pre.rows != size.height) {
  
                      resize(pre, frame, size);
  
                      pre.release();
  
                  }
  
                  else {
  
                      frame = pre;
  
                  }
  
  
  
                  dPacket* pack = new dPacket{id,&frame};
  
                  for (auto i : clients) {
  
                      i->onPNewFrame(pack);
  
                  }
  
                  frame.release();
  
                  delete pack;
  
              }
  
          }
  
  
  
          catch (int e) {
  
              cout << endl << "-----Exception during capture process! CODE " << e << endl;
  
          }
  
          // End camera manipulations
  
      }
  
  
  
      cout << "Camera timed out, or connection is closed..." << endl;
  
      if (tryResetConnection) {
  
          cout << "Reconnection flag is set, retrying after 3 seconds..." << endl;
  
          for (FrameListener* i : clients) {
  
              i->onNotification(-1); // Notify clients about this shit
  
          }
  
          this_thread::sleep_for(chrono::milliseconds(3000));
  
          stream();
  
      }
  
  
  
      return true;
  
  }

  This is my onPNewFrame function. The conversion is still being done on camera’s thread because it was called within stream() and therefore is within that scope(and I also checked) :

  void GLWidget::onPNewFrame(dPacket* inPack) {
  
      lastFlag = 0;
  
  
  
      if (bufferEnabled) {
  
          buffer.push(QPixmap::fromImage(toQImageFromPMat(inPack->frame)));
  
      }
  
      else {
  
          if (playing) {
  
              /* Only process if this widget is playing */
  
              frameProcessing = true;
  
              lastImage.convertFromImage(toQImageFromPMat(inPack->frame));
  
              frameProcessing = false;
  
          }
  
      }
  
  
  
      if (lastFlag != -1 && !lastImage.isNull()) {
  
          connecting = false;
  
      }
  
      else {
  
          connecting = true;
  
      }
  
  }

  This is my Mat to QImage :

  QImage GLWidget::toQImageFromPMat(cv::Mat* mat) {
  
  
  
  
  
  
  
      return QImage(mat->data, mat->cols, mat->rows, QImage::Format_RGB888).rgbSwapped();

  NOTE : not converting does not result in CPU boost (at least not a significant one).

  Minimal verifiable example

  This program is large. I am going to paste GLWidget.cpp and GLWidget.h as well as Camera.h and Camera.cpp. You can put GLWidget into anything just as long as you spawn more than 6 of it. Camera relies on the CamUtils, but it is possible to just paste url in videocapture

  I also supplied CamUtils, just in case

  Camera.h :

  #pragma once
  
  #include <iostream>
  
  #include <vector>
  
  #include <fstream>
  
  #include <map>
  
  #include <string>
  
  #include <sstream>
  
  #include <algorithm>
  
  #include "FrameListener.h"
  
  #include 
  
  #include <thread>
  
  #include "CamUtils.h"
  
  #include <ctime>
  
  #include "dPacket.h"
  
  
  
  using namespace std;
  
  using namespace cv;
  
  
  
  class Camera
  
  {
  
  
  
      /*
  
          CLEANED UP!
  
          Camera now is only responsible for streaming and echoing captured frames.
  
          Frames are now wrapped into dPacket struct.
  
      */
  
  
  
  
  
  private:
  
      string id;
  
      vector clients;
  
      VideoCapture ipCam;
  
      string streamUrl;
  
      Size size;
  
      bool tryResetConnection = false;
  
  
  
      //TODO: Remove these as they are not going to be used going on:
  
      bool isPlaying = true;
  
      bool capture = true;
  
  
  
      //SECRET FEATURES:
  
      bool detect = false;
  
  
  
  
  
  public:
  
      Camera(string url, int width = 480, int height = 240, bool detect_=false);
  
      bool stream();
  
      void setReconnectable(bool newReconStatus);
  
      void addListener(FrameListener* client);
  
      vector<bool> getState();    // Returns current state: vector[0] stream state; vector[1] stream state; TODO: Remove this as this is no longer should control behaviour
  
      void killStream();
  
      bool getReconnectable();
  
  };
  
  
  
  </bool></ctime></thread></algorithm></sstream></string></map></fstream></vector></iostream>

  Camera.cpp

  #include "Camera.h"
  
  
  
  
  
  Camera::Camera(string url, int width, int height, bool detect_) // Default 240p
  
  {
  
      streamUrl = url; // Prepare url
  
      size = Size(width, height);
  
      detect = detect_;
  
  
  
  }
  
  
  
  void Camera::addListener(FrameListener* client) {
  
      clients.push_back(client);
  
  }
  
  
  
  
  
  /*
  
                  TEST CAMERAS(Paste into cameras.dViewer):
  
                  {"id":"96a73796-c129-46fc-9c01-40acd8ed7122","ip":"176.57.73.231","password":"null","username":"null"},
  
                  {"id":"96a73796-c129-46fc-9c01-40acd8ed7122","ip":"176.57.73.231","password":"null","username":"null"},
  
                  {"id":"96a73796-c129-46fc-9c01-40acd8ed7144","ip":"172.20.101.13","password":"admin","username":"root"}
  
                  {"id":"96a73796-c129-46fc-9c01-40acd8ed7144","ip":"172.20.101.13","password":"admin","username":"root"}
  
  
  
  */
  
  
  
  
  
  
  
  bool Camera::stream() {
  
      /* This function is meant to run on a separate thread and fill up the buffer independantly of
  
      main stream thread */
  
      //cv::setNumThreads(100);
  
      /* Rules for these slightly changed! */
  
      Mat pre;  // Grab initial undoctored frame
  
      //pre = Mat::zeros(size, CV_8UC1);
  
      Mat frame; // Final modified frame
  
      frame = Mat::zeros(size, CV_8UC1);
  
      if (!pre.isContinuous()) pre = pre.clone();
  
  
  
      ipCam.open(streamUrl, CAP_FFMPEG);
  
  
  
      while (ipCam.isOpened() && capture) {
  
          // If camera is opened wel need to capture and process the frame
  
          try {
  
              auto start = std::chrono::system_clock::now();
  
  
  
              ipCam >> pre;
  
  
  
              if (pre.empty()) {
  
                  /* Check for blank frame, return error if there is a blank frame*/
  
                  cerr << id << ": ERROR! blank frame grabbed\n";
  
                  for (FrameListener* i : clients) {
  
                      i->onNotification(1); // Notify clients about this shit
  
                  }
  
                  break;
  
              }
  
  
  
              else {
  
                  // Only continue if frame not empty
  
  
  
                  if (pre.cols != size.width && pre.rows != size.height) {
  
                      resize(pre, frame, size);
  
                      pre.release();
  
                  }
  
                  else {
  
                      frame = pre;
  
                  }
  
  
  
                  auto end = std::chrono::system_clock::now();
  
                  std::time_t ts = std::chrono::system_clock::to_time_t(end);
  
                  dPacket* pack = new dPacket{ id,&frame};
  
                  for (auto i : clients) {
  
                      i->onPNewFrame(pack);
  
                  }
  
                  frame.release();
  
                  delete pack;
  
              }
  
          }
  
  
  
          catch (int e) {
  
              cout << endl << "-----Exception during capture process! CODE " << e << endl;
  
          }
  
          // End camera manipulations
  
      }
  
  
  
      cout << "Camera timed out, or connection is closed..." << endl;
  
      if (tryResetConnection) {
  
          cout << "Reconnection flag is set, retrying after 3 seconds..." << endl;
  
          for (FrameListener* i : clients) {
  
              i->onNotification(-1); // Notify clients about this shit
  
          }
  
          this_thread::sleep_for(chrono::milliseconds(3000));
  
          stream();
  
      }
  
  
  
      return true;
  
  }
  
  
  
  
  
  void Camera::killStream(){
  
      tryResetConnection = false;
  
      capture = false;
  
      ipCam.release();
  
  }
  
  
  
  void Camera::setReconnectable(bool reconFlag) {
  
      tryResetConnection = reconFlag;
  
  }
  
  
  
  bool Camera::getReconnectable() {
  
      return tryResetConnection;
  
  }
  
  
  
  vector<bool> Camera::getState() {
  
      vector<bool> states;
  
      states.push_back(isPlaying);
  
      states.push_back(ipCam.isOpened());
  
      return states;
  
  }
  
  
  
  
  
  
  
  </bool></bool>

  GLWidget.h :

  #ifndef GLWIDGET_H
  
  #define GLWIDGET_H
  
  
  
  #include <qopenglwidget>
  
  #include <qmouseevent>
  
  #include "FrameListener.h"
  
  #include "Camera.h"
  
  #include "FrameListener.h"
  
  #include 
  
  #include "Camera.h"
  
  #include "CamUtils.h"
  
  #include 
  
  #include "dPacket.h"
  
  #include <chrono>
  
  #include <ctime>
  
  #include 
  
  #include "FullScreenVideo.h"
  
  #include <qmovie>
  
  #include "helper.h"
  
  #include <iostream>
  
  #include <qpainter>
  
  #include <qtimer>
  
  
  
  class Helper;
  
  
  
  class GLWidget : public QOpenGLWidget, public FrameListener
  
  {
  
      Q_OBJECT
  
  
  
  public:
  
      GLWidget(std::string camId, CamUtils *cUtils, int width, int height, bool denyFullScreen_ = false, bool detectFlag_=false, QWidget* parent = nullptr);
  
      void killStream();
  
      ~GLWidget();
  
  
  
  public slots:
  
      void animate();
  
      void setBufferEnabled(bool setState);
  
      void setCameraRetryConnection(bool setState);
  
      void GLUpdate();            // Call to update the widget
  
      void onRightClickMenu(const QPoint&amp; point);
  
  
  
  protected:
  
      void paintEvent(QPaintEvent* event) override;
  
      void onPNewFrame(dPacket* frame);
  
      void onNotification(int alert_code);
  
  
  
  
  
  private:
  
      // Objects and resourses
  
      Helper* helper;
  
      Camera* cam;
  
      CamUtils* camUtils;
  
      QTimer* timer; // Keep track of update
  
      QPixmap lastImage;
  
      QMovie* connMov;
  
      QMovie* test;
  
  
  
      QPixmap logo;
  
  
  
      // Control fields
  
      int width;
  
      int height;
  
      int camUtilsAddr;
  
      int elapsed;
  
      std::thread* camThread;
  
      std::string camId;
  
      bool denyFullScreen = false;
  
      bool playing = true;
  
      bool streaming = true;
  
      bool debug = false;
  
      bool connecting = true;
  
      int lastFlag = 0;
  
  
  
  
  
      // Debug fields
  
      std::chrono::high_resolution_clock::time_point lastFrameAt;
  
      std::chrono::high_resolution_clock::time_point now;
  
      std::chrono::duration<double> painTime; // time took to draw last frame
  
  
  
      //Buffer stuff
  
      std::queue<qpixmap> buffer;
  
      bool bufferEnabled = false;
  
      bool initialBuffer = false;
  
      bool buffering = true;
  
      bool frameProcessing = false;
  
  
  
  
  
  
  
      //Functions
  
      QImage toQImageFromPMat(cv::Mat* inFrame);
  
      void mousePressEvent(QMouseEvent* event) override;
  
      void drawImageGLLatest(QPainter* painter, QPaintEvent* event, int elapsed);
  
      void drawOnPaused(QPainter* painter, QPaintEvent* event, int elapsed);
  
      void drawOnStatus(int statusFlag, QPainter* painter, QPaintEvent* event, int elapsed);
  
  };
  
  
  
  #endif
  
  
  
  </qpixmap></double></qtimer></qpainter></iostream></qmovie></ctime></chrono></qmouseevent></qopenglwidget>

  GLWidget.cpp :

  #include "glwidget.h"
  
  #include <future>
  
  
  
  
  
  FullScreenVideo* fullScreen;
  
  
  
  GLWidget::GLWidget(std::string camId_, CamUtils* cUtils, int width_, int height_,  bool denyFullScreen_, bool detectFlag_, QWidget* parent)
  
      : QOpenGLWidget(parent), helper(helper)
  
  {
  
      cout &lt;&lt; "Player for CAMERA " &lt;&lt; camId_ &lt;&lt; endl;
  
  
  
      /* Underlying properties */
  
      camUtils = cUtils;
  
      cout &lt;&lt; "GLWidget Incoming CamUtils addr " &lt;&lt; camUtils &lt;&lt; endl;
  
      cout &lt;&lt; "GLWidget Set CamUtils addr " &lt;&lt; camUtils &lt;&lt; endl;
  
      camId = camId_;
  
      elapsed = 0;
  
      width = width_ + 5;
  
      height = height_ + 5;
  
      helper = new Helper();
  
      setFixedSize(width, height);
  
      denyFullScreen = denyFullScreen_;
  
  
  
      /* Camera capture thread */
  
      cam = new Camera(camUtils->getCameraStreamURL(camId), width_, height_, detectFlag_);
  
      cam->addListener(this);
  
  
  
      /* Sync states */
  
      vector<bool> initState = cam->getState();
  
      playing = initState[0];
  
      streaming = initState[1];
  
      cout &lt;&lt; "Initial states: " &lt;&lt; playing &lt;&lt; " " &lt;&lt; streaming &lt;&lt; endl;
  
      camThread = new std::thread(&amp;Camera::stream, cam);
  
      cout &lt;&lt; "================================================" &lt;&lt; endl;
  
  
  
      // Right click set up
  
      setContextMenuPolicy(Qt::CustomContextMenu);
  
  
  
  
  
      /* Loading gif */
  
      connMov = new QMovie("establishingConnection.gif");
  
      connMov->start();
  
      QString url = R"(RLC-logo.png)";
  
      logo = QPixmap(url);
  
      QTimer* timer = new QTimer(this);
  
      connect(timer, SIGNAL(timeout()), this, SLOT(GLUpdate()));
  
      timer->start(1000/30);
  
      playing = true;
  
  
  
  }
  
  
  
  /* SYSTEM */
  
  void GLWidget::animate()
  
  {
  
      elapsed = (elapsed + qobject_cast(sender())->interval()) % 1000;
  
      std::cout &lt;&lt; elapsed &lt;&lt; "\n";
  
  }
  
  
  
  
  
  void GLWidget::GLUpdate() {
  
      /* Process descisions before update call */
  
      if (bufferEnabled) {
  
          /* Process buffer before update */
  
          now = chrono::high_resolution_clock::now();
  
          std::chrono::duration timeSinceLastUpdate = now - lastFrameAt;
  
          if (timeSinceLastUpdate.count() > 25) {
  
              if (buffer.size() > 1 &amp;&amp; playing) {
  
                  lastImage.swap(buffer.front());
  
                  buffer.pop();
  
                  lastFrameAt = chrono::high_resolution_clock::now();
  
              }
  
          }
  
          //update(); // Update
  
      }
  
      else {
  
          /* No buffer */
  
      }
  
      repaint();
  
  }
  
  
  
  
  
  /* EVENTS */
  
  void GLWidget::onRightClickMenu(const QPoint&amp; point) {
  
      cout &lt;&lt; "Right click request got" &lt;&lt; endl;
  
  
  
      QPoint globPos = this->mapToGlobal(point);
  
      QMenu myMenu;
  
  
  
      if (!denyFullScreen) {
  
          myMenu.addAction("Open Full Screen");
  
      }
  
      myMenu.addAction("Toggle Debug Info");
  
  
  
  
  
      QAction* selected = myMenu.exec(globPos);
  
  
  
      if (selected) {
  
          string optiontxt = selected->text().toStdString();
  
  
  
          if (optiontxt == "Open Full Screen") {
  
              cout &lt;&lt; "Chose to open full screen of " &lt;&lt; camId &lt;&lt; endl;
  
              fullScreen = new FullScreenVideo(bufferEnabled, this);
  
              fullScreen->setUpView(camUtils, camId);
  
              fullScreen->show();
  
              playing = false;
  
          }
  
  
  
          if (optiontxt == "Toggle Debug Info") {
  
              cout &lt;&lt; "Chose to toggle debug of " &lt;&lt; camId &lt;&lt; endl;
  
              debug = !debug;
  
          }
  
      }
  
      else {
  
          cout &lt;&lt; "Chose nothing!" &lt;&lt; endl;
  
      }
  
  
  
  
  
  }
  
  
  
  
  
  
  
  void GLWidget::onPNewFrame(dPacket* inPack) {
  
      lastFlag = 0;
  
  
  
      if (bufferEnabled) {
  
          buffer.push(QPixmap::fromImage(toQImageFromPMat(inPack->frame)));
  
      }
  
      else {
  
          if (playing) {
  
              /* Only process if this widget is playing */
  
              frameProcessing = true;
  
              lastImage.convertFromImage(toQImageFromPMat(inPack->frame));
  
              frameProcessing = false;
  
          }
  
      }
  
  
  
      if (lastFlag != -1 &amp;&amp; !lastImage.isNull()) {
  
          connecting = false;
  
      }
  
      else {
  
          connecting = true;
  
      }
  
  }
  
  
  
  
  
  void GLWidget::onNotification(int alert) {
  
      lastFlag = alert;   
  
  }
  
  
  
  
  
  /* Paint events*/
  
  
  
  
  
  void GLWidget::paintEvent(QPaintEvent* event)
  
  {
  
      QPainter painter(this);
  
  
  
          if (lastFlag != 0 || connecting) {
  
              drawOnStatus(lastFlag, &amp;painter, event, elapsed);
  
          }
  
          else {
  
  
  
              /* Actual frame drawing */
  
              if (playing) {
  
                  if (!frameProcessing) {
  
                      drawImageGLLatest(&amp;painter, event, elapsed);
  
                  }
  
              }
  
              else {
  
                  drawOnPaused(&amp;painter, event, elapsed);
  
              }
  
          }
  
      painter.end();
  
  
  
  }
  
  
  
  
  
  /* DRAWING STUFF */
  
  
  
  void GLWidget::drawOnStatus(int statusFlag, QPainter* bgPaint, QPaintEvent* event, int elapsed) {
  
  
  
      QString str;
  
      QFont font("times", 15);
  
      bgPaint->eraseRect(QRect(0, 0, width, height));
  
      if (!lastImage.isNull()) {
  
          bgPaint->drawPixmap(QRect(0, 0, width, height), lastImage);
  
      }
  
      /* Test background painting */
  
      if (connecting) {
  
          string k = "Connecting to " + camUtils->getIp(camId);
  
          str.append(k.c_str());
  
      }
  
      else {
  
          switch (statusFlag) {
  
          case 1:
  
              str = "Blank frame received...";
  
              break;
  
  
  
          case -1:
  
              if (cam->getReconnectable()) {
  
                  str = "Connection lost, will try to reconnect.";
  
                  bgPaint->setOpacity(0.3);
  
              }
  
              else {
  
                  str = "Connection lost...";
  
                  bgPaint->setOpacity(0.3);
  
              }
  
  
  
              break;
  
          }
  
      }
  
  
  
      bgPaint->drawPixmap(QRect(0, 0, width, height), QPixmap::fromImage(connMov->currentImage()));
  
      bgPaint->setPen(Qt::red);
  
      bgPaint->setFont(font);
  
      QFontMetrics fm(font);
  
      const QRect kek(0, 0, fm.width(str), fm.height());
  
      QRect bound;
  
      bgPaint->setOpacity(1);
  
      bgPaint->drawText(bgPaint->viewport().width()/2 - kek.width()/2, bgPaint->viewport().height()/2 - kek.height(), str);
  
  
  
      bgPaint->drawPixmap(bgPaint->viewport().width() / 2 - logo.width()/2, height - logo.width() - 15, logo);
  
  
  
  }
  
  
  
  
  
  
  
  void GLWidget::drawOnPaused(QPainter* painter, QPaintEvent* event, int elapsed) {
  
      painter->eraseRect(0, 0, width, height);
  
      QFont font = painter->font();
  
      font.setPointSize(18);
  
      painter->setPen(Qt::red);
  
      QFontMetrics fm(font);
  
      QString str("Paused");
  
      painter->drawPixmap(QRect(0, 0, width, height),lastImage);
  
      painter->drawText(QPoint(painter->viewport().width() - fm.width(str), 50), str);
  
  
  
      if (debug) {
  
          QFont font = painter->font();
  
          font.setPointSize(25);
  
          painter->setPen(Qt::red);
  
          string camMess = "CAMID: " + camId;
  
          QString mess(camMess.c_str());
  
          string camIp = "IP: " + camUtils->getIp(camId);
  
          QString ipMess(camIp.c_str());
  
          QString bufferSize("Buffer size: " + QString::number(buffer.size()));
  
          QString lastFrameText("Last frame draw time: " + QString::number(painTime.count()) + "s");
  
          painter->drawText(QPoint(10, 50), mess);
  
          painter->drawText(QPoint(10, 60), ipMess);
  
          QString bufferState;
  
          if (bufferEnabled) {
  
              bufferState = QString("Experimental BUFFER is enabled!");
  
              QString currentBufferSize("Current buffer load: " + QString::number(buffer.size()));
  
              painter->drawText(QPoint(10, 80), currentBufferSize);
  
          }
  
          else {
  
              bufferState = QString("Experimental BUFFER is disabled!");
  
          }
  
          painter->drawText(QPoint(10, 70), bufferState);
  
          painter->drawText(QPoint(10, height - 25), lastFrameText);
  
      }
  
  }
  
  
  
  
  
  void GLWidget::drawImageGLLatest(QPainter* painter, QPaintEvent* event, int elapsed) {
  
      auto start = chrono::high_resolution_clock::now();
  
      painter->drawPixmap(QRect(0, 0, width, height), lastImage);
  
      if (debug) {
  
          QFont font = painter->font();
  
          font.setPointSize(25);
  
          painter->setPen(Qt::red);
  
          string camMess = "CAMID: " + camId;
  
          QString mess(camMess.c_str());
  
          string camIp = "IP: " + camUtils->getIp(camId);
  
          QString ipMess(camIp.c_str());
  
          QString bufferSize("Buffer size: " + QString::number(buffer.size()));
  
          QString lastFrameText("Last frame draw time: " + QString::number(painTime.count()) + "s");
  
          painter->drawText(QPoint(10, 50), mess);
  
          painter->drawText(QPoint(10, 60), ipMess);
  
          QString bufferState;
  
          if(bufferEnabled){
  
              bufferState = QString("Experimental BUFFER is enabled!");
  
              QString currentBufferSize("Current buffer load: " + QString::number(buffer.size()));
  
              painter->drawText(QPoint(10,80), currentBufferSize);
  
          }
  
          else {
  
              bufferState = QString("Experimental BUFFER is disabled!");
  
              QString currentBufferSize("Current buffer load: " + QString::number(buffer.size()));
  
              painter->drawText(QPoint(10, 80), currentBufferSize);
  
          }
  
          painter->drawText(QPoint(10, 70), bufferState);
  
          painter->drawText(QPoint(10, height - 25), lastFrameText);
  
  
  
      }
  
      auto end = chrono::high_resolution_clock::now();
  
      painTime = end - start;
  
  }
  
  
  
  
  
  
  
  /* END DRAWING STUFF */
  
  
  
  
  
  
  
  /* UI EVENTS */
  
  
  
  void GLWidget::mousePressEvent(QMouseEvent* e) {
  
  
  
      if (e->button() == Qt::LeftButton) {
  
          if (fullScreen == nullptr || !fullScreen->isVisible()) { // Do not unpause if window is opened
  
              playing = !playing;
  
          }
  
      }
  
  
  
      if (e->button() == Qt::RightButton) {
  
          onRightClickMenu(e->pos());
  
      }
  
  }
  
  
  
  
  
  
  
  /* Utilities */
  
  QImage GLWidget::toQImageFromPMat(cv::Mat* mat) {
  
  
  
  
  
  
  
      return QImage(mat->data, mat->cols, mat->rows, QImage::Format_RGB888).rgbSwapped();
  
  
  
  
  
  
  
  }
  
  
  
  /* State control */
  
  
  
  void GLWidget::killStream() {
  
      cam->killStream();
  
      camThread->join();
  
  }
  
  
  
  void GLWidget::setBufferEnabled(bool newBufferState) {
  
      cout &lt;&lt; "Player: " &lt;&lt; camId &lt;&lt; ", buffer state updated: " &lt;&lt; newBufferState &lt;&lt; endl;
  
      bufferEnabled = newBufferState;
  
      buffer.empty();
  
  }
  
  
  
  void GLWidget::setCameraRetryConnection(bool newState) {
  
      cam->setReconnectable(newState);
  
  }
  
  
  
  /* Destruction */
  
  GLWidget::~GLWidget() {
  
      cam->killStream();
  
      camThread->join();
  
  }
  
  </bool></future>

  CamUtils.h :

  #pragma once
  
  #include <iostream>
  
  #include <vector>
  
  #include <fstream>
  
  #include <map>
  
  #include <string>
  
  #include <sstream>
  
  #include <algorithm>
  
  #include <nlohmann></nlohmann>json.hpp>
  
  
  
  using namespace std;
  
  using json = nlohmann::json;
  
  
  
  class CamUtils
  
  {
  
  private:
  
  
  
      string camDb = "cameras.dViewer";
  
      map> cameraList; // Legacy
  
      json cameras;
  
      ofstream dbFile;
  
      bool dbExists(); // Always hard coded
  
  
  
      /* Old IMPLEMENTATION */
  
      void writeLineToDb_(const string&amp; content, bool append = false);
  
      void loadCameras_();
  
  
  
      /* JSON based */
  
      void loadCameras();
  
  
  
  public:
  
      CamUtils();
  
      string generateRandomString(size_t length);
  
      string getCameraStreamURL(string cameraId) const;
  
      string saveCamera(string ip, string username, string pass); // Return generated id
  
      vector<string> listAllCameraIds();
  
      string getIp(string cameraId);
  
  };
  
  
  
  
  
  </string></algorithm></sstream></string></map></fstream></vector></iostream>

  CamUtils.cpp :

  #include "CamUtils.h"
  
  #pragma comment(lib, "rpcrt4.lib")  // UuidCreate - Minimum supported OS Win 2000
  
  #include 
  
  #include <iostream>
  
  
  
  CamUtils::CamUtils()
  
  {
  
      if (!dbExists()) {
  
          ofstream dbFile;
  
          dbFile.open(camDb);
  
          cameras["cameras"] = json::array();
  
          dbFile &lt;&lt; cameras &lt;&lt; std::endl;
  
          dbFile.close();
  
  
  
      }
  
      else {
  
          loadCameras();
  
      }
  
  }
  
  
  
  
  
  
  
  
  
  vector<string> CamUtils::listAllCameraIds() {
  
      vector<string> ids;
  
      cout &lt;&lt; "IN LIST " &lt;&lt; endl;
  
      for (auto&amp; cam : cameras["cameras"]) {
  
          ids.push_back(cam["id"].get<string>());
  
          //cout &lt;&lt; cam["id"].get<string>() &lt;&lt; std::endl;
  
      }
  
      return ids;
  
  }
  
  
  
  string CamUtils::getIp(string id) {
  
      vector<string> camDetails = cameraList[id];
  
      string ip = "NO IP WILL DISPLAYED UNTIL I FIGURE OUT A BUG";
  
      for (auto&amp; cam : cameras["cameras"]) {
  
          if (id == cam["id"]) {
  
              ip = cam["ip"].get<string>();
  
          }
  
      }
  
  
  
      return ip;
  
  }
  
  
  
  string CamUtils::getCameraStreamURL(string id) const {
  
      string url = "err"; // err is the default, it will be overwritten in case id is found, dont forget to check for it
  
  
  
      for (auto&amp; cam : cameras["cameras"]) {
  
          if (id == cam["id"]) {
  
              if (cam["username"].get<string>() == "null") {
  
                  url = "rtsp://" + cam["ip"].get<string>() + ":554/axis-media/media.amp?tcp";
  
              }
  
              else {
  
                  url = "rtsp://" + cam["username"].get<string>() + ":" + cam["password"].get<string>() + "@" + cam["ip"].get<string>() + ":554/axis-media/media.amp?streamprofile=720_30";
  
              }
  
          }
  
      }
  
  
  
      return url;  // Dont forget to check for err when using this shit
  
  }
  
  
  
  
  
  string CamUtils::saveCamera(string ip, string username, string password) {
  
      UUID uid;
  
      UuidCreate(&amp;uid);
  
      char* str;
  
      UuidToStringA(&amp;uid, (RPC_CSTR*)&amp;str);
  
      string id = str;
  
      cout &lt;&lt; "GEN: " &lt;&lt; id &lt;&lt; endl;
  
      json cam = json({}); //Create emtpy object
  
      cam["id"] = id;
  
      cam["ip"] = ip;
  
      cam["username"] = username;
  
      cam["password"] = password;
  
      cameras["cameras"].push_back(cam);
  
      std::ofstream out(camDb);
  
      out &lt;&lt; cameras &lt;&lt; std::endl;
  
      cout &lt;&lt; cameras["cameras"] &lt;&lt; endl;
  
  
  
      cout &lt;&lt; "Saved camera as " &lt;&lt; id &lt;&lt; endl;
  
      return id;
  
  }
  
  
  
  
  
  bool CamUtils::dbExists() {
  
      ifstream dbFile(camDb);
  
      return (bool)dbFile;
  
  }
  
  
  
  
  
  
  
  
  
  
  
  void CamUtils::loadCameras() {
  
      cout &lt;&lt; "Load call" &lt;&lt; endl;
  
      ifstream dbFile(camDb);
  
      string line;
  
      string wholeFile;
  
  
  
      while (std::getline(dbFile, line)) {
  
          cout &lt;&lt; line &lt;&lt; endl;
  
          wholeFile += line;
  
      }
  
      try {
  
          cameras = json::parse(wholeFile);
  
          //cout &lt;&lt; cameras["cameras"] &lt;&lt; endl;
  
  
  
      }
  
      catch (exception e) {
  
          cout &lt;&lt; e.what() &lt;&lt; endl;
  
      }
  
      dbFile.close();
  
  }
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  /*
  
      LEGACY CODE, TO BE REMOVED!
  
  
  
  */
  
  
  
  
  
  
  
  void CamUtils::loadCameras_() {
  
      /* 
  
          LEGACY CODE:
  
          This used to be the way to load cameras, but I moved on to JSON based configuration so this is no longer needed and will be removed soon
  
      */
  
  
  
      ifstream dbFile(camDb);
  
      string line;
  
      while (std::getline(dbFile, line)) {
  
          /*
  
              This function load camera data to the map:
  
              The order MUST be the following: 0:ID, 1:IP, 2:USERNAME, 3:PASSWORD.
  
              Always delimited with | no spaces between!
  
          */
  
          if (!line.empty()) {
  
              stringstream ss(line);
  
              string item;
  
              vector<string> splitString;
  
  
  
              while (std::getline(ss, item, '|')) {
  
                  splitString.push_back(item);
  
              }
  
              if (splitString.size() > 0) {
  
                  /* Dont even parse if the program didnt split right*/
  
                  //cout &lt;&lt; "Split string: " &lt;&lt; splitString.size() &lt;&lt; "\n";
  
                  for (int i = 0; i &lt; (splitString.size()); i++) cameraList[splitString[0]].push_back(splitString[i]);
  
              }
  
          }
  
      }
  
  }
  
  
  
  
  
  
  
  void CamUtils::writeLineToDb_(const string &amp; content, bool append) {
  
      ofstream dbFile;
  
      cout &lt;&lt; "Creating?";
  
      if (append) {
  
          dbFile.open(camDb, ios_base::app);
  
      }
  
      else {
  
          dbFile.open(camDb);
  
      }
  
  
  
      dbFile &lt;&lt; content.c_str() &lt;&lt; "\r\n";
  
      dbFile.flush();
  
  }
  
  
  
  /* JSON Reworx */
  
  
  
  
  
  
  
  
  
  string CamUtils::generateRandomString(size_t length)
  
  {
  
      const char* charmap = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
  
      const size_t charmapLength = strlen(charmap);
  
      auto generator = [&amp;]() { return charmap[rand() % charmapLength]; };
  
      string result;
  
      result.reserve(length);
  
      generate_n(back_inserter(result), length, generator);
  
      return result;
  
  }
  
  </string></string></string></string></string></string></string></string></string></string></string></string></iostream>

  End of example

  How would I go about decreasing CPU usage when dealing with large amount of streams ?

• Server-side tracking vs client-side tracking : What you need to know

  3 juillet, par Joe

  Server-side tracking vs client-side tracking : What you need to know

  Today, consumers are more aware of their online privacy rights, leading to an extensive use of ad blockers and stricter cookie policies. Organisations are facing some noteworthy challenges with this trend, including :

  • Limited data collection, which makes it harder to understand user behaviour and deliver personalised ads that resonate with customers
  • Rising compliance costs as businesses adapt to new regulations, straining resources and budgets.
  • Growing customer scepticism in data practices, affecting brand reputation.
  • Maintaining transparency and fostering trust with customers through clear communication about data practices.

  Server-side tracking can help resolve these problems. This article will cover server-side tracking, how it works, implementation methods and its benefits.

  What is server-side tracking ? 

  Server-side tracking refers to a method where user data is collected directly by a server rather than through a user’s browser.

  The key advantage of server-side tracking is that data collection, processing, and storage occur directly on the website’s server.

  For example, when a visitor interacts with any website, the server captures that activity through the backend system, allowing for greater data control and security. 

  Client-side tracking vs. server-side tracking 

  There are two methods to collect user data : client-side and server-side. 

  Let’s understand their differences. 

  Client-side tracking : Convenience with caveats

  Client-side tracking embeds JavaScript tags, pixels or other scripts directly into a website’s code. When a user interacts with the site, these tags fire, collecting data from their browser. This information might include page views, button clicks, form submissions and other user actions. 

  The collected data is then sent directly to third-party analytics platforms like Google Analytics or Adobe Analytics, or internal teams can also analyse it.

  This method is relatively easy to implement. That’s because marketers can often deploy these tags without needing extensive developer support, enabling quick adjustments and A/B testing. 

  However, there are some challenges. 

  Ad blockers and browser privacy settings, such as Intelligent Tracking Prevention (ITP), restrict the ability of third-party tags to collect data. 

  This results in data gaps and inaccuracies skewing analytics reports and potentially leading to misguided business decisions. 

  Reliance on numerous JavaScript tags can also negatively impact website performance, slowing down page load times and affecting user experience. This is especially true on mobile devices where processing power and network speeds are often limited.

  

  Now, let’s see how server-side tracking changes this.

  Server-side tracking : Control and reliability

  Server-side tracking shifts the burden of data collection from the user’s browser to a server controlled by the business. 

  Instead of relying on JavaScript tags firing directly from the user’s device, user interactions are first sent to the business’s own server. Here, the data can be processed, enriched, and analysed. 

  This method provides numerous advantages, including enhanced control over data integrity, improved privacy, and more, which we discuss in the next section.

  Benefits of server-side tracking 

  Server-side tracking offers a compelling alternative to traditional client-side methods, providing numerous business advantages. Let’s take a look at them.

  Improved data accuracy

  This method reduces inaccuracies caused by ad blockers or cookie restrictions by bypassing browser limitations. As a result, the data collected is more reliable, leading to better analytics and marketing attribution.

  Data minimisation

  Data minimisation is a fundamental principle in data protection. It emphasises that organisations should collect only data that is strictly needed for a specific purpose. 

  In server-side tracking, this translates into collecting just the essential data points and discarding anything extra before the data is sent to analytics platforms. It helps organisations avoid accumulating excessive personal information, reducing the risk of data breaches and misuse.

  For example, consider a scenario where a user purchases a product on an e-commerce website. 

  With client-side tracking scripts, the system might inadvertently collect a range of data, including the user’s IP address, browser type, operating system and even details about other websites they have visited. 

  However, for conversions, the organisation only needs to know the purchase amount, product IDs, user IDS, and timestamps. 

  Server-side tracking filters unnecessary information. This reduces the privacy impact and simplifies data analysis and storage.

  Cross-device tracking capabilities

  Server-side tracking provides a unified view of customer behaviour regardless of the device they use, allowing for more personalised and targeted marketing campaigns. 

  In-depth event tracking

  Server-side tracking helps businesses track events that occur outside their websites, such as payment confirmations. Companies gain insights into the entire customer journey, from initial interaction to final purchase, optimising every touchpoint. 

  Enhanced privacy compliance

  With increasing regulations like GDPR and CCPA, businesses can better manage user consent and data handling practices through server-side solutions. 

  Server-side setups make honouring user consent easier. If a user opts out, server-side logic can exclude their data from all outgoing analytics calls in one central place. 

  

  Server-side methods reassure users and regulators that data is collected and secured with minimal risk. 

  In sectors like government and banking, this level of control is often a non-negotiable part of their duty of care. 

  Extended cookie lifetime

  Traditional website tracking faces growing obstacles as modern browsers prioritise user privacy. Initiatives like Safari’s ITP block third-party cookies and also constrain the use of first-party cookies. 

  Other browsers, such as Firefox and Brave, are implementing similar methods, while Chrome is beginning to phase out third-party cookies. Retargeting and cross-site analytics, which rely on these cookies, encounter significant challenges.

  Server-side tracking overcomes this by allowing businesses to collect data over a longer duration. 

  When a website’s server directly sets a cookie, that cookie often lasts longer than cookies created by JavaScript code running inside the browser. This lets websites get around some of the limits browsers put on tracking and allows them to remember a visitor when they return to the site later, which gives better customer insights. Plus, server-side tracking typically classifies cookies as first-party data, which is less susceptible to blocking by browsers and ad blockers.

  Server-side tracking : Responsibilities and considerations

  While server-side tracking delivers powerful capabilities, remember that it also brings increased responsibility. Companies must remain vigilant in upholding privacy regulations and user consent. It’s up to the organisation to make sure the server follows user consent, for example, not sending data if someone has opted out.

  Server-side setups introduce technical complexity, which can potentially lead to data errors that are more difficult to identify and resolve. Therefore, monitoring processes and quality assurance practices are essential for data integrity. 

  How does server-side tracking work ? 

  When a user interacts with a website (e.g., clicking a button), this action triggers an event. The event could be anything from a page view to a form submission.

  The backend system captures relevant details such as the event type, user ID and timestamp. This information helps in understanding user behaviour and creating meaningful analytics.

  The captured data is processed directly on the organisation’s server, allowing for immediate validation. For example, organisations can add additional context or filter out irrelevant information.

  Instead of sending data to third-party endpoints, the organisation stores everything in its own database or data warehouse. This ensures full control over data privacy and security.

  Organisations can perform their own analysis using tools like SQL or Python. To visualise data, custom dashboards and reports can be created using self-hosted analytics tools. This way, businesses can present complex data in a clear and actionable manner.

  How to implement server-side tracking ?

  Server-side tracking can work in four common ways, each offering a different blend of control, flexibility and complexity.

  1. Server-side tag management

  In this method, organisations use platforms like Google Tag Manager Server-Side to manage tracking tags on the server, often using containers to isolate and manage different tagging environments. 

  

  (Image Source) 

  This approach offers a balance between control and ease of use. It allows for the deployment and management of tags without modifying the application code, which is particularly useful for marketers who want to adjust tracking configurations quickly.

  2. Direct server-to-server tracking via APIs

  This method involves sharing information between two servers without affecting the user’s browser or device. 

  A unique identifier is generated and stored on a server when a user interacts with an ad or webpage. 

  If a user takes some action, like making a purchase, the unique identifier is sent from the advertiser’s server directly to the platform’s server (Google or Facebook) via an API. 

  It requires more development effort but is ideal for organisations needing fine-grained data control.

  3. Using analytics platforms with built-in server SDKs

  Another way is to employ analytics platforms like Matomo that provide SDKs for various programming languages to instrument the server-side code. 

  This eases integration with the platform’s analytics features and is a good choice for organisations primarily using a single analytics platform and want to use its server-side capabilities.

  4. Hybrid approaches

  Finally, organisations can also combine client- and server-side tracking to capture different data types and maximise accuracy. 

  This method involves client-side scripts for specific interactions (like UI events) and server-side tracking for more sensitive or critical data (like transactions). 

  While these are general approaches, dedicated analytics platforms can also be helpful. Matomo, for example, facilitates server-side tracking through two specific methods.

  Using server logs

  Matomo can import existing web server logs, such as Apache or Nginx, that capture each request. Every page view or resource load becomes a data point. 

  Matomo’s log processing script reads log files, importing millions of hits. This removes the need to add code to the site, making it suitable for basic page analytics (like the URL) without client-side scripts, particularly on security-sensitive sites.

  Using the Matomo tracking API (Server-side SDKs)

  This method integrates application code with calls to Matomo’s API. For example, when a user performs a specific action, the server sends a request to Matomo.php, the tracking endpoint, which includes details like the user ID and action. 

  Matomo offers SDKs in PHP, Java C#, and community SDKs to simplify these calls. These allow tracking of not just page views but custom events such as downloads and transactions from the backend, functioning similarly to Google’s Measurement Protocol but sending data to the Matomo instance. 

  Data privacy, regulations and Matomo

  As privacy concerns grow and regulations like GDPR and CCPA become more stringent, businesses must adopt data collection methods that respect user consent and data protection rights. 

  Server-side tracking allows organisations to collect first-party data directly from their servers, which is generally considered more compliant with privacy regulations.

  Matomo is a popular open-source web analytics platform that is committed to privacy. It gives organisations 100% data ownership and control, and no data is sent to third parties by default.

  

  (Image Source) 

  Matomo is a full-featured analytics platform with dashboards and segmentation comparable to Google Analytics. It can self-host and provides DoNotTrack settings and the ability to anonymise IP addresses.

  Governments and organisations requiring data sovereignty, such as the EU Commission and the Swiss government, choose Matomo for web analytics due to its strong compliance posture.

  Balancing data collection and user privacy

  Ad blockers and other restrictions prevent data from being accurate. Server-side tracking helps get data on the server and makes it more reliable while respecting user privacy. Matomo supports server-side tracking, and over one million websites use Matomo to optimise their data strategies. 

  Get started today by trying Matomo for free for 21 days, no credit card required.

• Revision 32594 : plugins en minuscules, et alias pour les noms de sites

  1er novembre 2009, par fil@… — Log

  plugins en minuscules, et alias pour les noms de sites