Recherche avancée

Sur d’autres sites (6890)

• Capture from multiple streams concurrently, best way to do it and how to reduce CPU usage

  19 juin 2019, par DRONE_6969

  I am currently in the process of writing an application that will capture a lot of RTSP streams(in my case its 12) and display it on the QT widget. The problem arouses when I am going beyond around 6-7 streams, the CPU usage spikes and there is visible stutter.

  The reason why I think that it is not QT draw function is because I have done some checking to measure how much time it takes to draw an incoming image from camera and just sample images I had, it is always a lot less than 33 milliseconds(even if there are 12 widgets being updated).

  I also just ran opencv capture method without drawing and got pretty much the same CPU consumption as if I was drawing the frames (lost like 10% CPU at most and GPU usage went to zero).

  IMPORTANT : I am using RTSP stream which is a h264 stream.

  IF IT MATTERS MY SPECS :

  Intel Core i7-6700 @ 3.40GHZ(8 CPUS)
  Memory : 16gb
  GPU : Intel HD Graphics 530

  (Also I ran my code on a computer with dedicated Graphics card, it did eliminate some stutter but CPU usage is still pretty high)

  I am currently using OPENCV 4.1.0 with GSTREAMER enabled and built, I also have the OPENCV-WORLD version, there is no difference in performance.

  I have created a special class called Camera that holds its frame size constraints and various control functions as well stream function. The stream function is being ran on a separate thread, whenever stream() function is done with current frame it sends ready Mat via onNewFrame event I created which converts to QPixmap and updates widget’s lastImage variable. This way I can update image in a more thread safe way.

  I have tried to manipulate those VideoCapture.set() values, but it didn’t really help.

  This is my stream function (Ignore the bool return, it doesn’t do anything it is a remnant from couple of minutes ago when I was trying to use std::async) :

  bool Camera::stream() {
  
      /* This function is meant to run on a separate thread and fill up the buffer independantly of
  
      main stream thread */
  
      //cv::setNumThreads(100);
  
      /* Rules for these slightly changed! */
  
      Mat pre;  // Grab initial undoctored frame
  
      //pre = Mat::zeros(size, CV_8UC1);
  
      Mat frame; // Final modified frame
  
      frame = Mat::zeros(size, CV_8UC1);
  
      if (!pre.isContinuous()) pre = pre.clone();
  
  
  
      ipCam.open(streamUrl, CAP_FFMPEG);
  
  
  
  
  
      while (ipCam.isOpened() && capture) {
  
          // If camera is opened wel need to capture and process the frame
  
          try {
  
              auto start = std::chrono::system_clock::now();
  
  
  
              ipCam >> pre;
  
  
  
              if (pre.empty()) {
  
                  /* Check for blank frame, return error if there is a blank frame*/
  
                  cerr << id << ": ERROR! blank frame grabbed\n";
  
                  for (FrameListener* i : clients) {
  
                      i->onNotification(1); // Notify clients about this shit
  
                  }
  
                  break;
  
              }
  
  
  
              else {
  
                  // Only continue if frame not empty
  
  
  
                  if (pre.cols != size.width && pre.rows != size.height) {
  
                      resize(pre, frame, size);
  
                      pre.release();
  
                  }
  
                  else {
  
                      frame = pre;
  
                  }
  
  
  
                  dPacket* pack = new dPacket{id,&frame};
  
                  for (auto i : clients) {
  
                      i->onPNewFrame(pack);
  
                  }
  
                  frame.release();
  
                  delete pack;
  
              }
  
          }
  
  
  
          catch (int e) {
  
              cout << endl << "-----Exception during capture process! CODE " << e << endl;
  
          }
  
          // End camera manipulations
  
      }
  
  
  
      cout << "Camera timed out, or connection is closed..." << endl;
  
      if (tryResetConnection) {
  
          cout << "Reconnection flag is set, retrying after 3 seconds..." << endl;
  
          for (FrameListener* i : clients) {
  
              i->onNotification(-1); // Notify clients about this shit
  
          }
  
          this_thread::sleep_for(chrono::milliseconds(3000));
  
          stream();
  
      }
  
  
  
      return true;
  
  }

  This is my onPNewFrame function. The conversion is still being done on camera’s thread because it was called within stream() and therefore is within that scope(and I also checked) :

  void GLWidget::onPNewFrame(dPacket* inPack) {
  
      lastFlag = 0;
  
  
  
      if (bufferEnabled) {
  
          buffer.push(QPixmap::fromImage(toQImageFromPMat(inPack->frame)));
  
      }
  
      else {
  
          if (playing) {
  
              /* Only process if this widget is playing */
  
              frameProcessing = true;
  
              lastImage.convertFromImage(toQImageFromPMat(inPack->frame));
  
              frameProcessing = false;
  
          }
  
      }
  
  
  
      if (lastFlag != -1 && !lastImage.isNull()) {
  
          connecting = false;
  
      }
  
      else {
  
          connecting = true;
  
      }
  
  }

  This is my Mat to QImage :

  QImage GLWidget::toQImageFromPMat(cv::Mat* mat) {
  
  
  
  
  
  
  
      return QImage(mat->data, mat->cols, mat->rows, QImage::Format_RGB888).rgbSwapped();

  NOTE : not converting does not result in CPU boost (at least not a significant one).

  Minimal verifiable example

  This program is large. I am going to paste GLWidget.cpp and GLWidget.h as well as Camera.h and Camera.cpp. You can put GLWidget into anything just as long as you spawn more than 6 of it. Camera relies on the CamUtils, but it is possible to just paste url in videocapture

  I also supplied CamUtils, just in case

  Camera.h :

  #pragma once
  
  #include <iostream>
  
  #include <vector>
  
  #include <fstream>
  
  #include <map>
  
  #include <string>
  
  #include <sstream>
  
  #include <algorithm>
  
  #include "FrameListener.h"
  
  #include 
  
  #include <thread>
  
  #include "CamUtils.h"
  
  #include <ctime>
  
  #include "dPacket.h"
  
  
  
  using namespace std;
  
  using namespace cv;
  
  
  
  class Camera
  
  {
  
  
  
      /*
  
          CLEANED UP!
  
          Camera now is only responsible for streaming and echoing captured frames.
  
          Frames are now wrapped into dPacket struct.
  
      */
  
  
  
  
  
  private:
  
      string id;
  
      vector clients;
  
      VideoCapture ipCam;
  
      string streamUrl;
  
      Size size;
  
      bool tryResetConnection = false;
  
  
  
      //TODO: Remove these as they are not going to be used going on:
  
      bool isPlaying = true;
  
      bool capture = true;
  
  
  
      //SECRET FEATURES:
  
      bool detect = false;
  
  
  
  
  
  public:
  
      Camera(string url, int width = 480, int height = 240, bool detect_=false);
  
      bool stream();
  
      void setReconnectable(bool newReconStatus);
  
      void addListener(FrameListener* client);
  
      vector<bool> getState();    // Returns current state: vector[0] stream state; vector[1] stream state; TODO: Remove this as this is no longer should control behaviour
  
      void killStream();
  
      bool getReconnectable();
  
  };
  
  
  
  </bool></ctime></thread></algorithm></sstream></string></map></fstream></vector></iostream>

  Camera.cpp

  #include "Camera.h"
  
  
  
  
  
  Camera::Camera(string url, int width, int height, bool detect_) // Default 240p
  
  {
  
      streamUrl = url; // Prepare url
  
      size = Size(width, height);
  
      detect = detect_;
  
  
  
  }
  
  
  
  void Camera::addListener(FrameListener* client) {
  
      clients.push_back(client);
  
  }
  
  
  
  
  
  /*
  
                  TEST CAMERAS(Paste into cameras.dViewer):
  
                  {"id":"96a73796-c129-46fc-9c01-40acd8ed7122","ip":"176.57.73.231","password":"null","username":"null"},
  
                  {"id":"96a73796-c129-46fc-9c01-40acd8ed7122","ip":"176.57.73.231","password":"null","username":"null"},
  
                  {"id":"96a73796-c129-46fc-9c01-40acd8ed7144","ip":"172.20.101.13","password":"admin","username":"root"}
  
                  {"id":"96a73796-c129-46fc-9c01-40acd8ed7144","ip":"172.20.101.13","password":"admin","username":"root"}
  
  
  
  */
  
  
  
  
  
  
  
  bool Camera::stream() {
  
      /* This function is meant to run on a separate thread and fill up the buffer independantly of
  
      main stream thread */
  
      //cv::setNumThreads(100);
  
      /* Rules for these slightly changed! */
  
      Mat pre;  // Grab initial undoctored frame
  
      //pre = Mat::zeros(size, CV_8UC1);
  
      Mat frame; // Final modified frame
  
      frame = Mat::zeros(size, CV_8UC1);
  
      if (!pre.isContinuous()) pre = pre.clone();
  
  
  
      ipCam.open(streamUrl, CAP_FFMPEG);
  
  
  
      while (ipCam.isOpened() && capture) {
  
          // If camera is opened wel need to capture and process the frame
  
          try {
  
              auto start = std::chrono::system_clock::now();
  
  
  
              ipCam >> pre;
  
  
  
              if (pre.empty()) {
  
                  /* Check for blank frame, return error if there is a blank frame*/
  
                  cerr << id << ": ERROR! blank frame grabbed\n";
  
                  for (FrameListener* i : clients) {
  
                      i->onNotification(1); // Notify clients about this shit
  
                  }
  
                  break;
  
              }
  
  
  
              else {
  
                  // Only continue if frame not empty
  
  
  
                  if (pre.cols != size.width && pre.rows != size.height) {
  
                      resize(pre, frame, size);
  
                      pre.release();
  
                  }
  
                  else {
  
                      frame = pre;
  
                  }
  
  
  
                  auto end = std::chrono::system_clock::now();
  
                  std::time_t ts = std::chrono::system_clock::to_time_t(end);
  
                  dPacket* pack = new dPacket{ id,&frame};
  
                  for (auto i : clients) {
  
                      i->onPNewFrame(pack);
  
                  }
  
                  frame.release();
  
                  delete pack;
  
              }
  
          }
  
  
  
          catch (int e) {
  
              cout << endl << "-----Exception during capture process! CODE " << e << endl;
  
          }
  
          // End camera manipulations
  
      }
  
  
  
      cout << "Camera timed out, or connection is closed..." << endl;
  
      if (tryResetConnection) {
  
          cout << "Reconnection flag is set, retrying after 3 seconds..." << endl;
  
          for (FrameListener* i : clients) {
  
              i->onNotification(-1); // Notify clients about this shit
  
          }
  
          this_thread::sleep_for(chrono::milliseconds(3000));
  
          stream();
  
      }
  
  
  
      return true;
  
  }
  
  
  
  
  
  void Camera::killStream(){
  
      tryResetConnection = false;
  
      capture = false;
  
      ipCam.release();
  
  }
  
  
  
  void Camera::setReconnectable(bool reconFlag) {
  
      tryResetConnection = reconFlag;
  
  }
  
  
  
  bool Camera::getReconnectable() {
  
      return tryResetConnection;
  
  }
  
  
  
  vector<bool> Camera::getState() {
  
      vector<bool> states;
  
      states.push_back(isPlaying);
  
      states.push_back(ipCam.isOpened());
  
      return states;
  
  }
  
  
  
  
  
  
  
  </bool></bool>

  GLWidget.h :

  #ifndef GLWIDGET_H
  
  #define GLWIDGET_H
  
  
  
  #include <qopenglwidget>
  
  #include <qmouseevent>
  
  #include "FrameListener.h"
  
  #include "Camera.h"
  
  #include "FrameListener.h"
  
  #include 
  
  #include "Camera.h"
  
  #include "CamUtils.h"
  
  #include 
  
  #include "dPacket.h"
  
  #include <chrono>
  
  #include <ctime>
  
  #include 
  
  #include "FullScreenVideo.h"
  
  #include <qmovie>
  
  #include "helper.h"
  
  #include <iostream>
  
  #include <qpainter>
  
  #include <qtimer>
  
  
  
  class Helper;
  
  
  
  class GLWidget : public QOpenGLWidget, public FrameListener
  
  {
  
      Q_OBJECT
  
  
  
  public:
  
      GLWidget(std::string camId, CamUtils *cUtils, int width, int height, bool denyFullScreen_ = false, bool detectFlag_=false, QWidget* parent = nullptr);
  
      void killStream();
  
      ~GLWidget();
  
  
  
  public slots:
  
      void animate();
  
      void setBufferEnabled(bool setState);
  
      void setCameraRetryConnection(bool setState);
  
      void GLUpdate();            // Call to update the widget
  
      void onRightClickMenu(const QPoint&amp; point);
  
  
  
  protected:
  
      void paintEvent(QPaintEvent* event) override;
  
      void onPNewFrame(dPacket* frame);
  
      void onNotification(int alert_code);
  
  
  
  
  
  private:
  
      // Objects and resourses
  
      Helper* helper;
  
      Camera* cam;
  
      CamUtils* camUtils;
  
      QTimer* timer; // Keep track of update
  
      QPixmap lastImage;
  
      QMovie* connMov;
  
      QMovie* test;
  
  
  
      QPixmap logo;
  
  
  
      // Control fields
  
      int width;
  
      int height;
  
      int camUtilsAddr;
  
      int elapsed;
  
      std::thread* camThread;
  
      std::string camId;
  
      bool denyFullScreen = false;
  
      bool playing = true;
  
      bool streaming = true;
  
      bool debug = false;
  
      bool connecting = true;
  
      int lastFlag = 0;
  
  
  
  
  
      // Debug fields
  
      std::chrono::high_resolution_clock::time_point lastFrameAt;
  
      std::chrono::high_resolution_clock::time_point now;
  
      std::chrono::duration<double> painTime; // time took to draw last frame
  
  
  
      //Buffer stuff
  
      std::queue<qpixmap> buffer;
  
      bool bufferEnabled = false;
  
      bool initialBuffer = false;
  
      bool buffering = true;
  
      bool frameProcessing = false;
  
  
  
  
  
  
  
      //Functions
  
      QImage toQImageFromPMat(cv::Mat* inFrame);
  
      void mousePressEvent(QMouseEvent* event) override;
  
      void drawImageGLLatest(QPainter* painter, QPaintEvent* event, int elapsed);
  
      void drawOnPaused(QPainter* painter, QPaintEvent* event, int elapsed);
  
      void drawOnStatus(int statusFlag, QPainter* painter, QPaintEvent* event, int elapsed);
  
  };
  
  
  
  #endif
  
  
  
  </qpixmap></double></qtimer></qpainter></iostream></qmovie></ctime></chrono></qmouseevent></qopenglwidget>

  GLWidget.cpp :

  #include "glwidget.h"
  
  #include <future>
  
  
  
  
  
  FullScreenVideo* fullScreen;
  
  
  
  GLWidget::GLWidget(std::string camId_, CamUtils* cUtils, int width_, int height_,  bool denyFullScreen_, bool detectFlag_, QWidget* parent)
  
      : QOpenGLWidget(parent), helper(helper)
  
  {
  
      cout &lt;&lt; "Player for CAMERA " &lt;&lt; camId_ &lt;&lt; endl;
  
  
  
      /* Underlying properties */
  
      camUtils = cUtils;
  
      cout &lt;&lt; "GLWidget Incoming CamUtils addr " &lt;&lt; camUtils &lt;&lt; endl;
  
      cout &lt;&lt; "GLWidget Set CamUtils addr " &lt;&lt; camUtils &lt;&lt; endl;
  
      camId = camId_;
  
      elapsed = 0;
  
      width = width_ + 5;
  
      height = height_ + 5;
  
      helper = new Helper();
  
      setFixedSize(width, height);
  
      denyFullScreen = denyFullScreen_;
  
  
  
      /* Camera capture thread */
  
      cam = new Camera(camUtils->getCameraStreamURL(camId), width_, height_, detectFlag_);
  
      cam->addListener(this);
  
  
  
      /* Sync states */
  
      vector<bool> initState = cam->getState();
  
      playing = initState[0];
  
      streaming = initState[1];
  
      cout &lt;&lt; "Initial states: " &lt;&lt; playing &lt;&lt; " " &lt;&lt; streaming &lt;&lt; endl;
  
      camThread = new std::thread(&amp;Camera::stream, cam);
  
      cout &lt;&lt; "================================================" &lt;&lt; endl;
  
  
  
      // Right click set up
  
      setContextMenuPolicy(Qt::CustomContextMenu);
  
  
  
  
  
      /* Loading gif */
  
      connMov = new QMovie("establishingConnection.gif");
  
      connMov->start();
  
      QString url = R"(RLC-logo.png)";
  
      logo = QPixmap(url);
  
      QTimer* timer = new QTimer(this);
  
      connect(timer, SIGNAL(timeout()), this, SLOT(GLUpdate()));
  
      timer->start(1000/30);
  
      playing = true;
  
  
  
  }
  
  
  
  /* SYSTEM */
  
  void GLWidget::animate()
  
  {
  
      elapsed = (elapsed + qobject_cast(sender())->interval()) % 1000;
  
      std::cout &lt;&lt; elapsed &lt;&lt; "\n";
  
  }
  
  
  
  
  
  void GLWidget::GLUpdate() {
  
      /* Process descisions before update call */
  
      if (bufferEnabled) {
  
          /* Process buffer before update */
  
          now = chrono::high_resolution_clock::now();
  
          std::chrono::duration timeSinceLastUpdate = now - lastFrameAt;
  
          if (timeSinceLastUpdate.count() > 25) {
  
              if (buffer.size() > 1 &amp;&amp; playing) {
  
                  lastImage.swap(buffer.front());
  
                  buffer.pop();
  
                  lastFrameAt = chrono::high_resolution_clock::now();
  
              }
  
          }
  
          //update(); // Update
  
      }
  
      else {
  
          /* No buffer */
  
      }
  
      repaint();
  
  }
  
  
  
  
  
  /* EVENTS */
  
  void GLWidget::onRightClickMenu(const QPoint&amp; point) {
  
      cout &lt;&lt; "Right click request got" &lt;&lt; endl;
  
  
  
      QPoint globPos = this->mapToGlobal(point);
  
      QMenu myMenu;
  
  
  
      if (!denyFullScreen) {
  
          myMenu.addAction("Open Full Screen");
  
      }
  
      myMenu.addAction("Toggle Debug Info");
  
  
  
  
  
      QAction* selected = myMenu.exec(globPos);
  
  
  
      if (selected) {
  
          string optiontxt = selected->text().toStdString();
  
  
  
          if (optiontxt == "Open Full Screen") {
  
              cout &lt;&lt; "Chose to open full screen of " &lt;&lt; camId &lt;&lt; endl;
  
              fullScreen = new FullScreenVideo(bufferEnabled, this);
  
              fullScreen->setUpView(camUtils, camId);
  
              fullScreen->show();
  
              playing = false;
  
          }
  
  
  
          if (optiontxt == "Toggle Debug Info") {
  
              cout &lt;&lt; "Chose to toggle debug of " &lt;&lt; camId &lt;&lt; endl;
  
              debug = !debug;
  
          }
  
      }
  
      else {
  
          cout &lt;&lt; "Chose nothing!" &lt;&lt; endl;
  
      }
  
  
  
  
  
  }
  
  
  
  
  
  
  
  void GLWidget::onPNewFrame(dPacket* inPack) {
  
      lastFlag = 0;
  
  
  
      if (bufferEnabled) {
  
          buffer.push(QPixmap::fromImage(toQImageFromPMat(inPack->frame)));
  
      }
  
      else {
  
          if (playing) {
  
              /* Only process if this widget is playing */
  
              frameProcessing = true;
  
              lastImage.convertFromImage(toQImageFromPMat(inPack->frame));
  
              frameProcessing = false;
  
          }
  
      }
  
  
  
      if (lastFlag != -1 &amp;&amp; !lastImage.isNull()) {
  
          connecting = false;
  
      }
  
      else {
  
          connecting = true;
  
      }
  
  }
  
  
  
  
  
  void GLWidget::onNotification(int alert) {
  
      lastFlag = alert;   
  
  }
  
  
  
  
  
  /* Paint events*/
  
  
  
  
  
  void GLWidget::paintEvent(QPaintEvent* event)
  
  {
  
      QPainter painter(this);
  
  
  
          if (lastFlag != 0 || connecting) {
  
              drawOnStatus(lastFlag, &amp;painter, event, elapsed);
  
          }
  
          else {
  
  
  
              /* Actual frame drawing */
  
              if (playing) {
  
                  if (!frameProcessing) {
  
                      drawImageGLLatest(&amp;painter, event, elapsed);
  
                  }
  
              }
  
              else {
  
                  drawOnPaused(&amp;painter, event, elapsed);
  
              }
  
          }
  
      painter.end();
  
  
  
  }
  
  
  
  
  
  /* DRAWING STUFF */
  
  
  
  void GLWidget::drawOnStatus(int statusFlag, QPainter* bgPaint, QPaintEvent* event, int elapsed) {
  
  
  
      QString str;
  
      QFont font("times", 15);
  
      bgPaint->eraseRect(QRect(0, 0, width, height));
  
      if (!lastImage.isNull()) {
  
          bgPaint->drawPixmap(QRect(0, 0, width, height), lastImage);
  
      }
  
      /* Test background painting */
  
      if (connecting) {
  
          string k = "Connecting to " + camUtils->getIp(camId);
  
          str.append(k.c_str());
  
      }
  
      else {
  
          switch (statusFlag) {
  
          case 1:
  
              str = "Blank frame received...";
  
              break;
  
  
  
          case -1:
  
              if (cam->getReconnectable()) {
  
                  str = "Connection lost, will try to reconnect.";
  
                  bgPaint->setOpacity(0.3);
  
              }
  
              else {
  
                  str = "Connection lost...";
  
                  bgPaint->setOpacity(0.3);
  
              }
  
  
  
              break;
  
          }
  
      }
  
  
  
      bgPaint->drawPixmap(QRect(0, 0, width, height), QPixmap::fromImage(connMov->currentImage()));
  
      bgPaint->setPen(Qt::red);
  
      bgPaint->setFont(font);
  
      QFontMetrics fm(font);
  
      const QRect kek(0, 0, fm.width(str), fm.height());
  
      QRect bound;
  
      bgPaint->setOpacity(1);
  
      bgPaint->drawText(bgPaint->viewport().width()/2 - kek.width()/2, bgPaint->viewport().height()/2 - kek.height(), str);
  
  
  
      bgPaint->drawPixmap(bgPaint->viewport().width() / 2 - logo.width()/2, height - logo.width() - 15, logo);
  
  
  
  }
  
  
  
  
  
  
  
  void GLWidget::drawOnPaused(QPainter* painter, QPaintEvent* event, int elapsed) {
  
      painter->eraseRect(0, 0, width, height);
  
      QFont font = painter->font();
  
      font.setPointSize(18);
  
      painter->setPen(Qt::red);
  
      QFontMetrics fm(font);
  
      QString str("Paused");
  
      painter->drawPixmap(QRect(0, 0, width, height),lastImage);
  
      painter->drawText(QPoint(painter->viewport().width() - fm.width(str), 50), str);
  
  
  
      if (debug) {
  
          QFont font = painter->font();
  
          font.setPointSize(25);
  
          painter->setPen(Qt::red);
  
          string camMess = "CAMID: " + camId;
  
          QString mess(camMess.c_str());
  
          string camIp = "IP: " + camUtils->getIp(camId);
  
          QString ipMess(camIp.c_str());
  
          QString bufferSize("Buffer size: " + QString::number(buffer.size()));
  
          QString lastFrameText("Last frame draw time: " + QString::number(painTime.count()) + "s");
  
          painter->drawText(QPoint(10, 50), mess);
  
          painter->drawText(QPoint(10, 60), ipMess);
  
          QString bufferState;
  
          if (bufferEnabled) {
  
              bufferState = QString("Experimental BUFFER is enabled!");
  
              QString currentBufferSize("Current buffer load: " + QString::number(buffer.size()));
  
              painter->drawText(QPoint(10, 80), currentBufferSize);
  
          }
  
          else {
  
              bufferState = QString("Experimental BUFFER is disabled!");
  
          }
  
          painter->drawText(QPoint(10, 70), bufferState);
  
          painter->drawText(QPoint(10, height - 25), lastFrameText);
  
      }
  
  }
  
  
  
  
  
  void GLWidget::drawImageGLLatest(QPainter* painter, QPaintEvent* event, int elapsed) {
  
      auto start = chrono::high_resolution_clock::now();
  
      painter->drawPixmap(QRect(0, 0, width, height), lastImage);
  
      if (debug) {
  
          QFont font = painter->font();
  
          font.setPointSize(25);
  
          painter->setPen(Qt::red);
  
          string camMess = "CAMID: " + camId;
  
          QString mess(camMess.c_str());
  
          string camIp = "IP: " + camUtils->getIp(camId);
  
          QString ipMess(camIp.c_str());
  
          QString bufferSize("Buffer size: " + QString::number(buffer.size()));
  
          QString lastFrameText("Last frame draw time: " + QString::number(painTime.count()) + "s");
  
          painter->drawText(QPoint(10, 50), mess);
  
          painter->drawText(QPoint(10, 60), ipMess);
  
          QString bufferState;
  
          if(bufferEnabled){
  
              bufferState = QString("Experimental BUFFER is enabled!");
  
              QString currentBufferSize("Current buffer load: " + QString::number(buffer.size()));
  
              painter->drawText(QPoint(10,80), currentBufferSize);
  
          }
  
          else {
  
              bufferState = QString("Experimental BUFFER is disabled!");
  
              QString currentBufferSize("Current buffer load: " + QString::number(buffer.size()));
  
              painter->drawText(QPoint(10, 80), currentBufferSize);
  
          }
  
          painter->drawText(QPoint(10, 70), bufferState);
  
          painter->drawText(QPoint(10, height - 25), lastFrameText);
  
  
  
      }
  
      auto end = chrono::high_resolution_clock::now();
  
      painTime = end - start;
  
  }
  
  
  
  
  
  
  
  /* END DRAWING STUFF */
  
  
  
  
  
  
  
  /* UI EVENTS */
  
  
  
  void GLWidget::mousePressEvent(QMouseEvent* e) {
  
  
  
      if (e->button() == Qt::LeftButton) {
  
          if (fullScreen == nullptr || !fullScreen->isVisible()) { // Do not unpause if window is opened
  
              playing = !playing;
  
          }
  
      }
  
  
  
      if (e->button() == Qt::RightButton) {
  
          onRightClickMenu(e->pos());
  
      }
  
  }
  
  
  
  
  
  
  
  /* Utilities */
  
  QImage GLWidget::toQImageFromPMat(cv::Mat* mat) {
  
  
  
  
  
  
  
      return QImage(mat->data, mat->cols, mat->rows, QImage::Format_RGB888).rgbSwapped();
  
  
  
  
  
  
  
  }
  
  
  
  /* State control */
  
  
  
  void GLWidget::killStream() {
  
      cam->killStream();
  
      camThread->join();
  
  }
  
  
  
  void GLWidget::setBufferEnabled(bool newBufferState) {
  
      cout &lt;&lt; "Player: " &lt;&lt; camId &lt;&lt; ", buffer state updated: " &lt;&lt; newBufferState &lt;&lt; endl;
  
      bufferEnabled = newBufferState;
  
      buffer.empty();
  
  }
  
  
  
  void GLWidget::setCameraRetryConnection(bool newState) {
  
      cam->setReconnectable(newState);
  
  }
  
  
  
  /* Destruction */
  
  GLWidget::~GLWidget() {
  
      cam->killStream();
  
      camThread->join();
  
  }
  
  </bool></future>

  CamUtils.h :

  #pragma once
  
  #include <iostream>
  
  #include <vector>
  
  #include <fstream>
  
  #include <map>
  
  #include <string>
  
  #include <sstream>
  
  #include <algorithm>
  
  #include <nlohmann></nlohmann>json.hpp>
  
  
  
  using namespace std;
  
  using json = nlohmann::json;
  
  
  
  class CamUtils
  
  {
  
  private:
  
  
  
      string camDb = "cameras.dViewer";
  
      map> cameraList; // Legacy
  
      json cameras;
  
      ofstream dbFile;
  
      bool dbExists(); // Always hard coded
  
  
  
      /* Old IMPLEMENTATION */
  
      void writeLineToDb_(const string&amp; content, bool append = false);
  
      void loadCameras_();
  
  
  
      /* JSON based */
  
      void loadCameras();
  
  
  
  public:
  
      CamUtils();
  
      string generateRandomString(size_t length);
  
      string getCameraStreamURL(string cameraId) const;
  
      string saveCamera(string ip, string username, string pass); // Return generated id
  
      vector<string> listAllCameraIds();
  
      string getIp(string cameraId);
  
  };
  
  
  
  
  
  </string></algorithm></sstream></string></map></fstream></vector></iostream>

  CamUtils.cpp :

  #include "CamUtils.h"
  
  #pragma comment(lib, "rpcrt4.lib")  // UuidCreate - Minimum supported OS Win 2000
  
  #include 
  
  #include <iostream>
  
  
  
  CamUtils::CamUtils()
  
  {
  
      if (!dbExists()) {
  
          ofstream dbFile;
  
          dbFile.open(camDb);
  
          cameras["cameras"] = json::array();
  
          dbFile &lt;&lt; cameras &lt;&lt; std::endl;
  
          dbFile.close();
  
  
  
      }
  
      else {
  
          loadCameras();
  
      }
  
  }
  
  
  
  
  
  
  
  
  
  vector<string> CamUtils::listAllCameraIds() {
  
      vector<string> ids;
  
      cout &lt;&lt; "IN LIST " &lt;&lt; endl;
  
      for (auto&amp; cam : cameras["cameras"]) {
  
          ids.push_back(cam["id"].get<string>());
  
          //cout &lt;&lt; cam["id"].get<string>() &lt;&lt; std::endl;
  
      }
  
      return ids;
  
  }
  
  
  
  string CamUtils::getIp(string id) {
  
      vector<string> camDetails = cameraList[id];
  
      string ip = "NO IP WILL DISPLAYED UNTIL I FIGURE OUT A BUG";
  
      for (auto&amp; cam : cameras["cameras"]) {
  
          if (id == cam["id"]) {
  
              ip = cam["ip"].get<string>();
  
          }
  
      }
  
  
  
      return ip;
  
  }
  
  
  
  string CamUtils::getCameraStreamURL(string id) const {
  
      string url = "err"; // err is the default, it will be overwritten in case id is found, dont forget to check for it
  
  
  
      for (auto&amp; cam : cameras["cameras"]) {
  
          if (id == cam["id"]) {
  
              if (cam["username"].get<string>() == "null") {
  
                  url = "rtsp://" + cam["ip"].get<string>() + ":554/axis-media/media.amp?tcp";
  
              }
  
              else {
  
                  url = "rtsp://" + cam["username"].get<string>() + ":" + cam["password"].get<string>() + "@" + cam["ip"].get<string>() + ":554/axis-media/media.amp?streamprofile=720_30";
  
              }
  
          }
  
      }
  
  
  
      return url;  // Dont forget to check for err when using this shit
  
  }
  
  
  
  
  
  string CamUtils::saveCamera(string ip, string username, string password) {
  
      UUID uid;
  
      UuidCreate(&amp;uid);
  
      char* str;
  
      UuidToStringA(&amp;uid, (RPC_CSTR*)&amp;str);
  
      string id = str;
  
      cout &lt;&lt; "GEN: " &lt;&lt; id &lt;&lt; endl;
  
      json cam = json({}); //Create emtpy object
  
      cam["id"] = id;
  
      cam["ip"] = ip;
  
      cam["username"] = username;
  
      cam["password"] = password;
  
      cameras["cameras"].push_back(cam);
  
      std::ofstream out(camDb);
  
      out &lt;&lt; cameras &lt;&lt; std::endl;
  
      cout &lt;&lt; cameras["cameras"] &lt;&lt; endl;
  
  
  
      cout &lt;&lt; "Saved camera as " &lt;&lt; id &lt;&lt; endl;
  
      return id;
  
  }
  
  
  
  
  
  bool CamUtils::dbExists() {
  
      ifstream dbFile(camDb);
  
      return (bool)dbFile;
  
  }
  
  
  
  
  
  
  
  
  
  
  
  void CamUtils::loadCameras() {
  
      cout &lt;&lt; "Load call" &lt;&lt; endl;
  
      ifstream dbFile(camDb);
  
      string line;
  
      string wholeFile;
  
  
  
      while (std::getline(dbFile, line)) {
  
          cout &lt;&lt; line &lt;&lt; endl;
  
          wholeFile += line;
  
      }
  
      try {
  
          cameras = json::parse(wholeFile);
  
          //cout &lt;&lt; cameras["cameras"] &lt;&lt; endl;
  
  
  
      }
  
      catch (exception e) {
  
          cout &lt;&lt; e.what() &lt;&lt; endl;
  
      }
  
      dbFile.close();
  
  }
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  /*
  
      LEGACY CODE, TO BE REMOVED!
  
  
  
  */
  
  
  
  
  
  
  
  void CamUtils::loadCameras_() {
  
      /* 
  
          LEGACY CODE:
  
          This used to be the way to load cameras, but I moved on to JSON based configuration so this is no longer needed and will be removed soon
  
      */
  
  
  
      ifstream dbFile(camDb);
  
      string line;
  
      while (std::getline(dbFile, line)) {
  
          /*
  
              This function load camera data to the map:
  
              The order MUST be the following: 0:ID, 1:IP, 2:USERNAME, 3:PASSWORD.
  
              Always delimited with | no spaces between!
  
          */
  
          if (!line.empty()) {
  
              stringstream ss(line);
  
              string item;
  
              vector<string> splitString;
  
  
  
              while (std::getline(ss, item, '|')) {
  
                  splitString.push_back(item);
  
              }
  
              if (splitString.size() > 0) {
  
                  /* Dont even parse if the program didnt split right*/
  
                  //cout &lt;&lt; "Split string: " &lt;&lt; splitString.size() &lt;&lt; "\n";
  
                  for (int i = 0; i &lt; (splitString.size()); i++) cameraList[splitString[0]].push_back(splitString[i]);
  
              }
  
          }
  
      }
  
  }
  
  
  
  
  
  
  
  void CamUtils::writeLineToDb_(const string &amp; content, bool append) {
  
      ofstream dbFile;
  
      cout &lt;&lt; "Creating?";
  
      if (append) {
  
          dbFile.open(camDb, ios_base::app);
  
      }
  
      else {
  
          dbFile.open(camDb);
  
      }
  
  
  
      dbFile &lt;&lt; content.c_str() &lt;&lt; "\r\n";
  
      dbFile.flush();
  
  }
  
  
  
  /* JSON Reworx */
  
  
  
  
  
  
  
  
  
  string CamUtils::generateRandomString(size_t length)
  
  {
  
      const char* charmap = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
  
      const size_t charmapLength = strlen(charmap);
  
      auto generator = [&amp;]() { return charmap[rand() % charmapLength]; };
  
      string result;
  
      result.reserve(length);
  
      generate_n(back_inserter(result), length, generator);
  
      return result;
  
  }
  
  </string></string></string></string></string></string></string></string></string></string></string></string></iostream>

  End of example

  How would I go about decreasing CPU usage when dealing with large amount of streams ?

• Only '-vf scale.....' read, ignoring remaining -vf options : Use ',' to separate filters even though no other vf present

  24 septembre 2019, par Vitalis Hommel

  I am using

  ffmpeg -y -ss 00:02:24.643 -fflags +igndts -t 00:00:39.337 -i 1.mp4 -r 60 -c:v libx264 -copytb 1 -vf scale=-2:1080,pad=width=1920:height=1080:x=0:y=656:color=black -af afade=t=in:ss=0:d=1,afade=t=out:st='00\:00\:38.337':d=1 2.mp4

  and I get

  Only '-vf scale=-2:1080,pad=width=1920:height=1080:x=0:y=656:color=black' read, ignoring remaining -vf options: Use ',' to separate filters
  
  Only '-af afade=t=in:ss=0:d=1,afade=t=out:st='00\:00\:38.337':d=1' read, ignoring remaining -af options: Use ',' to separate filters

  How to solve this ?

• How can I convert multiple images to a video file while keeping the same file name ? [duplicate]

  13 juillet 2015, par Ertzi

  This question already has an answer here :

  • execute ffmpeg command in a loop
    
    2 answers
    

  My intention would be to get at the same time, picture1.jpg, picture2.jpg, picture3.jpg to video format such as picture1.mp4, picture2.mp4, picture3.mp4.

  I am currently using Mencoder and Linux bash code mencoder mf: //*.jpg -mf w = 1366: h = 768: fps = 6/60: type = jpg -ovc copy -oac copy -o images.mp4. But this command makes all the images into one video file (images.mp4).

  Can I do it with mencoder or ffmpeg ?
  My linux bash coding skills are basic.