Recherche avancée

Sur d’autres sites (2019)

• FFmpegFrameGrabber video artefacts from RTSP network camera

  2 février 2015, par UncleChris

  I’m using JavaCV FFmpegFrameGrabber to grab frames from my network camera through RTSP protocol. Simplified code looks like this :

  /* from ini method */
  
  // url like: rtsp://ip:port/stream1
  
  grabber = new FFmpegFrameGrabber(stream.getUrl());
  
  // type: RTP
  
  grabber.setFormat(stream.getMediaType());
  
  grabber.start();
  
  
  
  /* it's called in while loop from outside */
  
  public void grab() throws FrameProcessorsException {
  
  
  
      try {
  
          LOGGER.info(grabber.getFrameNumber());
  
          frame = grabber.grab();
  
      } catch (FrameGrabber.Exception e) {
  
          throw new FrameProcessorsException(e);
  
      }
  
  
  
      // I save my frames to other grabber, to make mp4 file to watch it later
  
      try {
  
          videoRecorder.recordFrame(frame, grabber.getTimestamp(), grabber.getImageWidth(), grabber.getImageHeight(), grabber.getAudioChannels());
  
      } catch (FrameRecorder.Exception e) {
  
          throw new FrameProcessorsException(e);
  
      }
  
  
  
      // my processing, the troublemaker
  
      long currentFrameNum = grabber.getFrameNumber();
  
      if (processing && currentFrameNum - lastFrameWithAnalysis >= PROCESS_FREQUENCY) {
  
  
  
          lastFrameWithAnalysis = currentFrameNum;
  
  
  
          Mat frameMat = new Mat(frame, false);
  
          try {
  
              LOGGER.info("Processing :" + grabber.getFrameNumber());
  
              AnalysisResult result = frameAnalyzer.processFrame(frameMat, (int) currentFrameNum);
  
              videoAnalysisSaver.saveFrameAnalysisResult(frameMat, result, (int) currentFrameNum);
  
          } catch (ServerErrorException | NotExistException e) {
  
              LOGGER.warn(e);
  
          }

  In code You can see processing variable. If it’s set to false, I can watch my network streams on page with no problems. But if I set it to true, suddenly, I got visual artefacts, looking like this :

  http://answers.opencv.org/upfiles/1400931120927032.png

  And also I can see some infos on my logs :

  [libx264 @ 0x7fe2a7e2ae00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
  
  [libx264 @ 0x7fe2a7e2ae00] profile High, level 4.0
  
  [libx264 @ 0x7fe2a7e2ae00] 264 - core 142 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1,00:0,00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=400 ratetol=1,0 qcomp=0,60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1,40 aq=1:1,00
  
  [mp4 @ 0x7fe2909feee0] Using AVStream.codec.time_base as a timebase hint to the muxer is deprecated. Set AVStream.time_base instead.
  
  2015-02-02 10:34:31,986 INFO  [img.StreamGrabber] 6
  
  2015-02-02 10:34:31,998 INFO  [img.StreamGrabber] Processing :1
  
  2015-02-02 10:34:32,881 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1524, 1564, 678, 718) is above threshold
  
  2015-02-02 10:34:32,882 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1538, 1577, 678, 717) is above threshold
  
  2015-02-02 10:34:32,884 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1320, 1420, 298, 398) is above threshold
  
  2015-02-02 10:34:33,199 INFO  [img.StreamGrabber] 1
  
  2015-02-02 10:34:33,212 INFO  [img.StreamGrabber] 2
  
  2015-02-02 10:34:33,222 INFO  [img.StreamGrabber] 3
  
  2015-02-02 10:34:33,232 INFO  [img.StreamGrabber] 4
  
  2015-02-02 10:34:33,244 INFO  [img.StreamGrabber] 5
  
  2015-02-02 10:34:33,255 INFO  [img.StreamGrabber] Processing :6
  
  2015-02-02 10:34:33,870 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1537, 1578, 678, 719) is above threshold
  
  2015-02-02 10:34:33,871 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1315, 1422, 298, 405) is above threshold
  
  2015-02-02 10:34:34,318 INFO  [img.StreamGrabber] 6
  
  2015-02-02 10:34:34,338 INFO  [img.StreamGrabber] 7
  
  2015-02-02 10:34:34,347 INFO  [img.StreamGrabber] 8
  
  2015-02-02 10:34:34,357 INFO  [img.StreamGrabber] 9
  
  2015-02-02 10:34:34,368 INFO  [img.StreamGrabber] 10
  
  2015-02-02 10:34:34,379 INFO  [img.StreamGrabber] Processing :11
  
  2015-02-02 10:34:35,025 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1561, 1618, 477, 534) is above threshold
  
  2015-02-02 10:34:35,027 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1318, 1421, 300, 403) is above threshold
  
  2015-02-02 10:34:35,185 INFO  [img.StreamGrabber] 11
  
  2015-02-02 10:34:35,202 INFO  [img.StreamGrabber] 12
  
  2015-02-02 10:34:35,213 INFO  [img.StreamGrabber] 13
  
  2015-02-02 10:34:35,223 INFO  [img.StreamGrabber] 14
  
  2015-02-02 10:34:35,235 INFO  [img.StreamGrabber] 15
  
  2015-02-02 10:34:35,286 INFO  [img.StreamGrabber] Processing :16
  
  2015-02-02 10:34:35,952 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1429, 1470, 703, 744) is above threshold
  
  2015-02-02 10:34:35,954 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1315, 1422, 295, 402) is above threshold
  
  2015-02-02 10:34:36,218 INFO  [img.StreamGrabber] 16
  
  2015-02-02 10:34:36,237 INFO  [img.StreamGrabber] 17
  
  2015-02-02 10:34:36,246 INFO  [img.StreamGrabber] 18
  
  2015-02-02 10:34:36,257 INFO  [img.StreamGrabber] 19
  
  2015-02-02 10:34:36,268 INFO  [img.StreamGrabber] 20
  
  2015-02-02 10:34:36,279 INFO  [img.StreamGrabber] Processing :21
  
  2015-02-02 10:34:36,967 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1562, 1616, 480, 534) is above threshold
  
  2015-02-02 10:34:36,968 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1314, 1420, 296, 402) is above threshold
  
  2015-02-02 10:34:37,186 INFO  [img.StreamGrabber] 21
  
  2015-02-02 10:34:37,206 INFO  [img.StreamGrabber] 22
  
  2015-02-02 10:34:37,217 INFO  [img.StreamGrabber] 23
  
  2015-02-02 10:34:37,227 INFO  [img.StreamGrabber] 24
  
  [h264 @ 0x7fe2915b30a0] RTP: missed 1514 packets
  
  [h264 @ 0x7fe2f1050ea0] Cannot use next picture in error concealment
  
  [h264 @ 0x7fe2f1050ea0] concealing 4608 DC, 4608 AC, 4608 MV errors in P frame
  
  2015-02-02 10:34:37,238 INFO  [img.StreamGrabber] 25
  
  2015-02-02 10:34:37,250 INFO  [img.StreamGrabber] Processing :26
  
  2015-02-02 10:34:37,944 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1562, 1616, 479, 533) is above threshold
  
  2015-02-02 10:34:37,945 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1315, 1422, 297, 404) is above threshold
  
  2015-02-02 10:34:38,107 INFO  [img.StreamGrabber] 26
  
  [h264 @ 0x7fe2915b30a0] RTP: missed 295 packets
  
  [h264 @ 0x7fe2a5713e00] Cannot use next picture in error concealment
  
  [h264 @ 0x7fe2a5713e00] concealing 1996 DC, 1996 AC, 1996 MV errors in P frame
  
  2015-02-02 10:34:38,120 INFO  [img.StreamGrabber] 27
  
  2015-02-02 10:34:38,130 INFO  [img.StreamGrabber] 28
  
  2015-02-02 10:34:38,143 INFO  [img.StreamGrabber] 29
  
  2015-02-02 10:34:38,231 INFO  [img.StreamGrabber] 30
  
  2015-02-02 10:34:38,249 INFO  [img.StreamGrabber] Processing :31
  
  2015-02-02 10:34:38,962 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1170, 1211, 322, 363) is above threshold
  
  2015-02-02 10:34:38,964 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1316, 1421, 298, 403) is above threshold
  
  2015-02-02 10:34:39,329 INFO  [img.StreamGrabber] 31
  
  [h264 @ 0x7fe2915b30a0] RTP: missed 232 packets
  
  [h264 @ 0x7fe2a4203d80] Cannot use next picture in error concealment
  
  [h264 @ 0x7fe2a4203d80] concealing 1142 DC, 1142 AC, 1142 MV errors in P frame
  
  2015-02-02 10:34:39,342 INFO  [img.StreamGrabber] 32
  
  2015-02-02 10:34:39,352 INFO  [img.StreamGrabber] 33
  
  [h264 @ 0x7fe2915b30a0] RTP: missed 1 packets
  
  [h264 @ 0x7fe2915b43c0] corrupted macroblock 86 67 (total_coeff=-1)
  
  [h264 @ 0x7fe2915b43c0] error while decoding MB 86 67
  
  [h264 @ 0x7fe2915b43c0] Cannot use next picture in error concealment
  
  [h264 @ 0x7fe2915b43c0] concealing 83 DC, 83 AC, 83 MV errors in P frame
  
  2015-02-02 10:34:39,362 INFO  [img.StreamGrabber] Processing :144
  
  2015-02-02 10:34:40,071 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1563, 1614, 480, 531) is above threshold
  
  2015-02-02 10:34:40,074 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1318, 1423, 296, 401) is above threshold
  
  2015-02-02 10:34:40,462 INFO  [img.StreamGrabber] 144
  
  2015-02-02 10:34:40,482 INFO  [img.StreamGrabber] 145
  
  [h264 @ 0x7fe2915b30a0] RTP: missed 377 packets
  
  [h264 @ 0x7fe2a515baa0] Cannot use next picture in error concealment
  
  [h264 @ 0x7fe2a515baa0] concealing 6822 DC, 6822 AC, 6822 MV errors in P frame
  
  2015-02-02 10:34:40,494 INFO  [img.StreamGrabber] Processing :167
  
  2015-02-02 10:34:41,222 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1563, 1615, 479, 531) is above threshold
  
  2015-02-02 10:34:41,230 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1319, 1421, 295, 397) is above threshold
  
  2015-02-02 10:34:41,930 INFO  [img.StreamGrabber] 167
  
  2015-02-02 10:34:41,947 INFO  [img.StreamGrabber] 168
  
  2015-02-02 10:34:41,958 INFO  [img.StreamGrabber] 169
  
  2015-02-02 10:34:41,970 INFO  [img.StreamGrabber] 170
  
  2015-02-02 10:34:41,985 INFO  [img.StreamGrabber] 171
  
  [h264 @ 0x7fe2915b30a0] RTP: missed 311 packets
  
  [h264 @ 0x7fe2f10506c0] Cannot use next picture in error concealment
  
  [h264 @ 0x7fe2f10506c0] concealing 1409 DC, 1409 AC, 1409 MV errors in P frame
  
  2015-02-02 10:34:41,997 INFO  [img.StreamGrabber] Processing :190
  
  2015-02-02 10:34:42,715 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1322, 1384, 340, 402) is above threshold
  
  2015-02-02 10:34:42,717 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1312, 1425, 290, 403) is above threshold
  
  2015-02-02 10:34:42,929 INFO  [img.StreamGrabber] 190
  
  [h264 @ 0x7fe2915b30a0] RTP: missed 13 packets
  
  [h264 @ 0x7fe2f1050ea0] Cannot use next picture in error concealment
  
  [h264 @ 0x7fe2f1050ea0] concealing 6489 DC, 6489 AC, 6489 MV errors in P frame
  
  2015-02-02 10:34:42,943 INFO  [img.StreamGrabber] 191
  
  [h264 @ 0x7fe2915b30a0] RTP: missed 484 packets
  
  [h264 @ 0x7fe2915b43c0] concealing 6609 DC, 6609 AC, 6609 MV errors in I frame
  
  2015-02-02 10:34:42,957 INFO  [img.StreamGrabber] 192
  
  2015-02-02 10:34:42,970 INFO  [img.StreamGrabber] 193
  
  [h264 @ 0x7fe2915b30a0] RTP: missed 313 packets
  
  [h264 @ 0x7fe2a51a0fc0] Cannot use next picture in error concealment
  
  [h264 @ 0x7fe2a51a0fc0] concealing 1666 DC, 1666 AC, 1666 MV errors in P frame
  
  2015-02-02 10:34:43,271 INFO  [img.StreamGrabber] 194
  
  2015-02-02 10:34:43,314 INFO  [img.StreamGrabber] Processing :249
  
  2015-02-02 10:34:44,099 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1322, 1384, 340, 402) is above threshold
  
  2015-02-02 10:34:44,100 INFO  [img.analysis.face.PersonFaceRecognizer] Predicted face (1313, 1403, 300, 390) is above threshold
  
  2015-02-02 10:34:45,473 INFO  [img.save.event.EventMaker] Creating 1 face recognition events
  
  2015-02-02 10:34:45,618 INFO  [core.task.StreamRecordingTaskExecutor] Stream recording task ended: rtsp://MYURL

  My guess is, that my computer is simply too busy to catch all packages from camera stream. I’m operating on two streams, one is low quality with like 3 fps, and other is 30. Of course problems show up all the time on fast one, and rarely on slow stream.
  I’m wondering if there is possibility to force somehow FFmpegFrameGraber not to create artifacts, but simply drop current frame and go to next one ? Fps and frame continuity is not so important. I was trying to use grabber’s setfps, settimestamp, delayedGrab method’s to somehow slow down 30-fps stream, but it didn’t even react to that. I’m sure I’m doing something wrong.

  I’ve found some topics related to my problem, but they did not helped me, maybe You will see more :
  http://answers.opencv.org/question/34012/ip-camera-h264-error-while-decoding/
  How to deal with cv::VideoCapture decode errors ?
  http://superuser.com/questions/663928/ffmpeg-to-capture-stills-from-h-264-stream

  Thank You for Your help.

• Evolution #4427 (En cours) : Permettre (via une constante) de passer de 65 536 à 4096 le nombre ma...

  24 janvier 2020, par b b

  Salut, peux-tu fournir un diff du patch que tu proposes afin de faciliter la compréhension de ta proposition ?

  PS : version cible 3.3 mini

• How to Convert 16:9 Video to 9:16 Ratio While Ensuring Speaker Presence in Frame ?

  28 avril 2024, par shreesha

  I am tried so many time to figure out the problem in detecting the face and also it's not so smooth enough to like other tools out there.

  


  So basically I am using python and Yolo in this project but I want the person who is talking and who the ROI (region of interest) is.

  


  Here is the code :

  


  from ultralytics import YOLO
from ultralytics.engine.results import Results
from moviepy.editor import VideoFileClip, concatenate_videoclips
from moviepy.video.fx.crop import crop

# Load the YOLOv8 model
model = YOLO("yolov8n.pt")

# Load the input video
clip = VideoFileClip("short_test.mp4")

tacked_clips = []

for frame_no, frame in enumerate(clip.iter_frames()):
    # Process the frame
    results: list[Results] = model(frame)

    # Get the bounding box of the main object
    if results[0].boxes:
        objects = results[0].boxes
        main_obj = max(
            objects, key=lambda x: x.conf
        )  # Assuming the first detected object is the main one

        x1, y1, x2, y2 = [int(val) for val in main_obj.xyxy[0].tolist()]

        # Calculate the crop region based on the object's position and the target aspect ratio
        w, h = clip.size
        new_w = int(h * 9 / 16)
        new_h = h

        x_center = x2 - x1
        y_center = y2 - y1

        # Adjust x_center and y_center if they would cause the crop region to exceed the bounds
        if x_center + (new_w / 2) > w:
            x_center -= x_center + (new_w / 2) - w
        elif x_center - (new_w / 2) < 0:
            x_center += abs(x_center - (new_w / 2))

        if y_center + (new_h / 2) > h:
            y_center -= y_center + (new_h / 2) - h
        elif y_center - (new_h / 2) < 0:
            y_center += abs(y_center - (new_h / 2))

        # Create a subclip for the current frame
        start_time = frame_no / clip.fps
        end_time = (frame_no + 1) / clip.fps
        subclip = clip.subclip(start_time, end_time)

        # Apply cropping using MoviePy
        cropped_clip = crop(
            subclip, x_center=x_center, y_center=y_center, width=new_w, height=new_h
        )

        tacked_clips.append(cropped_clip)

reframed_clip = concatenate_videoclips(tacked_clips, method="compose")
reframed_clip.write_videofile("output_video.mp4")


  


  So basically I want to fix the face detection with ROI detection where it can detect the face and make that face and the body on to the frame and making sure that the speaker who is speaking is brought to the frame