Recherche avancée

Sur d’autres sites (8242)

• ffmpeg decoding yuv420p to rgb shifts luminance/brightness down

  6 avril 2013, par glopes

  I'm recording video from a grayscale camera into an MPEG4 video file using FFMPEG. Recently I noticed a weird effect : if I playback the file using FFMPEG or windows media player, the output frames are noticeably darker (by about 10 brightness values) than the original source.

  I thought the encoding step was doing this until I opened the same file in VLC and it gave me back the correct result. I played around with FFMPEG command lines to decode a single frame and realized that if I decode a frame into GRAY8 pixel format, the brightness/luminance values are preserved. Here's the command and ffmpeg output :

  ffmpeg -ss 0.5 -i video.avi -vframes 1 -t 1 -s 1280x680 -pix_fmt gray gray.bmp
  
  
  
  ffmpeg version 1.2 Copyright (c) 2000-2013 the FFmpeg developers
  
    built on Apr  4 2013 12:40:58 with gcc 4.6.2 (GCC)
  
    configuration: --enable-w32threads
  
    libavutil      52. 18.100 / 52. 18.100
  
    libavcodec     54. 92.100 / 54. 92.100
  
    libavformat    54. 63.104 / 54. 63.104
  
    libavdevice    54.  3.103 / 54.  3.103
  
    libavfilter     3. 42.103 /  3. 42.103
  
    libswscale      2.  2.100 /  2.  2.100
  
    libswresample   0. 17.102 /  0. 17.102
  
  Input #0, avi, from 'video.avi':
  
    Metadata:
  
      encoder         : Lavf53.32.100
  
    Duration: 00:00:07.47, start: 0.000000, bitrate: 16532 kb/s
  
      Stream #0:0: Video: mpeg4 (Simple Profile) (FMP4 / 0x34504D46), yuv420p, 128
  
  0x720 [SAR 1:1 DAR 16:9], 30 tbr, 30 tbn, 30 tbc
  
  Output #0, image2, to 'gray.bmp':
  
    Metadata:
  
      encoder         : Lavf54.63.104
  
      Stream #0:0: Video: bmp, gray, 1280x680 [SAR 17:18 DAR 16:9], q=2-31, 200 kb
  
  /s, 90k tbn, 30 tbc
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mpeg4 -> bmp)
  
  Press [q] to stop, [?] for help
  
  frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.03 bitrate=N/A
  
  video:851kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.002524%

  However, if I decode the same frame into an rgb color pixel format, I end up again with the darker frames :

  ffmpeg -ss 0.5 -i video.avi -vframes 1 -t 1 -s 1280x680 -pix_fmt bgr24 rgb.bmp
  
  
  
  ffmpeg version 1.2 Copyright (c) 2000-2013 the FFmpeg developers
  
    built on Apr  4 2013 12:40:58 with gcc 4.6.2 (GCC)
  
    configuration: --enable-w32threads
  
    libavutil      52. 18.100 / 52. 18.100
  
    libavcodec     54. 92.100 / 54. 92.100
  
    libavformat    54. 63.104 / 54. 63.104
  
    libavdevice    54.  3.103 / 54.  3.103
  
    libavfilter     3. 42.103 /  3. 42.103
  
    libswscale      2.  2.100 /  2.  2.100
  
    libswresample   0. 17.102 /  0. 17.102
  
  Input #0, avi, from 'video.avi':
  
    Metadata:
  
      encoder         : Lavf53.32.100
  
    Duration: 00:00:07.47, start: 0.000000, bitrate: 16532 kb/s
  
      Stream #0:0: Video: mpeg4 (Simple Profile) (FMP4 / 0x34504D46), yuv420p, 128
  
  0x720 [SAR 1:1 DAR 16:9], 30 tbr, 30 tbn, 30 tbc
  
  Output #0, image2, to 'rgb.bmp':
  
    Metadata:
  
      encoder         : Lavf54.63.104
  
      Stream #0:0: Video: bmp, bgr24, 1280x680 [SAR 17:18 DAR 16:9], q=2-31, 200 k
  
  b/s, 90k tbn, 30 tbc
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mpeg4 -> bmp)
  
  Press [q] to stop, [?] for help
  
  frame=    1 fps=0.0 q=0.0 Lsize=N/A time=00:00:00.03 bitrate=N/A
  
  video:2550kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.000843%

  This happens with every single grayscale video that I encode to MPEG4 with FFMPEG. My best guess so far is it has to do with how the container pixel format gets converted to/from. Since I'm using MPEG4, the file pixel format is YUV420P. I have no idea how ffmpeg encodes from GRAY8 to YUV420P, but maybe it stores just the luminance values in Y... if this happens, then decoding from this to RGB could produce darker pixels by the scaling factor that is applied to the luminance matrix ?

  To sum it up :

  1) How can encoding a grayscale video to YUV420P with FFMPEG and decoding back produce wrong (darker) brightness values when decoding to RGB versus GRAY8 ? Presumably once the frames are in YUV420P format it shouldn't matter whether the source is actually grayscale or not so the result should be equivalent, no ?

  2) How does VLC avoid this situation ? I was under the impression that VLC used FFMPEG as well for video decoding, but somehow they managed to figure out how to produce the correct values without requiring me to indicate explicitly that the video was grayscale.

• How to obtain time markers for video splitting using python/OpenCV

  30 mars 2016, par Bleddyn Raw-Rees

  Hi I’m new to the world of programming and computer vision so please bare with me.

  I’m working on my MSc project which is researching automated deletion of low value content in digital file stores. I’m specifically looking at the sort of long shots that often occur in natural history filming whereby a static camera is left rolling in order to capture the rare snow leopard or whatever. These shots may only have some 60s of useful content with perhaps several hours of worthless content either side.

  As a first step I have a simple motion detection program from Adrian Rosebrock’s tutorial [http://www.pyimagesearch.com/2015/05/25/basic-motion-detection-and-tracking-with-python-and-opencv/#comment-393376]. Next I intend to use FFMPEG to split the video.

  What I would like help with is how to get in and out points based on the first and last points that motion is detected in the video.

  Here is the code should you wish to see it...

  # import the necessary packages
  
  import argparse
  
  import datetime
  
  import imutils
  
  import time
  
  import cv2
  
  
  
  # construct the argument parser and parse the arguments
  
  ap = argparse.ArgumentParser()
  
  ap.add_argument("-v", "--video", help="path to the video file")
  
  ap.add_argument("-a", "--min-area", type=int, default=500, help="minimum area size")
  
  args = vars(ap.parse_args())
  
  
  
  # if the video argument is None, then we are reading from webcam
  
  if args.get("video", None) is None:
  
  camera = cv2.VideoCapture(0)
  
  time.sleep(0.25)
  
  
  
  # otherwise, we are reading from a video file
  
  else:
  
      camera = cv2.VideoCapture(args["video"])
  
  
  
  # initialize the first frame in the video stream
  
  firstFrame = None
  
  
  
  # loop over the frames of the video
  
  while True:
  
      # grab the current frame and initialize the occupied/unoccupied
  
      # text
  
      (grabbed, frame) = camera.read()
  
      text = "Unoccupied"
  
  
  
      # if the frame could not be grabbed, then we have reached the end
  
      # of the video
  
      if not grabbed:
  
          break
  
  
  
      # resize the frame, convert it to grayscale, and blur it
  
      frame = imutils.resize(frame, width=500)
  
      gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
  
      gray = cv2.GaussianBlur(gray, (21, 21), 0)
  
  
  
      # if the first frame is None, initialize it
  
      if firstFrame is None:
  
          firstFrame = gray
  
          continue
  
  
  
      # compute the absolute difference between the current frame and
  
      # first frame
  
      frameDelta = cv2.absdiff(firstFrame, gray)
  
      thresh = cv2.threshold(frameDelta, 25, 255, cv2.THRESH_BINARY)[1]
  
  
  
      # dilate the thresholded image to fill in holes, then find contours
  
      # on thresholded image
  
      thresh = cv2.dilate(thresh, None, iterations=2)
  
      (_, cnts, _) = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
  
  
  
      # loop over the contours
  
      for c in cnts:
  
          # if the contour is too small, ignore it
  
          if cv2.contourArea(c) < args["min_area"]:
  
              continue
  
  
  
          # compute the bounding box for the contour, draw it on the frame,
  
          # and update the text
  
          (x, y, w, h) = cv2.boundingRect(c)
  
          cv2.rectangle(frame, (x, y), (x + w, y + h), (0, 255, 0), 2)
  
          text = "Occupied"
  
  
  
      # draw the text and timestamp on the frame
  
      cv2.putText(frame, "Room Status: {}".format(text), (10, 20),
  
          cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)
  
      cv2.putText(frame, datetime.datetime.now().strftime("%A %d %B %Y %I:%M:%S%p"),
  
          (10, frame.shape[0] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1)
  
  
  
      # show the frame and record if the user presses a key
  
      cv2.imshow("Security Feed", frame)
  
      cv2.imshow("Thresh", thresh)
  
      cv2.imshow("Frame Delta", frameDelta)
  
      key = cv2.waitKey(1) & 0xFF
  
  
  
      # if the `q` key is pressed, break from the lop
  
      if key == ord("q"):
  
          break
  
  
  
  # cleanup the camera and close any open windows
  
  camera.release()
  
  cv2.destroyAllWindows()

  Thanks !

• Stream webm to node.js from c# application in chunks

  7 août 2015, par Dan-Levi Tømta

  I am in the process of learning about streaming between node.js with socket.io and c#.

  I have code that successfully records the screen with ffmpeg, redirects it StandardOutput.BaseStream and stores it into a Memorybuffer, when i click stop in my application it sends the memorystream as a byte array to the node.js server which are storing the file so the clients can play it. This are working just fine and here are my setup for that :

  C#

  bool ffWorkerIsWorking = false;
  
  private void btnFFMpeg_Click(object sender, RoutedEventArgs e)
  
  {
  
      BackgroundWorker ffWorker = new BackgroundWorker();
  
      ffWorker.WorkerSupportsCancellation = true;
  
      ffWorker.DoWork += ((ffWorkerObj,ffWorkerEventArgs) =>
  
      {
  
          ffWorkerIsWorking = true;
  
          using (var FFProcess = new Process())
  
          {
  
              var processStartInfo = new ProcessStartInfo
  
              {
  
                  FileName = "ffmpeg.exe",
  
                  RedirectStandardInput = true,
  
                  RedirectStandardOutput = true,
  
                  UseShellExecute = false,
  
                  CreateNoWindow = false,
  
                  Arguments = " -loglevel panic -hide_banner -y -f gdigrab -draw_mouse 1 -i desktop -threads 2 -deadline realtime  -f webm -"
  
              };
  
              FFProcess.StartInfo = processStartInfo;
  
              FFProcess.Start();
  
  
  
              byte[] buffer = new byte[32768];
  
              using (MemoryStream ms = new MemoryStream())
  
              {
  
                  while (!FFProcess.HasExited)
  
                  {
  
                      int read = FFProcess.StandardOutput.BaseStream.Read(buffer, 0, buffer.Length);
  
                      if (read <= 0)
  
                          break;
  
                      ms.Write(buffer, 0, read);
  
                      Console.WriteLine(ms.Length);
  
                      if (!ffWorkerIsWorking)
  
                      {                                
  
                          clientSocket.Emit("video", ms.ToArray());                                 
  
                          ffWorker.CancelAsync();
  
                          break;
  
                      }
  
                  }
  
              }
  
          }
  
      });
  
      ffWorker.RunWorkerAsync();
  
  }

  JS (Server)

  socket.on('video', function(data) {
  
      fs.appendFile('public/fooTest.webm', data, function (err) {
  
        if (err) throw err;
  
        console.log('File uploaded');
  
      });
  
  });

  Now i need to change this code so it instead of sending the whole file it should sends chunks of byte arrays instead of the whole video, and node will then initially create a file and then append those chunks of byte arrays as they are received. Ok sound easy enough, but apparently not.

  I need to somehow instruct the code to use a offset and just the bytes after that offset and then update the offset.

  On the server side i think the best approach is to create a file and append the byte arrays to that file as they are received.

  On the server side i would do something like this :

  JS (Server)

  var buffer = new Buffer(32768);
  
  var isBuffering = false;
  
  socket.on('video', function(data) {
  
      //concatenate the buffer with the incoming data and broadcast.emit to clients
  
  
  
  });

  How am i able to setup the offset for the bytes to be sent and update that offset, and how would i approach the way of concatenating the data to the initialized buffer ?

  I have tried to write some code that only reads from the offset to the end and it seems like this is working although the video when added up in node is just black :

  C#

  while (!FFProcess.HasExited)
  
  {
  
      int read = FFProcess.StandardOutput.BaseStream.Read(buffer, 0, buffer.Length);
  
      if (read <= 0)
  
          break;
  
      int offset = (read - buffer.Length > 0 ? read - buffer.Length : 0);
  
      ms.Write(buffer, offset, read);
  
      clientSocket.Emit("videoChunk", buffer.ToArray());
  
      if (!ffWorkerIsWorking)
  
      {                                
  
          ffWorker.CancelAsync();
  
          break;
  
      }
  
  }

  Node console output

  

  JS (Server)

  socket.on('videoChunk', function(data) {
  
      if (!isBufferingDone) {
  
          buffer = Buffer.concat([buffer, data]);
  
          console.log(data.length);
  
      }
  
  });
  
  
  
  socket.on('cancelVideo', function() {
  
      isBufferingDone = true;
  
      setTimeout(function() {
  
          fs.writeFile("public/test.webm", buffer, function(err) {
  
              if(err) {
  
                  return console.log(err);
  
              }
  
              console.log("The file was saved!");
  
              buffer = new Buffer(32768);
  
          }); 
  
      }, 1000); 
  
  });

  JS (Client)

  socket.on('video', function(filePath) {
  
     console.log('path: ' + filePath);
  
     $('#videoSource').attr('src',filePath);
  
     $('#video').play();
  
  });

  Thanks !