Recherche avancée

Sur d’autres sites (2604)

• Lossless avi encoding on linux

  31 août 2012, par JohnSavage

  I am trying to write video using opencv. It is important for me to do this precisely - so it has to be a lossless codec. I am working with OpenCV 2.4.1 on Ubuntu 12.04

  Previously, I was using the fourcc code 0. This gave me the exact result I wanted, and I was able to recover the images perfectly.

  I am not sure what happened, but as of a recent update (around Jul 20th 2012), something went wrong and I am no longer able to write files with this fourcc code. I really don't remember what it was, but it could have come from doing an update, removing some software from my software center, and some other things I did during general cleaning...

  When I check an older file with mediainfo (http://www.fourcc.org/identifier/) I see the following result :

  Complete name                            : oldsample.avi
  
  Format                                   : AVI
  
  Format/Info                              : Audio Video Interleave
  
  Format profile                           : OpenDML
  
  File size                                : 1.07 GiB
  
  Duration                                 : 41s 467ms
  
  Overall bit rate                         : 221 Mbps
  
  Writing application                      : Lavf53.5.0
  
  Video
  
  ID                                       : 0
  
  Format                                   : RGB
  
  Codec ID                                 : 0x00000000
  
  Codec ID/Info                            : Basic Windows bitmap format. 1, 4 and 8 bpp     versions are palettised. 16, 24 and 32bpp contain raw RGB samples
  
  Duration                                 : 41s 467ms
  
  Bit rate                                 : 221 Mbps
  
  Width                                    : 640 pixels
  
  Height                                   : 4294966 816 pixels
  
  Display aspect ratio                     : 0.000
  
  Frame rate                               : 30.000 fps
  
  Bit depth                                : 8 bits
  
  Stream size                              : 1.07 GiB (100%)

  Now, I see that when I write using the 0 fourcc codec, the program actually defaults to the i420 codec. Here is the output from one of the files I try to write now :

  Complete name                            : newsample.avi
  
  Format                                   : AVI
  
  Format/Info                              : Audio Video Interleave
  
  File size                                : 73.0 MiB
  
  Duration                                 : 5s 533ms
  
  Overall bit rate                         : 111 Mbps
  
  Writing application                      : Lavf54.6.100
  
  Video
  
  ID                                       : 0
  
  Format                                   : YUV
  
  Codec ID                                 : I420
  
  Codec ID/Info                            : 8 bit Y plane followed by 8 bit 2x2 subsampled U and V planes.
  
  Duration                                 : 5s 533ms
  
  Bit rate                                 : 111 Mbps
  
  Width                                    : 640 pixels
  
  Height                                   : 480 pixels
  
  Display aspect ratio                     : 4:3
  
  Frame rate                               : 30.000 fps
  
  Compression mode                         : Lossless
  
  Bits/(Pixel*Frame)                       : 12.000
  
  Stream size                              : 72.9 MiB (100%)

  This format, and other formats I try to use (like huffyuv HFYU), do not work for me because I end up with effects like this http://imgur.com/a/0OC4y - you see the bright artifacts coming in due to what I assume is either lossy compression or chroma subsampling in the case of HFYU which is supposed to be lossless. What you are looking at is the red channel from one of my videos. The perceptual effect is negligible when you look at all 3 channels simultaneously but it is essential that I reconstruct the images exactly.

  Furthermore, while I am able to play my old files in media players like vlc, I suddenly find them to be completely incompatible with opencv. When I try to open the older files with a videocapture, the open step works fine, but trying to do a read operation results in a segfault. Furthermore, When I try to write with either :

  CV_FOURCC(0,0,0,0)
  
  0

  Opencv defaults to I420 for some reason.

  Next, I tried using some alternate codecs. 'DIB ' seems like something that should work for me, and on the opencv website (http://opencv.willowgarage.com/wiki/VideoCodecs) it is listed as a 'recommended' codec. However, trying to use this results in the following message :

  OpenCV-2.4.1/modules/highgui/src/cap_gstreamer.cpp:483: error: (-210) Gstreamer Opencv backend doesn't support this codec acutally. in function CvVideoWriter_GStreamer::open
  
  
  
  Aborted (core dumped)

  I checked the opencv source for this codec, and stumbled across the following :

  cd OpenCV-2.4.1/modules
  
  grep -i -r "CV_FOURCC" ./*
  
  ...
  
  ./highgui/src/cap_qt.cpp:    /*if( fourcc == CV_FOURCC( 'D', 'I', 'B', ' ' ))
  
  ./highgui/include/opencv2/highgui/highgui_c.h:#define CV_FOURCC_DEFAULT CV_FOURCC('I', 'Y', 'U', 'V') /* Use default codec for specified filename (Linux only) */

  I tried installing qt4 and reconfiguring with the WITH_QT flag, but that did not change anything. I also tried uncommenting that part of the code and reinstalling opencv, but that also did not work.

  My ultimate goal is for any way to efficiently store and retrieve a video stream with 16 bits for every pixel (like 32float would work fine, and then it wouldn't need to be perfect). Right now I am unpacking the 16 bits into the red and green channels, which is why I need it to be perfect - since an error of 1 in the red channel is multiplied by 256 in the final result. I am not having success with any of the fourcc codes available to me.

• Java FFmpeg decoding AVPacket result of avcodec_decode_video2 negative

  19 octobre 2017, par nh_

  I’m new to ffmpeg. I’m using FFmpegFrameGrabber and JavaCPP (version 1.3.3) to grab packets of an mp4-video-file. I would like to save the packet’s bytestream in a database in order to decode the packets at the time the data is requested and to process the image.

  public static void saveVideoToDB(String pathToVideo){
  
  FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(pathToVideo);
  
  grabber.start();
  
  AVPacket pkt;
  
  while ((pkt = grabber.grabPacket()) != null) {
  
      BytePointer data = pkt.data();
  
      data.capacity(pkt.size());
  
      byte[] arr = data.getStringBytes();  
  
      //arr = [0, 0, 0, 62, 39, 100, 0, 40, -83, -124.....]
  
       if (pkt.stream_index() == AVMEDIA_TYPE_VIDEO) {
  
          //ToDo: save arr to database
  
           testDecode(arr);
  
       }
  
  
  
  
  
  }
  
  grabber.close();
  
  logger.info("Import video finished.");
  
  }   

  In my code I first tried to decode a packet’s data for a proof-of-concept, but I’m not sure if it’s working like this :

   public static void testDecode(byte[] data){
  
      AVCodec avCodec = avcodec_find_decoder(AV_CODEC_ID_H264);
  
      AVCodecContext avCodecContext = avcodec_alloc_context3(avCodec);
  
      AVDictionary opts = new AVDictionary();
  
      avcodec_open2(avCodecContext, avCodec, opts);
  
      av_dict_free(opts);
  
      AVFrame avFrame = av_frame_alloc();
  
      AVPacket avPacket = new AVPacket();
  
      av_init_packet(avPacket);
  
      Frame frame = new Frame();
  
      avPacket.pts(AV_NOPTS_VALUE);
  
      avPacket.dts(AV_NOPTS_VALUE);
  
      BytePointer bp = new BytePointer(data);
  
      bp.capacity(data.length);
  
      avPacket.data(bp);
  
      avPacket.size(data.length);
  
      avPacket.pos(-1);
  
  
  
      IntBuffer gotPicture = IntBuffer.allocate(1);
  
      boolean doVideo = true;
  
      boolean keyFrames = false;
  
      boolean processImage = true;
  
      AVPacket pkt = avPacket; 
  
      if (doVideo) {
  
        int len = avcodec_decode_video2(avCodecContext, avFrame, gotPicture, avPacket);
  
        if (len >= 0 && gotPicture.get(0) != 0
  
            && (!keyFrames || avFrame.pict_type() == AV_PICTURE_TYPE_I)) {
  
          //ToDo: process image
  
          logger.info("decode success");
  
        }else{
  
          logger.info("decode failed");
  
        }
  
      }
  
    }

  The result of avcodec_decode_video2 is always negative (-1094995529 => Invalid data found when processing input) and I receive following errors :

  [h264 @ 00000000214d11a0] No start code is found.

  [h264 @ 00000000214d11a0] Error splitting the input into NAL units.

  Here are some metadata from the input :

  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\user01\Documents\fullstream.mp4':
  
    Metadata:
  
      major_brand     : mp42
  
      minor_version   : 0
  
      compatible_brands: mp42mp41isomiso2
  
      creation_time   : 2017-07-27T11:17:19.000000Z
  
    Duration: 00:55:55.48, start: 0.000000, bitrate: 5126 kb/s
  
      Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 4996 kb/s, 25 fps, 25 tbr, 2500 tbn, 5k tbc (default)
  
      Metadata:
  
        creation_time   : 2017-07-27T11:17:19.000000Z
  
        handler_name    : VideoHandler
  
      Stream #0:1(eng): Audio: mp3 (mp4a / 0x6134706D), 24000 Hz, mono, s16p, 125 kb/s (default)
  
      Metadata:
  
        creation_time   : 2017-07-27T11:17:19.000000Z
  
        handler_name    : SoundHandler

• Live audio using ffmpeg, javascript and nodejs

  8 novembre 2017, par klaus

  I am new to this thing. Please don’t hang me for the poor grammar. I am trying to create a proof of concept application which I will later extend. It does the following : We have a html page which asks for permission to use the microphone. We capture the microphone input and send it via websocket to a node js app.

  JS (Client) :

  var bufferSize = 4096;
  
  var socket = new WebSocket(URL);
  
  var myPCMProcessingNode = context.createScriptProcessor(bufferSize, 1, 1);
  
  myPCMProcessingNode.onaudioprocess = function(e) {
  
    var input = e.inputBuffer.getChannelData(0);
  
    socket.send(convertFloat32ToInt16(input));
  
  }
  
  
  
  function convertFloat32ToInt16(buffer) {
  
    l = buffer.length;
  
    buf = new Int16Array(l);
  
    while (l--) {
  
      buf[l] = Math.min(1, buffer[l])*0x7FFF;
  
    }
  
    return buf.buffer;
  
  }
  
  
  
  navigator.mediaDevices.getUserMedia({audio:true, video:false})
  
                                  .then(function(stream){
  
                                    var microphone = context.createMediaStreamSource(stream);
  
                                    microphone.connect(myPCMProcessingNode);
  
                                    myPCMProcessingNode.connect(context.destination);
  
                                  })
  
                                  .catch(function(e){});

  In the server we take each incoming buffer, run it through ffmpeg, and send what comes out of the std out to another device using the node js ’http’ POST. The device has a speaker. We are basically trying to create a 1 way audio link from the browser to the device.

  Node JS (Server) :

  var WebSocketServer = require('websocket').server;
  
  var http = require('http');
  
  var children = require('child_process');
  
  
  
  wsServer.on('request', function(request) {
  
    var connection = request.accept(null, request.origin);
  
    connection.on('message', function(message) {
  
      if (message.type === 'utf8') { /*NOP*/ }
  
      else if (message.type === 'binary') {
  
        ffm.stdin.write(message.binaryData);
  
      }
  
    });
  
    connection.on('close', function(reasonCode, description) {});
  
    connection.on('error', function(error) {});
  
  });
  
  
  
  var ffm = children.spawn(
  
      './ffmpeg.exe'
  
     ,'-stdin -f s16le -ar 48k -ac 2 -i pipe:0 -acodec pcm_u8 -ar 48000 -f aiff pipe:1'.split(' ')
  
  );
  
  
  
  ffm.on('exit',function(code,signal){});
  
  
  
  ffm.stdout.on('data', (data) => {
  
    req.write(data);
  
  });
  
  
  
  var options = {
  
    host: 'xxx.xxx.xxx.xxx',
  
    port: xxxx,
  
    path: '/path/to/service/on/device',
  
    method: 'POST',
  
    headers: {
  
     'Content-Type': 'application/octet-stream',
  
     'Content-Length': 0,
  
     'Authorization' : 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx',
  
     'Transfer-Encoding' : 'chunked',
  
     'Connection': 'keep-alive'
  
    }
  
  };
  
  
  
  var req = http.request(options, function(res) {});

  The device supports only continuous POST and only a couple of formats (ulaw, aiff, wav)

  This solution doesn’t seem to work. In the device speaker we only hear something like white noise.

  Also, I think I may have a problem with the buffer I am sending to the ffmpeg std in -> Tried to dump whatever comes out of the websocket to a .wav file then play it with VLC -> it plays everything in the record very fast -> 10 seconds of recording played in about 1 second.

  I am new to audio processing and have searched for about 3 days now for solutions on how to improve this and found nothing.

  I would ask from the community for 2 things :

  1. Is something wrong with my approach ? What more can I do to make this work ? I will post more details if required.

  2. If what I am doing is reinventing the wheel then I would like to know what other software / 3rd party service (like amazon or whatever) can accomplish the same thing.

  Thank you.