Recherche avancée

Sur d’autres sites (10492)

• Simply beyond ridiculous

  7 mai 2010, par Dark Shikari — H.265, speed

  For the past few years, various improvements on H.264 have been periodically proposed, ranging from larger transforms to better intra prediction. These finally came together in the JCT-VC meeting this past April, where over two dozen proposals were made for a next-generation video coding standard. Of course, all of these were in very rough-draft form ; it will likely take years to filter it down into a usable standard. In the process, they’ll pick the most useful features (hopefully) from each proposal and combine them into something a bit more sane. But, of course, it all has to start somewhere.

  A number of features were common : larger block sizes, larger transform sizes, fancier interpolation filters, improved intra prediction schemes, improved motion vector prediction, increased internal bit depth, new entropy coding schemes, and so forth. A lot of these are potentially quite promising and resolve a lot of complaints I’ve had about H.264, so I decided to try out the proposal that appeared the most interesting : the Samsung+BBC proposal (A124), which claims compression improvements of around 40%.

  The proposal combines a bouillabaisse of new features, ranging from a 12-tap interpolation filter to 12thpel motion compensation and transforms as large as 64×64. Overall, I would say it’s a good proposal and I don’t doubt their results given the sheer volume of useful features they’ve dumped into it. I was a bit worried about complexity, however, as 12-tap interpolation filters don’t exactly scream “fast”.

  I prepared myself for the slowness of an unoptimized encoder implementation, compiled their tool, and started a test encode with their recommended settings.

  I waited. The first frame, an I-frame, completed.

  I took a nap.

  I waited. The second frame, a P-frame, was done.

  I played a game of Settlers.

  I waited. The third frame, a B-frame, was done.

  I worked on a term paper.

  I waited. The fourth frame, a B-frame, was done.

  After a full 6 hours, 8 frames had encoded. Yes, at this rate, it would take a full two weeks to encode 10 seconds of HD video. On a Core i7. This is not merely slow ; this is over 1000 times slower than x264 on “placebo” mode. This is so slow that it is not merely impractical ; it is impossible to even test. This encoder is apparently designed for some sort of hypothetical future computer from space. And word from other developers is that the Intel proposal is even slower.

  This has led me to suspect that there is a great deal of cheating going on in the H.265 proposals. The goal of the proposals, of course, is to pick the best feature set for the next generation video compression standard. But there is an extra motivation : organizations whose features get accepted get patents on the resulting standard, and thus income. With such large sums of money in the picture, dishonesty becomes all the more profitable.

  There is a set of rules, of course, to limit how the proposals can optimize their encoders. If different encoders use different optimization techniques, the results will no longer be comparable — remember, they are trying to compare compression features, not methods of optimizing encoder-side decisions. Thus all encoders are required to use a constant quantizer, specified frame types, and so forth. But there are no limits on how slow an encoder can be or what algorithms it can use.

  It would be one thing if the proposed encoder was a mere 10 times slower than the current reference ; that would be reasonable, given the low level of optimization and higher complexity of the new standard. But this is beyond ridiculous. With the prize given to whoever can eke out the most PSNR at a given quantizer at the lowest bitrate (with no limits on speed), we’re just going to get an arms race of slow encoders, with every company trying to use the most ridiculous optimizations possible, even if they involve encoding the frame 100,000 times over to choose the optimal parameters. And the end result will be as I encountered here : encoders so slow that they are simply impossible to even test.

  Such an arms race certainly does little good in optimizing for reality where we don’t have 30 years to encode an HD movie : a feature that gives great compression improvements is useless if it’s impossible to optimize for in a reasonable amount of time. Certainly once the standard is finalized practical encoders will be written — but it makes no sense to optimize the standard for a use-case that doesn’t exist. And even attempting to “optimize” anything is difficult when encoding a few seconds of video takes weeks.

  Update : The people involved have contacted me and insist that there was in fact no cheating going on. This is probably correct ; the problem appears to be that the rules that were set out were simply not strict enough, making many changes that I would intuitively consider “cheating” to be perfectly allowed, and thus everyone can do it.

  I would like to apologize if I implied that the results weren’t valid ; they are — the Samsung-BBC proposal is definitely one of the best, which is why I picked it to test with. It’s just that I think any situation in which it’s impossible to test your own software is unreasonable, and thus the entire situation is an inherently broken one, given the lax rules, slow baseline encoder, and no restrictions on compute time.

• Access Violation at avcodec_encode_video2()

  23 mars 2016, par bot1131357

  I am trying to understand the FFmpeg API by following online examples available but it seems that the FFmpeg API has changed over time, making most of the examples obsolete ; I hope some of you can help me make more sense of the FFmpeg API examples.

  I am currently trying to understand the encoding-example from FFmpeg, but I am getting an Access Violation error at this line :

  out_size = avcodec_encode_video2(codecCtx, &avpkt, picture, &got_packet);

  where I get "Unhandled exception at 0x77c29e42 in test01_encode.exe : 0xC0000005 : Access violation reading location 0xccccccc8." from Visual Studio.

  I understand that avcodec_encode_video() is deprecated in favour of avcodec_encode_video2(), which uses AVPacket. I’ve allocated a buffer to data member of AVPacket and set its size, but still the same. What did I miss ?

  The library that I’m using is ffmpeg-20160219-git-98a0053-win32-dev. I would really really appreciate if you could help me out of this confusion.

  (Side : What does it mean by "get delayed frames" and why are we encoding by specifying AVFrame* parameter as NULL ?)

  /*
  
   * Video encoding example
  
   */
  
  char filename[] = "test.mpg";
  
  int main(int argc, char** argv)
  
  {
  
      AVCodec *codec;
  
      AVCodecContext *codecCtx= NULL;
  
      int i, out_size, size, x, y, outbuf_size;
  
      FILE *f;
  
      AVFrame *picture;
  
      uint8_t *outbuf, *picture_buf;
  
  
  
      printf("Video encoding\n");
  
  
  
      // Register all formats and codecs
  
      av_register_all();
  
  
  
      /* find the mpeg1 video encoder */
  
      codec = avcodec_find_encoder(AV_CODEC_ID_MPEG1VIDEO);
  
      if (!codec) {
  
          fprintf(stderr, "codec not found\n");
  
          exit(1);
  
      }
  
  
  
      codecCtx= avcodec_alloc_context3(codec);
  
      picture= av_frame_alloc();
  
  
  
      /* put sample parameters */
  
      codecCtx->bit_rate = 400000;
  
      /* resolution must be a multiple of two */
  
      codecCtx->width = 352;
  
      codecCtx->height = 288;
  
      /* frames per second */
  
      //codecCtx->time_base= (AVRational){1,25};
  
      codecCtx->time_base.num = 1;
  
      codecCtx->time_base.den = 25;
  
  
  
      codecCtx->gop_size = 10; /* emit one intra frame every ten frames */
  
      codecCtx->max_b_frames=1;
  
      codecCtx->pix_fmt = AV_PIX_FMT_YUV420P;
  
  
  
      /* open it */
  
      if (avcodec_open2(codecCtx, codec, NULL) < 0) {
  
          fprintf(stderr, "could not open codec\n");
  
          exit(1);
  
      }
  
  
  
      fopen_s(&f,filename, "wb");
  
      if (!f) {
  
          fprintf(stderr, "could not open %s\n", filename);
  
          exit(1);
  
      }
  
  
  
      /* alloc image and output buffer */
  
      outbuf_size = 100000;
  
      outbuf = (uint8_t*) malloc(outbuf_size);
  
      size = codecCtx->width * codecCtx->height;
  
      picture_buf = (uint8_t*) malloc((size * 3) / 2); /* size for YUV 420 */
  
  
  
      picture->data[0] = picture_buf;
  
      picture->data[1] = picture->data[0] + size;
  
      picture->data[2] = picture->data[1] + size / 4;
  
      picture->linesize[0] = codecCtx->width;
  
      picture->linesize[1] = codecCtx->width / 2;
  
      picture->linesize[2] = codecCtx->width / 2;
  
  
  
      picture->width = codecCtx->width;
  
      picture->height = codecCtx->height;
  
      picture->format = codecCtx->pix_fmt;
  
  
  
      AVPacket avpkt;
  
      int got_packet;
  
  
  
      avpkt.size=av_image_get_buffer_size(codecCtx->pix_fmt, codecCtx->width,
  
                      codecCtx->height,1);    
  
      avpkt.data = (uint8_t *)av_malloc(avpkt.size*sizeof(uint8_t));
  
  
  
      /* encode 1 second of video */
  
      for(i=0;i<25;i++) {
  
          fflush(stdout);
  
          /* prepare a dummy image */
  
          /* Y */
  
          for(y=0;yheight;y++) {
  
              for(x=0;xwidth;x++) {
  
                  picture->data[0][y * picture->linesize[0] + x] = x + y + i * 3;
  
              }
  
          }
  
  
  
          /* Cb and Cr */
  
          for(y=0;yheight/2;y++) {
  
              for(x=0;xwidth/2;x++) {
  
                  picture->data[1][y * picture->linesize[1] + x] = 128 + y + i * 2;
  
                  picture->data[2][y * picture->linesize[2] + x] = 64 + x + i * 5;
  
              }
  
          }
  
  
  
          /* encode the image */
  
          //out_size = avcodec_encode_video(codecCtx, outbuf, outbuf_size, picture);
  
          // <access violation="violation"> 
  
          out_size = avcodec_encode_video2(codecCtx, &amp;avpkt, picture, &amp;got_packet); 
  
          printf("encoding frame %3d (size=%5d)\n", i, out_size);
  
          //fwrite(outbuf, 1, out_size, f);
  
          fwrite(avpkt.data, 1, avpkt.size, f);
  
      }
  
  
  
      /* get the delayed frames */
  
      for(; out_size; i++) {
  
          fflush(stdout);
  
  
  
          //out_size = avcodec_encode_video(codecCtx, outbuf, outbuf_size, NULL);
  
          out_size = avcodec_encode_video2(codecCtx, &amp;avpkt, NULL, &amp;got_packet); 
  
          printf("write frame %3d (size=%5d)\n", i, out_size);
  
          //fwrite(outbuf, 1, out_size, f);
  
          fwrite(avpkt.data, 1, avpkt.size, f);
  
      }
  
  
  
      /* add sequence end code to have a real mpeg file */
  
      outbuf[0] = 0x00;
  
      outbuf[1] = 0x00;
  
      outbuf[2] = 0x01;
  
      outbuf[3] = 0xb7;
  
      fwrite(outbuf, 1, 4, f);
  
      fclose(f);
  
      free(picture_buf);
  
      free(outbuf);
  
  
  
      avcodec_close(codecCtx);
  
      av_free(codecCtx);
  
      av_free(picture);
  
      printf("\n");
  
  }
  
  </access>

• Decoding the h.264 stream from a COM port

  18 mars, par Peter

  I would like to know if there is a reliable way to decode an H.264 NAL stream coming through a serial port using software.

  


  So far, I have managed to decode a single frame using a python script. In this script, I first write the incoming data to a file, and when the end-of-frame marker 00_00_00_01 appears, I display the frame using ffplay.

  


  import serial
import subprocess
import os
import time

ser = serial.Serial('COM3', 115200, timeout=1)
output_file = "output.264"

# Variable to store the ffplay process
ffplay_process = None

# Open the file for writing in binary mode
with open(output_file, "wb") as file:

    print("Writing bytes to output.264. Waiting for the end-of-frame marker 0x00000001.")

    buffer = bytearray()
    marker = b'\x00\x00\x00\x01'

    try:
        while True:
            if ser.in_waiting:  # If there is data in the buffer
                data = ser.read(ser.in_waiting)  # Read all available bytes
                buffer.extend(data)

                # Check if the end-of-frame marker is in the buffer
                while marker in buffer:
                    index = buffer.index(marker) + len(marker)  # Position after the marker
                    frame = buffer[:index]  # Extract the frame
                    buffer = buffer[index:]  # Keep the remaining data

                    print(f"Frame recorded: {len(frame)} bytes")
                    file.write(frame)  # Write the frame to the file
                    file.flush()  # Force writing to disk

                    # Close the ffplay window if it is already open
                    if ffplay_process and ffplay_process.poll() is None:
                        ffplay_process.terminate()
                        ffplay_process.wait()  # Wait for the process to terminate

                    # Play the recorded frame, reopening the window
                    ffplay_process = subprocess.Popen(["ffplay", "-f", "h264", "-i", output_file])

    except KeyboardInterrupt:
        print("\nRecording stopped.")
    finally:
        # Close the serial port and the ffplay process
        ser.close()


  


  However, each time a new end-of-frame marker is detected, the ffplay window closes and reopens to show the next frame. It will flicker when transferring the video. Is there a way to display the frames in the same window for seamless playback when streaming video ?

  


  Or is there a better approach or software that is more suited for this task ? I do not know where to start, so I will be glad for any hints.