Recherche avancée

Sur d’autres sites (8283)

• How to encode the input images from camera into H.264 stream ?

  22 avril 2015, par kuu

  I’m trying to encode the input images from MacBook Pro’s built-in FaceTime HD Camera into an H.264 video stream in real time using the libx264 on Mac OS X 10.9.5.

  Below are the steps I took :

  1. Get 1280x720 32BGRA images from camera at 15fps using AVFoundation API (AVCaptureDevice class, etc.)
  2. Convert the images into 320x180 YUV420P format using libswscale.
  3. Encode the images into an H.264 video stream (baseline profile) using libx264.

  I apply the above steps each time the image is obtained from the camera, believing that the encoder keeps track of the encoding state and generates a NAL unit when it’s available.

  As I wanted to get the encoded frames while providing the input images to the encoder, I decided to flush the encoder (calling x264_encoder_delayed_frames()) every 30 frames (2 seconds).

  However, when I restart the encoding, the encoder stops after a while (x264_encoder_encode() never returns.) I tried changing the number of frames before flushing, but the situation didn’t change.

  Below are the related code (I omitted the image capture code because it looks no problem.)

  Can you point out anything I might be doing wrong ?

  x264_t *encoder;
  
  x264_param_t param;
  
  
  
  // Will be called only first time.
  
  int initEncoder() {
  
    int ret;
  
  
  
    if ((ret = x264_param_default_preset(&param, "medium", NULL)) < 0) {
  
      return ret;
  
    }
  
  
  
    param.i_csp = X264_CSP_I420;
  
    param.i_width  = 320;
  
    param.i_height = 180;
  
    param.b_vfr_input = 0;
  
    param.b_repeat_headers = 1;
  
    param.b_annexb = 1;
  
  
  
    if ((ret = x264_param_apply_profile(&param, "baseline")) < 0) {
  
      return ret;
  
    }
  
  
  
    encoder = x264_encoder_open(&param);
  
    if (!encoder) {
  
      return AVERROR_UNKNOWN;
  
    }
  
  
  
    return 0;
  
  }
  
  
  
  // Will be called from encodeFrame() defined below.
  
  int convertImage(const enum AVPixelFormat srcFmt, const int srcW, const int srcH, const uint8_t *srcData, const enum AVPixelFormat dstFmt, const int dstW, const int dstH, x264_image_t *dstData) {
  
    struct SwsContext *sws_ctx;
  
    int ret;
  
    int src_linesize[4];
  
    uint8_t *src_data[4];
  
  
  
    sws_ctx = sws_getContext(srcW, srcH, srcFmt,
  
                         dstW, dstH, dstFmt,
  
                         SWS_BILINEAR, NULL, NULL, NULL);
  
  
  
    if (!sws_ctx) {
  
      return AVERROR_UNKNOWN;
  
    }
  
  
  
    if ((ret = av_image_fill_linesizes(src_linesize, srcFmt, srcW)) < 0) {
  
      sws_freeContext(sws_ctx);
  
      return ret;
  
    }
  
  
  
    if ((ret = av_image_fill_pointers(src_data, srcFmt, srcH, (uint8_t *) srcData, src_linesize)) < 0) {
  
      sws_freeContext(sws_ctx);
  
      return ret;
  
    }
  
  
  
    sws_scale(sws_ctx, (const uint8_t * const*)src_data, src_linesize, 0, srcH, dstData->plane, dstData->i_stride);
  
    sws_freeContext(sws_ctx);
  
    return 0;
  
  }
  
  
  
  // Will be called for each frame.
  
  int encodeFrame(const uint8_t *data, const int width, const int height) {
  
    int ret;
  
    x264_picture_t pic;
  
    x264_picture_t pic_out;
  
    x264_nal_t *nal;
  
    int i_nal;
  
  
  
    if ((ret = x264_picture_alloc(&pic, param.i_csp, param.i_width, param.i_height)) < 0) {
  
      return ret;
  
    }
  
  
  
    if ((ret = convertImage(AV_PIX_FMT_RGB32, width, height, data, AV_PIX_FMT_YUV420P, 320, 180, &pic.img)) < 0) {
  
      x264_picture_clean(&pic);
  
      return ret;
  
    }
  
  
  
    if ((ret = x264_encoder_encode(encoder, &nal, &i_nal, &pic, &pic_out)) < 0) {
  
      x264_picture_clean(&pic);
  
      return ret;
  
    }
  
  
  
    if(ret) {
  
      for (int i = 0; i < i_nal; i++) {
  
        printNAL(nal + i);
  
      }
  
    }
  
  
  
    x264_picture_clean(&pic);
  
    return 0;
  
  }
  
  
  
  // Will be called every 30 frames.
  
  int flushEncoder() {
  
    int ret;
  
    x264_nal_t *nal;
  
    int i_nal;
  
    x264_picture_t pic_out;
  
  
  
    /* Flush delayed frames */
  
    while (x264_encoder_delayed_frames(encoder)) {
  
      if ((ret = x264_encoder_encode(encoder, &nal, &i_nal, NULL, &pic_out)) < 0) {
  
        return ret;
  
      }
  
  
  
      if (ret) {
  
        for (int j = 0; j < i_nal; j++) {
  
          printNAL(nal + j);
  
        }
  
      }
  
    }
  
  }

• Better Image Quality with Python OpenCV

  6 avril 2023, par bozolino

  I use OpenCV to capture webcam video as frames to a list variable, then write that list to disk both as image sequence and as a video file. The image quality is fine. But the video is horribly compressed, with a bitrate of less than 4mbit, sometimes even less than 3mbit.

  


  I've spent days and nights on Google finding solutions, tried many fourccs and other wrappers like WriteGear and ffmpeg-python, but to no avail.

  


  Can anyone please tell me how I

  


  a) specify the bitrate for my H.264 file, and

  


  b) choose a lossless compression codec ?

  


  (Python 3.10 on MacOS MacBook Pro M1 / MacOS 13.0.1)

  


  # ::::: import libraries and set variables

import cv2, time, os, shutil, datetime as dt
frames = []
frame_dimensions_known = False

# ::::: define file and folder names

output_folder_name = "output"
frame_folder_name = "frames"
video_file_name = "video.mp4"
frame_file_name = "frame_x.jpg"

# ::::: create output folders and paths

path_script = os.path.abspath(os.path.dirname(__file__))
path_output = os.path.join(path_script,output_folder_name)
if not os.path.exists(path_output):
    os.mkdir(path_output)
    path_session_fold
path_session_folder  = os.path.join(
    path_script,
    output_folder_name,
    dt.datetime.now().strftime("%y%m%d_%H%M%S"))
os.mkdir(path_session_folder)
path_video =  os.path.join(path_session_folder,video_file_name)
path_frames = os.path.join(path_session_folder,frame_folder_name)
os.mkdir(path_frames)


# ::::: open webcam stream

vcap = cv2.VideoCapture(0)
start_time = (time.time() * 1000)
if vcap.isOpened() is False:
    print("[Exiting]: Error accessing webcam stream.")
    exit(0)

# ::::: read frames

while True:
    _grabbed, _frame = vcap.read()
    if _grabbed is False:
        print('[Exiting] No more frames to read')
        exit(0)

    # ::::: get frame dimensions on first read
    if not frame_dimensions_known:
        height, width, channels = _frame.shape
        frame_dimensions_known = True

    # ::::: append frame to list
    frames.append(_frame)

    # ::::: process frames
    pass

    # ::::: display frame
    cv2.imshow("cam", _frame)
    if cv2.waitKey(1) == ord("q"):
        break

# ::::: close webcam stream

vcap.release()
cv2.destroyAllWindows()

# ::::: calculate frame rate

stop_time = (time.time() * 1000)
total_seconds = (stop_time - start_time) / 1000
fps = len(frames) / total_seconds

# ::::: open video writer

fourcc = cv2.VideoWriter_fourcc(*'mp4v')
video_writer = cv2.VideoWriter(
    path_video,
    fourcc,
    fps,
    (width, height)
)

# ::::: write frames to disk as files and video

frame_number = 0
for frame in frames:

    # ::::: write video

    video_writer.write(frame)  # Write out frame to video

    # ::::: construct image file path

    frame_number = frame_number + 1
    frame_number_file = str(frame_number).zfill(4)
    frame_name = frame_file_name.replace('x',frame_number_file)
    frame_path = os.path.join(path_frames,frame_name)

    # ::::: write image

    cv2.imwrite(
        frame_path,
        frame,
        [cv2.IMWRITE_JPEG_QUALITY, 100]
    )

# ::::: clean up

video_writer.release
print (f"Written {frame_number} frames at {fps} FPS.")



  


• How can I remove the zero padding at the start of a .mp3 file generated by ffmpeg conversion ?

  22 juin 2022, par Miguel de Sousa

  After reading this and this, I understand that the .mp3 encoder appends zeros at the start and at the end of an audio.

  


  With this approach, the encoder can pass a sliding analysis window through the initial and final audio samples, just like in the rest of the audio.

  


  This sliding window is associated with a Fourier Transform-like algorithm, but I will not get into this technical detail.

  


  My problem is : When I generate an .mp3 with ffmpeg like :

  


  MacBook-Air-de-Miguel:faixas miguel$ ffmpeg -i primeira_caf.caf primeira_agora_vai.mp3
ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with Apple clang version 11.0.3 (clang-1103.0.32.62)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.3.1_1 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
[caf @ 0x7fe205009000] Estimating duration from bitrate, this may be inaccurate
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, caf, from 'primeira_caf.caf':
  Duration: 00:00:08.73, start: 0.000000, bitrate: 1414 kb/s
    Stream #0:0: Audio: pcm_s16le (lpcm / 0x6D63706C), 44100 Hz, stereo, s16, 1411 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (pcm_s16le (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
Output #0, mp3, to 'primeira_agora_vai.mp3':
  Metadata:
    TSSE            : Lavf58.45.100
    Stream #0:0: Audio: mp3 (libmp3lame), 44100 Hz, stereo, s16p
    Metadata:
      encoder         : Lavc58.91.100 libmp3lame
size=     137kB time=00:00:08.75 bitrate= 128.6kbits/s speed=  61x    
video:0kB audio:137kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.180156%


  


  my generated .mp3 has +- 50ms of latency. I'm not 100% sure that all this latency is caused by the zero padding that I mentioned.

  


  Converting .caf to other audio formats like .wav and .ogg do not give me this problem.

  


  Considering .mp3 as a constraint, is there any simple way to generate a .mp3 with zero latency ? Maybe a ffmpeg argument that cuts off the zero padding at the start ?

  


  If not, is it possible to do it manually ? How can I know or calculate how many samples should be cut off ?

  


  I have also tried sox. It adds 25ms of latency instead of 50ms.