Recherche avancée

Sur d’autres sites (7246)

• Why is ffmpeg biased when converting to 16-bit integer or 32-bit float RGB output ?

  8 mars 2023, par F.X.

  Consider the following Python invocations, that exercise ffmpeg's ability to convert a 8-bit input into 16-bit integer values or 32-bit float values :

  


  import cv2
import subprocess
import numpy as np

ffmpeg = "ffmpeg -hide_banner -loglevel error -y"
flags = "-sws_flags accurate_rnd+bitexact+full_chroma_int+neighbor -sws_dither none"

for input_color_range in ("tv", "pc"):

    # Generate YUV444 video and encode it losslessly
    subprocess.check_call(rf"{ffmpeg} -y {flags} -color_range {input_color_range} -f lavfi -i yuvtestsrc {flags} -pix_fmt yuv444p -color_range {input_color_range} -x264-params qp=0 -frames 1 -c:v libx264 video_temp.mp4")

    # Extract YUV444 frame, as well as 8-bit int, 16-bit int and 32-bit float frame
    subprocess.check_call(rf"{ffmpeg} -y {flags} -i video_temp.mp4 {flags} -f rawvideo video_temp_444_frame1.yuv")
    subprocess.check_call(rf"{ffmpeg} -y {flags} -i video_temp.mp4 {flags} -color_range pc -pix_fmt rgb24 video_temp_444_frame1_8u.png")
    subprocess.check_call(rf"{ffmpeg} -y {flags} -i video_temp.mp4 {flags} -color_range pc -pix_fmt rgb48be video_temp_444_frame1_16u.png")
    subprocess.check_call(rf"{ffmpeg} -y {flags} -i video_temp.mp4 {flags} -color_range pc -pix_fmt gbrpf32be -f rawvideo video_temp_444_frame1_32f.gbrpf32be")

    # Reead these frames into (height, width, 3) Numpy arrays containing YUV or RGB data
    data_8u = cv2.imread("video_temp_444_frame1_8u.png", cv2.IMREAD_UNCHANGED)[..., ::-1]
    data_16u = cv2.imread("video_temp_444_frame1_16u.png", cv2.IMREAD_UNCHANGED)[..., ::-1]
    data_yuv = np.rollaxis(np.frombuffer(open("video_temp_444_frame1.yuv", "rb").read()).view(np.uint8).reshape((3, 240, 320)), 0, 3).copy()
    data_32f = np.rollaxis(np.frombuffer(open("video_temp_444_frame1_32f.gbrpf32be", "rb").read()).view(np.dtype(">f4")).reshape((3, 240, 320)), 0, 3).copy()
    data_32f[..., (0, 1, 2)] = data_32f[..., (2, 0, 1)]

    # This pixel in yuvtestsrc corresponds to limited-range YUV=(235, 128,
    # 128)=(100%, 0.0, 0.0), which should correspond to 100% white, i.e.
    # RGB=(100%, 100%, 100%).
    if input_color_range == "tv":
        i, j = 10, 294
    # This pixel in yuvtestsrc corresponds to full-range YUV=(255, 128,
    # 128)=(100%, 0.0, 0.0), which should correspond to 100% white, i.e.
    # RGB=(100%, 100%, 100%).
    elif input_color_range == "pc":
        i, j = 10, 319
    else:
        raise Exception(input_color_range)

    # Print pixel values
    print("")
    print(f"Values for {input_color_range}-range input video at ({i}, {j}):")
    print("- 8-bit YUV input   = %s" % data_yuv[i, j, :])
    print("- 8-bit RGB output  = %s (= %s)" % (data_8u[i, j, :], data_8u[i, j, :] / 255))
    print("- 16-bit RGB output = %s (= %s)" % (data_16u[i, j, :], data_16u[i, j, :] / 65535))
    print("- Float RGB output  = %s" % data_32f[i, j, :])


  


  The script first generates a video frame from ffmpeg's YUV test source, encoded with no chroma subsampling (4:4:4) as losslessly as possible.

  


  That video is then used as a reference source to extract the following frames :

  


  

  • The input reference YUV data

  


  • A 8-bit RGB conversion

  


  • A 16-bit RGB conversion

  


  • A 32-bit floating-point RGB conversion

  


  


  One pixel is extracted from each frame, which should contain a 100% white value. The output of the last series of print statements is the following :

  


  Values for tv-range input video at (10, 294):
- 8-bit YUV input   = [235 128 128]
- 8-bit RGB output  = [255 255 255] (= [1. 1. 1.])
- 16-bit RGB output = [65283 65283 65283] (= [0.99615473 0.99615473 0.99615473])
- Float RGB output  = [0.9961547 0.9961547 0.9961547]

Values for pc-range input video at (10, 319):
- 8-bit YUV input   = [255 128 128]
- 8-bit RGB output  = [255 255 255] (= [1. 1. 1.])
- 16-bit RGB output = [65280 65280 65280] (= [0.99610895 0.99610895 0.99610895])
- Float RGB output  = [0.99610895 0.99610895 0.99610895]


  


  While the 8-bit RGB output values are correct, none of the others correctly output a 100% white signal. Why is that ?

  


• VP8 for Real-time Video Applications

  15 février 2011, par noreply@blogger.com (John Luther)

  With the growing interest in videoconferencing on the web platform, it’s a good time to explore the features of VP8 that make it an exceptionally good codec for real-time applications like videoconferencing.

  VP8 Design History & Features

  Real-time applications were a primary use case when VP8 was designed. The VP8 encoder has features specifically engineered to overcome the challenges inherent in compressing and transmitting real-time video data.
  

  • Processor-adaptive encoding. 16 encoder complexity levels automatically (or manually) adjust encoder features such as motion search strategy, quantizer optimizations, and loop filtering strength.
  • Encoder can be configured to use a target percentage of the host CPU.
    Ability to measure the time taken to encode each frame and adjust encoder complexity dynamically to keep the encoding time per frame constant
  • Robust error recovery (packet retransmission, forward error correction, recovery frame/new keyframe requests)
  • Temporal scalability (i.e., a single video bitstream that can degrade as needed depending on a participant’s available bandwidth)
  • Highly efficient decoding performance on low-power devices. Conventional video technology has grown to a state of complexity where dedicated hardware chips are needed to make it work well. With VP8, software-based solutions have proven to meet customer needs without requiring specialized hardware.

  For a more information about real-time video features in VP8, see the slide presentation by WebM Project engineer Paul Wilkins (PDF file).

  Commercially Available Products

  Millions of people around the world have been using VP7/8 for video chat for years. VP8 is deployed in some of today’s most popular consumer videoconferencing applications, including Skype (group video calling), Sightspeed, ooVoo and Logitech Vid. All of these vendors are active WebM project supporters. VP8’s predecessor, VP7, has been used in Skype video calling since 2005 and is supported in the new Skype app for iPhone. Other real-time VP8 implementations are coming soon, including ooVoo, and VP8 will play a leading role in Google’s plans for real-time applications on the web platform.

  Real-time applications will be extremely important as the web platform matures. The WebM community has made significant improvements in VP8 for real-time use cases since our launch and will continue to do so in the future.

  John Luther is Product Manager of the WebM Project.

  

• Revised FATE Test Spec System

  9 juin 2010, par Multimedia Mike — FATE Server

  FATE involves some database tables that define the test specifications. Like everything else in FATE, the concept could use some improvement. After I prototyped an improved, multithreaded testing client, the next logical revision seemed to be the test spec system.

  History
  The test spec system has been handled by a single table that includes an FFmpeg command line (with a few possible modifiers thrown in), an integer ID, a human-friendly ID, a description, the expected command line return code, the expected command output, a maximum runtime, and a Boolean to indicate whether the test is to be considered active.

  Adjunct to this test database is a large corpus of test media named the FATE suite.

  At first, the FATE testing script used a direct MySQL database protocol to query the test specs from the server before every build/test cycle. I soon realized this was ludicrously inefficient since the test specs don’t change that often. So I cached the tests in a static file to be retrieved via HTTP, first in Python’s "pickled" (serialized) format, then in an SQLite database.

  Planned Upgrades
  There are 2 major features I would like to build into the system going forward :

  1. The ability to version the entire suite so that it’s possible to test old branches of FFmpeg
  2. Another database field to indicate which, if any, other test specs must be executed before this spec can be executed

  I think I will take this opportunity to switch the test cache serialization format to JSON. I switched from Python pickling to SQLite because the latter was more portable between languages. JSON has that same benefit. Further, working with JSON data doesn’t require a round trip to disk (i.e., want to generate an SQLite database for sending via HTTP ? It needs to go onto disk first. It’s possible to create and manipulate a database entirely in memory but not fetch the bits).

  Things To Research

  • Pondering how version control systems operate and what they have to teach regarding how to version this data (including the question of whether I can just use an existing version control mechanism instead of creating my own system)
  • Efficient caching mechanism
  • Tagging test specs for alternate purposes such as longevity testing
  • Learn about web form programming in the 21st century so that it’s not quite as painful to maintain the system.

  Preliminary Versioning Concept
  Here is one approach I am thinking of : Create test groups. Each test spec is assigned to at least one test group. I can think of at least 2 groups : functional (the base test set in existence that validates functionality) and profiling (the projected test set that will be used for ongoing performance and memory profiling). The web frontend will allow for the creation of labels that will apply to a single group. Doing so will apply that label to all active tests in the group.