Recherche avancée

Sur d’autres sites (5473)

• FFMPEG ignores encoding level parameter

  2 juillet 2023, par jakebird451

  I need to be able to encode video to a specific profile-level-id in accordance to a negotiated session via WebRTC. I can readily change the profile via code, however, I have not been able to change the level. I am using the hardware encoder on the RaspberryPi 4, which uses a BCM2835 via Video4Linux2 drivers. By default, the driver is leaving the encoding level set to level 4, which I can verify by taking a sample I am generating with the encoder and running the command mediainfo sample.h264 | grep profile over the sample. Everything I have tried has been setting the level back to the default level 4 encoding.

  


  The code below is ffmpeg-next, an ffmpeg wrapper in rust. All the commands can be traced back to C api calls for ffmpeg.

  


  pub fn create_encoder() -> ffmpeg_next::codec::encoder::video::Video {&#xA;    let codec = ffmpeg_next::encoder::find_by_name("h264_v4l2m2m").unwrap();&#xA;    let ctx = new_context_from(codec).encoder();&#xA;    let mut ctx = ctx.video().unwrap();&#xA;    ctx.set_width(1280);&#xA;    ctx.set_height(720);&#xA;    ctx.set_format(ffmpeg_next::format::pixel::Pixel::YUV420P);&#xA;    let fps = ffmpeg_next::Rational::new(30, 1);&#xA;    let tb = fps.invert();&#xA;    ctx.set_frame_rate(Some(fps));&#xA;    ctx.set_time_base(tb);&#xA;    ctx.set_bit_rate(4_000_000);&#xA;    ctx.set_max_bit_rate(4_000_000);&#xA;    ctx.set_aspect_ratio(ffmpeg_next::util::rational::Rational(16, 9));&#xA;    ctx.set_flags(ffmpeg_next::codec::Flags::GLOBAL_HEADER);&#xA;    ctx.set_gop(0);&#xA;    ctx.set_max_b_frames(0);&#xA;&#xA;    // ----- Below here is my attempt to set profile and level -----&#xA;    // Below modifies `compression_level`, but tried it as well just in case&#xA;    ctx.set_compression(Some(9)); // Does not work (tried 9 &amp; 31)&#xA;    unsafe {&#xA;        let mut p = ctx.as_mut_ptr();&#xA;        (*p).profile = FF_PROFILE_H264_HIGH; // &lt;-- Changing this will change the profile (works)&#xA;        (*p).level = 9; // Does not work (tried 9 &amp; 31)&#xA;    }&#xA;&#xA;    // Does not work (Errors with OptionNotFound)&#xA;    if let Err(e) = video_opt_set(&amp;mut ctx, "level", "9", 0) {&#xA;        log::warn!("Could not set the level: {:?}", e);&#xA;    }&#xA;&#xA;    // Does not work (Errors with OptionNotFound)&#xA;    if let Err(e) = video_opt_set(&amp;mut ctx, "h264_level", "9", 0) {&#xA;        log::warn!("Could not set the h264_level: {:?}", e);&#xA;    }&#xA;&#xA;    // This is an attempt to make an AVDictionary for avcodec_open2&#xA;    // (tried 9, 31, &amp; 3.1)&#xA;    let settings = make_avdict("level=9,h264_level=9").unwrap();&#xA;    ctx.open_with(settings).unwrap().0&#xA;}&#xA;&#xA;/// Modified from:&#xA;/// https://github.com/zmwangx/rust-ffmpeg/blob/master/examples/transcode-x264.rs#L154&#xA;pub fn make_avdict&lt;&#x27;a, S: AsRef<str>>(s: S) -> Option> {&#xA;    let s = s.as_ref();&#xA;    let mut dict = Dictionary::new();&#xA;    for keyval in s.split_terminator(&#x27;,&#x27;) {&#xA;        let tokens: Vec&lt;&amp;str> = keyval.split(&#x27;=&#x27;).collect();&#xA;        match tokens[..] {&#xA;            [key, val] => dict.set(key, val),&#xA;            _ => return None,&#xA;        }&#xA;    }&#xA;    Some(dict)&#xA;}&#xA;&#xA;/// Helper function to wrap `av_opt_set` and properly return an exception on failures&#xA;pub fn video_opt_set(video: &amp;mut Video, key: &amp;str, value: &amp;str, search: i32) -> Result&lt;(), OptError> {&#xA;    let err = unsafe {&#xA;        av_opt_set(&#xA;            (*video.as_ptr()).priv_data,&#xA;            key.as_ptr(),&#xA;            value.as_ptr(),&#xA;            search,&#xA;        )&#xA;    };&#xA;    match err {&#xA;        0 => Ok(()),&#xA;        AVERROR_OPTION_NOT_FOUND => Err(OptError::OptionNotFound),&#xA;        -34 => Err(OptError::RangeError),&#xA;        -22 => Err(OptError::Invalid),&#xA;        _ => panic!("Not a valid error code")&#xA;    }&#xA;}&#xA;</str>

  


  The V4L2 hardware encoder is device 11 (/dev/video11). I can use v4l-ctl to interrogate the options it has available :

  


  $ v4l2-ctl -d 11 -l

Codec Controls

                 video_b_frames 0x009909ca (int)    : min=0 max=0 step=1 default=0 value=0 flags=update
                 video_gop_size 0x009909cb (int)    : min=0 max=2147483647 step=1 default=60 value=60
             video_bitrate_mode 0x009909ce (menu)   : min=0 max=1 default=0 value=0 flags=update
                  video_bitrate 0x009909cf (int)    : min=25000 max=25000000 step=25000 default=10000000 value=10000000
           sequence_header_mode 0x009909d8 (menu)   : min=0 max=1 default=1 value=1
         repeat_sequence_header 0x009909e2 (bool)   : default=0 value=0
                force_key_frame 0x009909e5 (button) : flags=write-only, execute-on-write
          h264_minimum_qp_value 0x00990a61 (int)    : min=0 max=51 step=1 default=20 value=20
          h264_maximum_qp_value 0x00990a62 (int)    : min=0 max=51 step=1 default=51 value=51
            h264_i_frame_period 0x00990a66 (int)    : min=0 max=2147483647 step=1 default=60 value=60
                     h264_level 0x00990a67 (menu)   : min=0 max=15 default=11 value=11
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=4 value=4


  


  And specifically looking at profile and level I have the following options provided by V4L2 :

  


                       h264_level 0x00990a67 (menu)   : min=0 max=15 default=11 value=11
                                0: 1
                                1: 1b
                                2: 1.1
                                3: 1.2
                                4: 1.3
                                5: 2
                                6: 2.1
                                7: 2.2
                                8: 3
                                9: 3.1
                                10: 3.2
                                11: 4
                                12: 4.1
                                13: 4.2
                                14: 5
                                15: 5.1
                   h264_profile 0x00990a6b (menu)   : min=0 max=4 default=4 value=4
                                0: Baseline
                                1: Constrained Baseline
                                2: Main
                                4: High


  


  I have even tried overwriting the default for V4L with the command v4l2-ctl -d 11 --set-ctrl=h264_level=9. But this does nothing. It doesn't error, which would be the case where I would enter a setting that doesn't exist such as h264_levelx. Also, if I try to change it to something like 999, the command errors out with "Numerical result out of range". So it understands the parameter I am trying to change and has a grasp on what values it can modify to. But instead of seeing any changes, the command simply executes without error and applies no lasting changes.

  


• My journey to Coviu

  27 octobre 2015, par silvia

  My new startup just released our MVP – this is the story of what got me here.

  I love creating new applications that let people do their work better or in a manner that wasn’t possible before.

  My first such passion was as a student intern when I built a system for a building and loan association’s monthly customer magazine. The group I worked with was managing their advertiser contacts through a set of paper cards and I wrote a dBase based system (yes, that long ago) that would manage their customer relationships. They loved it – until it got replaced by an SAP system that cost 100 times what I cost them, had really poor UX, and only gave them half the functionality. It was a corporate system with ongoing support, which made all the difference to them.

  The story repeated itself with a CRM for my Uncle’s construction company, and with a resume and quotation management system for Accenture right after Uni, both of which I left behind when I decided to go into research.

  Even as a PhD student, I never lost sight of challenges that people were facing and wanted to develop technology to overcome problems. The aim of my PhD thesis was to prepare for the oncoming onslaught of audio and video on the Internet (yes, this was 1994 !) by developing algorithms to automatically extract and locate information in such files, which would enable users to structure, index and search such content.

  Many of the use cases that we explored are now part of products or continue to be challenges : finding music that matches your preferences, identifying music or video pieces e.g. to count ads on the radio or to mark copyright infringement, or the automated creation of video summaries such as trailers.

  

  This continued when I joined the CSIRO in Australia – I was working on segmenting speech into words or talk spurts since that would simplify captioning & subtitling, and on MPEG-7 which was a (slightly over-engineered) standard to structure metadata about audio and video.

  In 2001 I had the idea of replicating the Web for videos : i.e. creating hyperlinked and searchable video-only experiences. We called it “Annodex” for annotated and indexed video and it needed full-screen hyperlinked video in browsers – man were we ahead of our time ! It was my first step into standards, got several IETF RFCs to my name, and started my involvement with open codecs through Xiph.

  Around the time that YouTube was founded in 2006, I founded Vquence – originally a video search company for the Web, but pivoted to a video metadata mining company. Vquence still exists and continues to sell its data to channel partners, but it lacks the user impact that has always driven my work.

  As the video element started being developed for HTML5, I had to get involved. I contributed many use cases to the W3C, became a co-editor of the HTML5 spec and focused on video captioning with WebVTT while contracting to Mozilla and later to Google. We made huge progress and today the technology exists to publish video on the Web with captions, making the Web more inclusive for everybody. I contributed code to YouTube and Google Chrome, but was keen to make a bigger impact again.

  The opportunity came when a couple of former CSIRO colleagues who now worked for NICTA approached me to get me interested in addressing new use cases for video conferencing in the context of WebRTC. We worked on a kiosk-style solution to service delivery for large service organisations, particularly targeting government. The emerging WebRTC standard posed many technical challenges that we addressed by building rtc.io , by contributing to the standards, and registering bugs on the browsers.

  Fast-forward through the development of a few further custom solutions for customers in health and education and we are starting to see patterns of need emerge. The core learning that we’ve come away with is that to get things done, you have to go beyond “talking heads” in a video call. It’s not just about seeing the other person, but much more about having a shared view of the things that need to be worked on and a shared way of interacting with them. Also, we learnt that the things that are being worked on are quite varied and may include multiple input cameras, digital documents, Web pages, applications, device data, controls, forms.

  So we set out to build a solution that would enable productive remote collaboration to take place. It would need to provide an excellent user experience, it would need to be simple to work with, provide for the standard use cases out of the box, yet be architected to be extensible for specialised data sharing needs that we knew some of our customers had. It would need to be usable directly on Coviu.com, but also able to integrate with specialised applications that some of our customers were already using, such as the applications that they spend most of their time in (CRMs, practice management systems, learning management systems, team chat systems). It would need to require our customers to sign up, yet their clients to join a call without sign-up.

  Collaboration is a big problem. People are continuing to get more comfortable with technology and are less and less inclined to travel distances just to get a service done. In a country as large as Australia, where 12% of the population lives in rural and remote areas, people may not even be able to travel distances, particularly to receive or provide recurring or specialised services, or to achieve work/life balance. To make the world a global village, we need to be able to work together better remotely.

  The need for collaboration is being recognised by specialised Web applications already, such as the LiveShare feature of Invision for Designers, Codassium for pair programming, or the recently announced Dropbox Paper. Few go all the way to video – WebRTC is still regarded as a complicated feature to support.

  

  With Coviu, we’d like to offer a collaboration feature to every Web app. We now have a Web app that provides a modern and beautifully designed collaboration interface. To enable other Web apps to integrate it, we are now developing an API. Integration may entail customisation of the data sharing part of Coviu – something Coviu has been designed for. How to replicate the data and keep it consistent when people collaborate remotely – that is where Coviu makes a difference.

  We have started our journey and have just launched free signup to the Coviu base product, which allows individuals to own their own “room” (i.e. a fixed URL) in which to collaborate with others. A huge shout out goes to everyone in the Coviu team – a pretty amazing group of people – who have turned the app from an idea to reality. You are all awesome !

  With Coviu you can share and annotate :

  • images (show your mum photos of your last holidays, or get feedback on an architecture diagram from a customer),
  • pdf files (give a presentation remotely, or walk a customer through a contract),
  • whiteboards (brainstorm with a colleague), and
  • share an application window (watch a YouTube video together, or work through your task list with your colleagues).

  All of these are regarded as “shared documents” in Coviu and thus have zooming and annotations features and are listed in a document tray for ease of navigation.

  This is just the beginning of how we want to make working together online more productive. Give it a go and let us know what you think.

  http://coviu.com/

• Xbox Sphinx Protocol

  21 octobre 2013, par Multimedia Mike — DRM, xbox

  I’ve gone down the rabbit hole of trying to read the Xbox DVD drive from Linux. Honestly, I’m trying to remember why I even care at this point. Perhaps it’s just my metagame of trying to understand how games and related technologies operate. In my last post of the matter, I determined that it is possible to hook an Xbox drive up to a PC using a standard 40-pin IDE interface and read data sectors. However, I learned that just because the Xbox optical drive is reading an Xbox disc, that doesn’t mean it’s just going to read the sectors in response to a host request.

  Oh goodness, no. The drive is going to make the host work for those sectors.

  To help understand the concept of locked/unlocked sectors on an Xbox disc, I offer this simplistic diagram :

  
  
  

  Any DVD drive (including the Xbox drive) is free to read those first 6992 sectors (about 14 MB of data) which just contain a short DVD video asking the user to insert the disc into a proper Xbox console. Reading the remaining sectors involves performing a sequence of SCSI commands that I have taken to calling the “Sphinx Protocol” for reasons I will explain later in this post.

  References
  Doing a little Googling after my last post on the matter produced this site hosting deep, technical Xbox information. It even has a page about exactly what I am trying to achieve : Use an Xbox DVD Drive in Your PC. The page provides a tool named dvdunlocker written by “The Specialist” to perform the necessary unlocking. The archive includes a compiled Windows binary as well as its source code. The source code is written in Delphi Pascal and leverages Windows SCSI APIs. Still, it is well commented and provides a roadmap, which I will try to describe in this post.

  Sphinx Protocol
  Here is a rough flowchart of the steps that are (probably) involved in the unlocking of those remaining sectors. I reverse engineered this based on the Pascal tool described in the previous section. Disclaimer : at the time of this writing, I haven’t tested all of the steps due to some Linux kernel problems, described later.

  
  
  

  Concerning the challenge/response table that the drive sends back, it’s large (0×664 / 1636 bytes), and not all of the bytes’ meanings are known. However, these are the bytes that seem to be necessary (all multi-byte numbers are big endian) :

   bytes 0-1        Size of mode page payload data (should be 0x0662)
   bytes 2-771      Unknown
   byte  772        Should be 1
   byte  773        Number of entries in challenge/response table
   bytes 774-1026   Encrypted challenge/response table
   bytes 1027-1186  Unknown
   bytes 1187-1230  Key basis (44 bytes)
   bytes 1231-1635  Unknown
  

  The challenge/response table is the interesting part, but it’s encrypted with RC4 a.k.a. ARCFOUR. The key is derived from the 44 bytes I have labeled “key basis”– cryptographic literature probably has a better term for it ; chime in if you know what that might be. An SHA-1 hash is computed over the 44 bytes.

  The resulting SHA-1 hash — the first part of it, to be exact — is fed as the key into the RC4 decryption. The output of SHA-1 contains 160 bits of information. 160 / 8 = 20 bytes of information. To express this as a printable hex digest requires 40 characters. The SHA-1 hash is converted to a hex digest and then the first 7 of the characters are fed into the RC4 initialization function as the key. Then, the RC4 decrypter does its work on the 253 bytes of the challenge/response table.

  So that’s why I took to calling this the “Sphinx Protocol” — I felt like I was being challenged with a bizarre riddle. Perhaps that describes a lot of cryptosystems, though You have to admit it sounds kind of cool.

  The challenge/response table contains 23 11-byte records. The format of this table is (again, multi-byte numbers are big-endian) :

   byte  0     This is 1 if this challenge/response pair is valid
   byte  1     Challenge ID
   bytes 2-5   Challenge
   byte  6     Response ID
   bytes 7-10  Response
  

  Example
  It’s useful to note that the challenge/response table and associated key is different for every disc (at least all the ones I have looked at). So this might be data that comes from the disc, since the values will always be the same for a given disc.

  Let’s examine Official Xbox Magazine disc #16 (Indiana Jones and The Emperor’s Tomb) :

  
  
  

  Before I decrypt the challenge/response table, it looks like this :

     0 : 180, 172 : 0xEB100059 ;  66 : 0xD56AFB56
     1 :  34,  71 : 0x8F9BF03A ; 192 : 0xC32CBDF8
     2 : 226, 216 : 0xA29B77F2 ;  12 : 0x4474A6F1
     3 :  72, 122 : 0x9F5ABF33 ; 255 : 0xC5E3C304
     4 :   1, 103 : 0x76142ADA ; 233 : 0xDE145D42 ****
     5 :  49, 193 : 0xA1CD6192 ; 189 : 0x2169DBA5
     6 : 182, 250 : 0x9977894F ;  96 : 0x5A929E2B
     7 : 148,  71 : 0x6DD10A54 ; 115 : 0xF0BDAC4F
     8 :  12,  45 : 0x5D5EB6FD ; 148 : 0x84E60A00
     9 :  99, 121 : 0xFEAED372 ; 201 : 0xDA9986F9
    10 : 172, 230 : 0xE6C0D0B4 ; 214 : 0x9050C250
    11 :  84,  65 : 0x95CB8775 ; 104 : 0x550886C6
    12 : 210,  65 : 0x1ED23619 ; 171 : 0x6DF4A35B
    13 :   2, 155 : 0xD0AAE1E0 ; 130 : 0x00D1FFCF
    14 :  40,   2 : 0x172EFEB8 ; 159 : 0x37E03E50
    15 :  49,  15 : 0x43E5E378 ; 223 : 0x267F9C9A
    16 : 240, 173 : 0x357D5D1C ; 250 : 0x24965D67
    17 :  80, 184 : 0x5E7AF1A3 ;  81 : 0x3A8F69A7
    18 : 154, 186 : 0x6626BEAC ; 245 : 0xE639540A
    19 : 231, 249 : 0xFABAAFB7 ; 227 : 0x4C686A07
    20 : 150, 186 : 0x9A6D7AA3 ; 133 : 0x25971CF0
    21 : 236, 192 : 0x5CD97DD4 ; 247 : 0x26655EFB
    22 :  68, 173 : 0xE2D372E4 ; 207 : 0x103FBF94
  there are 1 valid pairs in the list : 4
  

  My best clue that it’s not right is that there is only 1 valid entry (denoted by my tool using ****). The source I reverse engineered for this data indicates that there needs to be at least 2 valid pairs. After running the RC4 decryption on the table, it looks like this and I get far more valid pairs :

     0 :   1, 174 : 0xBD628255 ;   0 : 0x9F0A31AF ****
     1 :   2, 176 : 0x3151B341 ;   2 : 0x9C87C180
     2 :   3, 105 : 0x018879E5 ;   1 : 0xFF068B5C
     3 :   2,   7 : 0x1F316AAF ;   3 : 0xF420D3ED
     4 :   3,  73 : 0xC2EBFBE9 ;   0 : 0x17062B5B
     5 : 252, 163 : 0xFF14B5CB ; 236 : 0xAF813FBC
     6 :   2, 233 : 0x5EE95C49 ;   1 : 0x37AA5511
     7 :   1, 126 : 0xBD628255 ;   0 : 0x5BA3FBD4 ****
     8 :   3,   4 : 0xB68BFEE6 ;   3 : 0xA8F3B918
     9 :   3,  32 : 0xEA614943 ;   2 : 0xA678D715
    10 :   2, 248 : 0x1BDD374E ;   0 : 0x8D2AC2C7
    11 :   3,  17 : 0x0EABCE81 ;   2 : 0xC90A7242
    12 :   1, 186 : 0xBD628255 ;   0 : 0xC4820242 ****
    13 :   3, 145 : 0xB178F942 ;   3 : 0x4D78AD62
    14 :   3,  37 : 0x4A6CE5E2 ;   2 : 0xBF94E1C6
    15 :   1, 102 : 0xBD628255 ;   0 : 0xFFB83D8D ****
    16 :   3, 122 : 0xF97B0905 ;   1 : 0x38533125
    17 :   3, 197 : 0x57A6865D ;   2 : 0xA61D31EF
    18 :   3,  27 : 0xC7227D7C ;   2 : 0xA3F9BA1E
    19 :   1,  16 : 0xBD628255 ;   0 : 0x8557CCC8 ****
    20 :   2,  53 : 0x1DA9D156 ;   3 : 0xC9051754
    21 :   2,  90 : 0x3CD66BEE ;   3 : 0xFD851D3E
    22 :   1, 252 : 0xBD628255 ;   0 : 0xB3F22701 ****
  there are 6 valid pairs in the list : 0 7 12 15 19 22
  

  So, hopefully, I have the decryption correct.

  Also of note is that you only get one chance to get this unlocking correct– fail, and the drive won’t return a valid DVD structure block again. You will either need to reboot the Xbox or eject & close the tray before you get to try again.

  Problems Making It Work In Linux
  There are a couple of ways to play with SCSI protocols under Linux. In more recent kernels, block devices are named /dev/sda, /dev/sdb, etc. Each of these block devices has a corresponding character device named /dev/sg0, /dev/sg1, etc. ‘sg’ stands for SCSI generic. This character devices can be opened as readable and/or writable and SCSI commands can be freely written with write() and data retrieved with read(). Pretty powerful.

  Except that the one machine I still possess which supports 40-pin IDE/ATAPI devices is running Linux kernel 2.6.24 which dates back to early 2008 and it still enumerates the IDE block devices as /dev/hda, /dev/hdb, etc. There are no corresponding /dev/sgX character devices. What to do ? It seems that a program can still issue SCSI commands using an ioctl() facility named SG_IO.

  I was able to make the SG_IO ioctl() work for the most part (except for the discovery that the Xbox drive doesn’t respond to a basic SCSI Inquiry command). However, I ran into a serious limitation– a program can only open a /dev/hdX block device in read-only mode if the device corresponds to a read-only drive like, for example, a DVD-ROM drive. This means that a program can’t issue SCSI mode select commands to the drive, which counts as writing. This means that my tool can’t unlock the drive.

  Current Status
  So this is where my experiment is blocked right now. I have been trying to compile various Linux kernels to remedy the situation. But I always seem to find myself stuck in one of 2 situations, depending on the configuration options I choose : Either the drives are enumerated with the /dev/hdX convention and I am stuck in read-only mode (with no mode select) ; or the drives are enumerated with /dev/sdX along with corresponding /dev/sgN character devices, in which case the kernel does not recognize the Xbox DVD-ROM drive.

  This makes me wonder if there’s a discrepancy between the legacy ATA/ATAPI drivers (which sees the drive) and the newer SATA/PATA subsystem (which doesn’t see the drive). I also wonder about hacking the kernel logic to allow SCSI mode select logic to proceed to the device for a read-only file handle.