Recherche avancée

Sur d’autres sites (4568)

• libswresample : Why does swr_init() change |in_ch_layout| order so it no longer matches my decoded AVFrames, causing resampling to fail ?

  20 novembre 2023, par CheekyChips

  I am trying to write some code that resamples an audio file to 16kHz and 1 channel and then encodes it to PCM, but I am having an issue with channel layouts.

  


  In a nutshell :

  


  My AVCodecContext and the frames I get from the stream via avcodec_receive_frame() have a channel layout order of AV_CHANNEL_ORDER_UNSPEC. But when I call swr_init() it changes the in_ch_layout order to AV_CHANNEL_ORDER_NATIVE. Then when I call swr_convert_frame() with my AVFrames, because the channel layout orders don't match, the resampling fails because it thinks the input changed.

  


  More details :

  


  I create an AVCodecContext from my audio stream's codec, and it has a channel layout of AV_CHANNEL_ORDER_UNSPEC with 2 channels, and any frames I decode from the stream via avcodec_receive_frame() also have a channel layout order of AV_CHANNEL_ORDER_UNSPEC.

  


  I set SwrContext's |in_ch_layout| to the sample channel layout from the codec context :

  


      AVChannelLayout in_ch_layout = in_codec_context->ch_layout,
    ...
    int ret = swr_alloc_set_opts2(&swr_ctx, ...
                      &in_ch_layout,
                      ...);


  


  But SwrContext->init() changes its internal in_ch_layout from AV_CHANNEL_ORDER_UNSPEC to AV_CHANNEL_ORDER_NATIVE meaning it fails the next time I call swr_convert_frame() because the input frame has a different channel layout to the SwrContext. When swr_init() is called (in my case indirectly by swr_convert_frame(), but also if I alternatively call it directly) the SwrContext->used_ch_layout and SwrContext->in_ch_layout are updated to have channel layout order of AV_CHANNEL_ORDER_NATIVE :

  


      // swresample.c
    av_cold int swr_init(struct SwrContext *s){
        ...
        if (!av_channel_layout_check(&s->used_ch_layout))       <-- This hits if I don't set anything for used_ch_layout
            av_channel_layout_default(&s->used_ch_layout, s->in.ch_count);      <-- default is AV_CHANNEL_ORDER_NATIVE
        ...
        if (s->used_ch_layout.order == AV_CHANNEL_ORDER_UNSPEC) <-- This hits if I do set used_ch_layout
            av_channel_layout_default(&s->used_ch_layout, s->used_ch_layout.nb_channels);   <-- default is AV_CHANNEL_ORDER_NATIVE


  


  Then when I next call swr_convert_frame(), because the frame has the same layout as the audio stream's codec (AV_CHANNEL_ORDER_UNSPEC), and this is different to SwrContext->in_ch_layout (AV_CHANNEL_ORDER_NATIVE), it early exits with ret |= AVERROR_INPUT_CHANGED.

  


  // swresample_frame.c
    int swr_convert_frame(SwrContext *s,
                      AVFrame *out, const AVFrame *in)
    {
        ...
        if ((ret = config_changed(s, out, in)))
            return ret;
        ...


  


      static int config_changed(SwrContext *s,
                            const AVFrame *out, const AVFrame *in)
    {
        ...
        if ((err = av_channel_layout_copy(&ch_layout, &in->ch_layout)) < 0)
            ...
        if (av_channel_layout_compare(&s->in_ch_layout, &ch_layout) || ...) {   <-- This hits the next time I call swr_convert_frame()
            ret |= AVERROR_INPUT_CHANGED;
        }


  


      // channel_layout.c
    int av_channel_layout_compare(const AVChannelLayout *chl, const AVChannelLayout *chl1)
    {
        ...
        // if only one is unspecified -> not equal
        if ((chl->order  == AV_CHANNEL_ORDER_UNSPEC) !=
            (chl1->order == AV_CHANNEL_ORDER_UNSPEC))
            return 1;


  


  If I hardcode the channel layout order of each input AVFrame to AV_CHANNEL_ORDER_NATIVE before resampling, then the resampling and subsequent encoding works, but this feels like a really bad idea and of course wouldn't work as soon as I resample an audio file with a different channel layout.

  


      avcodec_receive_frame(in_codec_context, input_frame);

    AVChannelLayout input_frame_ch_layout;
    av_channel_layout_default(&input_frame_ch_layout, 2 /* = nb_channels*/);
    input_frame->ch_layout = input_frame_ch_layout;
    // Bad idea - but "fixes" my issue!


  


  My questions

  


  What do I need to do to the resampler OR/AND the decoded audio frame to make sure they have the same channel layout order and the resampling works ?

  


  How can I make the channel order of the AVFrames that I get from avcodec_receive_frame() match the input channel order of SwrContext so the resampling works ? My understanding is that the decoded frames should be 'correct' already and I shouldn't need to change any of their values, only values of the output (resampled) frames that I create.

  


  Is there something I need to set on the AVFrame before I resample it ?

  


  Why does the SwrContext choose to change the channel order to AV_CHANNEL_ORDER_NATIVE ?

  


  Note :
A workaround could be to use swr_convert() with the raw data buffer instead of swr_convert_frame(), since it looks like it bypasses this check (since there are no frames involved). I haven't tried this but this shouldn't be necessary and I would like to use swr_convert_frame() as I am working with input and output frames.

  


  Unfortunately I can't find example code using swr_convert_frame() (not even the ffmpeg code seems to ever call it).

  


  My full c++ source code
(error handling omitted for readability) :

  


  std::string fileToUse = "/home/projects/audioFileProject/Audio files/14 Black Cadillacs.wma";
const std::string outputFilename = "out.wav";
const std::string PCMS16BE_encoder_name = "pcm_f32le";

int main()
{
    // Open audio file
    AVFormatContext* in_format_context = avformat_alloc_context();
    avformat_open_input(&in_format_context, fileToUse.c_str(), NULL, NULL);
    avformat_find_stream_info(in_format_context, NULL);
    
    // Get audio stream from file and corresponding decoder
    AVStream* in_stream = in_format_context->streams[0];
    AVCodecParameters* codec_params = in_stream->codecpar;
    const AVCodec* in_codec = avcodec_find_decoder(codec_params->codec_id);
    AVCodecContext *in_codec_context = avcodec_alloc_context3(in_codec);
    avcodec_parameters_to_context(in_codec_context, codec_params);
    avcodec_open2(in_codec_context, in_codec, NULL);

    // Prepare output stream and output encoder (PCM)
    AVFormatContext* out_format_context = nullptr;
    avformat_alloc_output_context2(&out_format_context, NULL, NULL, outputFilename.c_str());
    AVStream* out_stream = avformat_new_stream(out_format_context, NULL);
    const AVCodec* output_codec = avcodec_find_encoder_by_name(PCMS16BE_encoder_name.c_str());
    AVCodecContext* output_codec_context = avcodec_alloc_context3(output_codec);

    // -------------------------------
    
    AVChannelLayout output_ch_layout;
    av_channel_layout_default(&output_ch_layout, 1);    // AV_CHANNEL_LAYOUT_MONO
    output_codec_context->ch_layout = output_ch_layout;
    
    auto out_sample_rate = 16000;
    output_codec_context->sample_rate = out_sample_rate;
    output_codec_context->sample_fmt = output_codec->sample_fmts[0];
    //output_codec_context->bit_rate = output_codec_context->bit_rate;  // TODO Do we need to set the bit rate?
    output_codec_context->time_base = (AVRational){1, out_sample_rate};
    out_stream->time_base = output_codec_context->time_base;

    auto in_sample_rate = in_codec_context->sample_rate;
    AVChannelLayout in_ch_layout = in_codec_context->ch_layout,
                    out_ch_layout = output_ch_layout;   // AV_CHANNEL_LAYOUT_MONO;
    enum AVSampleFormat in_sample_fmt = in_codec_context->sample_fmt,
                        out_sample_fmt = in_codec_context->sample_fmt;

    SwrContext *swr_ctx = nullptr;
    int ret = swr_alloc_set_opts2(&swr_ctx,
                      &out_ch_layout,
                      out_sample_fmt,
                      out_sample_rate,
                      &in_ch_layout,
                      in_sample_fmt,
                      in_sample_rate,
                      0,                    // log_offset
                      NULL);                // log_ctx

    // Probably not necessary - documentation says "This option is
only used for special remapping."
    av_opt_set_chlayout(swr_ctx,    "used_chlayout",     &in_ch_layout, 0);

    // Open output file for writing
    avcodec_open2(output_codec_context, output_codec, NULL);
    avcodec_parameters_from_context(out_stream->codecpar, output_codec_context);
    
    if (out_format_context->oformat->flags & AVFMT_GLOBALHEADER)
        out_format_context->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

    avio_open(&out_format_context->pb, outputFilename.c_str(), AVIO_FLAG_WRITE);
    AVDictionary* muxer_opts = nullptr;
    avformat_write_header(out_format_context, &muxer_opts);

    AVFrame* input_frame = av_frame_alloc();
    AVPacket* in_packet = av_packet_alloc();

    // Loop through decoded input frames. Resample and get resulting samples in a new output frame.
    // I think PCM supports variable number of samples in frames so probably can immediately write out
    while (av_read_frame(in_format_context, in_packet) >= 0) {
        avcodec_send_packet(in_codec_context, in_packet);
        avcodec_receive_frame(in_codec_context, input_frame);

        // I don't want to do this, but it 'fixes' the error where channel layout of input frames
        // doesn't match what the resampler expects - hardcoded the number 2 to fit my sample audio file.
        AVChannelLayout input_frame_ch_layout;
        av_channel_layout_default(&input_frame_ch_layout, 2 /* = nb_channels*/);
        input_frame->ch_layout = input_frame_ch_layout;

        AVFrame* output_frame = av_frame_alloc();
        output_frame->sample_rate = out_sample_rate;
        output_frame->format = out_sample_fmt;
        output_frame->ch_layout = out_ch_layout;
        output_frame->nb_samples = output_codec_context->frame_size;
        
        // TODO Probably need to do maths to calculate new pts properly
        output_frame->pts = input_frame->pts;

        if (swr_convert_frame(swr_ctx, output_frame, input_frame))
            {   logging("Swr Convert failed");  return -1;   }          
            /// ^ Fails here, the second time (since the first time init() is called internally)

        AVPacket *output_packet = av_packet_alloc();
        int response = avcodec_send_frame(output_codec_context, output_frame);

        while (response >= 0) {
            response = avcodec_receive_packet(output_codec_context, output_packet);

            if (response == AVERROR(EAGAIN) || response == AVERROR_EOF) {
                break;
            }

            output_packet->stream_index = 0;
            av_packet_rescale_ts(output_packet, in_stream->time_base, out_stream->time_base);
            av_interleaved_write_frame(out_format_context, output_packet);
        }
        av_packet_unref(output_packet);
        av_packet_free(&output_packet);
        av_frame_unref(input_frame);    // Free references held by the frame before reading new data into it.
        av_frame_unref(output_frame);
    }
    // TODO write last output packet flushing the buffer

    avformat_close_input(&in_format_context);
    return 0;
}


  


• Homepage Design : Best Practices & Examples

  5 octobre 2022, par Erin

  Did you know users spend about 50 milliseconds deciding if they like your website’s homepage design or not ?

  With billions of websites and scrolling often done on the go, you have to make a strong first impression because the chances for a once-over are slim. 

  Learn how to design magnetically-appealing website homepages from this guide. 

  What is a homepage in web design ?

  Homepage is the front page of your website — a destination where users land when typing your website URL address. It’s located at the root of the website’s domain (e.g., matomo.org) or a subdomain (e.g., university.webflow.com).

  Design-wise a homepage has two goals :

  • Explain the purpose of the website and present overview information 
  • Provide top-level navigation to lower-level web pages (e.g., blog, sales pages, etc.) 

  Separately, a homepage is also the place where users will return each time they’ll feel stuck and want to start anew. Thus, your homepage website design should provide obvious navigation paths to other website areas.

  6 Must-Know Website Homepage Design Best Practices

  Behind every winning homepage design stands a detailed customer journey map. 

  A customer journey is a schematic representation of how site visitors will move around your website to accomplish various goals. 

  A good customer journey map lists different actions a user will take after landing on your website (e.g., browse product pages, save items to a wishlist, register an account, etc.) — and it does so for different audience segments. 

  

  Your homepage design should help users move from the first step on their journey (e.g., learning about your website) to the final one (e.g., converting to a paid customer). At the same time, your homepage should serve the needs of both new and returning visitors — prospects who may be at a different stage of their journey (e.g., consideration). 

  With the above in mind, let’s take a look at several website homepage design ideas and the reasons why they work. 

  1. Use Familiar Design Elements

  Whether you’re designing a new website or refreshing an old one, it’s always tempting to go “out of the box” — use horizontal scrolling, skip header navigation or include arty animations. 

  Bold design choices work for some brands, mainly those who aren’t using their website as a primary sales channel (e.g., luxury brands). 

  But unfamiliar design patterns can also intimidate a lot of shoppers. In one observational study, people were asked to guess where specific content (e.g., information on international calls) would be placed on a telecom website. 75% of users picked the same location. This means two things :

  • People already have expectations of where specific website information is typically placed 
  • Yet, one in four users struggles to identify the right areas even within standard website layouts

  So why make the job harder for them ? As UX consultant Peter Ramsey rightfully notes : 

  The truth is : designing the best experience isn’t about being unique, it’s about being easy. And guess what feels really easy to use ? Things that feel familiar.

  Therefore, analyse other homepage layout designs in your industry. Pay attention to the number and type of homepage screens and approaches to designing header/footer navigation. 

  Take some of those ideas as your “base”. Then make your homepage design on-brand with unique typography, icons, visuals and other graphic design elements.

  Take a cue from ICAM — a steel manufacturing company. Their niche isn’t typically exciting. Yet, their homepage design stops you in your tracks and tinkers your curiosity to discover more (even if you aren’t shopping for metalware). 

  

  The interesting part is that ICAM uses a rather standard homepage layout. You have a hero image in the first screen, followed by a multi-column layout of their industry expertise and an overview of manufacturers. 

  But this homepage design feels fresh because the company uses plenty of white space, bold typography and vibrant visuals. Also, they delay the creative twist (horizontal scrolling area) to the bottom of the homepage, meaning that it’s less likely to intimidate less confident web users. 

  2. Decide On The Optimal Homepage Layout 

  In web design, a homepage layout is your approach to visually organising different information on the screen. 

  Observant folks will notice that good homepage designs often have the same layout. For example, include a split-view “hero” screen with a call to action on the left and visuals (photo or video) on the left. 

  

  The reason for using similar layouts for website homepage design isn’t a lack of creativity. On the contrary, some layouts have become the “best practice” because they :

  • Offer a great user experience (UX) and don’t confuse first-time visitors 
  • Feel familiar and create a pleasurable sense of deja-vu among users 
  • Have proven to drive higher conversion rates through benchmarks and tests 

  Popular types of website homepage layouts : 

  • Single column – a classic option of presenting main content in a single, vertical column. Good choice for blogs, personal websites and simple corporate sites. 
  • Split screen layout divides the page in two equal areas with different information present. Works best for Ecommerce homepages (e.g., to separate different types of garments) or SaaS websites, offering two product types (e.g., a free personal product version and a business edition). 
  • Asymmetrical layout assumes dividing your homepage into areas of different size and styles. Asymmetry helps create specific focal points for users to draw their attention to the most prominent information. 
  • Grid of cards layout helps present a lot of information in a more digestible manner by breaking down bigger bulks of text into smaller cards — a graphic element, featuring an image and some texts. By tapping a card, users can then access extra content. 
  • Boxes are visually similar to cards, but can be of varying shape. For example, you can have a bigger header-width box area, followed by four smaller boxes within it. Both of these website layouts work well for Ecommerce. 
  • Featured image layout gives visuals (photos and videos) the most prominent placement on the homepage, with texts and other graphic design elements serving a secondary purpose. 
  • F-pattern layout is based on the standard eye movement most people have when reading content on the website. Eye tracking studies found that we usually pay the most attention to information atop of the page (header area), then scan horizontally before dripping down to the next vertical line until we find content that captures our attention. 

  User behaviour analytics (UBA) tools are the best way to determine what type of layout will work for your homepage. 

  For example, you can use Matomo Heatmaps and Session Recording to observe how users navigate your homepage, which areas or links they click and what blockers they face during navigation.

  

  Matomo can capture accurate behavioural insights because we track relative positions to elements within your websites. This approach allows us to provide accurate data for users with different browsers, operating systems, zoom-in levels and fonts. 

  The best part ? You can collect behavioural data from up to 100 different user segments to understand how different audience cohorts engage with your product.

  3. Include a One-Sentence Tagline

  A tagline is a one-line summary of what your company does and what its unique sales proposition (USP) is. It should be short, catchy and distinguish you from competitors.

  A modern homepage design practice is to include a call to action in the first screen. Why ? Because you then instantly communicate or remind of your value proposition to every user — and provide them with an easy way to convert whenever they are ready to do business with you. 

  Here’s how three companies with a similar product, a project management app, differentiate themselves through homepage taglines. 

  Monday.com positions itself as an operating system (OS) for work. 

  

  Basecamp emphasises its product simplicity and openly says that they are different from other overly-complex software. 

  

  Asana, in turn, addresses a familiar user pain point (siloed communication) that it attempts to fix with its product. 

  

  Coming up with the perfect homepage tagline is a big task. You may have plenty of ideas, but little confidence in what version will stick. 

  The best approach ? Let a series of A/B tests decide. You can test a roaster of homepage slogans on a rotating bi-weekly/monthly schedule and track how copy changes affect conversion rates. 

  With Matomo A/B test feature, you can create, track and manage all experiments straight from your web analytics app — and get consolidated reports on total page visitors and conversion rates per each tested variation. 

  

  Beyond slogans, you can also run A/B tests to validate submission form placements, button texts or the entire page layout. 

  For instance, you can benchmark how your new homepage design performs compared to the old version with a subset of users before making it publicly available. 

  4. Highlight The Main Tasks For The User

  Though casual browsing is a thing, most of us head to specific websites with a clear agenda — find information, compare prices, obtain services, etc. 

  Thus, your homepage should provide clear starting points for users’ main tasks (those you’ve also identified as conversion goals on your customer journey maps !).

  These tasks can include : 

  • Account registration 
  • Product demo request 
  • Newsletter sign-up 

  The best website homepage designs organically guide users through a set number of common tasks, one screen at a time. 

  Let’s analyse Sable homepage design. The company offers a no-fee bank account and a credit card product for soon-to-be US transplants. The main task a user has : Decide if they want to try Sable and hopefully open an account with them. 

  

  This mono-purpose page focuses on persuading a prospect that Sable is right for them. 

  The first screen hosts the main CTA with an animated drop-down arrow to keep scrolling. This is likely aimed at first-time visitors that just landed on the page from an online ad or social media post. 

  The second screen serves the main pitch — no-fee, no-hassle access to a US banking account that also helps you build your credit score. 

  The third screen encourages users to learn more about Sable Credit — the flagship product. For the sceptics, the fourth screen offers several more reasons to sign up for the credit product. 

  Then Sable moves on to pitching its second offering — a no-fee debit card with a cashback. Once again, the follow-up screen sweetens the deal by bringing up other perks (higher cashback for popular services like Amazon) and overcoming objections (no SSN required and multi-language support available). 

  The sequence ends with side-by-side product comparison and some extra social proof. 

  In Sable’s case, each homepage screen has a clear purpose and is designed to facilitate one specific user action — account opening. 

  For multi-product companies, the above strategy works great for designing individual landing pages. 

  5. Design Proper Navigation Paths

  All websites have two areas reserved for navigation : 

  • Header menu 
  • Footer menu 

  Designing an effective header menu is more important since it’s the primary tool visitors will use to discover other pages. 

  Your header menu can be :

  • Sticky — always visible as the person keeps scrolling. 
  • Static — e.g., a hidden drop-down menu. 

  If you go for a static header and have a longer homepage layout (e.g., 5+ screens), you also need to add extra navigation elements somewhere mid-page. Or else users might not figure out where to go next and merely bounce off. 

  You can do this by : 

  • Promoting other areas of your website (e.g., sub-category pages) by linking out to them 
  • Adding a carousel of “recent posts”, “recommended reads” and “latest products” 
  • Using buttons and CTAs to direct users towards specific actions (e.g., account registration) or assets (free eBook)

  For instance, cosmetics brand Typology doesn’t have a sticky header on the homepage. Instead, they prompt discovery by promoting different product categories (best sellers, bundles, latest arrivals) and their free skin diagnostic quiz — a great engagement mechanism to retain first time users.

  

  Once the user scrolls down to the bottom of the page, they should have an extra set of navigational options — aka footer links. 

  Again, these help steer the visitor towards discovering more content without scrolling back up to the top of your homepage. 

  Nielsen Norman Group says that people mostly use footers as :

  • A second chance to be convinced — after reading the entire homepage, the user is ready to give your product a go.
    
  • The last resort for hard-to-find content that’s not displayed in global header navigation (e.g., Terms and Conditions or shipping information pages).

  As a rule of thumb, you should designate the following information to the footer : 

  • Utility links (Contact page, Terms & Conditions, Privacy Policy, etc.) 
  • Secondary-task links (e.g., Career page, Investor Details, Media contacts, etc.) 
  • Brands within the organisation (if you operate several) 
  • Customer engagement link (email newsletters and social media buttons)

  The key is to keep the area compact — not more than one standard user screen resolution of 1280×720. 

  6. Show Users What’s Clickable (Or Not) 

  A homepage invites your site visitors on a journey. But if they don’t know which elements to click, they aren’t going to get anywhere.

  Good homepage design makes it obvious which page elements are clickable, i.e., can take the user to a new page or another segment of the homepage. 

  Here are several must-know homepage design tips for better on-page navigation : 

  • Use colour and underline or bold to highlight clickable words. Alternatively, you can change the browser cursor from a standard arrow into another element (e.g., a larger dot or a pointy finger) to indicate when the cursor hovers over a clickable website area. 
  • Make descriptive button texts that imply what will happen when a user clicks the page. Instead of using abstract and generic button texts like “see more” or “learn more”, try a more vibrant language like “dive in” for clicking through to a spa page. 
  • Use a unified hover area to show how different homepage design elements represent a single path or multiple navigation paths. When multiple items are encapsulated in one visual element (e.g., a box), users may be reluctant to click the image because they aren’t sure if it’s one large hit area leading to a single page or if there are multiple hit areas, leading to different pages. 

  Homepage of BEAUSiTE — a whimsical hotel in the Swiss Alps – embodies all of the above design principles. They change the cursor style whenever you scroll into a hit area, use emotive and creative micro-copy for all button texts and clearly distinguish between different homepage elements.

  

  How to Make Your Homepage Design Even More Impactful ? 

  Website homepage design is roughly 20% of pure design work and 80% of behind-the-scenes research. 

  To design a high-performing homepage you need to have data-backed answers to the following questions : 

  • Who are your primary and secondary target audiences ? 
  • Which tasks (1 to 4) you’d want to help them solve through your homepage ?

  You can get the answers to both questions from your web analytics data by using audience segmentation and page transition (behaviour flow) reports in Matomo. 

  Based on these, you can determine common user journeys and tasks people look to accomplish when visiting your website. Next, you can collect even more data with UBA tools  like heatmaps and user session recordings. Then translated the observed patterns into working homepage design ideas. 

  Improve your homepage design and conversion rates with Matomo. Start your free 21-day trial now ! 

  21 day free trial. No credit card required.

  Your email address
  

  Your website address
  

  Your Analytics subdomain will be .matomo.cloud (

• avdevice/decklink_enc : Add support for compressed AC-3 output over SDI

  7 avril 2023, par Devin Heitmueller

  avdevice/decklink_enc : Add support for compressed AC-3 output over SDI
  Extend the decklink output to include support for compressed AC-3,
  
  encapsulated using the SMPTE ST 377:2015 standard.
  This functionality can be exercised by using the "copy" codec when
  
  the input audio stream is AC-3.  For example :
  ./ffmpeg -i /foo.ts -codec:a copy -f decklink 'UltraStudio Mini Monitor'
  Note that the default behavior continues to be to do PCM output,
  
  which means without specifying the copy codec a stream containing
  
  AC-3 will be decoded and downmixed to stereo audio before output.
  Thanks to Marton Balint for providing feedback.
  Signed-off-by : Devin Heitmueller <dheitmueller@ltnglobal.com>
  
  Signed-off-by : Marton Balint <cus@passwd.hu>
  

  • [DH] libavdevice/decklink_enc.cpp