Recherche avancée

Sur d’autres sites (13637)

• checkasm : Add vc1dsp inverse transform tests

  31 mars 2022, par Ben Avison

  checkasm : Add vc1dsp inverse transform tests
  This test deliberately doesn't exercise the full range of inputs described in
  
  the committee draft VC-1 standard. It says :
  input coefficients in frequency domain, D, satisfy   -2048 <= D < 2047
  
  intermediate coefficients, E, satisfy                -4096 <= E < 4095
  
  fully inverse-transformed coefficients, R, satisfy    -512 <= R <  511
  For one thing, the inequalities look odd. Did they mean them to go the
  
  other way round ? That would make more sense because the equations generally
  
  both add and subtract coefficients multiplied by constants, including powers
  
  of 2. Requiring the most-negative values to be valid extends the number of
  
  bits to represent the intermediate values just for the sake of that one case !
  For another thing, the extreme values don't look to occur in real streams -
  
  both in my experience and supported by the following comment in the AArch32
  
  decoder :
      tNhalf is half of the value of tN (as described in vc1_inv_trans_8x8_c).
  
      This is done because sometimes files have input that causes tN + tM to
  
      overflow. To avoid this overflow, we compute tNhalf, then compute
  
      tNhalf + tM (which doesn't overflow), and then we use vhadd to compute
  
      (tNhalf + (tNhalf + tM)) >> 1 which does not overflow because it is
  
      one instruction.
  My AArch64 decoder goes further than this. It calculates tNhalf and tM
  
  then does an SRA (essentially a fused halve and add) to compute
  
  (tN + tM) >> 1 without ever having to hold (tNhalf + tM) in a 16-bit element
  
  without overflowing. It only encounters difficulties if either tNhalf or
  
  tM overflow in isolation.
  I haven't had sight of the final standard, so it's possible that these
  
  issues were dealt with during finalisation, which could explain the lack
  
  of usage of extreme inputs in real streams. Or a preponderance of decoders
  
  that only support 16-bit intermediate values in their inverse transforms
  
  might have caused encoders to steer clear of such cases.
  I have effectively followed this approach in the test, and limited the
  
  scale of the coefficients sufficient that both the existing AArch32 decoder
  
  and my new AArch64 decoder both pass.
  Signed-off-by : Ben Avison <bavison@riscosopen.org>
  
  Signed-off-by : Martin Storsjö <martin@martin.st>
  

  • [DH] tests/checkasm/vc1dsp.c

• Upsample and encode audio stream

  8 juin 2022, par Anton Issaikin

  Basically after transcoding pcm_alaw 8khz to mp3, I can hear only some brief or even swift sound in first 1-2 seconds, unrecognizable sound. So something is wrong with pts/dts, packed to planar convertion, or upsampling.

  


  My application does transcoding rtsp camera stream to file. Video and audio. Video works fine and audio remuxing as well. Now I have pcm_alaw 8khz audio stream and want to transcode it to mp4 file along with video.

  


  Code is quite cumbersome to construct reproducible part, so firstly I want to know if my logic is right. Here is my draft process (assume all error are checked and handled) :

  


  create encoder :

  


      codec_ = avcodec_find_encoder(AV_CODEC_ID_MP3);

    enc_ctx_ = avcodec_alloc_context3(codec_);

    enc_ctx_->bit_rate = 64000;
    enc_ctx_->codec_type = AVMEDIA_TYPE_AUDIO;

    enc_ctx_->sample_fmt   = codec_->sample_fmts ? codec_->sample_fmts[0] : AV_SAMPLE_FMT_S32P;

    // functions from here https://www.ffmpeg.org/doxygen/4.1/encode_audio_8c-example.html
    enc_ctx_->sample_rate    = select_sample_rate(codec_);
    enc_ctx_->channel_layout = select_channel_layout(codec_);
    enc_ctx_->channels       = av_get_channel_layout_nb_channels(enc_ctx_->channel_layout);
    enc_ctx_->time_base = (AVRational){1, enc_ctx_->sample_rate};
    enc_ctx_->strict_std_compliance = FF_COMPLIANCE_EXPERIMENTAL;

    if (is_global_header) {
        enc_ctx_->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
    }

    avcodec_open2(enc_ctx_, codec_, nullptr);


  


  create resampler (in_frame) :

  


      audio_fifo_ = av_audio_fifo_alloc(enc_ctx_->sample_fmt, enc_ctx_->channels, 1));
       
    in_ch_layout_ = in_frame->channel_layout;
    in_sample_fmt = in_frame->format;
    in_sample_rate_ = in_frame->sample_rate;

    swr_ctx_ = swr_alloc_set_opts(NULL,                       // we're allocating a new context
                             enc_ctx_->channel_layout,        // out_ch_layout
                             enc_ctx_->sample_fmt,            // out_sample_fmt
                             enc_ctx_->sample_rate,           // out_sample_rate
                             in_frame->channel_layout,        // in_ch_layout
                             (AVSampleFormat)in_frame->format, // in_sample_fmt
                             in_frame->sample_rate,            // in_sample_rate
                             0,                                // log_offset
                             NULL);                            // log_ctx
                             
    swr_init(swr_ctx_);


  


  resample (in_frame, start_pts, start_dts) :

  


      auto resampled_frame = av_frame_alloc();&#xA;&#xA;    auto dst_nb_samples = av_rescale_rnd(swr_get_delay(swr_ctx_, in_frame->sample_rate) &#x2B;&#xA;                                    in_frame->nb_samples, enc_ctx_->sample_rate, in_frame->sample_rate, AV_ROUND_UP);&#xA;&#xA;    // resampled_frame->nb_samples     = dst_nb_samples;&#xA;    resampled_frame->format         = enc_ctx_->sample_fmt;&#xA;    resampled_frame->channel_layout = enc_ctx_->channel_layout;&#xA;    // resampled_frame->channels       = enc_ctx_->channels;&#xA;    resampled_frame->sample_rate    = enc_ctx_->sample_rate;&#xA;&#xA;    error = swr_convert_frame(swr_ctx_, resampled_frame, in_frame);&#xA;&#xA;    /* Make the FIFO as large as it needs to be to hold both,&#xA;     * the old and the new samples. */&#xA;    if (av_audio_fifo_size(audio_fifo_) &lt; dst_nb_samples) {&#xA;        av_audio_fifo_realloc(audio_fifo_, dst_nb_samples);&#xA;    }&#xA;&#xA;    /* Store the new samples in the FIFO buffer. */&#xA;    auto nb_samples = av_audio_fifo_write(audio_fifo_,&#xA;                                          reinterpret_cast<void>(resampled_frame->extended_data),&#xA;                                          resampled_frame->nb_samples);&#xA;&#xA;&#xA;    int delay = 0;&#xA;    // trying to split resampled frame to desired chunks&#xA;    while (av_audio_fifo_size(audio_fifo_) > 0) {&#xA;        const int frame_size = FFMIN(av_audio_fifo_size(audio_fifo_), enc_ctx_->frame_size);&#xA;&#xA;        auto out_frame = av_frame_alloc();&#xA;&#xA;&#xA;        out_frame->nb_samples       = frame_size;&#xA;        out_frame->format           = enc_ctx_->sample_fmt;&#xA;        out_frame->channel_layout   = enc_ctx_->channel_layout;&#xA;        out_frame->channels         = enc_ctx_->channels;&#xA;        out_frame->sample_rate      = enc_ctx_->sample_rate;&#xA;&#xA;        av_frame_get_buffer(out_frame, 0);&#xA;        &#xA;        av_audio_fifo_read(audio_fifo_, (void **)out_frame->data, frame_size) &lt; frame_size);&#xA;&#xA;    // ***** tried both cases&#xA;        out_frame->pts = in_frame->pts &#x2B; delay;&#xA;        out_frame->pkt_dts = in_frame->pkt_dts &#x2B; delay;&#xA;        // swr_next_pts(swr_ctx_, in_frame->pts) &#x2B; delay;&#xA;        // swr_next_pts(swr_ctx_, in_frame->pkt_dts) &#x2B; delay;&#xA;&#xA;        result.push_back(out_frame);&#xA;&#xA;        delay &#x2B;= frame_size;&#xA;    }&#xA;&#xA;    return result;&#xA;</void>

  


  encoding and muxing (in_frame) :

  


      bool DoesNeedResample(const AVFrame * in_frame) {
        assert(("DoesNeedResample: in_frame is empty", in_frame));
        assert(("DoesNeedResample: encoder is not started", is_init_));

        if (in_frame->sample_rate != enc_ctx_->sample_rate ||
        in_frame->channel_layout != enc_ctx_->channel_layout ||
        in_frame->channels != enc_ctx_->channels ||
        in_frame->format != enc_ctx_->sample_fmt) {
        return true;
        }

        return false;
    }

    av_frame_make_writable(in_frame);


    streamserver::AVFrames encoding_frames;
    if (DoesNeedResample(in_frame)) {
        encoding_frames = Resample(in_frame, 
        av_rescale_q(in_frame->pts, in_audio_stream_timebase_, out_audio_stream_->time_base),
        av_rescale_q(in_frame->pkt_dts, in_audio_stream_timebase_, out_audio_stream_->time_base));
    } else {
        encoding_frames.push_back(av_frame_clone(in_frame));
    }


    for (auto frame : encoding_frames) {
        if ((err = avcodec_send_frame(encoder_ctx, frame)) < 0) {
            AVFrameFree(&frame);
        }

        while (err >= 0) {
            pkt_->data = NULL;
            pkt_->size = 0;
            av_init_packet(pkt_);

            err = avcodec_receive_packet(encoder_ctx, pkt_);
            if (err == AVERROR(EAGAIN) || err == AVERROR_EOF) {
                break;
            } else if (err < 0) {
                break;
            }

            pkt_->stream_index = out_audio_stream_->index;

            av_interleaved_write_frame(ofmt_ctx_, pkt_);
        }

        av_packet_unref(pkt_);
    }


  


  Sound in resulted video is corrupted, see first paragraph for description.

  


  In https://www.ffmpeg.org/doxygen/4.1/transcode_aac_8c-example.html
there are lines :

  


          /*
        * Perform a sanity check so that the number of converted samples is
        * not greater than the number of samples to be converted.
        * If the sample rates differ, this case has to be handled differently
        */
        av_assert0(output_codec_context->sample_rate == input_codec_context->sample_rate);


  


  How to handle such cases ? I tried to split resampled frames via fifo in example above !

  


• Extract Video Frames from SDP Output

  1er août 2022, par user1446797

  Does anyone know how to extract image frames from a SDP video output ? I'm using a Nest battery camera. The wired version gave me an RTSP stream which was easy to extract frames. However, the battery version gave me a SDP output which is hard to make sense of. I've looked at a few posts on stackoverflow but none seemed too promising :

  


  How to use the answerSDP returned from sdm.devices.commands.CameraLiveStream.GenerateWebRtcStream to establish a stream with google nest cam

  


  Executing FFmpeg recording using in-line SDP

  


  Even being able to stream SDP to a mp4 file using ffplay would be a nice start. But ultimately I would like to run a python script to extract frames from SDP output.

  


  I must admit, SDP (session description protocol) seems pretty long and complicated compared to working with RTSP streams. Anyway to simply convert an SDP stream to a RTSP stream ?

  


  https://andrewjprokop.wordpress.com/2013/09/30/understanding-session-description-protocol-sdp/

  


  Thanks !
Jacob

  


  SDP output looks something like this :

  


  v=0\r\no=- 0 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0 2 1\r\na=msid-semantic : WMS 16733765853514488918/633697675 virtual-6666\r\na=ice-lite\r\nm=audio 19305 UDP/TLS/RTP/SAVPF 111\r\nc=IN IP4 142.250.9.127\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=candidate : 1 udp 2113939711 2607:f8b0:4002:c11::7f 19305 typ host generation 0\r\na=candidate : 1 tcp 2113939710 2607:f8b0:4002:c11::7f 19305 typ host tcptype passive generation 0\r\na=candidate : 1 ssltcp 2113939709 2607:f8b0:4002:c11::7f 443 typ host generation 0\r\na=candidate : 1 udp 2113932031 142.250.9.127 19305 typ host generation 0\r\na=candidate : 1 tcp 2113932030 142.250.9.127 19305 typ host tcptype passive generation 0\r\na=candidate : 1 ssltcp 2113932029 142.250.9.127 443 typ host generation 0\r\na=ice-ufrag:UVDO0GOJASABT95E\r\na=ice-pwd:FRILJDCJZCH+51YNWDGZIN0K\r\na=fingerprint:sha-256 24:53:14:34:59:50:89:52:72:58:04:57:71:BB:C4:89:91:3A:52:EF:C0:5A:A5:EC:B5:51:64:80:AC:13:89:8A\r\na=setup:passive\r\na=mid:0\r\na=extmap:1 urn:ietf:params:rtp-hdrext:ssrc-audio-level\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=sendrecv\r\na=msid:virtual-6666 virtual-6666\r\na=rtcp-mux\r\na=rtpmap:111 opus/48000/2\r\na=rtcp-fb:111 transport-cc\r\na=fmtp:111 minptime=10 ;useinbandfec=1\r\na=ssrc:6666 cname:6666\r\nm=video 9 UDP/TLS/RTP/SAVPF 108 109\r\nc=IN IP4 0.0.0.0\r\na=rtcp:9 IN IP4 0.0.0.0\r\na=ice-ufrag:UVDO0GOJASABT95E\r\na=ice-pwd:FRILJDCJZCH+51YNWDGZIN0K\r\na=fingerprint:sha-256 24:53:14:34:59:50:89:52:72:58:04:57:71:BB:C4:89:91:3A:52:EF:C0:5A:A5:EC:B5:51:64:80:AC:13:89:8A\r\na=setup:passive\r\na=mid:1\r\na=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time\r\na=extmap:13 urn:3gpp:video-orientation\r\na=extmap:3 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01\r\na=sendrecv\r\na=msid:16733765853514488918/633697675 16733765853514488918/633697675\r\na=rtcp-mux\r\na=rtpmap:108 H264/90000\r\na=rtcp-fb:108 transport-cc\r\na=rtcp-fb:108 ccm fir\r\na=rtcp-fb:108 nack\r\na=rtcp-fb:108 nack pli\r\na=rtcp-fb:108 goog-remb\r\na=fmtp:108 level-asymmetry-allowed=1 ;packetization-mode=1 ;profile-level-id=42e01f\r\na=rtpmap:109 rtx/90000\r\na=fmtp:109 apt=108\r\na=ssrc-group:FID 633697675 3798748564\r\na=ssrc:633697675 cname:633697675\r\na=ssrc:3798748564 cname:633697675\r\nm=application 9 DTLS/SCTP 5000\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:UVDO0GOJASABT95E\r\na=ice-pwd:FRILJDCJZCH+51YNWDGZIN0K\r\na=fingerprint:sha-256 24:53:14:34:59:50:89:52:72:58:04:57:71:BB:C4:89:91:3A:52:EF:C0:5A:A5:EC:B5:51:64:80:AC:13:89:8A\r\na=setup:passive\r\na=mid:2\r\na=sctpmap:5000 webrtc-datachannel 1024\r\n