Recherche avancée

Sur d’autres sites (8600)

• libavfilter/scale2ref : Add constants for the primary input

  30 mai 2017, par Kevin Mark

  libavfilter/scale2ref : Add constants for the primary input
  Variables pertaining to the main video are now available when
  
  using the scale2ref filter. This allows, as an example, scaling a
  
  video with another as a reference point while maintaining the
  
  original aspect ratio of the primary/non-reference video.
  Consider the following graph : scale2ref=iw/6 :-1 [main][ref]
  
  This will scale [main] to 1/6 the width of [ref] while maintaining
  
  the aspect ratio. This works well when the AR of [ref] is equal to
  
  the AR of [main] only. What the above filter really does is
  
  maintain the AR of [ref] when scaling [main]. So in all non-same-AR
  
  situations [main] will appear stretched or compressed to conform to
  
  the same AR of the reference video. Without doing this calculation
  
  externally there is no way to scale in reference to another input
  
  while maintaining AR in libavfilter.
  To make this possible, we introduce eight new constants to be used
  
  in the w and h expressions only in the scale2ref filter :
   * main_w/main_h : width/height of the main input video
  
   * main_a : aspect ratio of the main input video
  
   * main_sar : sample aspect ratio of the main input video
  
   * main_dar : display aspect ratio of the main input video
  
   * main_hsub/main_vsub : horiz/vert chroma subsample vals of main
  
   * mdar : a shorthand alias of main_dar
  Of course, not all of these constants are needed for maintaining the
  
  AR, but adding additional constants in line of what is available for
  
  in/out allows for other scaling possibilities I have not imagined.
  So to now scale a video to 1/6 the size of another video using the
  
  width and maintaining its own aspect ratio you can do this :
  scale2ref=iw/6:ow/mdar [main][ref]
  This is ideal for picture-in-picture configurations where you could
  
  have a square or 4:3 video overlaid on a corner of a larger 16:9
  
  feed all while keeping the scaled video in the corner at its correct
  
  aspect ratio and always the same size relative to the larger video.
  I've tried to re-use as much code as possible. I could not find a way
  
  to avoid duplication of the var_names array. It must now be kept in
  
  sync with the other (the normal one and the scale2ref one) for
  
  everything to work which does not seem ideal. For every new variable
  
  introduced/removed into/from the normal scale filter one must be
  
  added/removed to/from the scale2ref version. Suggestions on how to
  
  avoid var_names duplication are welcome.
  var_values has been increased to always be large enough for the
  
  additional scale2ref variables. I do not forsee this being a problem
  
  as the names variable will always be the correct size. From my
  
  understanding of av_expr_parse_and_eval it will stop processing
  
  variables when it runs out of names even though there may be
  
  additional (potentially uninitialized) entries in the values array.
  
  The ideal solution here would be using a variable-length array but
  
  that is unsupported in C90.
  This patch does not remove any functionality and is strictly a
  
  feature patch. There are no API changes. Behavior does not change for
  
  any previously valid inputs.
  The applicable documentation has also been updated.
  Signed-off-by : Kevin Mark <kmark937@gmail.com>
  
  Signed-off-by : Michael Niedermayer <michael@niedermayer.cc>
  

  • [DH] doc/filters.texi
  • [DH] libavfilter/scale.c

• FFmpeg avcodec_open2 throws -22 ("Invalid Argument")

  14 avril 2023, par stupidbutseeking

  I got stuck trying to write a simple video conversion using C++ and ffmpeg.

  


  When trying to convert a video using FFmpeg, calling avcodec_open2 fails with the code "-22" which seems to be an "Invalid Argument"-error.

  


  I can't figure out why it fails, nor what the invalid argument is. In the following snippet I create the output codec and pass its context the information from the source (code further down below).

  


  The check for the "outputCodec" works and does not throw an error. As far as I know an "AVDictionary"-argument is optional. So I guess the reason for the error is the context.

  


      const AVCodec* outputCodec = avcodec_find_encoder_by_name(codecName.c_str());

    if (!outputCodec)
    {
        std::cout << "Zielformat-Codec nicht gefunden" << std::endl;
        return -1;
    }

    AVCodecContext* outputCodecContext = avcodec_alloc_context3(outputCodec);
    outputCodecContext->bit_rate = bitRate;
    outputCodecContext->width = inputCodecContext->width;
    outputCodecContext->height = inputCodecContext->height;
    outputCodecContext->pix_fmt = outputCodec->pix_fmts[0];
    outputCodecContext->time_base = inputCodecContext->time_base;

    **int errorCode = avcodec_open2(outputCodecContext, outputCodec, NULL); //THIS RETURNS -22**
    if (errorCode != 0)
    {
        std::cout << "Fehler beim Öffnen des Zielformat-Codecs" << std::endl;
        return -1;
    }


  


  Here is the code for getting the input file & context :

  


      std::string inputFilename = "input_video.mp4";
    std::string outputFilename = "output.avi";
    std::string codecName = "mpeg4";
    int bitRate = 400000;

    AVFormatContext* inputFormatContext = NULL;
    if (avformat_open_input(&inputFormatContext, inputFilename.c_str(), NULL, NULL) != 0)
    {
        std::cout << "Fehler beim Öffnen der Eingabedatei" << std::endl;
        return -1;
    }

    [Do Video Stream Search)

    AVCodecParameters* inputCodecParameters = inputFormatContext->streams[videoStreamIndex]->codecpar;
    const AVCodec* inputCodec = avcodec_find_decoder(inputCodecParameters->codec_id);
    AVCodecContext* inputCodecContext = avcodec_alloc_context3(inputCodec);
    if (avcodec_parameters_to_context(inputCodecContext, inputCodecParameters) != 0)
    {
        std::cout << "Fehler beim Setzen des Eingabecodecs" << std::endl;
        return -1;
    }
    if (avcodec_open2(inputCodecContext, inputCodec, NULL) != 0)
    {
        std::cout << "Fehler beim Öffnen des Eingabecodecs" << std::endl;
        return -1;
    }


  


  The purpose was simply to get started with ffmpeg in an own C++ project.

  


  If it is of any need, I downloaded the ffmpeg libs from here. I used the gpl shared ones. The architecture is win x64. I referenced them through the project properties (additional libraries and so on).

  


  I tried to convert a .mp4 video to an .avi video with an "mpeg4" compression.
I also tried other compressions like "libx264" but none worked.

  


  I searched for the problem on stackoverflow but could not find the exact same problem. While its purpose is different this post is about the same error code when calling avcodec_open2. But its solution is not working for me. I inspected the watch for the "outputContext" while running the code and the codec_id, codec_type and format is set. I use the time_base from the input file. According to my understanding, this should be equal to the source. So I can not find out what I am missing.

  


  Thanks in advance and sorry for my english.

  


  And for completion, here is the whole method :

  


  int TestConvert()
{
    std::string inputFilename = "input_video.mp4";
    std::string outputFilename = "output.avi";
    std::string codecName = "mpeg4";
    int bitRate = 400000;

    AVFormatContext* inputFormatContext = NULL;
    if (avformat_open_input(&inputFormatContext, inputFilename.c_str(), NULL, NULL) != 0)
    {
        std::cout << "Fehler beim Öffnen der Eingabedatei" << std::endl;
        return -1;
    }

    int videoStreamIndex = -1;
    for (unsigned int i = 0; i < inputFormatContext->nb_streams; i++)
    {
        if (inputFormatContext->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO)
        {
            videoStreamIndex = i;
            break;
        }
    }

    AVCodecParameters* inputCodecParameters = inputFormatContext->streams[videoStreamIndex]->codecpar;
    const AVCodec* inputCodec = avcodec_find_decoder(inputCodecParameters->codec_id);
    AVCodecContext* inputCodecContext = avcodec_alloc_context3(inputCodec);
    if (avcodec_parameters_to_context(inputCodecContext, inputCodecParameters) != 0)
    {
        std::cout << "Fehler beim Setzen des Eingabecodecs" << std::endl;
        return -1;
    }
    if (avcodec_open2(inputCodecContext, inputCodec, NULL) != 0)
    {
        std::cout << "Fehler beim Öffnen des Eingabecodecs" << std::endl;
        return -1;
    }

    const AVCodec* outputCodec = avcodec_find_encoder_by_name(codecName.c_str());

    if (!outputCodec)
    {
        std::cout << "Zielformat-Codec nicht gefunden" << std::endl;
        return -1;
    }

    AVCodecContext* outputCodecContext = avcodec_alloc_context3(outputCodec);
    outputCodecContext->bit_rate = bitRate;
    outputCodecContext->width = inputCodecContext->width;
    outputCodecContext->height = inputCodecContext->height;
    outputCodecContext->pix_fmt = outputCodec->pix_fmts[0];
    outputCodecContext->time_base = inputCodecContext->time_base;

    int errorCode = avcodec_open2(outputCodecContext, outputCodec, NULL);
    if (errorCode != 0)
    {
        std::cout << "Fehler beim Öffnen des Zielformat-Codecs" << std::endl;
        return -1;
    }

    AVFormatContext* outputFormatContext = NULL;
    if (avformat_alloc_output_context2(&outputFormatContext, NULL, NULL, outputFilename.c_str()) != 0)
    {
        std::cout << "Fehler beim Erstellen des Ausgabe-Formats" << std::endl;
        return -1;
    }

    AVStream* outputVideoStream = avformat_new_stream(outputFormatContext, outputCodec);
    if (outputVideoStream == NULL)
    {
        std::cout << "Fehler beim Hinzufügen des Video-Streams zum Ausgabe-Format" << std::endl;
        return -1;
    }
    outputVideoStream->id = outputFormatContext->nb_streams - 1;
    AVCodecParameters* outputCodecParameters = outputVideoStream->codecpar;
    if (avcodec_parameters_from_context(outputCodecParameters, outputCodecContext) != 0)
    {
        std::cout << "Fehler beim Setzen des Ausgabe-Codecs" << std::endl;
        return -1;
    }

    if (!(outputFormatContext->oformat->flags & AVFMT_NOFILE))
    {
        if (avio_open(&outputFormatContext->pb, outputFilename.c_str(), AVIO_FLAG_WRITE) != 0)
        {
            std::cout << "Fehler beim Öffnen der Ausgabedatei" << std::endl;
            return -1;
        }
    }

    if (avformat_write_header(outputFormatContext, NULL) != 0)
    {
        std::cout << "Fehler beim Schreiben des Ausgabe-Formats in die Ausgabedatei" << std::endl;
        return -1;
    }

    AVPacket packet;
    int response;
    AVFrame* frame = av_frame_alloc();
    AVFrame* outputFrame = av_frame_alloc();
    while (av_read_frame(inputFormatContext, &packet) == 0)
    {
        if (packet.stream_index == videoStreamIndex)
        {
            response = avcodec_send_packet(inputCodecContext, &packet);
            while (response >= 0)
            {
                response = avcodec_receive_frame(inputCodecContext, frame);
                if (response == AVERROR(EAGAIN) || response == AVERROR_EOF)
                {
                    break;
                }
                else if (response < 0)
                {
                    std::cout << "Fehler beim Dekodieren des Video-Pakets" << std::endl;
                    return -1;
                }

                struct SwsContext* swsContext = sws_getContext(inputCodecContext->width, inputCodecContext->height, inputCodecContext->pix_fmt, outputCodecContext->width, outputCodecContext->height, outputCodecContext->pix_fmt, SWS_BILINEAR, NULL, NULL, NULL); if (!swsContext)
                {
                    std::cout << "Fehler beim Erstellen des SwsContext" << std::endl;
                    return -1;
                }
                sws_scale(swsContext, frame->data, frame->linesize, 0, inputCodecContext->height, outputFrame->data, outputFrame->linesize);
                sws_freeContext(swsContext);

                outputFrame->pts = frame->pts;
                outputFrame->pkt_dts = frame->pkt_dts;
                //outputFrame->pkt_duration = frame->pkt_duration;
                response = avcodec_send_frame(outputCodecContext, outputFrame);
                while (response >= 0)
                {
                    response = avcodec_receive_packet(outputCodecContext, &packet);
                    if (response == AVERROR(EAGAIN) || response == AVERROR_EOF)
                    {
                        break;
                    }
                    else if (response < 0)
                    {
                        std::cout << "Fehler beim Kodieren des Ausgabe-Frames" << std::endl;
                        return -1;
                    }

                    packet.stream_index = outputVideoStream->id;
                    av_packet_rescale_ts(&packet, outputCodecContext->time_base, outputVideoStream->time_base);
                    if (av_interleaved_write_frame(outputFormatContext, &packet) != 0)
                    {
                        std::cout << "Fehler beim Schreiben des Ausgabe-Pakets" << std::endl;
                        return -1;
                    }
                    av_packet_unref(&packet);
                }
            }
        }
        av_packet_unref(&packet);
    }

    av_write_trailer(outputFormatContext);
    avcodec_free_context(&inputCodecContext);
    avcodec_free_context(&outputCodecContext);
    avformat_close_input(&inputFormatContext);
    avformat_free_context(inputFormatContext);
    avformat_free_context(outputFormatContext);
    av_frame_free(&frame);
    av_frame_free(&outputFrame);

    return 0;

}


  


• What's the most desireable way to capture system display and audio in the form of individual encoded audio and video packets in go (language) ? [closed]

  11 janvier 2023, par Tiger Yang

  Question (read the context below first) :

  


  For those of you familiar with the capabilities of go, Is there a better way to go about all this ? Since ffmpeg is so ubiquitous, I'm sure it's been optomized to perfection, but what's the best way to capture system display and audio in the form of individual encoded audio and video packets in go (language), so that they can be then sent via webtransport-go ? I wish for it to prioritize efficiency and low latency, and ideally capture and encode the framebuffer directly like ffmpeg does.

  


  Thanks ! I have many other questions about this, but I think it's best to ask as I go.

  


  Context and what I've done so far :

  


  I'm writing a remote desktop software for my personal use because of grievances with current solutions out there. At the moment, it consists of a web app that uses the webtransport API to send input datagrams and receive AV packets on two dedicated unidirectional streams, and the webcodecs API to decode these packets. On the serverside, I originally planned to use python with the aioquic library as a webtransport server. Upon connection and authentication, the server would start ffmpeg as a subprocess with this command :

  


  ffmpeg -init_hw_device d3d11va -filter_complex ddagrab=video_size=1920x1080:framerate=60 -vcodec hevc_nvenc -tune ll -preset p7 -spatial_aq 1 -temporal_aq 1 -forced-idr 1 -rc cbr -b:v 400K -no-scenecut 1 -g 216000 -f hevc -

  


  What I really appreciate about this is that it uses windows' desktop duplication API to copy the framebuffer of my GPU and hand that directly to the on-die hardware encoder with zero round trips to the CPU. I think it's about as efficient and elegant a solution as I can manage. It then outputs the encoded stream to the stdout, which python can read and send to the client.

  


  As for the audio, there is another ffmpeg instance :

  


  ffmpeg -f dshow -channels 2 -sample_rate 48000 -sample_size 16 -audio_buffer_size 15 -i audio="RD Audio (High Definition Audio Device)" -acodec libopus -vbr on -application audio -mapping_family 0 -apply_phase_inv true -b:a 25K -fec false -packet_loss 0 -map 0 -f data -

  


  which listens to a physical loopback interface, which is literally just a short wire bridging the front panel headphone and microphone jacks (I'm aware of the quality loss of converting to analog and back, but the audio is then crushed down to 25kbps so it's fine) ()

  


  Unfortunately, aioquic was not easy to work with IMO, and I found webtransport-go https://github.com/adriancable/webtransport-go, which was a hell of a lot better in both simplicity and documentation. However, now I'm dealing with a whole new language, and I wanna ask : (above)

  


  EDIT : Here's the code for my server so far :

  


  


  


  package main

import (
    "bytes"
    "context"
    "fmt"
    "log"
    "net/http"
    "os/exec"
    "time"

    "github.com/adriancable/webtransport-go"
)

func warn(str string) {
    fmt.Printf("\n===== WARNING ===================================================================================================\n   %s\n=================================================================================================================\n", str)
}

func main() {

    password := []byte("abc")

    videoString := []string{
        "ffmpeg",
        "-init_hw_device", "d3d11va",
        "-filter_complex", "ddagrab=video_size=1920x1080:framerate=60",
        "-vcodec", "hevc_nvenc",
        "-tune", "ll",
        "-preset", "p7",
        "-spatial_aq", "1",
        "-temporal_aq", "1",
        "-forced-idr", "1",
        "-rc", "cbr",
        "-b:v", "500K",
        "-no-scenecut", "1",
        "-g", "216000",
        "-f", "hevc", "-",
    }

    audioString := []string{
        "ffmpeg",
        "-f", "dshow",
        "-channels", "2",
        "-sample_rate", "48000",
        "-sample_size", "16",
        "-audio_buffer_size", "15",
        "-i", "audio=RD Audio (High Definition Audio Device)",
        "-acodec", "libopus",
        "-mapping_family", "0",
        "-b:a", "25K",
        "-map", "0",
        "-f", "data", "-",
    }

    connected := false

    http.HandleFunc("/", func(writer http.ResponseWriter, request *http.Request) {
        session := request.Body.(*webtransport.Session)

        session.AcceptSession()
        fmt.Println("\nAccepted incoming WebTransport connection.")
        fmt.Println("Awaiting authentication...")

        authData, err := session.ReceiveMessage(session.Context()) // Waits here till first datagram
        if err != nil {                                            // if client closes connection before sending anything
            fmt.Println("\nConnection closed:", err)
            return
        }

        if len(authData) >= 2 && bytes.Equal(authData[2:], password) {
            if connected {
                session.CloseSession()
                warn("Client has authenticated, but a session is already taking place! Connection closed.")
                return
            } else {
                connected = true
                fmt.Println("Client has authenticated!\n")
            }
        } else {
            session.CloseSession()
            warn("Client has failed authentication! Connection closed. (" + string(authData[2:]) + ")")
            return
        }

        videoStream, _ := session.OpenUniStreamSync(session.Context())

        videoCmd := exec.Command(videoString[0], videoString[1:]...)
        go func() {
            videoOut, _ := videoCmd.StdoutPipe()
            videoCmd.Start()

            buffer := make([]byte, 15000)
            for {
                len, err := videoOut.Read(buffer)
                if err != nil {
                    break
                }
                if len > 0 {
                    videoStream.Write(buffer[:len])
                }
            }
        }()

        time.Sleep(50 * time.Millisecond)

        audioStream, err := session.OpenUniStreamSync(session.Context())

        audioCmd := exec.Command(audioString[0], audioString[1:]...)
        go func() {
            audioOut, _ := audioCmd.StdoutPipe()
            audioCmd.Start()

            buffer := make([]byte, 15000)
            for {
                len, err := audioOut.Read(buffer)
                if err != nil {
                    break
                }
                if len > 0 {
                    audioStream.Write(buffer[:len])
                }
            }
        }()

        for {
            data, err := session.ReceiveMessage(session.Context())
            if err != nil {
                videoCmd.Process.Kill()
                audioCmd.Process.Kill()

                connected = false

                fmt.Println("\nConnection closed:", err)
                break
            }

            if len(data) == 0 {

            } else if data[0] == byte(0) {
                fmt.Printf("Received mouse datagram: %s\n", data)
            }
        }

    })

    server := &webtransport.Server{
        ListenAddr: ":1024",
        TLSCert:    webtransport.CertFile{Path: "SSL/fullchain.pem"},
        TLSKey:     webtransport.CertFile{Path: "SSL/privkey.pem"},
        QuicConfig: &webtransport.QuicConfig{
            KeepAlive:      false,
            MaxIdleTimeout: 3 * time.Second,
        },
    }

    fmt.Println("Launching WebTransport server at", server.ListenAddr)
    ctx, cancel := context.WithCancel(context.Background())
    if err := server.Run(ctx); err != nil {
        log.Fatal(err)
        cancel()
    }

}