Recherche avancée

Sur d’autres sites (6619)

• FFMPEG and DirectX Capture in C++

  13 décembre 2016, par tankyx

  I have a system that allows me to capture a window and save it as a mp4, using ffmpeg. I use gdigrab to capture the frame, but it is fairly slow (60ms per av_read_frame calls)

  I know I can capture a game using the DirectX API, but I don’t know how to convert the resulting BMP to an AVFrame.

  The following code is the DirectX code I use to capture the frame

  extern void* pBits;
  
  extern IDirect3DDevice9* g_pd3dDevice;
  
  IDirect3DSurface9* pSurface;
  
  g_pd3dDevice->CreateOffscreenPlainSurface(ScreenWidth, ScreenHeight,
  
                                        D3DFMT_A8R8G8B8, D3DPOOL_SCRATCH, 
  
                                        &pSurface, NULL);
  
  g_pd3dDevice->GetFrontBufferData(0, pSurface);
  
  D3DLOCKED_RECT lockedRect;
  
  pSurface->LockRect(&lockedRect,NULL,
  
                 D3DLOCK_NO_DIRTY_UPDATE|
  
                 D3DLOCK_NOSYSLOCK|D3DLOCK_READONLY)));
  
  for( int i=0 ; i < ScreenHeight ; i++)
  
  {
  
      memcpy( (BYTE*) pBits + i * ScreenWidth * BITSPERPIXEL / 8 , 
  
           (BYTE*) lockedRect.pBits + i* lockedRect.Pitch , 
  
           ScreenWidth * BITSPERPIXEL / 8);
  
  }
  
  g_pSurface->UnlockRect();
  
  pSurface->Release();

  And here is my read loop :

  {
  
      while (1) {
  
      if (av_read_frame(pFormatCtx, &packet) < 0 || exit)
  
          break;
  
      if (packet.stream_index == videoindex) {
  
          // Decode video frame
  
          av_packet_rescale_ts(&packet, { 1, std::stoi(pParser->GetVal("video-fps")) }, pCodecCtx->time_base);
  
          avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packet);
  
  
  
          if (frameFinished) {
  
              pFrame->pts = i;
  
              i++;
  
              sws_scale(sws_ctx, (uint8_t const * const *)pFrame->data, pFrame->linesize, 0, pCodecCtx->height, pFrameRGB->data, pFrameRGB->linesize);
  
              pFrameRGB->pts = pFrame->pts;
  
              enc.encodeFrame(pFrameRGB);
  
  
  
      }
  
      // Free the packet that was allocated by av_read_frame
  
      av_free_packet(&packet);
  
  }

  How can I create an AVFrame using the bmp I have, without using the av_read_frame ?

• FFmpeg : Decoding AVPackets received from UDP socket

  8 août 2022, par strong_kobayashi

  I am trying to stream video frames encoded with FFmpeg from an Unity game to a client UI via UDP. To be specific, I am sending AVPackets (which contain the compressed frames, as far as I understand) from the server, which are then, on the client side, extracted as follows :

  



  inline UDPpacket* SDLGameClient::receiveData()
{
    if(SDLNet_UDP_Recv(socket, packet))
        return packet;
    else
        return NULL;
}

int main()
{
    ...
    // Initialize UDP
    ...
    UDPpacket *udpPacket;

    int i = 0;

    for(;;)
    {
        udpPacket = client->receiveData();

        if(udpPacket != nullptr)
        {
            i++;
            std::cout << "Packet " << i << " received!" << std::endl;

            AVPacket packet;
            av_init_packet(&packet);

            packet.data = new uint8_t[udpPacket->len];
            memcpy(packet.data, udpPacket->data, udpPacket->len);
            packet.size = udpPacket->len;

            ...


  



  To realize networking, I use the SDL_net library. Fragmenting, sending and receiving the packets seem to work without any problem. My question is, how do I decode the incoming AVPackets and stream the frames recorded in Unity to the client ?
I am aware that I need to use avcodec_send_packet and avcodec_receive_frame for decoding (since avcodec_decode_video2, that is used in many example decoding code, is deprecated), but when I do something like this :

  



  int ret = avcodec_send_packet(codecContext, &packet);
if(ret < 0 || ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
    std::cout << "avcodec_send_packet: " << ret << std::endl;
else
{
    while(ret >= 0)
    {
        ret = avcodec_receive_frame(codecContext, frame);
        if(ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
            std::cout << "avcodec_receive_frame: " << ret << std::endl;

        std::cout << "Frame: " << codecContext->frame_number << std::endl;
    }
}

av_packet_unref(packet);


  



  ret always returns a negative value (-22), so perhaps something is wrong with the AVPackets, or I'm sending the frames way too fast, I really have no clue :/

  



  Thanks in advance for any help !

  


• Converting mp3 files to ogg vorbis while retaining cover art using FFmpeg

  22 juin 2017, par refi64

  Title says it all. I’ve tried :

  ffmpeg -i 01_FFXV_OST.mp3 -c:a libvorbis 01_FFXV_OST.ogg

  but I get :

  ffmpeg version git-2017-01-22-f1214ad Copyright (c) 2000-2017 the FFmpeg developers
  
    built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  
    configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --mandir=/usr/share/man --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libfreetype --enable-gnutls --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvidstab --enable-libwavpack --enable-nvenc
  
    libavutil      55. 44.100 / 55. 44.100
  
    libavcodec     57. 75.100 / 57. 75.100
  
    libavformat    57. 63.100 / 57. 63.100
  
    libavdevice    57.  2.100 / 57.  2.100
  
    libavfilter     6. 69.100 /  6. 69.100
  
    libavresample   3.  2.  0 /  3.  2.  0
  
    libswscale      4.  3.101 /  4.  3.101
  
    libswresample   2.  4.100 /  2.  4.100
  
    libpostproc    54.  2.100 / 54.  2.100
  
  Input #0, mp3, from '01_FFXV_OST.mp3':
  
    Metadata:
  
      album           : FINAL FANTASY XV Original Soundtrack
  
      artist          : Yoko Shimomura
  
      album_artist    : SQUARE ENIX MUSIC
  
      composer        : Yoko Shimomura
  
      disc            : 1
  
      genre           : Game
  
      title           : Somnus (Instrumental Version)
  
      track           : 01
  
      date            : 2016
  
    Duration: 00:02:29.47, start: 0.023021, bitrate: 306 kb/s
  
      Stream #0:0: Audio: mp3, 48000 Hz, stereo, s16p, 301 kb/s
  
      Metadata:
  
        encoder         : LAME3.98r
  
      Stream #0:1: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 500x500 [SAR 100:100 DAR 1:1], 90k tbr, 90k tbn, 90k tbc
  
      Metadata:
  
        comment         : Cover (front)
  
  Automatic encoder selection failed for output stream #0:0. Default encoder for format ogg (codec theora) is probably disabled. Please choose an encoder manually.
  
  Error selecting an encoder for stream 0:0

  Adding on -c:v copy does...nothing :

  ffmpeg version git-2017-01-22-f1214ad Copyright (c) 2000-2017 the FFmpeg developers
  
    built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  
    configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --mandir=/usr/share/man --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libfreetype --enable-gnutls --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvidstab --enable-libwavpack --enable-nvenc
  
    libavutil      55. 44.100 / 55. 44.100
  
    libavcodec     57. 75.100 / 57. 75.100
  
    libavformat    57. 63.100 / 57. 63.100
  
    libavdevice    57.  2.100 / 57.  2.100
  
    libavfilter     6. 69.100 /  6. 69.100
  
    libavresample   3.  2.  0 /  3.  2.  0
  
    libswscale      4.  3.101 /  4.  3.101
  
    libswresample   2.  4.100 /  2.  4.100
  
    libpostproc    54.  2.100 / 54.  2.100
  
  Input #0, mp3, from '01_FFXV_OST.mp3':
  
    Metadata:
  
      album           : FINAL FANTASY XV Original Soundtrack
  
      artist          : Yoko Shimomura
  
      album_artist    : SQUARE ENIX MUSIC
  
      composer        : Yoko Shimomura
  
      disc            : 1
  
      genre           : Game
  
      title           : Somnus (Instrumental Version)
  
      track           : 01
  
      date            : 2016
  
    Duration: 00:02:29.47, start: 0.023021, bitrate: 306 kb/s
  
      Stream #0:0: Audio: mp3, 48000 Hz, stereo, s16p, 301 kb/s
  
      Metadata:
  
        encoder         : LAME3.98r
  
      Stream #0:1: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 500x500 [SAR 100:100 DAR 1:1], 90k tbr, 90k tbn, 90k tbc
  
      Metadata:
  
        comment         : Cover (front)
  
  [ogg @ 0x2ac6e00] Unsupported codec id in stream 0
  
  Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
  
  Stream mapping:
  
    Stream #0:1 -> #0:0 (copy)
  
    Stream #0:0 -> #0:1 (mp3 (native) -> vorbis (libvorbis))
  
      Last message repeated 1 times

  I have absolutely no clue what to do next. I’ve also tried adding -c:v copy to no avail :

  ffmpeg version git-2017-01-22-f1214ad Copyright (c) 2000-2017 the FFmpeg developers
  
    built with gcc 4.8 (Ubuntu 4.8.4-2ubuntu1~14.04.3)
  
    configuration: --extra-libs=-ldl --prefix=/opt/ffmpeg --mandir=/usr/share/man --enable-avresample --disable-debug --enable-nonfree --enable-gpl --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --disable-decoder=amrnb --disable-decoder=amrwb --enable-libpulse --enable-libfreetype --enable-gnutls --enable-libx264 --enable-libx265 --enable-libfdk-aac --enable-libvorbis --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libspeex --enable-libass --enable-avisynth --enable-libsoxr --enable-libxvid --enable-libvidstab --enable-libwavpack --enable-nvenc
  
    libavutil      55. 44.100 / 55. 44.100
  
    libavcodec     57. 75.100 / 57. 75.100
  
    libavformat    57. 63.100 / 57. 63.100
  
    libavdevice    57.  2.100 / 57.  2.100
  
    libavfilter     6. 69.100 /  6. 69.100
  
    libavresample   3.  2.  0 /  3.  2.  0
  
    libswscale      4.  3.101 /  4.  3.101
  
    libswresample   2.  4.100 /  2.  4.100
  
    libpostproc    54.  2.100 / 54.  2.100
  
  Input #0, mp3, from '01_FFXV_OST.mp3':
  
    Metadata:
  
      album           : FINAL FANTASY XV Original Soundtrack
  
      artist          : Yoko Shimomura
  
      album_artist    : SQUARE ENIX MUSIC
  
      composer        : Yoko Shimomura
  
      disc            : 1
  
      genre           : Game
  
      title           : Somnus (Instrumental Version)
  
      track           : 01
  
      date            : 2016
  
    Duration: 00:02:29.47, start: 0.023021, bitrate: 306 kb/s
  
      Stream #0:0: Audio: mp3, 48000 Hz, stereo, s16p, 301 kb/s
  
      Metadata:
  
        encoder         : LAME3.98r
  
      Stream #0:1: Video: mjpeg, yuvj444p(pc, bt470bg/unknown/unknown), 500x500 [SAR 100:100 DAR 1:1], 90k tbr, 90k tbn, 90k tbc
  
      Metadata:
  
        comment         : Cover (front)
  
  [ogg @ 0x2b6ce00] Unsupported codec id in stream 0
  
  Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
  
  Stream mapping:
  
    Stream #0:1 -> #0:0 (copy)
  
    Stream #0:0 -> #0:1 (mp3 (native) -> vorbis (libvorbis))
  
      Last message repeated 1 times

  I get the same results if I change the stream order (by passing -map 0:0 -map 0:1). I’m not sure what to do next...