Recherche avancée

Sur d’autres sites (8326)

• ffmpeg - spark - azure databricks - error writing trailer of "filename.mp3" : Operation not supported

  4 juillet 2021, par CRAFTY DBA

  I have been trying to figure out this tough problem.

  


  I am trying to convert *.mp4 files to *.mp3 files.

  


  I tried using MoviePy but I found out that is uses ffmpeg and was having the same issue.

  


  I used these two articles to get the latest version of ffmpeg installed on the Azure Databricks Cluster during startup. I am using a single node cluster for this POC code.

  


  Pyspark : Use ffmpeg on the driver and workers
  
https://ubuntuhandbook.org/index.php/2020/06/install-ffmpeg-4-3-via-ppa-ubuntu-18-04-16-04
  

  


  The issue is that even the simplest command results in errors.

  


  %%bash ffmpeg -i /dbfs/Craftydba/recording.mp4 /dbfs/Craftydba/recording.mp3

  


  I even tried .wav as an output format and still the same issue.

  


  I retested this command on a Data Science VM in Azure with Python and FFMPEG. It works fine on that OS/Build.

  


  It has something to do with the version of the code on the spark cluster.

  


  Any help is appreciated.

  


  Sincerely

  


  John Miner

  


  PS : I am add a dump of the OS version as well as a ffmpeg error.

  


  Os Version Dump

  


  NAME="Ubuntu"
VERSION="18.04.5 LTS (Bionic Beaver)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 18.04.5 LTS"
VERSION_ID="18.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=bionic
UBUNTU_CODENAME=bionic

  


  FFMPEG Dump

  


  ffmpeg version 4.3.2-0york0~18.04 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: --prefix=/usr --extra-version='0york0~18.04' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libzimg --enable-pocketsphinx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/dbfs/Craftydba/recording.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2
    creation_time   : 2021-06-18T19:07:17.000000Z
  Duration: 00:04:48.64, start: 0.000000, bitrate: 1065 kb/s
    Stream #0:0(eng): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p, 1920x1080, 1000 kb/s, 7.96 fps, 8 tbr, 10k tbn, 20k tbc (default)
    Metadata:
      creation_time   : 2021-06-18T19:07:17.000000Z
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 16000 Hz, mono, fltp, 63 kb/s (default)
    Metadata:
      creation_time   : 2021-06-18T19:07:17.000000Z
Stream mapping:
  Stream #0:1 -> #0:0 (aac (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
Output #0, mp3, to '/dbfs/Craftydba/recording.mp3':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2
    TSSE            : Lavf58.45.100
    Stream #0:0(eng): Audio: mp3 (libmp3lame), 16000 Hz, mono, fltp (default)
    Metadata:
      creation_time   : 2021-06-18T19:07:17.000000Z
      encoder         : Lavc58.91.100 libmp3lame
Error writing trailer of /dbfs/Craftydba/recording.mp3: Operation not supported
size=     846kB time=00:04:48.65 bitrate=  24.0kbits/s speed=86.9x    
video:0kB audio:846kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.036945%


  


• "connection reset by peer" error when streaming from ffmpeg to ffserver

  7 juillet 2021, par g Kishore

  I'm trying to stream a static video file using ffmpeg to ffserver in androidv7.1.2 embedded board.

  


  

  1. Started ffserver with command "ffserver -d /etc/ffserver.conf &"

  


  2. ffmpeg command used to stream :
ffmpeg -i ./sample_960x400_ocean_with_audio.3gp -f flv http://192.168.47.174:8090/feed1.ffm

  


  


  ffserver.conf file content

  


  HTTPPort 8090&#xA;HTTPBindAddress 192.168.47.174&#xA;MaxHTTPConnections 2000&#xA;MaxClients 1000&#xA;MaxBandwidth 100000&#xA;&#xA;#NoDaemon&#xA;#UseDefaults&#xA;#NoDefaults&#xA;&#xA;<feed>&#xA;File /data/local/tmp/feed1.ffm&#xA;FileMaxSize 5M&#xA;</feed>&#xA;&#xA;<stream>&#xA;Feed feed1.ffm&#xA;Format flv&#xA;&#xA;VideoCodec libx264&#xA;VideoFrameRate 24&#xA;VideoBufferSize 80000&#xA;VideoBitRate 512&#xA;VideoQMin 1&#xA;VideoQMax 5&#xA;VideoSize 960x418&#xA;PreRoll 0&#xA;Noaudio&#xA;</stream>&#xA;

  


  Error :

  


  Thu Jan  1 00:20:04 2015 192.168.47.174 - - [POST] "/feed1.ffm HTTP/1.1" 200 415
av_interleaved_write_frame(): Connection reset by peer
    Last message repeated 1 times
[flv @ 0x41be1d40] Failed to update header with correct duration.
[flv @ 0x41be1d40] Failed to update header with correct filesize.
Error writing trailer of http://192.168.47.174:8090/feed1.ffm: Connection reset by peer
frame=    1 fps=0.0 q=1.6 Lsize=       0kB time=00:00:00.09 bitrate=  33.0kbits/s speed=1.68x    
video:10kB audio:2kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!


  


  Any help is greatly appreciated.

  


• FFMPEG using AV_PIX_FMT_D3D11 gives "Error registering the input resource" from NVENC

  13 novembre 2024, par nbabcock

  Input frames start on the GPU as ID3D11Texture2D pointers.

  


  I encode them to H264 using FFMPEG + NVENC. NVENC works perfectly if I download the textures to CPU memory as format AV_PIX_FMT_BGR0, but I'd like to cut out the CPU texture download entirely, and pass the GPU memory pointer directly into the encoder in native format. I write frames like this :

  


  int write_gpu_video_frame(ID3D11Texture2D* gpuTex, AVFormatContext* oc, OutputStream* ost) {
    AVFrame *hw_frame = ost->hw_frame;

    printf("gpuTex address = 0x%x\n", &gpuTex);

    hw_frame->data[0] = (uint8_t *) gpuTex;
    hw_frame->data[1] = (uint8_t *) (intptr_t) 0;
    hw_frame->pts     = ost->next_pts++;

    return write_frame(oc, ost->enc, ost->st, hw_frame);
    // write_frame is identical to sample code in ffmpeg repo
}


  


  Running the code with this modification gives the following error :

  


  gpuTex address = 0x4582f6d0
[h264_nvenc @ 00000191233e1bc0] Error registering an input resource: invalid call (9):
[h264_nvenc @ 00000191233e1bc0] Could not register an input HW frame
Error sending a frame to the encoder: Unknown error occurred


  


  ---

  


  Here's some supplemental code used in setting up and configuring the hw context and encoder :

  


  /* A few config flags */
#define ENABLE_NVENC TRUE
#define USE_D3D11 TRUE // Skip downloading textures to CPU memory and send it straight to NVENC


  


  /* Init hardware frame context */
static int set_hwframe_ctx(AVCodecContext* ctx, AVBufferRef* hw_device_ctx) {
    AVBufferRef*       hw_frames_ref;
    AVHWFramesContext* frames_ctx = NULL;
    int                err        = 0;

    if (!(hw_frames_ref = av_hwframe_ctx_alloc(hw_device_ctx))) {
        fprintf(stderr, "Failed to create HW frame context.\n");
        throw;
    }
    frames_ctx                    = (AVHWFramesContext*) (hw_frames_ref->data);
    frames_ctx->format            = AV_PIX_FMT_D3D11;
    frames_ctx->sw_format         = AV_PIX_FMT_NV12;
    frames_ctx->width             = STREAM_WIDTH;
    frames_ctx->height            = STREAM_HEIGHT;
    //frames_ctx->initial_pool_size = 20;
    if ((err = av_hwframe_ctx_init(hw_frames_ref)) < 0) {
        fprintf(stderr, "Failed to initialize hw frame context. Error code: %s\n", av_err2str(err));
        av_buffer_unref(&hw_frames_ref);
        throw;
    }
    ctx->hw_frames_ctx = av_buffer_ref(hw_frames_ref);
    if (!ctx->hw_frames_ctx)
        err = AVERROR(ENOMEM);

    av_buffer_unref(&hw_frames_ref);
    return err;
}


  


  /* Add an output stream. */
static void add_video_stream(
    OutputStream* ost,
    AVFormatContext* oc,
    const AVCodec** codec,
    enum AVCodecID  codec_id,
    int width,
    int height
) {
    AVCodecContext* c;
    int             i;
    bool            nvenc = false;

    /* find the encoder */
    if (ENABLE_NVENC) {
        printf("Getting nvenc encoder\n");
        *codec = avcodec_find_encoder_by_name("h264_nvenc");
        nvenc  = true;
    }
    
    if (!ENABLE_NVENC || *codec == NULL) {
        printf("Getting standard encoder\n");
        avcodec_find_encoder(codec_id);
        nvenc = false;
    }
    if (!(*codec)) {
        fprintf(stderr, "Could not find encoder for '%s'\n",
                avcodec_get_name(codec_id));
        exit(1);
    }

    ost->st = avformat_new_stream(oc, NULL);
    if (!ost->st) {
        fprintf(stderr, "Could not allocate stream\n");
        exit(1);
    }
    ost->st->id = oc->nb_streams - 1;
    c           = avcodec_alloc_context3(*codec);
    if (!c) {
        fprintf(stderr, "Could not alloc an encoding context\n");
        exit(1);
    }
    ost->enc = c;

    printf("Using video codec %s\n", avcodec_get_name(codec_id));

    c->codec_id = codec_id;
    c->bit_rate = 4000000;
    /* Resolution must be a multiple of two. */
    c->width  = STREAM_WIDTH;
    c->height = STREAM_HEIGHT;
    /* timebase: This is the fundamental unit of time (in seconds) in terms
        * of which frame timestamps are represented. For fixed-fps content,
        * timebase should be 1/framerate and timestamp increments should be
        * identical to 1. */
    ost->st->time_base = {1, STREAM_FRAME_RATE};
    c->time_base       = ost->st->time_base;
    c->gop_size = 12; /* emit one intra frame every twelve frames at most */

    if (nvenc && USE_D3D11) {
        const std::string hw_device_name = "d3d11va";
        AVHWDeviceType    device_type    = av_hwdevice_find_type_by_name(hw_device_name.c_str());

        // set up hw device context
        AVBufferRef *hw_device_ctx;
        // const char*  device = "0"; // Default GPU (may be integrated in the case of switchable graphics!)
        const char*  device = "1";
        ret = av_hwdevice_ctx_create(&hw_device_ctx, device_type, device, nullptr, 0);

        if (ret < 0) {
            fprintf(stderr, "Could not create hwdevice context; %s", av_err2str(ret));
        }

        set_hwframe_ctx(c, hw_device_ctx);
        c->pix_fmt = AV_PIX_FMT_D3D11;
    } else if (nvenc && !USE_D3D11)
        c->pix_fmt = AV_PIX_FMT_BGR0;
    else
        c->pix_fmt = STREAM_PIX_FMT;

    if (c->codec_id == AV_CODEC_ID_MPEG2VIDEO) {
        /* just for testing, we also add B-frames */
        c->max_b_frames = 2;
    }

    if (c->codec_id == AV_CODEC_ID_MPEG1VIDEO) {
        /* Needed to avoid using macroblocks in which some coeffs overflow.
            * This does not happen with normal video, it just happens here as
            * the motion of the chroma plane does not match the luma plane. */
        c->mb_decision = 2;
    }

    /* Some formats want stream headers to be separate. */
    if (oc->oformat->flags & AVFMT_GLOBALHEADER)
        c->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
}