Sur d’autres sites (7244)

• Why my code that based on ffmpeg can't sync video' time and audio's time ?

  6 juillet 2021, par ZSpirytus

  Background

  


  Recently, I use ffmpeg to write my first Android video player. But video channel's time is faster than audio channel's time about 2 3 times.

  


  Code

  


  In short, I use PacketDispatcher to read AVPacket from http hlv source :

  


  void PacketDispatcher::RealDispatch() {
    while (GetStatus() != DISPATCHER_STOP) {
        while (GetStatus() == DISPATCHER_PAUSE) {
            LOGD(TAG, "wait signal");
            pthread_mutex_lock(&mutex);
            pthread_cond_wait(&cond, &mutex);
            pthread_mutex_unlock(&mutex);
        }

        AVPacket *av_packet = av_packet_alloc();
        int ret = av_read_frame(av_format_context, av_packet);
        if (ret) {
            LOGE(TAG, "av_read_frame ret=%d", ret);
            break;
        }

        // PacketDispatcher is who read the AVPacket from http hlv source 
        // and dispatch to decoder by its stream index.
        decoder_map[av_packet->stream_index]->Push(av_packet);
    }
}


  


  And then, Decoder written by Producer-Consumer Pattern, Decoder maintain a queue that store all the AVPacket received from PacketDispatcher. The code like this :

  


  // write to the queue
void BaseDecoder::Push(AVPacket *av_packet) {
    pthread_mutex_lock(&av_packet_queue_mutex);
    av_packet_queue.push(av_packet);
    pthread_cond_signal(&av_packet_queue_cond);
    pthread_mutex_unlock(&av_packet_queue_mutex);
}

// real decode logic
void BaseDecoder::RealDecode() {
    SetDecoderState(START);
    LOGI(LogSpec(), "start decode");

    while (true) {
        // 1. check decoder status and queue size to decide if call thread.wait

        // 2. send packet to codec
        AVPacket* av_packet = av_packet_queue.front();
        int ret = avcodec_send_packet(av_codec_ctx, av_packet);

        // 3. read frame from codec
        AVFrame *av_frame = av_frame_alloc();
        ret = avcodec_receive_frame(av_codec_ctx, av_frame);

        if (m_render) {
            // 3. custom decode logic overrided by child class
            void *decode_result = DecodeFrame(av_frame);
            if (decode_result) {
                // 4. dispatch to render
                m_render->Render(decode_result);
            } else {
                LOGD("BaseDecoder", "decode_result=nullptr");
            }
        }
    }
}


  


  Finally, I do rendering logic in Render. Render also written by Producer-Consumer Pattern, it maintain a queue that store AVFrame received from Decoder, the code like this :

  


  // write AVFrame
void BaseRender::Render(void *frame_data) {
    Lock();
    frame_queue.push(frame_data);
    Signal();
    UnLock();
}

// render to surface or Open SL
void BaseRender::RealRender() {
    // frame data that contain frame pts and other metadata
    frame_data->pts = av_frame->pts = av_frame->best_effort_timestamp * av_q2d(GetTimeBase());
    // video only
    frame_data->video_extra_delay = av_frame->repeat_pict * 1.0 / fps * 2.0;
    if (m_render_synchronizer && m_render_synchronizer->Sync(frame_data)) {
        continue;
    }
}


  


  And then, the synchronizer will decide to sleep time or drop video frame according to the frame pts, frame pts is :

  


  frame_data->pts = av_frame->best_effort_timestamp * av_q2d(GetTimeBase());


  


  Also, video extra delay is :

  


  frame_data->video_extra_delay = av_frame->repeat_pict * 1.0 / fps * 2.0;


  


  RenderSynchronizer code like this :

  


  bool RenderSynchronizer::Sync(void *frame_data) {&#xA;    auto base_frame_data = static_cast<baseframedata>(frame_data);&#xA;    if (base_frame_data->media_type == AVMEDIA_TYPE_AUDIO) {&#xA;        return ReceiveAudioFrame(static_cast<pcmdata>(frame_data));&#xA;    } else if (base_frame_data->media_type == AVMEDIA_TYPE_VIDEO) {&#xA;        return ReceiveVideoFrame(static_cast<rgbadata>(frame_data));&#xA;    }&#xA;    return false;&#xA;}&#xA;&#xA;bool RenderSynchronizer::ReceiveAudioFrame(PCMData *pcm_data) {&#xA;    audio_pts = pcm_data->pts;&#xA;    return false;&#xA;}&#xA;&#xA;bool RenderSynchronizer::ReceiveVideoFrame(RGBAData *rgba_data) {&#xA;    video_pts = rgba_data->pts;&#xA;&#xA;    if (audio_pts &lt;= 0 || video_pts &lt;= 0) {&#xA;        return false;&#xA;    }&#xA;&#xA;    double diff = video_pts - audio_pts;&#xA;    if (diff > 0) {&#xA;        if (diff > 1) {&#xA;            av_usleep((unsigned int) (rgba_data->extra_delay * 1000000.0));&#xA;        } else {&#xA;            av_usleep((unsigned int) ((diff &#x2B; rgba_data->extra_delay) * 1000000.0));&#xA;        }&#xA;        return false;&#xA;    } else if (diff &lt; 0) {&#xA;        LOGD(TAG, "drop video frame");&#xA;        return true;&#xA;    } else {&#xA;        return false;&#xA;    }&#xA;}&#xA;</rgbadata></pcmdata></baseframedata>

  


  Why my code can not sync video time and audio time ? Thanks for your reading and answers.

  


• react vite ffmpeg how to resolve worker module returning 404

  27 août, par pmkob

  I have imported the js, wasm, and worker modules into my project
https://unpkg.com/@ffmpeg/core@0.12.6/dist/esm/ffmpeg-core.js

  


  https://unpkg.com/@ffmpeg/core@0.12.6/dist/esm/ffmpeg-core.wasm
https://unpkg.com/@ffmpeg/core-mt@0.12.2/dist/esm/ffmpeg-core.worker.js

  


  and upon calling the function that the load is in, i get the error in the worker module
Request URL :
http://localhost:3000/node_modules/.vite/deps/worker.js?worker_file&type=module Status Code :
404 Not Found
Referrer Policy :
strict-origin-when-cross-origin

  


  In order to resolve this issue i have added these to my config and looked at similar issues with different threads https://github.com/ffmpegwasm/ffmpeg.wasm/issues/532
but with no luck. The load also never completes.

  


  CONFIG:
export default defineConfig(({ mode }) => ({
//other config stuff
  plugins: [react()],
  optimizeDeps: {
    exclude: ["@ffmpeg/ffmpeg", "@ffmpeg/util"]
  },
  server: {
    headers: {
      "Cross-Origin-Opener-Policy": "same-origin",
      "Cross-Origin-Embedder-Policy": "require-corp"
    }
  }
}));

FILE:
import { FFmpeg } from "@ffmpeg/ffmpeg";
import { fetchFile, toBlobURL } from "@ffmpeg/util";
 const ffmpeg = new FFmpeg();
    await ffmpeg.load({
      coreURL: await toBlobURL(
        `${importedURL}/ffmpeg/ffmpeg-core.js`,
        "text/javascript"
      ),
      wasmURL: await toBlobURL(
        `${importedURL}/ffmpeg/ffmpeg-core.wasm`,
        "application/wasm"
      ),
      workerURL: await toBlobURL(
        `${importedURL}/ffmpeg/ffmpeg-core.worker.js`,
        "text/javascript"
      )
    });


  


• avisynth : adapt 239d02eff3ffe9f7d40caa21dde50fb4a0e94c24 to dlsym

  30 mars 2019, par Stephen Hutchinson

  avisynth : adapt 239d02eff3ffe9f7d40caa21dde50fb4a0e94c24 to dlsym
  This commit was merged in a couple years ago as a no-op because we
  
  had already switched from GetProcAddress to dlsym some time before
  
  that.  However, not applying the actual cast causes warnings about
  
  FARPROC and when attempting to build FFmpeg in MSVC with AviSynth-GCC
  
  32-bit compatibility, those FARPROC warnings turn into FARPROC errors.
  

  • [DH] libavformat/avisynth.c