Sur d’autres sites (12973)

• Kivy Crashes when Loading Multiple Audio Files

  26 mars 2022, par Omid Ketabollahi

  I have a music folder and in it I have a script that loads all the files using SoundLoader and assigns them to a variable. I have 9 audio files that I load in the script but it seems like that kivy cannot handle loading many audio files ? When I run my main script I get this error and the program crashes :

  


  Assertion 'pa_atomic_load(&(c)->_ref) >= 1' failed at pulse/context.c:1056, function pa_context_disconnect(). Aborting.


  


  I don't think that this is an issue with the audio files ( I may be wrong ) because I tried loading other files instead of the original ones that I used and I still got this error.

  


  In case you are wondering, this is the code :

  


  import os.path
from typing import Final
from constants import MUSIC_DIRECTORY
from kivy.core.audio import SoundLoader

__all__ = (
    "A_DARK_TIME",
    "BLOOD_PUMPING",
    "BROKEN_VILLAGE",
    "DRUM_COMMERCIAL_B",
    "SUPER_SUSPENSE",
    "THE_HERO_WE_NEED",
    "TROUBLE_IS_BREWING",
    "VIKING_FEAST",
    "WARRIOR_GODS",
)


A_DARK_TIME: Final = SoundLoader.load(
    os.path.join(MUSIC_DIRECTORY, "A-Dark-Time-David-Fesliyan.mp3")
)
BLOOD_PUMPING: Final = SoundLoader.load(
    os.path.join(MUSIC_DIRECTORY, "Blood-Pumping-David-Fesliyan.mp3")
)
BROKEN_VILLAGE: Final = SoundLoader.load(
    os.path.join(MUSIC_DIRECTORY, "Broken-Village-David_Fesliyan.mp3")
)
DRUM_COMMERCIAL_B: Final = SoundLoader.load(
    os.path.join(MUSIC_DIRECTORY, "Drum-Commercial-B-David-Fesliyan.mp3")
)
SUPER_SUSPENSE: Final = SoundLoader.load(
    os.path.join(MUSIC_DIRECTORY, "Super-Suspense-David-Fesliyan.mp3")
)
THE_HERO_WE_NEED: Final = SoundLoader.load(
    os.path.join(MUSIC_DIRECTORY, "The-Hero-We-Need-Steve-Oxen.mp3")
)
TROUBLE_IS_BREWING: Final = SoundLoader.load(
    os.path.join(MUSIC_DIRECTORY, "Trouble-Is-Brewing-David-Fesliyan.mp3")
)
VIKING_FEAST: Final = SoundLoader.load(
    os.path.join(MUSIC_DIRECTORY, "Viking-Feast-David-Fesliyan.mp3")
)
WARRIOR_GODS: Final = SoundLoader.load(
    os.path.join(MUSIC_DIRECTORY, "Warrior-Gods-Steve-Oxen.mp3")
)


  


• ffmpeg and streaming video - frame time issue

  6 mai 2014, par 4ntoine

  I’ve compiled ffmpeg-android (https://github.com/appunite/AndroidFFmpeg) and it works for files.
  The problem is that it shows nothing for network streams (both rtmp and hls) as frame timestamp is too big and it seems to be not from video beginning (as for file). Then it waits for the first frame time which for network stream is too big :

  player_decode_video copying...
  
  05-05 18:11:26.994: INFO/player.c(16998): player_decode_video Decoded video frame: 568.233000, time_base: 51140970
  
  05-05 18:11:26.994: INFO/player.c(16998): player_wait_for_frame[0] start
  
  ...
  
  05-05 18:11:30.587: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (0.000977)
  
  05-05 18:11:30.587: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 568232023
  
  05-05 18:11:31.088: INFO/player.c(16998): player_wait_for_frame[0] timeout
  
  05-05 18:11:31.088: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (0.501542)
  
  05-05 18:11:31.088: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 567731458
  
  05-05 18:11:31.588: INFO/player.c(16998): player_wait_for_frame[0] timeout
  
  05-05 18:11:31.588: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (1.002778)
  
  05-05 18:11:31.588: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 567230222
  
  05-05 18:11:32.089: INFO/player.c(16998): player_wait_for_frame[0] timeout
  
  05-05 18:11:32.089: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (1.504563)
  
  05-05 18:11:32.089: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 566728437
  
  05-05 18:11:32.590: INFO/player.c(16998): player_wait_for_frame[0] timeout
  
  05-05 18:11:32.590: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (568.233000) - (2.005708)
  
  05-05 18:11:32.590: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: 566227292

  also i can’t figure out what timestamp for frame for network stream means (for file frames it’s a time from video beginning)

  for file it’s clear : time_base is frame number and timestamp is time from video beginning and it plays the files :

  player_decode_video Decoded video frame: 0.320000, time_base: 8
  
  05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0] start
  
  05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.320000) - (0.344337)
  
  05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -24337
  
  05-05 18:32:42.344: INFO/player.c(16998): player_wait_for_frame[0] finish[0]
  
  05-05 18:32:42.344: INFO/player.c(16998): player_update_time: 0.346169/4.000000
  
  05-05 18:32:42.344: INFO/player.c(16998): player_decode waiting for frame[0]
  
  05-05 18:32:42.344: INFO/player.c(16998): player_decode decoding frame[0]
  
  05-05 18:32:42.344: INFO/player.c(16998): player_decode_video decoding
  
  05-05 18:32:42.344: INFO/player.c(16998): player_decode_video copy wait
  
  05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream Read frame
  
  05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream looking for stream
  
  05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream stream found [0]
  
  05-05 18:32:42.344: INFO/player.c(16998): player_read_from_stream waiting for queue
  
  05-05 18:32:42.344: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888
  
  05-05 18:32:42.344: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280
  
  05-05 18:32:42.344: INFO/player.c(16998): player_decode_video copying...
  
  05-05 18:32:42.425: INFO/player.c(16998): player_decode_video Decoded video frame: 0.360000, time_base: 9
  
  05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0] start
  
  05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.360000) - (0.427994)
  
  05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -67994
  
  05-05 18:32:42.425: INFO/player.c(16998): player_wait_for_frame[0] finish[0]
  
  05-05 18:32:42.425: INFO/player.c(16998): player_update_time: 0.429214/4.000000
  
  05-05 18:32:42.425: INFO/player.c(16998): player_decode waiting for frame[0]
  
  05-05 18:32:42.425: INFO/player.c(16998): player_decode decoding frame[0]
  
  05-05 18:32:42.425: INFO/player.c(16998): player_decode_video decoding
  
  05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream Read frame
  
  05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream looking for stream
  
  05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream stream found [0]
  
  05-05 18:32:42.425: INFO/player.c(16998): player_read_from_stream waiting for queue
  
  05-05 18:32:42.425: INFO/player.c(16998): player_decode_video copy wait
  
  05-05 18:32:42.435: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888
  
  05-05 18:32:42.435: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280
  
  05-05 18:32:42.435: INFO/player.c(16998): player_decode_video copying...
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode_video Decoded video frame: 0.400000, time_base: 10
  
  05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0] start
  
  05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.400000) - (0.494742)
  
  05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -94742
  
  05-05 18:32:42.495: INFO/player.c(16998): player_wait_for_frame[0] finish[0]
  
  05-05 18:32:42.495: INFO/player.c(16998): player_update_time: 0.495993/4.000000
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode waiting for frame[0]
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode decoding frame[0]
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode_video decoding
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode_video copy wait
  
  05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream Read frame
  
  05-05 18:32:42.495: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888
  
  05-05 18:32:42.495: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280
  
  05-05 18:32:42.495: INFO/player.c(16998): player_decode_video copying...
  
  05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream looking for stream
  
  05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream stream found [0]
  
  05-05 18:32:42.495: INFO/player.c(16998): player_read_from_stream waiting for queue
  
  05-05 18:32:42.555: INFO/player.c(16998): player_decode_video Decoded video frame: 0.440000, time_base: 11
  
  05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0] start
  
  05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0 = Video] = (0.440000) - (0.556698)
  
  05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0] Waiting for frame: sleeping: -116698
  
  05-05 18:32:42.555: INFO/player.c(16998): player_wait_for_frame[0] finish[0]
  
  05-05 18:32:42.555: INFO/player.c(16998): player_update_time: 0.557858/4.000000
  
  05-05 18:32:42.555: INFO/player.c(16998): player_decode waiting for frame[0]
  
  05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream Read frame
  
  05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream looking for stream
  
  05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream stream found [0]
  
  05-05 18:32:42.555: INFO/player.c(16998): player_read_from_stream waiting for queue
  
  05-05 18:32:42.555: INFO/player.c(16998): player_decode decoding frame[0]
  
  05-05 18:32:42.555: INFO/player.c(16998): player_decode_video decoding
  
  05-05 18:32:42.555: INFO/player.c(16998): player_decode_video copy wait
  
  05-05 18:32:42.565: INFO/player.c(16998): Format: WINDOW_FORMAT_RGBA_8888
  
  05-05 18:32:42.565: INFO/player.c(16998): Buffer: width: 1280, height: 720, stride: 1280
  
  05-05 18:32:42.565: INFO/player.c(16998): player_decode_video copying...
  
  05-05 18:32:42.625: INFO/player.c(16998): player_decode_video Decoded video frame: 0.480000, time_base: 12

  So what is the meaning of timestamp for stream and what should i change to make it playing streams (rtmp/hls - its can be opened now but it wait for frame time which is far in the future) ?

  UPDATE :

      int64_t pts = av_frame_get_best_effort_timestamp(frame);
  
  if (pts == AV_NOPTS_VALUE) {
  
      pts = 0;
  
  }
  
  int64_t time = av_rescale_q(pts, stream->time_base, AV_TIME_BASE_Q);
  
  LOGI(10,
  
          "player_decode_video Decoded video frame: %f, time_base: %" SCNd64,
  
          time/1000000.0, pts);
  
  player_wait_for_frame(player, time, stream_no);

  it extracts time which is far in future :

  player_wait_for_frame[0 = Video] = (568.233000) - (0.000977)

  Is it correct ?

• Why my code that based on ffmpeg can't sync video' time and audio's time ?

  6 juillet 2021, par ZSpirytus

  Background

  


  Recently, I use ffmpeg to write my first Android video player. But video channel's time is faster than audio channel's time about 2 3 times.

  


  Code

  


  In short, I use PacketDispatcher to read AVPacket from http hlv source :

  


  void PacketDispatcher::RealDispatch() {
    while (GetStatus() != DISPATCHER_STOP) {
        while (GetStatus() == DISPATCHER_PAUSE) {
            LOGD(TAG, "wait signal");
            pthread_mutex_lock(&mutex);
            pthread_cond_wait(&cond, &mutex);
            pthread_mutex_unlock(&mutex);
        }

        AVPacket *av_packet = av_packet_alloc();
        int ret = av_read_frame(av_format_context, av_packet);
        if (ret) {
            LOGE(TAG, "av_read_frame ret=%d", ret);
            break;
        }

        // PacketDispatcher is who read the AVPacket from http hlv source 
        // and dispatch to decoder by its stream index.
        decoder_map[av_packet->stream_index]->Push(av_packet);
    }
}


  


  And then, Decoder written by Producer-Consumer Pattern, Decoder maintain a queue that store all the AVPacket received from PacketDispatcher. The code like this :

  


  // write to the queue
void BaseDecoder::Push(AVPacket *av_packet) {
    pthread_mutex_lock(&av_packet_queue_mutex);
    av_packet_queue.push(av_packet);
    pthread_cond_signal(&av_packet_queue_cond);
    pthread_mutex_unlock(&av_packet_queue_mutex);
}

// real decode logic
void BaseDecoder::RealDecode() {
    SetDecoderState(START);
    LOGI(LogSpec(), "start decode");

    while (true) {
        // 1. check decoder status and queue size to decide if call thread.wait

        // 2. send packet to codec
        AVPacket* av_packet = av_packet_queue.front();
        int ret = avcodec_send_packet(av_codec_ctx, av_packet);

        // 3. read frame from codec
        AVFrame *av_frame = av_frame_alloc();
        ret = avcodec_receive_frame(av_codec_ctx, av_frame);

        if (m_render) {
            // 3. custom decode logic overrided by child class
            void *decode_result = DecodeFrame(av_frame);
            if (decode_result) {
                // 4. dispatch to render
                m_render->Render(decode_result);
            } else {
                LOGD("BaseDecoder", "decode_result=nullptr");
            }
        }
    }
}


  


  Finally, I do rendering logic in Render. Render also written by Producer-Consumer Pattern, it maintain a queue that store AVFrame received from Decoder, the code like this :

  


  // write AVFrame
void BaseRender::Render(void *frame_data) {
    Lock();
    frame_queue.push(frame_data);
    Signal();
    UnLock();
}

// render to surface or Open SL
void BaseRender::RealRender() {
    // frame data that contain frame pts and other metadata
    frame_data->pts = av_frame->pts = av_frame->best_effort_timestamp * av_q2d(GetTimeBase());
    // video only
    frame_data->video_extra_delay = av_frame->repeat_pict * 1.0 / fps * 2.0;
    if (m_render_synchronizer && m_render_synchronizer->Sync(frame_data)) {
        continue;
    }
}


  


  And then, the synchronizer will decide to sleep time or drop video frame according to the frame pts, frame pts is :

  


  frame_data->pts = av_frame->best_effort_timestamp * av_q2d(GetTimeBase());


  


  Also, video extra delay is :

  


  frame_data->video_extra_delay = av_frame->repeat_pict * 1.0 / fps * 2.0;


  


  RenderSynchronizer code like this :

  


  bool RenderSynchronizer::Sync(void *frame_data) {&#xA;    auto base_frame_data = static_cast<baseframedata>(frame_data);&#xA;    if (base_frame_data->media_type == AVMEDIA_TYPE_AUDIO) {&#xA;        return ReceiveAudioFrame(static_cast<pcmdata>(frame_data));&#xA;    } else if (base_frame_data->media_type == AVMEDIA_TYPE_VIDEO) {&#xA;        return ReceiveVideoFrame(static_cast<rgbadata>(frame_data));&#xA;    }&#xA;    return false;&#xA;}&#xA;&#xA;bool RenderSynchronizer::ReceiveAudioFrame(PCMData *pcm_data) {&#xA;    audio_pts = pcm_data->pts;&#xA;    return false;&#xA;}&#xA;&#xA;bool RenderSynchronizer::ReceiveVideoFrame(RGBAData *rgba_data) {&#xA;    video_pts = rgba_data->pts;&#xA;&#xA;    if (audio_pts &lt;= 0 || video_pts &lt;= 0) {&#xA;        return false;&#xA;    }&#xA;&#xA;    double diff = video_pts - audio_pts;&#xA;    if (diff > 0) {&#xA;        if (diff > 1) {&#xA;            av_usleep((unsigned int) (rgba_data->extra_delay * 1000000.0));&#xA;        } else {&#xA;            av_usleep((unsigned int) ((diff &#x2B; rgba_data->extra_delay) * 1000000.0));&#xA;        }&#xA;        return false;&#xA;    } else if (diff &lt; 0) {&#xA;        LOGD(TAG, "drop video frame");&#xA;        return true;&#xA;    } else {&#xA;        return false;&#xA;    }&#xA;}&#xA;</rgbadata></pcmdata></baseframedata>

  


  Why my code can not sync video time and audio time ? Thanks for your reading and answers.