Recherche avancée
Médias (39)
-
Stereo master soundtrack
17 octobre 2011, par
Mis à jour : Octobre 2011
Langue : English
Type : Audio
-
ED-ME-5 1-DVD
11 octobre 2011, par
Mis à jour : Octobre 2011
Langue : English
Type : Audio
-
1,000,000
27 septembre 2011, par
Mis à jour : Septembre 2011
Langue : English
Type : Audio
-
Demon Seed
26 septembre 2011, par
Mis à jour : Septembre 2011
Langue : English
Type : Audio
-
The Four of Us are Dying
26 septembre 2011, par
Mis à jour : Septembre 2011
Langue : English
Type : Audio
-
Corona Radiata
26 septembre 2011, par
Mis à jour : Septembre 2011
Langue : English
Type : Audio
Autres articles (42)
-
La file d’attente de SPIPmotion
28 novembre 2010, parUne file d’attente stockée dans la base de donnée
Lors de son installation, SPIPmotion crée une nouvelle table dans la base de donnée intitulée spip_spipmotion_attentes.
Cette nouvelle table est constituée des champs suivants : id_spipmotion_attente, l’identifiant numérique unique de la tâche à traiter ; id_document, l’identifiant numérique du document original à encoder ; id_objet l’identifiant unique de l’objet auquel le document encodé devra être attaché automatiquement ; objet, le type d’objet auquel (...) -
Support de tous types de médias
10 avril 2011Contrairement à beaucoup de logiciels et autres plate-formes modernes de partage de documents, MediaSPIP a l’ambition de gérer un maximum de formats de documents différents qu’ils soient de type : images (png, gif, jpg, bmp et autres...) ; audio (MP3, Ogg, Wav et autres...) ; vidéo (Avi, MP4, Ogv, mpg, mov, wmv et autres...) ; contenu textuel, code ou autres (open office, microsoft office (tableur, présentation), web (html, css), LaTeX, Google Earth) (...)
-
Supporting all media types
13 avril 2011, parUnlike most software and media-sharing platforms, MediaSPIP aims to manage as many different media types as possible. The following are just a few examples from an ever-expanding list of supported formats : images : png, gif, jpg, bmp and more audio : MP3, Ogg, Wav and more video : AVI, MP4, OGV, mpg, mov, wmv and more text, code and other data : OpenOffice, Microsoft Office (Word, PowerPoint, Excel), web (html, CSS), LaTeX, Google Earth and (...)
Sur d’autres sites (7316)
-
Frame sliced into 2 under some degree. How fix it ?
26 novembre 2020, par Алекс АникейI trying use h264 codec in videochat application. And in some reason frame sliced into 2 triangle (picture below). I try send my desktop image to another person and get this image on another client.


What settings i set wrong ?
My code :


Init :


VCSession *vc_new_x264(Logger *log, ToxAV *av, uint32_t friend_number, toxav_video_receive_frame_cb *cb, void *cb_data,
 VCSession *vc)
{

 if (x264_param_default_preset(&param, "slow", "zerolatency") < 0) {
 // goto fail;
 }

 param.i_csp = X264_CSP_I420;
 param.i_width = 1920;
 param.i_height = 1080;
 vc->h264_enc_width = param.i_width;
 vc->h264_enc_height = param.i_height;

 param.i_keyint_max = 30;

 param.b_vfr_input = 1; /* VFR input. If 1, use timebase and timestamps for ratecontrol purposes.
 * If 0, use fps only. */
 param.i_timebase_num = 1; // 1 ms = timebase units = (1/1000)s
 param.i_timebase_den = 1000; // 1 ms = timebase units = (1/1000)s
 param.b_repeat_headers = 1;
 param.b_annexb = 1;

 param.rc.f_rate_tolerance = VIDEO_F_RATE_TOLERANCE_H264;
 param.rc.i_vbv_buffer_size = 1500;
 param.rc.i_vbv_max_bitrate = VIDEO_BITRATE_INITIAL_VALUE_H264 * 1;

 vc->h264_enc_bitrate = VIDEO_BITRATE_INITIAL_VALUE_H264;

 param.rc.i_qp_min = 13;
 param.rc.i_qp_max = 35; // max quantizer for x264

 vc->h264_enc_bitrate = VIDEO_BITRATE_INITIAL_VALUE_H264;

 param.rc.b_stat_read = 0;
 param.rc.b_stat_write = 0;


 if (x264_param_apply_profile(&param,
 "high") < 0) { // "baseline", "main", "high", "high10", "high422", "high444"
 // goto fail;
 }


 if (x264_picture_alloc(&(vc->h264_in_pic), param.i_csp, param.i_width, param.i_height) < 0) {
 // goto fail;
 }

 vc->h264_encoder = x264_encoder_open(&param);

 AVCodec *codec = NULL;
 vc->h264_decoder = NULL;
 avcodec_register_all();
 codec = NULL;

 codec = avcodec_find_decoder(AV_CODEC_ID_H264);

 if (!codec) {
 LOGGER_WARNING(log, "codec not found H264 on decoder");
 }

 vc->h264_decoder = avcodec_alloc_context3(codec);

 if (codec->capabilities & AV_CODEC_CAP_TRUNCATED) {
 vc->h264_decoder->flags |= AV_CODEC_FLAG_TRUNCATED; /* we do not send complete frames */
 }


 vc->h264_decoder->delay = 5;

 if (avcodec_open2(vc->h264_decoder, codec, NULL) < 0) {
 LOGGER_WARNING(log, "could not open codec H264 on decoder");
 }


 return vc;
}


Get frame and decoding him :


void vc_iterate_x264(VCSession *vc)
{

 if (!vc) {
 return;
 }

 pthread_mutex_lock(vc->queue_mutex);

 struct RTPMessage *p;

 if (!rb_read(vc->vbuf_raw, (void **)&p)) {
 LOGGER_TRACE(vc->log, "no Video frame data available");
 pthread_mutex_unlock(vc->queue_mutex);
 return;
 }

 pthread_mutex_unlock(vc->queue_mutex);
 const struct RTPHeader *const header = &p->header;

 uint32_t full_data_len;

 if (header->flags & RTP_LARGE_FRAME) {
 full_data_len = header->data_length_full;
 LOGGER_WARNING(vc->log, "vc_iterate:001:full_data_len=%d", (int)full_data_len);
 } else {
 full_data_len = p->len;
 if (header->data_length_lower != full_data_len)
 {
 LOGGER_ERROR("Data header and packet don't equal: %d - header %d - packet", header->data_length_lower, full_data_len);
 }
 LOGGER_DEBUG(vc->log, "vc_iterate:002");
 }

 decode_frame_h264(vc, p, full_data_len);
}

void decode_frame_h264(VCSession *vc,
 struct RTPMessage *p,
 uint32_t full_data_len)
{

 AVPacket *compr_data;
 compr_data = av_packet_alloc();


 uint8_t *tmp_buf = calloc(1, full_data_len + FF_INPUT_BUFFER_PADDING_SIZE);
 memcpy(tmp_buf, p->data, full_data_len);

 compr_data->data = tmp_buf; // p->data;
 compr_data->size = (int)full_data_len; // hmm, "int" again

 avcodec_send_packet(vc->h264_decoder, compr_data);

 int ret_ = 0;
 while (ret_ >= 0) {
 AVFrame *frame = av_frame_alloc();
 ret_ = avcodec_receive_frame(vc->h264_decoder, frame);
 if (ret_ == AVERROR(EAGAIN) || ret_ == AVERROR_EOF) {
 // error
 break;
 } else if (ret_ < 0) {
 // Error during decoding
 break;
 } else if (ret_ == 0) {
 vc->vcb(vc->av, vc->friend_number, frame->width, frame->height,
 (const uint8_t *)frame->data[0],
 (const uint8_t *)frame->data[1],
 (const uint8_t *)frame->data[2],
 frame->linesize[0], frame->linesize[1],
 frame->linesize[2], vc->vcb_user_data);
 } else {
 // some other error
 }
 av_frame_free(&frame);
 }
 av_packet_free(&compr_data);
 free(tmp_buf);
 free(p);
}


Send frame and encoding :


bool toxav_video_send_frame(ToxAV *av, uint32_t friend_number, uint16_t width, uint16_t height, const uint8_t *y,
 const uint8_t *u, const uint8_t *v, Toxav_Err_Send_Frame *error, int16_t kf_max_dist, vpx_codec_er_flags_t error_resilient,
 unsigned int my_lag_in_frames, uint16_t kf_mode, uint16_t quality_mode)
{
 Toxav_Err_Send_Frame rc = TOXAV_ERR_SEND_FRAME_OK;
 ToxAVCall *call;
 uint64_t video_frame_record_timestamp = current_time_monotonic(av->m->mono_time);

 int vpx_encode_flags = 0;

 pthread_mutex_lock(call->mutex_video);
 pthread_mutex_unlock(av->mutex);

 if (y == nullptr || u == nullptr || v == nullptr) {
 pthread_mutex_unlock(call->mutex_video);
 rc = TOXAV_ERR_SEND_FRAME_NULL;
 goto RETURN;
 }


 if (call->video_rtp->ssrc < VIDEO_SEND_X_KEYFRAMES_FIRST) {
 // Key frame flag for first frames
 vpx_encode_flags = VPX_EFLAG_FORCE_KF;
 LOGGER_INFO(av->m->log, "I_FRAME_FLAG:%d only-i-frame mode", call->video_rtp->ssrc);

 ++call->video_rtp->ssrc;
 } else if (call->video_rtp->ssrc == VIDEO_SEND_X_KEYFRAMES_FIRST) {
 // normal keyframe placement
 vpx_encode_flags = 0;
 LOGGER_INFO(av->m->log, "I_FRAME_FLAG:%d normal mode", call->video_rtp->ssrc);

 ++call->video_rtp->ssrc;
 }


 x264_nal_t *nal = NULL;
 int i_frame_size = 0;

 uint32_t result = encode_frame_h264(av, friend_number, width, height,
 y, u, v,
 &video_frame_record_timestamp,
 vpx_encode_flags,
 &nal,
 &i_frame_size);
 if (result != 0) {
 pthread_mutex_unlock(call->mutex_video);
 rc = TOXAV_ERR_SEND_FRAME_INVALID;
 goto RETURN;
 }

 ++call->video->frame_counter;

 rc = send_frames_h264(av, friend_number, width, height,
 y, u, v, call,
 &video_frame_record_timestamp,
 vpx_encode_flags,
 &nal,
 &i_frame_size,
 &rc);

 pthread_mutex_unlock(call->mutex_video);

RETURN:

 if (error) {
 *error = rc;
 }

 return rc == TOXAV_ERR_SEND_FRAME_OK;
}

uint32_t send_frames_h264(ToxAV *av, uint32_t friend_number, uint16_t width, uint16_t height,
 const uint8_t *y,
 const uint8_t *u, const uint8_t *v, ToxAVCall *call,
 uint64_t *video_frame_record_timestamp,
 int vpx_encode_flags,
 x264_nal_t **nal,
 int *i_frame_size,
 TOXAV_ERR_SEND_FRAME *rc)
{

 if (*i_frame_size > 0) {

 // use the record timestamp that was actually used for this frame
 *video_frame_record_timestamp = (uint64_t)call->video->h264_in_pic.i_pts;
 const uint32_t frame_length_in_bytes = *i_frame_size;
 const int keyframe = (int)call->video->h264_out_pic.b_keyframe;

 LOGGER_DEBUG(av->m->log, "video packet record time: %lu", (*video_frame_record_timestamp));

 int res = rtp_send_data
 (
 call->video_rtp,
 (const uint8_t *)((*nal)->p_payload),
 frame_length_in_bytes,
 keyframe,
 *video_frame_record_timestamp,
 av->m->log
 );

 (*video_frame_record_timestamp)++;

 if (res < 0) {
 LOGGER_WARNING(av->m->log, "Could not send video frame: %s", strerror(errno));
 *rc = TOXAV_ERR_SEND_FRAME_RTP_FAILED;
 return 1;
 }

 return 0;
 } else {
 *rc = TOXAV_ERR_SEND_FRAME_RTP_FAILED;
 return 1;
 }

}


I get image like this :




-
Audio Video Mixing - Sync issue in Android with FFMPEG, Media Codec in different devices
24 novembre 2020, par khushbuI have already tried everything for Audio Video mixing and it's not working perfectly as in processing while mixing audio into the recorded video, sometimes the audio is ahead of video and vice-versa.


Using FFMPEG :


This is for add an Audio file to the Video file and generated the final Video where audio is replaced in the video.


val cmd ="-i $inputVideoPath -i ${inputAudio.absolutePath} -map 0:v -map 1:a -c:v copy -shortest ${outputVideo.absolutePath}"


After generating the final video, found some delay based on device performance so added delay in the below two cases :


1)Added delay in Audio if audio is ahead of the video.


val cmd = "-i ${tmpVideo.absolutePath} -itsoffset $hms -i ${tmpVideo.absolutePath} -map 0:v -map 1:a -c copy -preset veryfast ${createdVideo1?.absolutePath}"


2)Added delay in Video if the video is ahead of the audio.


val cmd = "-i ${tmpVideo.absolutePath} -itsoffset $hms -i ${tmpVideo.absolutePath} -map 1:v -map 0:a -c copy -preset veryfast ${createdVideo1?.absolutePath}"


NOTE : Here $hms is delay in 00:00:00.000 formate


but still, it's not working on all the devices like readmi, oneplus etc.


Using Media Codec :


Found some better performance in this solution but still not working on all the devices.


In this process, It's supporting .aac format so first if the user selected .mp3 formate than i have to convert it into .aac format using the below function :


fun Convert_Mp3_to_acc() {

 
 AndroidAudioConverter.load(requireActivity(), object : ILoadCallback {
 override fun onSuccess() {

 val callback: IConvertCallback = object : IConvertCallback {
 override fun onSuccess(convertedFile: File) {
 toggleLoader(false)
 audioLink = convertedFile.absolutePath
 append()
 

 }

 override fun onFailure(error: java.lang.Exception) {
 toggleLoader(false)
 Toast.makeText(requireActivity(), "" + error, Toast.LENGTH_SHORT).show()
 }
 }
 AndroidAudioConverter.with(requireActivity())
 .setFile(File(audioLink))
 .setFormat(AudioFormat.AAC)
 .setCallback(callback)
 .convert()
 }

 override fun onFailure(error: java.lang.Exception) {
 toggleLoader(false)
 }
 })
}


After successful conversion from .mp3 to .aac formate, It's extracting audio track and video track for merge


private fun append(): Boolean {

 val progressDialog = ProgressDialog(requireContext())
 Thread {
 requireActivity().runOnUiThread {
 progressDialog.setMessage("Please wait..")
 progressDialog.show()
 }
 val video_list = ArrayList<string>()
 for (i in videopaths.indices) {
 val file: File = File(videopaths.get(i))
 if (file.exists()) {
 val retriever = MediaMetadataRetriever()
 retriever.setDataSource(requireActivity(), Uri.fromFile(file))
 val hasVideo =
 retriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_HAS_VIDEO)
 val isVideo = "yes" == hasVideo
 if (isVideo /*&& file.length() > 1000*/) {
 Log.d("resp", videopaths.get(i))
 video_list.add(videopaths.get(i))
 }
 }
 }
 try {
 val inMovies = arrayOfNulls<movie>(video_list.size)
 for (i in video_list.indices) {
 inMovies[i] = MovieCreator.build(video_list[i])
 }
 val videoTracks: MutableList<track> =
 LinkedList()
 val audioTracks: MutableList<track> =
 LinkedList()
 for (m in inMovies) {
 for (t in m!!.tracks) {
 if (t.handler == "soun") {
 audioTracks.add(t)
 }
 if (t.handler == "vide") {
 videoTracks.add(t)
 }
 }
 }
 val result = Movie()
 if (audioTracks.size > 0) {
 result.addTrack(AppendTrack(*audioTracks.toTypedArray()))
 }
 if (videoTracks.size > 0) {
 result.addTrack(AppendTrack(*videoTracks.toTypedArray()))
 }
 val out = DefaultMp4Builder().build(result)
 var outputFilePath: String? = null
 outputFilePath = Variables.outputfile

 /*if (audio != null) {
 Variables.outputfile
 } else {
 Variables.outputfile2
 }*/

 val fos = FileOutputStream(File(outputFilePath))
 out.writeContainer(fos.channel)
 fos.close()

 requireActivity().runOnUiThread {
 progressDialog.dismiss()

 Merge_withAudio()

 /* if (audio != null) else {
 //Go_To_preview_Activity()
 }*/
 }
 } catch (e: java.lang.Exception) {
 }
 }.start()

 return true
}
</track></track></movie></string>

This will add the selected audio with the recorded video


fun Merge_withAudio() {
 val root = Environment.getExternalStorageDirectory().toString()

 // Uri mediaPath = Uri.parse("android.resource://" + getPackageName() + "/" + R.raw.file_copy);
 //String audio_file =Variables.app_folder+Variables.SelectedAudio_AAC;

 //String filename = "android.resource://" + getPackageName() + "/raw/file_copy.aac";
 val audio_file: String = audioLink!!
 Log.e("Merge ", audio_file)
 val video = "$root/output.mp4"

 val bundle=Bundle()
 bundle.putString("FinalVideo", createdVideo?.absolutePath)

 val merge_video_audio = Merge_Video_Audio(this, bundle, object : AsyncResponse {
 override fun processFinish(output: Bundle?) {

 requireActivity().runOnUiThread {
 finalVideo = bundle.getString("FinalVideo")
 createdVideo = File(finalVideo)

 Log.e("Final Path ", finalVideo)

 createThumb {
 setUpExoPlayer()
 }
 }

 }
 })
 merge_video_audio.doInBackground(audio_file, video, createdVideo?.absolutePath)
}


 public class Merge_Video_Audio extends AsyncTask {

 ProgressDialog progressDialog;
 RecentCompletedVideoFragment context;
 public AsyncResponse delegate = null;


Bundle bundleValue;

String audio,video,output;

public Merge_Video_Audio(RecentCompletedVideoFragment context, Bundle bundle , AsyncResponse delegate ){
 this.context=context;
 this.bundleValue=bundle;
 this.delegate=delegate;
 progressDialog=new ProgressDialog(context.requireContext());
 progressDialog.setMessage("Please Wait...");
}

@Override
protected void onPreExecute() {
 super.onPreExecute();
}

@Override
public String doInBackground(String... strings) {
 try {
 progressDialog.show();
 }catch (Exception e){

 }
 audio=strings[0];
 video=strings[1];
 output=strings[2];

 Log.d("resp",audio+"----"+video+"-----"+output);

 Thread thread = new Thread(runnable);
 thread.start();

 return null;
}


@Override
protected void onPostExecute(String s) {
 super.onPostExecute(s);
 Log.e("On Post Execute ", "True");


}


 public void Go_To_preview_Activity(){

 delegate.processFinish(bundleValue);
 }

 public Track CropAudio(String videopath, Track fullAudio){
 try {

 IsoFile isoFile = new IsoFile(videopath);

 double lengthInSeconds = (double)
 isoFile.getMovieBox().getMovieHeaderBox().getDuration() /
 isoFile.getMovieBox().getMovieHeaderBox().getTimescale();


 Track audioTrack = (Track) fullAudio;


 double startTime1 = 0;
 double endTime1 = lengthInSeconds;


 long currentSample = 0;
 double currentTime = 0;
 double lastTime = -1;
 long startSample1 = -1;
 long endSample1 = -1;


 for (int i = 0; i < audioTrack.getSampleDurations().length; i++) {

 long delta = audioTrack.getSampleDurations()[i];

 if (currentTime > lastTime && currentTime <= startTime1) {
 // current sample is still before the new starttime
 startSample1 = currentSample;
 }
 if (currentTime > lastTime && currentTime <= endTime1) {
 // current sample is after the new start time and still before the new endtime
 endSample1 = currentSample;
 }

 lastTime = currentTime;
 currentTime += (double) delta / (double) audioTrack.getTrackMetaData().getTimescale();
 currentSample++;
 }

 CroppedTrack cropperAacTrack = new CroppedTrack(fullAudio, startSample1, endSample1);

 return cropperAacTrack;

 } catch (IOException e) {
 e.printStackTrace();
 }

 return fullAudio;
}



 public Runnable runnable =new Runnable() {
 @Override
 public void run() {

 try {

 Movie m = MovieCreator.build(video);


 List nuTracks = new ArrayList<>();

 for (Track t : m.getTracks()) {
 if (!"soun".equals(t.getHandler())) {

 Log.e("Track ",t.getName());
 nuTracks.add(t);
 }
 }

 Log.e("Path ",audio.toString());


 try {
 // Track nuAudio = new AACTrackImpl();
 Track nuAudio = new AACTrackImpl(new FileDataSourceImpl(audio));

 Track crop_track = CropAudio(video, nuAudio);

 nuTracks.add(crop_track);

 m.setTracks(nuTracks);

 Container mp4file = new DefaultMp4Builder().build(m);

 FileChannel fc = new FileOutputStream(new File(output)).getChannel();
 mp4file.writeContainer(fc);
 fc.close();

 }catch (FileNotFoundException fnfe){
 fnfe.printStackTrace();
 }catch(IOException ioe){
 ioe.printStackTrace();
 }


 try {

 progressDialog.dismiss();
 }catch (Exception e){
 Log.d("resp",e.toString());

 }finally {
 Go_To_preview_Activity();

 }

 } catch (IOException e) {
 e.printStackTrace();
 Log.d("resp",e.toString());

 }

 }

 };

 }


This solution is also not working in all the devices.


Can anyone suggest where i am going wrong or any solution for it ?


-
How to play and seek fragmented MP4 audio using MSE SourceBuffer ?
29 juin 2024, par Stefan FalkNote :




If you end up here, you might want to take a look at shaka-player and the accompanying shaka-streamer. Use it. Don't implement this yourself unless you really have to.




I am trying for quite some time now to be able to play an audio track on Chrome, Firefox, Safari, etc. but I keep hitting brick walls. My problem is currently that I am just not able to seek within a fragmented MP4 (or MP3).


At the moment I am converting audio files such as MP3 to fragmented MP4 (fMP4) and send them chunk-wise to the client. What I do is defining a
CHUNK_DURACTION_SEC(chunk duration in seconds) and compute a chunk size like this :

chunksTotal = Math.ceil(this.track.duration / CHUNK_DURATION_SEC);
chunkSize = Math.ceil(this.track.fileSize / this.chunksTotal);


With this I partition the audio file and can fetch it entirely jumping
chunkSize-many bytes for each chunk :

-----------------------------------------
| chunk 1 | chunk 2 | ... | chunk n |
-----------------------------------------


How audio files are converted to fMP4


ffmpeg -i input.mp3 -acodec aac -b:a 256k -f mp4 \
 -movflags faststart+frag_every_frame+empty_moov+default_base_moof \
 output.mp4


This seems to work with Chrome and Firefox (so far).


How chunks are appended


After following this example, and realizing that it's simply not working as it is explained here, I threw it away and started over from scratch. Unfortunately without success. It's still not working.


The following code is supposed to play a track from the very beginning to the very end. However, I also need to be able to seek. So far, this is simply not working. Seeking will just stop the audio after the
seekingevent got triggered.

The code


/* Desired chunk duration in seconds. */
const CHUNK_DURATION_SEC = 20;

const AUDIO_EVENTS = [
 'ended',
 'error',
 'play',
 'playing',
 'seeking',
 'seeked',
 'pause',
 'timeupdate',
 'canplay',
 'loadedmetadata',
 'loadstart',
 'updateend',
];


class ChunksLoader {

 /** The total number of chunks for the track. */
 public readonly chunksTotal: number;

 /** The length of one chunk in bytes */
 public readonly chunkSize: number;

 /** Keeps track of requested chunks. */
 private readonly requested: boolean[];

 /** URL of endpoint for fetching audio chunks. */
 private readonly url: string;

 constructor(
 private track: Track,
 private sourceBuffer: SourceBuffer,
 private logger: NGXLogger,
 ) {

 this.chunksTotal = Math.ceil(this.track.duration / CHUNK_DURATION_SEC);
 this.chunkSize = Math.ceil(this.track.fileSize / this.chunksTotal);

 this.requested = [];
 for (let i = 0; i < this.chunksTotal; i++) {
 this.requested[i] = false;
 }

 this.url = `${environment.apiBaseUrl}/api/tracks/${this.track.id}/play`;
 }

 /**
 * Fetch the first chunk.
 */
 public begin() {
 this.maybeFetchChunk(0);
 }

 /**
 * Handler for the "timeupdate" event. Checks if the next chunk should be fetched.
 *
 * @param currentTime
 * The current time of the track which is currently played.
 */
 public handleOnTimeUpdate(currentTime: number) {

 const nextChunkIndex = Math.floor(currentTime / CHUNK_DURATION_SEC) + 1;
 const hasAllChunks = this.requested.every(val => !!val);

 if (nextChunkIndex === (this.chunksTotal - 1) && hasAllChunks) {
 this.logger.debug('Last chunk. Calling mediaSource.endOfStream();');
 return;
 }

 if (this.requested[nextChunkIndex] === true) {
 return;
 }

 if (currentTime < CHUNK_DURATION_SEC * (nextChunkIndex - 1 + 0.25)) {
 return;
 }

 this.maybeFetchChunk(nextChunkIndex);
 }

 /**
 * Fetches the chunk if it hasn't been requested yet. After the request finished, the returned
 * chunk gets appended to the SourceBuffer-instance.
 *
 * @param chunkIndex
 * The chunk to fetch.
 */
 private maybeFetchChunk(chunkIndex: number) {

 const start = chunkIndex * this.chunkSize;
 const end = start + this.chunkSize - 1;

 if (this.requested[chunkIndex] == true) {
 return;
 }

 this.requested[chunkIndex] = true;

 if ((end - start) == 0) {
 this.logger.warn('Nothing to fetch.');
 return;
 }

 const totalKb = ((end - start) / 1000).toFixed(2);
 this.logger.debug(`Starting to fetch bytes ${start} to ${end} (total ${totalKb} kB). Chunk ${chunkIndex + 1} of ${this.chunksTotal}`);

 const xhr = new XMLHttpRequest();
 xhr.open('get', this.url);
 xhr.setRequestHeader('Authorization', `Bearer ${AuthenticationService.getJwtToken()}`);
 xhr.setRequestHeader('Range', 'bytes=' + start + '-' + end);
 xhr.responseType = 'arraybuffer';
 xhr.onload = () => {
 this.logger.debug(`Range ${start} to ${end} fetched`);
 this.logger.debug(`Requested size: ${end - start + 1}`);
 this.logger.debug(`Fetched size: ${xhr.response.byteLength}`);
 this.logger.debug('Appending chunk to SourceBuffer.');
 this.sourceBuffer.appendBuffer(xhr.response);
 };
 xhr.send();
 };

}

export enum StreamStatus {
 NOT_INITIALIZED,
 INITIALIZING,
 PLAYING,
 SEEKING,
 PAUSED,
 STOPPED,
 ERROR
}

export class PlayerState {
 status: StreamStatus = StreamStatus.NOT_INITIALIZED;
}


/**
 *
 */
@Injectable({
 providedIn: 'root'
})
export class MediaSourcePlayerService {

 public track: Track;

 private mediaSource: MediaSource;

 private sourceBuffer: SourceBuffer;

 private audioObj: HTMLAudioElement;

 private chunksLoader: ChunksLoader;

 private state: PlayerState = new PlayerState();

 private state$ = new BehaviorSubject<playerstate>(this.state);

 public stateChange = this.state$.asObservable();

 private currentTime$ = new BehaviorSubject<number>(null);

 public currentTimeChange = this.currentTime$.asObservable();

 constructor(
 private httpClient: HttpClient,
 private logger: NGXLogger
 ) {
 }

 get canPlay() {
 const state = this.state$.getValue();
 const status = state.status;
 return status == StreamStatus.PAUSED;
 }

 get canPause() {
 const state = this.state$.getValue();
 const status = state.status;
 return status == StreamStatus.PLAYING || status == StreamStatus.SEEKING;
 }

 public playTrack(track: Track) {
 this.logger.debug('playTrack');
 this.track = track;
 this.startPlayingFrom(0);
 }

 public play() {
 this.logger.debug('play()');
 this.audioObj.play().then();
 }

 public pause() {
 this.logger.debug('pause()');
 this.audioObj.pause();
 }

 public stop() {
 this.logger.debug('stop()');
 this.audioObj.pause();
 }

 public seek(seconds: number) {
 this.logger.debug('seek()');
 this.audioObj.currentTime = seconds;
 }

 private startPlayingFrom(seconds: number) {
 this.logger.info(`Start playing from ${seconds.toFixed(2)} seconds`);
 this.mediaSource = new MediaSource();
 this.mediaSource.addEventListener('sourceopen', this.onSourceOpen);

 this.audioObj = document.createElement('audio');
 this.addEvents(this.audioObj, AUDIO_EVENTS, this.handleEvent);
 this.audioObj.src = URL.createObjectURL(this.mediaSource);

 this.audioObj.play().then();
 }

 private onSourceOpen = () => {

 this.logger.debug('onSourceOpen');

 this.mediaSource.removeEventListener('sourceopen', this.onSourceOpen);
 this.mediaSource.duration = this.track.duration;

 this.sourceBuffer = this.mediaSource.addSourceBuffer('audio/mp4; codecs="mp4a.40.2"');
 // this.sourceBuffer = this.mediaSource.addSourceBuffer('audio/mpeg');

 this.chunksLoader = new ChunksLoader(
 this.track,
 this.sourceBuffer,
 this.logger
 );

 this.chunksLoader.begin();
 };

 private handleEvent = (e) => {

 const currentTime = this.audioObj.currentTime.toFixed(2);
 const totalDuration = this.track.duration.toFixed(2);
 this.logger.warn(`MediaSource event: ${e.type} (${currentTime} of ${totalDuration} sec)`);

 this.currentTime$.next(this.audioObj.currentTime);

 const currentStatus = this.state$.getValue();

 switch (e.type) {
 case 'playing':
 currentStatus.status = StreamStatus.PLAYING;
 this.state$.next(currentStatus);
 break;
 case 'pause':
 currentStatus.status = StreamStatus.PAUSED;
 this.state$.next(currentStatus);
 break;
 case 'timeupdate':
 this.chunksLoader.handleOnTimeUpdate(this.audioObj.currentTime);
 break;
 case 'seeking':
 currentStatus.status = StreamStatus.SEEKING;
 this.state$.next(currentStatus);
 if (this.mediaSource.readyState == 'open') {
 this.sourceBuffer.abort();
 }
 this.chunksLoader.handleOnTimeUpdate(this.audioObj.currentTime);
 break;
 }
 };

 private addEvents(obj, events, handler) {
 events.forEach(event => obj.addEventListener(event, handler));
 }

}
</number></playerstate>

Running it will give me the following output :






Apologies for the screenshot but it's not possible to just copy the output without all the stack traces in Chrome.




What I also tried was following this example and call
sourceBuffer.abort()but that didn't work. It looks more like a hack that used to work years ago but it's still referenced in the docs (see "Example" -> "You can see something similar in action in Nick Desaulnier's bufferWhenNeeded demo ..").

case 'seeking':
 currentStatus.status = StreamStatus.SEEKING;
 this.state$.next(currentStatus); 
 if (this.mediaSource.readyState === 'open') {
 this.sourceBuffer.abort();
 } 
 break;


Trying with MP3


I have tested the above code under Chrome by converting tracks to MP3 :


ffmpeg -i input.mp3 -acodec aac -b:a 256k -f mp3 output.mp3


and creating a
SourceBufferusingaudio/mpegas type :

this.mediaSource.addSourceBuffer('audio/mpeg')


I have the same problem when seeking.


The issue wihout seeking


The above code has another issue :


After two minutes of playing, the audio playback starts to stutter and comes to a halt prematurely. So, the audio plays up to a point and then it stops without any obvious reason.


For whatever reason there is another
canplayandplayingevent. A few seconds after, the audio simply stops..



Syndiquer tout le site
