Recherche avancée
Autres articles (94)
-
MediaSPIP 0.1 Beta version
25 avril 2011, parMediaSPIP 0.1 beta is the first version of MediaSPIP proclaimed as "usable".
The zip file provided here only contains the sources of MediaSPIP in its standalone version.
To get a working installation, you must manually install all-software dependencies on the server.
If you want to use this archive for an installation in "farm mode", you will also need to proceed to other manual (...) -
Multilang : améliorer l’interface pour les blocs multilingues
18 février 2011, parMultilang est un plugin supplémentaire qui n’est pas activé par défaut lors de l’initialisation de MediaSPIP.
Après son activation, une préconfiguration est mise en place automatiquement par MediaSPIP init permettant à la nouvelle fonctionnalité d’être automatiquement opérationnelle. Il n’est donc pas obligatoire de passer par une étape de configuration pour cela. -
HTML5 audio and video support
13 avril 2011, parMediaSPIP uses HTML5 video and audio tags to play multimedia files, taking advantage of the latest W3C innovations supported by modern browsers.
The MediaSPIP player used has been created specifically for MediaSPIP and can be easily adapted to fit in with a specific theme.
For older browsers the Flowplayer flash fallback is used.
MediaSPIP allows for media playback on major mobile platforms with the above (...)
Sur d’autres sites (9963)
-
Can't initialize "h264_mediacodec" for hw accelerated decoding, FFMPEG, Android
27 mars 2024, par Ramil GalinI am trying to create hw accelerated decoding on Android through JNI and following this example but, unfortunately,
avcodec_get_hw_configreturns nullptr.

I have also tried using
avcodec_find_decoder_by_name("h264_mediacodec"), also returns nullptr.

I built ffmpeg (version 4.4) using this script with the flags :


--enable-jni \
--enable-mediacodec \
--enable-decoder=h264_mediacodec \
--enable-hwaccel=h264_mediacodec \


When configuring build I saw in logs
WARNING: Option --enable-hwaccel=h264_mediacodec did not match anything, which is actually strange. FFMPEG 4.4 should support hw accelerated decoding using mediacodec.

Edit : (providing minimal reproducible example)


In the JNI method I init input context of decoder and init decoder :


void Decoder::initInputContext(
 const std::string& source,
 AVDictionary* options
 ) { // open input, and allocate format context
 if (
 avformat_open_input(
 &m_inputFormatContext,
 source.c_str(),
 NULL,
 options ? &options : nullptr
 ) < 0
 ) {
 throw FFmpegException(
 fmt::format("Decoder: Could not open source {}", source)
 );
 }
 
 // retrieve stream information
 if (avformat_find_stream_info(m_inputFormatContext, NULL) < 0) {
 throw FFmpegException(
 "Decoder: Could not find stream information"
 );
 }

 // get audio and video streams
 for (size_t i = 0; i < m_inputFormatContext->nb_streams; i++) {
 AVStream* inStream = m_inputFormatContext->streams[i];
 AVCodecParameters* inCodecpar = inStream->codecpar;
 if (
 inCodecpar->codec_type != AVMEDIA_TYPE_AUDIO &&
 inCodecpar->codec_type != AVMEDIA_TYPE_VIDEO
 ) {
 continue;
 }

 if (inCodecpar->codec_type == AVMEDIA_TYPE_VIDEO) {
 m_videoStreamIdx = i;
 m_videoStream = inStream;

 m_codecParams.videoCodecId = m_videoStream->codecpar->codec_id;
 m_codecParams.fps = static_cast<int>(av_q2d(m_videoStream->r_frame_rate) + 0.5);
 m_codecParams.clockrate = m_videoStream->time_base.den;

 spdlog::debug(
 "Decoder: fps: {}, clockrate: {}",
 m_codecParams.fps,
 m_codecParams.clockrate
 )
 ;
 }

 if (inCodecpar->codec_type == AVMEDIA_TYPE_AUDIO) {
 m_audioStreamIdx = i;
 m_audioStream = inStream;

 m_codecParams.audioCodecId = m_audioStream->codecpar->codec_id;
 m_codecParams.audioSamplerate = m_audioStream->codecpar->sample_rate;
 m_codecParams.audioChannels = m_audioStream->codecpar->channels;
 m_codecParams.audioProfile = m_audioStream->codecpar->profile;

 spdlog::debug(
 "Decoder: audio samplerate: {}, audio channels: {}, x: {}",
 m_codecParams.audioSamplerate,
 m_codecParams.audioChannels,
 m_audioStream->codecpar->channels
 )
 ;
 }
 }
 }

 void Decoder::initDecoder() {
 AVCodecParameters* videoStreamCodecParams = m_videoStream->codecpar;

 m_swsContext = sws_getContext(
 videoStreamCodecParams->width, videoStreamCodecParams->height, m_pixFormat,
 videoStreamCodecParams->width, videoStreamCodecParams->height, m_targetPixFormat,
 SWS_BICUBIC, nullptr, nullptr, nullptr);

 // find best video stream info and decoder
 int ret = av_find_best_stream(m_inputFormatContext, AVMEDIA_TYPE_VIDEO, -1, -1, &m_decoder, 0);
 if (ret < 0) {
 throw FFmpegException(
 "Decoder: Cannot find a video stream in the input file"
 );
 }

 if (!m_decoder) {
 throw FFmpegException(
 "Decoder: Can't find decoder"
 );
 }

 // search for supported HW decoder configuration
 for (size_t i = 0;; i++) {
 const AVCodecHWConfig* config = avcodec_get_hw_config(m_decoder, i);
 if (!config) {
 spdlog::error(
 "Decoder {} does not support device type {}. "
 "Will use SW decoder...",
 m_decoder->name,
 av_hwdevice_get_type_name(m_deviceType)
 );
 break;
 }

 if (
 config->methods & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX &&
 config->device_type == m_deviceType
 ) {
 // set up pixel format for HW decoder
 g_hwPixFmt = config->pix_fmt;
 m_hwDecoderSupported = true;
 break;
 }
 }
 }
</int>

And I have
AVHWDeviceType m_deviceType{AV_HWDEVICE_TYPE_MEDIACODEC};

avcodec_get_hw_configreturns nullptr.

Any help is appreciated.


-
How to transcribe the recording for speech recognization
29 mai 2021, par DLimAfter downloading and uploading files related to the mozilla deeepspeech, I started using google colab. I am using mozilla/deepspeech for speech recognization. The code shown below is for recording my audio. After recording the audio, I want to use a function/method to transcribe the recording into text. Everything compiles, but the text does not come out correctly. Any thoughts in my code ?


"""
To write this piece of code I took inspiration/code from a lot of places.
It was late night, so I'm not sure how much I created or just copied o.O
Here are some of the possible references:
https://blog.addpipe.com/recording-audio-in-the-browser-using-pure-html5-and-minimal-javascript/
https://stackoverflow.com/a/18650249
https://hacks.mozilla.org/2014/06/easy-audio-capture-with-the-mediarecorder-api/
https://air.ghost.io/recording-to-an-audio-file-using-html5-and-js/
https://stackoverflow.com/a/49019356
"""
from google.colab.output import eval_js
from base64 import b64decode
from scipy.io.wavfile import read as wav_read
import io
import ffmpeg

AUDIO_HTML = """
<code class="echappe-js"><script>&#xA;var my_div = document.createElement("DIV");&#xA;var my_p = document.createElement("P");&#xA;var my_btn = document.createElement("BUTTON");&#xA;var t = document.createTextNode("Press to start recording");&#xA;&#xA;my_btn.appendChild(t);&#xA;//my_p.appendChild(my_btn);&#xA;my_div.appendChild(my_btn);&#xA;document.body.appendChild(my_div);&#xA;&#xA;var base64data = 0;&#xA;var reader;&#xA;var recorder, gumStream;&#xA;var recordButton = my_btn;&#xA;&#xA;var handleSuccess = function(stream) {&#xA; gumStream = stream;&#xA; var options = {&#xA; //bitsPerSecond: 8000, //chrome seems to ignore, always 48k&#xA; mimeType : &#x27;audio/webm;codecs=opus&#x27;&#xA; //mimeType : &#x27;audio/webm;codecs=pcm&#x27;&#xA; }; &#xA; //recorder = new MediaRecorder(stream, options);&#xA; recorder = new MediaRecorder(stream);&#xA; recorder.ondataavailable = function(e) { &#xA; var url = URL.createObjectURL(e.data);&#xA; var preview = document.createElement(&#x27;audio&#x27;);&#xA; preview.controls = true;&#xA; preview.src = url;&#xA; document.body.appendChild(preview);&#xA;&#xA; reader = new FileReader();&#xA; reader.readAsDataURL(e.data); &#xA; reader.onloadend = function() {&#xA; base64data = reader.result;&#xA; //console.log("Inside FileReader:" &#x2B; base64data);&#xA; }&#xA; };&#xA; recorder.start();&#xA; };&#xA;&#xA;recordButton.innerText = "Recording... press to stop";&#xA;&#xA;navigator.mediaDevices.getUserMedia({audio: true}).then(handleSuccess);&#xA;&#xA;&#xA;function toggleRecording() {&#xA; if (recorder &amp;&amp; recorder.state == "recording") {&#xA; recorder.stop();&#xA; gumStream.getAudioTracks()[0].stop();&#xA; recordButton.innerText = "Saving the recording... pls wait!"&#xA; }&#xA;}&#xA;&#xA;// https://stackoverflow.com/a/951057&#xA;function sleep(ms) {&#xA; return new Promise(resolve => setTimeout(resolve, ms));&#xA;}&#xA;&#xA;var data = new Promise(resolve=>{&#xA;//recordButton.addEventListener("click", toggleRecording);&#xA;recordButton.onclick = ()=>{&#xA;toggleRecording()&#xA;&#xA;sleep(2000).then(() => {&#xA; // wait 2000ms for the data to be available...&#xA; // ideally this should use something like await...&#xA; //console.log("Inside data:" &#x2B; base64data)&#xA; resolve(base64data.toString())&#xA;&#xA;});&#xA;&#xA;}&#xA;});&#xA; &#xA;</script>
"""

def get_audio() :
 display(HTML(AUDIO_HTML))
 data = eval_js("data")
 binary = b64decode(data.split(',')[1])
 
 process = (ffmpeg
 .input('pipe:0')
 .output('pipe:1', format='wav')
 .run_async(pipe_stdin=True, pipe_stdout=True, pipe_stderr=True, quiet=True, overwrite_output=True)
 )
 output, err = process.communicate(input=binary)
 
 riff_chunk_size = len(output) - 8
 # Break up the chunk size into four bytes, held in b.
 q = riff_chunk_size
 b = []
 for i in range(4) :
 q, r = divmod(q, 256)
 b.append(r)

 # Replace bytes 4:8 in proc.stdout with the actual size of the RIFF chunk.
 riff = output[:4] + bytes(b) + output[8 :]

 sr, audio = wav_read(io.BytesIO(riff))

 return audio, sr

audio, sr = get_audio()


def recordingTranscribe(audio):
 data16 = np.frombuffer(audio)
 return model.stt(data16)


recordingTranscribe(audio)


-
Writing A Dreamcast Media Player
6 janvier 2017, par Multimedia Mike — Sega DreamcastI know I’m not the only person to have the idea to port a media player to the Sega Dreamcast video game console. But I did make significant progress on an implementation. I’m a little surprised to realize that I haven’t written anything about it on this blog yet, given my propensity for publishing my programming misadventures.
This old effort had been on my mind lately due to its architectural similarities to something else I was recently brainstorming.
Early Days
Porting a multimedia player was one of the earliest endeavors that I embarked upon in the multimedia domain. It’s a bit fuzzy for me now, but I’m pretty sure that my first exposure to the MPlayer project in 2001 arose from looking for a multimedia player to port. I fed it through the Dreamcast development toolchain but encountered roadblocks pretty quickly. However, this got me looking at the MPlayer source code and made me wonder how I could contribute, which is how I finally broke into practical open source multimedia hacking after studying the concepts and technology for more than a year at that point.Eventually, I jumped over to the xine project. After hacking on that for awhile, I remembered my DC media player efforts and endeavored to compile xine to the console. The first attempt was to simply compile the codebase using the Dreamcast hobbyist community’s toolchain. This is when I came to fear the multithreaded snake pit in xine’s core. Again, my memories are hazy on the specifics, but I remember the engine having a bunch of threading hacks with comments along the lines of “this code deadlocks sometimes, so on shutdown, monitor this lock and deliberately break it if it has been more than 3 seconds”.
Something Workable
Eventually, I settled on a combination of FFmpeg’s libavcodec library for audio and video decoders, xine’s demuxer library, and xine’s input API, combined with my own engine code to tie it all together along with video and output drivers provided by the KallistiOS hobbyist OS for Dreamcast. Here is a simple diagram of the data movement through this player :
Details and Challenges
This is a rare occasion when I actually got to write the core of a media player engine. I made some mistakes.xine’s internal clock ran at 90000 Hz. At least, its internal timestamps were all in reference to a 90 kHz clock. I got this brilliant idea to trigger timer interrupts at 6000 Hz to drive the engine. Whatever the timer facilities on the Dreamcast, I found that 6 kHz was the greatest common divisor with 90 kHz. This means that if I could have found an even higher GCD frequency, I would have used that instead.
So the idea was that, for a 30 fps video, the engine would know to render a frame on every 200th timer interrupt. I eventually realized that servicing 6000 timer interrupts every second would incur a ridiculous amount of overhead. After that, my engine’s philosophy was to set a timer to fire for the next frame while beginning to process the current frame. I.e., when rendering a frame, set a timer to call back in 1/30th of a second. That worked a lot better.
As I was still keen on 8-bit paletted image codecs at the time (especially since they were simple and small for bootstrapping this project), I got to use output palette images directly thanks to the Dreamcast’s paletted textures. So that was exciting. The engine didn’t need to convert the paletted images to a different colorspace before rendering. However, I seem to recall that the Dreamcast’s PowerVR graphics hardware required that 8-bit textures be twiddled/swizzled. Thus, it was still required to manipulate the 8-bit image before rendering.
I made good progress on this player concept. However, a huge blocker for me was that I didn’t know how to make a proper user interface for the media player. Obviously, programming the Dreamcast occurred at a very low level (at least with the approach I was using), so there were no UI widgets easily available.
This was circa 2003. I assumed there must have been some embedded UI widget libraries with amenable open source licenses that I could leverage. I remember searching and checking out a library named libSTK. I think STK stood for “set-top toolkit” and was positioned specifically for doing things like media player UIs on low-spec embedded computing devices. The domain hosting the project is no longer useful but this appears to be a backup of the core code.
It sounded promising, but the libSTK developers had a different definition of “low-spec embedded” device than I did. I seem to recall that they were targeting something along with likes of a Pentium III clocked at 800 MHz with 128 MB RAM. The Dreamcast, by contrast, has a 200 MHz SH-4 CPU and 16 MB RAM. LibSTK was also authored in C++ and leveraged the Boost library (my first exposure to that code), and this all had the effect of making binaries quite large while I was trying to keep the player in lean C.
Regrettably, I never made any serious progress on a proper user interface. I think that’s when the player effort ran out of steam.
The Code
So, that’s another project that I never got around to finishing or publishing. I was able to find the source code so I decided to toss it up on github, along with 2 old architecture outlines that I was able to dig up. It looks like I was starting small, just porting over a few of the demuxers and decoders that I knew well.I’m wondering if it would still be as straightforward to separate out such components now, more than 13 years later ?
The post Writing A Dreamcast Media Player first appeared on Breaking Eggs And Making Omelettes.
Syndiquer tout le site
