Recherche avancée
Médias (33)
-
Stereo master soundtrack
17 octobre 2011, par
Mis à jour : Octobre 2011
Langue : English
Type : Audio
-
#7 Ambience
16 octobre 2011, par
Mis à jour : Juin 2015
Langue : English
Type : Audio
-
#6 Teaser Music
16 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
#5 End Title
16 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
#3 The Safest Place
16 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
#4 Emo Creates
15 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
Autres articles (29)
-
Support audio et vidéo HTML5
10 avril 2011MediaSPIP utilise les balises HTML5 video et audio pour la lecture de documents multimedia en profitant des dernières innovations du W3C supportées par les navigateurs modernes.
Pour les navigateurs plus anciens, le lecteur flash Flowplayer est utilisé.
Le lecteur HTML5 utilisé a été spécifiquement créé pour MediaSPIP : il est complètement modifiable graphiquement pour correspondre à un thème choisi.
Ces technologies permettent de distribuer vidéo et son à la fois sur des ordinateurs conventionnels (...) -
HTML5 audio and video support
13 avril 2011, parMediaSPIP uses HTML5 video and audio tags to play multimedia files, taking advantage of the latest W3C innovations supported by modern browsers.
The MediaSPIP player used has been created specifically for MediaSPIP and can be easily adapted to fit in with a specific theme.
For older browsers the Flowplayer flash fallback is used.
MediaSPIP allows for media playback on major mobile platforms with the above (...) -
De l’upload à la vidéo finale [version standalone]
31 janvier 2010, parLe chemin d’un document audio ou vidéo dans SPIPMotion est divisé en trois étapes distinctes.
Upload et récupération d’informations de la vidéo source
Dans un premier temps, il est nécessaire de créer un article SPIP et de lui joindre le document vidéo "source".
Au moment où ce document est joint à l’article, deux actions supplémentaires au comportement normal sont exécutées : La récupération des informations techniques des flux audio et video du fichier ; La génération d’une vignette : extraction d’une (...)
Sur d’autres sites (2468)
-
Create Panorama from Non-Sequential Video Frames
6 mai 2021, par M.InnatThere is a similar question (not that detailed and no exact solution).



I want to create a single panorama image from video frames. And for that, I need to get minimum non-sequential video frames at first. A demo video file is uploaded here.


What I Need


A mechanism that can produce not-only non-sequential video frames but also in such a way that can be used to create a panorama image. A sample is given below. As we can see to create a panorama image, all the input samples must contain minimum overlap regions to each other otherwise it can not be done.




So, if I have the following video frame's order


A, A, A, B, B, B, B, C, C, A, A, C, C, C, B, B, B ...


To create a panorama image, I need to get something as follows - reduced sequential frames (or adjacent frames) but with minimum overlapping.


[overlap] [overlap] [overlap] [overlap] [overlap]
 A, A,B, B,C, C,A, A,C, C,B, ...


What I've Tried and Stuck


A demo video clip is given above. To get non-sequential video frames, I primarily rely on
ffmpegsoftware.

Trial 1 Ref.


ffmpeg -i check.mp4 -vf mpdecimate,setpts=N/FRAME_RATE/TB -map 0:v out.mp4


After that, on the
out.mp4, I applied slice the video frames usingopencv

import cv2, os 
from pathlib import Path

vframe_dir = Path("vid_frames/")
vframe_dir.mkdir(parents=True, exist_ok=True)

vidcap = cv2.VideoCapture('out.mp4')
success,image = vidcap.read()
count = 0

while success:
 cv2.imwrite(f"{vframe_dir}/frame%d.jpg" % count, image) 
 success,image = vidcap.read()
 count += 1


Next, I rotated these saved images horizontally (as my video is a vertical view).


vframe_dir = Path("out/")
vframe_dir.mkdir(parents=True, exist_ok=True)

vframe_dir_rot = Path("vframe_dir_rot/")
vframe_dir_rot.mkdir(parents=True, exist_ok=True)

for i, each_img in tqdm(enumerate(os.listdir(vframe_dir))):
 image = cv2.imread(f"{vframe_dir}/{each_img}")[:, :, ::-1] # Read (with BGRtoRGB)
 
 image = cv2.rotate(image,cv2.cv2.ROTATE_180)
 image = cv2.rotate(image,cv2.ROTATE_90_CLOCKWISE)

 cv2.imwrite(f"{vframe_dir_rot}/{each_img}", image[:, :, ::-1]) # Save (with RGBtoBGR)


The output is ok for this method (with
ffmpeg) but inappropriate for creating the panorama image. Because it didn't give some overlapping frames sequentially in the results. Thus panorama can't be generated.


Trail 2 - Ref


ffmpeg -i check.mp4 -vf decimate=cycle=2,setpts=N/FRAME_RATE/TB -map 0:v out.mp4


didn't work at all.


Trail 3


ffmpeg -i check.mp4 -ss 0 -qscale 0 -f image2 -r 1 out/images%5d.png


No luck either. However, I've found this last
ffmpegcommand was close by far but wasn't enough. Comparatively to others, this gave me a small amount of non-duplicate frames (good) but the bad thing is stilldo not needframes, and I kinda manually pick some desired frames, and then theopecvstitching algorithm works. So, after picking some frames and rotating (as mentioned before) :

stitcher = cv2.Stitcher.create()
status, pano = stitcher.stitch(images) # images: manually picked video frames -_- 



Update


After some trials, I am kinda adopting the non-programming solution. But would love to see an efficient programmatic approach.


On the given demo video, I used
Adobeproducts (premiere proandphotoshop) to do this task, video instruction. But the issue was, I kind of took all video frames at first (without dropping to any frames and that will computationally cost further) viapremierand usephotoshopto stitching them (according to the youtube video instruction). It was too heavy for these editor tools and didn't look better way but the output was better than anything until now. Though I took few (400+ frames) video frames only out of 1200+.




Here are some big challenges. The original video clips have some conditions though, and it's too serious. Unlike the given demo video clips :


- 

- It's not straight forward, i.e. camera shaking
- Lighting condition, i.e. causes different visual look at the same spot
- Cameral flickering or banding








This scenario is not included in the given demo video. And this brings additional and heavy challenges to create panorama images from such videos. Even with the non-programming way (using
adobetools) I couldn't make it any good.


However, for now, all I'm interest to get a panorama image from the given demo video which is without the above condition. But I would love to know any comment or suggestion on that.


-
How can I record, encrypt (in memory), and mux audio and video on android without the files getting out of sync ?
9 février 2021, par RobertWe’re attempting to save video and audio from an Android device into an encrypted file. Our current implementation pipes the outputs from microphone and camera through the MediaEncoder class. As the data is output from MediaEncoder, we are encrypting and writing the contents of the byte buffer to disk. This approach works, however, when attempting to stitch the files back together with FFMPEG, we notice that the two streams seem to get out of sync somewhere mid stream. It appears that a lot of important metadata is lost with this method, specifically presentation timestamps and frame rate data as ffmpeg has to do some guess work to mux the files.


Are there techniques for keeping these streams in sync without using MediaMuxer ? The video is encoded with H.264 and the audio with AAC.


Other Approaches :
We attempted to use the MediaMuxer to mux the output data to a file, but our use case requires that we encrypt the bytes of data before they are saved to disk which eliminates the possibility of using the default constructor.


Additionally, we have attempted to use the newly added (API 26) constructor that takes a FileDescriptor instead and have that pointed to a ParcelFileDescriptor that wrapped an Encrypted Document (https://android.googlesource.com/platform/development/+/master/samples/Vault/src/com/example/android/vault/EncryptedDocument.java). However, this approach led to crashes at the native layer and we believe it may have to do with this comment from the source code (https://android.googlesource.com/platform/frameworks/base.git/+/master/media/java/android/media/MediaMuxer.java#353) about the native writer trying to memory map the output file.


import android.graphics.YuvImage
import android.media.MediaCodec
import android.media.MediaCodecInfo
import android.media.MediaFormat
import android.media.MediaMuxer
import com.callyo.video_10_21.Utils.YuvImageUtils.convertNV21toYUV420Planar
import java.io.FileDescriptor
import java.util.*
import java.util.concurrent.atomic.AtomicReference
import kotlin.properties.Delegates

class VideoEncoderProcessor(
 private val fileDescriptor: FileDescriptor,
 private val width: Int,
 private val height: Int,
 private val frameRate: Int
): MediaCodec.Callback() {
 private lateinit var videoFormat: MediaFormat
 private var trackIndex by Delegates.notNull<int>()
 private var mediaMuxer: MediaMuxer
 private val mediaCodec = createEncoder()
 private val pendingVideoEncoderInputBufferIndices = AtomicReference>(LinkedList())

 companion object {
 private const val VIDEO_FORMAT = "video/avc"
 }

 init {
 mediaMuxer = MediaMuxer(fileDescriptor, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4)
 mediaCodec.setCallback(this)
 mediaCodec.start()
 }

 private fun createEncoder(): MediaCodec {
 videoFormat = MediaFormat.createVideoFormat(VIDEO_FORMAT, width, height).apply {
 setInteger(MediaFormat.KEY_FRAME_RATE, frameRate)
 setInteger(MediaFormat.KEY_COLOR_FORMAT, MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Flexible)
 setInteger(MediaFormat.KEY_BIT_RATE, width * height * 5)
 setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 1)
 }

 return MediaCodec.createEncoderByType(VIDEO_FORMAT).apply {
 configure(videoFormat, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
 }
 }

 override fun onInputBufferAvailable(codec: MediaCodec, index: Int) {
 // logic for handling stream end omitted for clarity

 /* Video frames come in asynchronously from input buffer availability
 * so we need to keep track of available buffers in queue */
 pendingVideoEncoderInputBufferIndices.get().add(index)
 }

 override fun onError(codec: MediaCodec, e: MediaCodec.CodecException) {}

 override fun onOutputFormatChanged(codec: MediaCodec, format: MediaFormat) {
 trackIndex = mediaMuxer.addTrack(format)
 mediaMuxer.start()
 }

 override fun onOutputBufferAvailable(codec: MediaCodec, index: Int, bufferInfo: MediaCodec.BufferInfo) {
 val buffer = mediaCodec.getOutputBuffer(index)
 buffer?.apply {
 if (bufferInfo.size != 0) {
 limit(bufferInfo.offset + bufferInfo.size)
 rewind()
 mediaMuxer.writeSampleData(trackIndex, this, bufferInfo)
 }
 }

 mediaCodec.releaseOutputBuffer(index, false)

 if (bufferInfo.flags == MediaCodec.BUFFER_FLAG_END_OF_STREAM) {
 mediaCodec.stop()
 mediaCodec.release()
 mediaMuxer.stop()
 mediaMuxer.release()
 }
 }

 // Public method that receives raw unencoded video data
 fun encode(yuvImage: YuvImage) {
 // logic for handling stream end omitted for clarity

 pendingVideoEncoderInputBufferIndices.get().poll()?.let { index ->
 val buffer = mediaCodec.getInputBuffer(index)
 buffer?.clear()
 // converting frame to correct color format
 val input =
 yuvImage.convertNV21toYUV420Planar(ByteArray(yuvImage.yuvData.size), yuvImage.width, yuvImage.height)
 buffer?.put(input)
 buffer?.let {
 mediaCodec.queueInputBuffer(index, 0, input.size, System.nanoTime() / 1000, 0)
 }
 }
 }
}


</int>


Additional Info :
I’m using MediaCodec.Callback() (https://developer.android.com/reference/kotlin/android/media/MediaCodec.Callback?hl=en) to handle the encoding asynchronously.


-
Parsing The Clue Chronicles
30 décembre 2018, par Multimedia Mike — Game HackingA long time ago, I procured a 1999 game called Clue Chronicles : Fatal Illusion, based on the classic board game Clue, a.k.a. Cluedo. At the time, I was big into collecting old, unloved PC games so that I could research obscure multimedia formats.
Surveying the 3 CD-ROMs contained in the box packaging revealed only Smacker (SMK) videos for full motion video which was nothing new to me or the multimedia hacking community at the time. Studying the mix of data formats present on the discs, I found a selection of straightforward formats such as WAV for audio and BMP for still images. I generally find myself more fascinated by how computer games are constructed rather than by playing them, and this mix of files has always triggered a strong “I could implement a new engine for this !” feeling in me, perhaps as part of the ScummVM project which already provides the core infrastructure for reimplementing engines for 2D adventure games.
Tying all of the assets together is a custom high-level programming language. I have touched on this before in a blog post over a decade ago. The scripts are in a series of files bearing the extension .ini (usually reserved for configuration scripts, but we’ll let that slide). A representative sample of such a script can be found here :
What Is This Language ?
At the time I first analyzed this language, I was still primarily a C/C++-minded programmer, with a decent amount of Perl experience as a high level language, and had just started to explore Python. I assessed this language to be “mildly object oriented with C++-type comments (‘//’) and reliant upon a number of implicit library functions”. Other people saw other properties. When I look at it nowadays, it reminds me a bit more of JavaScript than C++. I think it’s sort of a Rorschach test for programming languages.Strangely, I sort of had this fear that I would put a lot of effort into figuring out how to parse out the language only for someone to come along and point out that it’s a well-known yet academic language that already has a great deal of supporting code and libraries available as open source. Google for “spanish dolphins far side comic” for an illustration of the feeling this would leave me with.
It doesn’t matter in the end. Even if such libraries exist, how easy would they be to integrate into something like ScummVM ? Time to focus on a workable approach to understanding and processing the format.
Problem Scope
So I set about to see if I can write a program to parse the language seen in these INI files. Some questions :- How large is the corpus of data that I need to be sure to support ?
- What parsing approach should I take ?
- What is the exact language format ?
- Other hidden challenges ?
To figure out how large the data corpus is, I counted all of the INI files on all of the discs. There are 138 unique INI files between the 3 discs. However, there are 146 unique INI files after installation. This leads to a hidden challenge described a bit later.
What parsing approach should I take ? I worried a bit too much that I might not be doing this the “right” way. I’m trying to ignore doubts like this, like how “SQL Shame” blocked me on a task for a little while a few years ago as I concerned myself that I might not be using the purest, most elegant approach to the problem. I know I covered language parsing a lot time ago in university computer science education and there is a lot of academic literature to the matter. But sometimes, you just have to charge in and experiment and prototype and see what falls out. In doing so, I expect to have a better understanding of the problems that need to solved and the right questions to ask, not unlike that time that I wrote a continuous integration system from scratch because I didn’t actually know that “continuous integration” was the keyword I needed.
Next, what is the exact language format ? I realized that parsing the language isn’t the first and foremost problem here– I need to know exactly what the language is. I need to know what the grammar are keywords are. In essence, I need to reverse engineer the language before I write a proper parser for it. I guess that fits in nicely with the historical aim of this blog (reverse engineering).
Now, about the hidden challenges– I mentioned that there are 8 more INI files after the game installs itself. Okay, so what’s the big deal ? For some reason, all of the INI files are in plaintext on the CD-ROM but get compressed (apparently, according to file size ratios) when installed to the hard drive. This includes those 8 extra INI files. I thought to look inside the CAB installation archive file on the CD-ROM and the files were there… but all in compressed form. I suspect that one of the files forms the “root” of the program and is the launching point for the game.
Parsing Approach
I took a stab at parsing an INI file. My approach was to first perform lexical analysis on the file and create a list of 4 types : symbols, numbers, strings, and language elements ([]{}()=., :). Apparently, this is the kind of thing that Lex/Flex are good at. This prototyping tool is written in Python, but when I port this to ScummVM, it might be useful to call upon the services of Lex/Flex, or another lexical analyzer, for there are many. I have a feeling it will be easier to use better tools when I understand the full structure of the language based on the data available.
The purpose of this tool is to explore all the possibilities of the existing corpus of INI files. To that end, I ran all 138 of the plaintext files through it, collected all of the symbols, and massaged the results, assuming that the symbols that occurred most frequently are probably core language features. These are all the symbols which occur more than 1000 times among all the scripts :6248 false 5734 looping 4390 scripts 3877 layer 3423 sequentialscript 3408 setactive 3360 file 3257 thescreen 3239 true 3008 autoplay 2914 offset 2599 transparent 2441 text 2361 caption 2276 add 2205 ge 2197 smackanimation 2196 graphicscript 2196 graphic 1977 setstate 1642 state 1611 skippable 1576 desc 1413 delayscript 1298 script 1267 seconds 1019 rect
About That Compression
I have sorted out at least these few details of the compression :bytes 0-3 "COMP" (a pretty strong sign that this is, in fact, compressed data) bytes 4-11 unknown bytes 12-15 size of uncompressed data bytes 16-19 size of compressed data (filesize - 20) bytes 20- compressed payload
The compression ratios are on the same order of gzip. I was hoping that it was stock zlib data. However, I have been unable to prove this. I wrote a Python script that scrubbed through the first 100 bytes of payload data and tried to get Python’s zlib.decompress to initialize– no luck. It’s frustrating to know that I’ll have to reverse engineer a compression algorithm that deals with just 8 total text files if I want to see this effort through to fruition.
Update, January 15, 2019
Some folks expressed interest in trying to sort out the details of the compression format. So I have posted a followup in which I post some samples and go into deeper details about things I have tried :Reverse Engineering Clue Chronicles Compression
The post Parsing The Clue Chronicles first appeared on Breaking Eggs And Making Omelettes.
Syndiquer tout le site
