Recherche avancée
Médias (91)
-
#3 The Safest Place
16 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
#4 Emo Creates
15 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
#2 Typewriter Dance
15 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
#1 The Wires
11 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Audio
-
ED-ME-5 1-DVD
11 octobre 2011, par
Mis à jour : Octobre 2011
Langue : English
Type : Audio
-
Revolution of Open-source and film making towards open film making
6 octobre 2011, par
Mis à jour : Juillet 2013
Langue : English
Type : Texte
Autres articles (95)
-
Publier sur MédiaSpip
13 juin 2013Puis-je poster des contenus à partir d’une tablette Ipad ?
Oui, si votre Médiaspip installé est à la version 0.2 ou supérieure. Contacter au besoin l’administrateur de votre MédiaSpip pour le savoir -
Support de tous types de médias
10 avril 2011Contrairement à beaucoup de logiciels et autres plate-formes modernes de partage de documents, MediaSPIP a l’ambition de gérer un maximum de formats de documents différents qu’ils soient de type : images (png, gif, jpg, bmp et autres...) ; audio (MP3, Ogg, Wav et autres...) ; vidéo (Avi, MP4, Ogv, mpg, mov, wmv et autres...) ; contenu textuel, code ou autres (open office, microsoft office (tableur, présentation), web (html, css), LaTeX, Google Earth) (...)
-
Ajouter notes et légendes aux images
7 février 2011, parPour pouvoir ajouter notes et légendes aux images, la première étape est d’installer le plugin "Légendes".
Une fois le plugin activé, vous pouvez le configurer dans l’espace de configuration afin de modifier les droits de création / modification et de suppression des notes. Par défaut seuls les administrateurs du site peuvent ajouter des notes aux images.
Modification lors de l’ajout d’un média
Lors de l’ajout d’un média de type "image" un nouveau bouton apparait au dessus de la prévisualisation (...)
Sur d’autres sites (5090)
-
Overthinking My Search Engine Problem
31 décembre 2013, par Multimedia Mike — GeneralI wrote a search engine for my Game Music Appreciation website, because the site would have been significantly less valuable without it (and I would eventually realize that the search feature is probably the most valuable part of this endeavor). I came up with a search solution that was a bit sketchy, but worked… until it didn’t. I thought of a fix but still searched for more robust and modern solutions (where ‘modern’ is defined as something that doesn’t require compiling a C program into a static CGI script and hoping that it works on a server I can’t debug on).
Finally, I realized that I was overthinking the problem– did you know that a bunch of relational database management systems (RDBMSs) support full text search (FTS) ? Okay, maybe you did, but I didn’t know this.
Problem Statement
My goal is to enable users to search the metadata (title, composer, copyright, other tags) attached to various games. To do this, I want to index a series of contrived documents that describe the metadata. 2 examples of these contrived documents, interesting because both of these games have very different titles depending on region, something the search engine needs to account for :system : Nintendo NES game : Snoopy’s Silly Sports Spectacular author : None ; copyright : 1988 Kemco ; dumped by : None additional tags : Donald Duck.nsf Donald Duck
system : Super Nintendo
game : Arcana
author : Jun Ishikawa, Hirokazu Ando ; copyright : 1992 HAL Laboratory ; dumped by : Datschge
additional tags : card.rsn.gamemusic Card Master CardmasterThe index needs to map these documents to various pieces of game music and the search solution needs to efficiently search these documents and find the various game music entries that match a user’s request.
Now that I’ve been looking at it for long enough, I’m able to express the problem surprisingly succinctly. If I had understood that much originally, this probably would have been simpler.
First Solution & Breakage
My original solution was based on SWISH-E. The CGI script was a C program that statically linked the SWISH-E library into a binary that miraculously ran on my web provider. At least, it ran until it decided to stop working a month ago when I added a new feature unrelated to search. It was a very bizarre problem, the details of which would probably bore you to tears. But if you care, the details are all there in the Stack Overflow question I asked on the matter.While no one could think of a direct answer to the problem, I eventually thought of a roundabout fix. The problem seemed to pertain to the static linking. Since I couldn’t count on the relevant SWISH-E library to be on my host’s system, I uploaded the shared library to the same directory as the CGI script and used dlopen()/dlsym() to fetch the functions I needed. It worked again, but I didn’t know for how long.
Searching For A Hosted Solution
I know that anything is possible in this day and age ; while my web host is fairly limited, there are lots of solutions for things like this and you can deploy any technology you want, and for reasonable prices. I figured that there must be a hosted solution out there.I have long wanted a compelling reason to really dive into Amazon Web Services (AWS) and this sounded like a good opportunity. After all, my script works well enough ; if I could just find a simple Linux box out there where I could install the SWISH-E library and compile the CGI script, I should be good to go. AWS has a free tier and I started investigating this approach. But it seems like a rabbit hole with a lot of moving pieces necessary for such a simple task.
I had heard that AWS had something in this area. Sure enough, it’s called CloudSearch. However, I’m somewhat discouraged by the fact that it would cost me around $75 per month to run the smallest type of search instance which is at the core of the service.
Finally, I came to another platform called Heroku. It’s supposed to be super-scalable while having a free tier for hobbyists. I started investigating FTS on Heroku and found this article which recommends using the FTS capabilities of their standard hosted PostgreSQL solution. However, the free tier of Postgres hosting only allows for 10,000 rows of data. Right now, my database has about 5400 rows. I expect it to easily overflow the 10,000 limit as soon as I incorporate the C64 SID music corpus.
However, this Postgres approach planted a seed.
RDBMS Revelation
I have 2 RDBMSs available on my hosting plan– MySQL and SQLite (the former is a separate service while SQLite is built into PHP). I quickly learned that both have FTS capabilities. Since I like using SQLite so much, I elected to leverage its FTS functionality. And it’s just this simple :CREATE VIRTUAL TABLE gamemusic_metadata_fts USING fts3 ( content TEXT, game_id INT, title TEXT ) ;
SELECT id, title FROM gamemusic_metadata_fts WHERE content MATCH "arcana" ;
479|ArcanaThe ‘content’ column gets the metadata pseudo-documents. The SQL gets wrapped up in a little PHP so that it queries this small database and turns the result into JSON. The script is then ready as a drop-in replacement for the previous script.
-
SegFault while freeing nvenc hwdevice_ctx
23 février 2024, par camelCaseFor a project, I created a class encoding the output of an OpenGL
renderbufferobject usingh264_nvenc. Unfortunately, tidying up doesn't work, and the program crashes with a SegFault. The reason is accessing an unaccessable memory region, happening twice in the final lines (see below) when callingav_buffer_unref( &_hwDeviceRefCtx )and implicitely also duringavcodec_free_context( &_pCodecCtx ), but both calls are required for shutting down.

The (in this case relevant)
valgrind-output is

Invalid read of size 8
 at 0x48AD987: UnknownInlinedFun (buffer.c:121)
 by 0x48AD987: UnknownInlinedFun (buffer.c:144)
 by 0x48AD987: av_buffer_unref (buffer.c:139)
 by 0x5D06D7A: avcodec_close (avcodec.c:486)
 by 0x628DD7D: avcodec_free_context (options.c:175)
 by 0x10A863: main (main.cpp:115)
 Address 0x17812700 is 0 bytes inside a block of size 24 free'd
 at 0x484488F: free (vg_replace_malloc.c:985)
 by 0x48AD98F: UnknownInlinedFun (buffer.c:127)
 by 0x48AD98F: UnknownInlinedFun (buffer.c:144)
 by 0x48AD98F: av_buffer_unref (buffer.c:139)
 by 0x48BE098: hwframe_ctx_free (hwcontext.c:240)
 by 0x48AD9A6: UnknownInlinedFun (buffer.c:133)
 by 0x48AD9A6: UnknownInlinedFun (buffer.c:144)
 by 0x48AD9A6: av_buffer_unref (buffer.c:139)
 by 0x5D06D0A: UnknownInlinedFun (decode.c:1261)
 by 0x5D06D0A: avcodec_close (avcodec.c:465)
 by 0x628DD7D: avcodec_free_context (options.c:175)
 by 0x10A863: main (main.cpp:115)
 Block was alloc'd at
 at 0x4849366: posix_memalign (vg_replace_malloc.c:2099)
 by 0x48D9BD5: av_malloc (mem.c:105)
 by 0x48D9DAD: av_mallocz (mem.c:256)
 by 0x48AD8DD: UnknownInlinedFun (buffer.c:44)
 by 0x48AD8DD: av_buffer_create (buffer.c:64)
 by 0x48BDDEB: av_hwdevice_ctx_alloc (hwcontext.c:179)
 by 0x48BDF29: av_hwdevice_ctx_create (hwcontext.c:622)
 by 0x10A482: main (main.cpp:43)

Invalid free() / delete / delete[] / realloc()
 at 0x484488F: free (vg_replace_malloc.c:985)
 by 0x48AD98F: UnknownInlinedFun (buffer.c:127)
 by 0x48AD98F: UnknownInlinedFun (buffer.c:144)
 by 0x48AD98F: av_buffer_unref (buffer.c:139)
 by 0x5D06D7A: avcodec_close (avcodec.c:486)
 by 0x628DD7D: avcodec_free_context (options.c:175)
 by 0x10A863: main (main.cpp:115)
 Address 0x17812700 is 0 bytes inside a block of size 24 free'd
 at 0x484488F: free (vg_replace_malloc.c:985)
 by 0x48AD98F: UnknownInlinedFun (buffer.c:127)
 by 0x48AD98F: UnknownInlinedFun (buffer.c:144)
 by 0x48AD98F: av_buffer_unref (buffer.c:139)
 by 0x48BE098: hwframe_ctx_free (hwcontext.c:240)
 by 0x48AD9A6: UnknownInlinedFun (buffer.c:133)
 by 0x48AD9A6: UnknownInlinedFun (buffer.c:144)
 by 0x48AD9A6: av_buffer_unref (buffer.c:139)
 by 0x5D06D0A: UnknownInlinedFun (decode.c:1261)
 by 0x5D06D0A: avcodec_close (avcodec.c:465)
 by 0x628DD7D: avcodec_free_context (options.c:175)
 by 0x10A863: main (main.cpp:115)
 Block was alloc'd at
 at 0x4849366: posix_memalign (vg_replace_malloc.c:2099)
 by 0x48D9BD5: av_malloc (mem.c:105)
 by 0x48D9DAD: av_mallocz (mem.c:256)
 by 0x48AD8DD: UnknownInlinedFun (buffer.c:44)
 by 0x48AD8DD: av_buffer_create (buffer.c:64)
 by 0x48BDDEB: av_hwdevice_ctx_alloc (hwcontext.c:179)
 by 0x48BDF29: av_hwdevice_ctx_create (hwcontext.c:622)
 by 0x10A482: main (main.cpp:43)


that is also duplicated (due to the calls to
avcodec_free_context()andav_buffer_unref()).

The question is : How can I fix this ?


The (more or less) minimal (not) working example reads


#include <string>

extern "C" {
 #include <libavutil></libavutil>opt.h>
 #include <libavcodec></libavcodec>avcodec.h>
 #include <libavformat></libavformat>avformat.h>
 #include <libavutil></libavutil>hwcontext.h>
 #include <libavutil></libavutil>pixdesc.h>
 #include <libavutil></libavutil>hwcontext_cuda.h>
}

//(former) libx264 encoding based on https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/muxing.c
//update to h264_nvenc with a lot of help from https://stackoverflow.com/questions/49862610/opengl-to-ffmpeg-encode
//and some additional info of https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/vaapi_encode.c

int main() {
 const int _SrcImageWidth=640;
 const int _SrcImageHeight=480;
 
 const AVOutputFormat *_oFmt = nullptr;
 AVFormatContext *_oFmtCtx = nullptr;
 
 const AVCodec *_pCodec = nullptr;
 AVCodecContext *_pCodecCtx = nullptr;
 
 AVFrame* _frame;
 AVPacket* _packet;
 AVStream* _stream;
 
 AVBufferRef *_hwDeviceRefCtx = nullptr;
 const CUcontext* _cudaCtx;
 
 const std::string _OutFileName = "output.mkv";
 
 //constructor part
 int ret;

 //output format context 
 avformat_alloc_output_context2( &_oFmtCtx, nullptr, nullptr, _OutFileName.c_str() );
 _oFmt = _oFmtCtx->oformat;

 //hardware format context
 ret = av_hwdevice_ctx_create( &_hwDeviceRefCtx, AV_HWDEVICE_TYPE_CUDA, "NVIDIA GeForce RTX 4070", nullptr, 0 );

 //hardware frame context for device buffer allocation
 AVBufferRef* hwFrameRefCtx = av_hwframe_ctx_alloc( _hwDeviceRefCtx );
 AVHWFramesContext* hwFrameCtx = (AVHWFramesContext*) (hwFrameRefCtx->data);
 hwFrameCtx->width = _SrcImageWidth;
 hwFrameCtx->height = _SrcImageHeight;
 hwFrameCtx->sw_format = AV_PIX_FMT_0BGR32;
 hwFrameCtx->format = AV_PIX_FMT_CUDA;
 hwFrameCtx->device_ref = _hwDeviceRefCtx;
 hwFrameCtx->device_ctx = (AVHWDeviceContext*) _hwDeviceRefCtx->data;

 ret = av_hwframe_ctx_init( hwFrameRefCtx );

 //get cuda context
 const AVHWDeviceContext* hwDeviceCtx = (AVHWDeviceContext*)(_hwDeviceRefCtx->data);
 const AVCUDADeviceContext* cudaDeviceCtx = (AVCUDADeviceContext*)(hwDeviceCtx->hwctx);
 _cudaCtx = &(cudaDeviceCtx->cuda_ctx);

 //codec context
 _pCodec = avcodec_find_encoder_by_name( "h264_nvenc" );

 _packet = av_packet_alloc();

 _stream = avformat_new_stream( _oFmtCtx, nullptr );
 _stream->id = _oFmtCtx->nb_streams - 1;
 _pCodecCtx = avcodec_alloc_context3( _pCodec );

 _pCodecCtx->qmin = 18;
 _pCodecCtx->qmax = 20;
 _pCodecCtx->width = _SrcImageWidth;
 _pCodecCtx->height = _SrcImageHeight;
 _pCodecCtx->framerate = (AVRational) {25,1};
 _pCodecCtx->time_base = (AVRational) {1,25};
 _stream->time_base = _pCodecCtx->time_base;
 _pCodecCtx->gop_size = 12; //I-Frame every at most 12 frames
 _pCodecCtx->max_b_frames = 2;
 _pCodecCtx->pix_fmt = AV_PIX_FMT_CUDA; //required to use renderbuffer as src
 _pCodecCtx->codec_type = AVMEDIA_TYPE_VIDEO;
 _pCodecCtx->sw_pix_fmt = AV_PIX_FMT_0BGR32; 
 _pCodecCtx->hw_device_ctx = _hwDeviceRefCtx;
 _pCodecCtx->hw_frames_ctx = av_buffer_ref( hwFrameRefCtx );
 av_opt_set(_pCodecCtx->priv_data, "preset", "p7", 0);
 av_opt_set(_pCodecCtx->priv_data, "rc", "vbr", 0);
 if( _oFmtCtx->oformat->flags & AVFMT_GLOBALHEADER ) {
 _pCodecCtx->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
 }

 ret = avcodec_open2( _pCodecCtx, _pCodec, nullptr );
 avcodec_parameters_from_context( _stream->codecpar, _pCodecCtx );

 if (!(_oFmtCtx->oformat->flags & AVFMT_NOFILE)) {
 ret = avio_open(&_oFmtCtx->pb, _OutFileName.c_str(), AVIO_FLAG_WRITE);
 }
 ret = avformat_write_header( _oFmtCtx, nullptr );

 //use hardware frame from above
 _frame = av_frame_alloc();
 ret = av_hwframe_get_buffer( _pCodecCtx->hw_frames_ctx, _frame, 0 );
 _frame->pts = 1;

 av_buffer_unref( &hwFrameRefCtx );

 //destructor part
 av_frame_free( &_frame );
 av_packet_free( &_packet );

 av_write_trailer( _oFmtCtx );
 avio_closep( &_oFmtCtx->pb );

 avformat_free_context( _oFmtCtx );

 avcodec_free_context( &_pCodecCtx );
 av_buffer_unref( &_hwDeviceRefCtx );

 return 0;
}
</string>

and compiles with (linux user)


g++ -lavutil -lavformat -lavcodec -lz -lavutil -lswscale -lswresample -lm -ggdb3 -I/opt/cuda/include main.cpp


Thanks in advance !


-
Is there a way to extract frames from a video file using ffmpeg to memory and make some manipulation on each frame ?
28 octobre 2022, par Rojer BriefThe goal is to extract each time a frame from the video file then make histogram from the image and then to move to the next frame. this way all the frames.


The frames extraction and the histogram manipulation is working fine when the frames have saved as images on the hard disk. but now i want to do it all in memory.


to extract the frames i'm using ffmpeg because i think it's fast enough :


ffmpeg -r 1 -i MyVid.mp4 -r 1 "$filename%03d.png


for now i'm using the ffmpeg in command prompt window.


with this command it will save on the hard disk over 65000 images(frames).
but instead saving them on the hard disk i wonder if i can make the histogram manipulation on each frame in memory instead saving all the 65000 frames to the hard disk.


then i want to find specific images using the histogram and save to the hard disk this frames.


the histogram part for now is also using files from the hard disk and not from the memory :


private void btnLoadHistogram_Click(object sender, System.EventArgs e)
 {
 string[] files = Directory.GetFiles(@"d:\screenshots\", "*.jpg");

 for (int i = 0; i < files.Length; i++)
 {
 sbInfo.Text = "Loading image";
 if (pbImage.Image != null)
 pbImage.Image.Dispose();

 pbImage.Image = Image.FromFile(files[i]);//txtFileName.Text);

 Application.DoEvents();

 sbInfo.Text = "Computing histogram";
 long[] myValues = GetHistogram(new Bitmap(pbImage.Image));

 Histogram.DrawHistogram(myValues);

 sbInfo.Text = ""; 
 } 
 }

 public long[] GetHistogram(System.Drawing.Bitmap picture)
 {
 long[] myHistogram = new long[256];

 for (int i=0;i3;
 myHistogram[Temp]++;
 }

 return myHistogram;
 }


and the code of the class of the constrol HistogramaDesenat :


using System;
using System.Collections;
using System.ComponentModel;
using System.Drawing;
using System.Data;
using System.Windows.Forms;

namespace Histograma
{
 /// <summary>
 /// Summary description for HistogramaDesenat.
 /// </summary>
 public class HistogramaDesenat : System.Windows.Forms.UserControl
 {
 /// <summary> 
 /// Required designer variable.
 /// </summary>
 private System.ComponentModel.Container components = null;

 public HistogramaDesenat()
 {
 // This call is required by the Windows.Forms Form Designer.
 InitializeComponent();

 // TODO: Add any initialization after the InitializeComponent call

 this.Paint += new PaintEventHandler(HistogramaDesenat_Paint);
 this.Resize+=new EventHandler(HistogramaDesenat_Resize);
 }

 /// <summary> 
 /// Clean up any resources being used.
 /// </summary>
 protected override void Dispose( bool disposing )
 {
 if( disposing )
 {
 if(components != null)
 {
 components.Dispose();
 }
 }
 base.Dispose( disposing );
 }

 #region Component Designer generated code
 /// <summary> 
 /// Required method for Designer support - do not modify 
 /// the contents of this method with the code editor.
 /// </summary>
 private void InitializeComponent()
 {
 // 
 // HistogramaDesenat
 // 
 this.Font = new System.Drawing.Font("Tahoma", 8.25F, System.Drawing.FontStyle.Regular, System.Drawing.GraphicsUnit.Point, ((System.Byte)(0)));
 this.Name = "HistogramaDesenat";
 this.Size = new System.Drawing.Size(208, 176);
 }
 #endregion

 private void HistogramaDesenat_Paint(object sender, PaintEventArgs e)
 {
 if (myIsDrawing)
 {

 Graphics g = e.Graphics;
 Pen myPen = new Pen(new SolidBrush(myColor),myXUnit);
 //The width of the pen is given by the XUnit for the control.
 for (int i=0;i/We draw each line 
 g.DrawLine(myPen,
 new PointF(myOffset + (i*myXUnit), this.Height - myOffset), 
 new PointF(myOffset + (i*myXUnit), this.Height - myOffset - myValues[i] * myYUnit));

 //We plot the coresponding index for the maximum value.
 if (myValues[i]==myMaxValue)
 {
 SizeF mySize = g.MeasureString(i.ToString(),myFont);

 g.DrawString(i.ToString(),myFont,new SolidBrush(myColor),
 new PointF(myOffset + (i*myXUnit) - (mySize.Width/2), this.Height - myFont.Height ),
 System.Drawing.StringFormat.GenericDefault);
 }
 }

 //We draw the indexes for 0 and for the length of the array beeing plotted
 g.DrawString("0",myFont, new SolidBrush(myColor),new PointF(myOffset,this.Height - myFont.Height),System.Drawing.StringFormat.GenericDefault);
 g.DrawString((myValues.Length-1).ToString(),myFont, 
 new SolidBrush(myColor),
 new PointF(myOffset + (myValues.Length * myXUnit) - g.MeasureString((myValues.Length-1).ToString(),myFont).Width,
 this.Height - myFont.Height),
 System.Drawing.StringFormat.GenericDefault);

 //We draw a rectangle surrounding the control.
 g.DrawRectangle(new System.Drawing.Pen(new SolidBrush(Color.Black),1),0,0,this.Width-1,this.Height-1);
 }

 }

 long myMaxValue;
 private long[] myValues;
 private bool myIsDrawing;

 private float myYUnit; //this gives the vertical unit used to scale our values
 private float myXUnit; //this gives the horizontal unit used to scale our values
 private int myOffset = 20; //the offset, in pixels, from the control margins.

 private Color myColor = Color.Black;
 private Font myFont = new Font("Tahoma",10);

 [Category("Histogram Options")]
 [Description ("The distance from the margins for the histogram")]
 public int Offset
 {
 set
 {
 if (value>0)
 myOffset= value;
 }
 get
 {
 return myOffset;
 }
 }

 [Category("Histogram Options")]
 [Description ("The color used within the control")]
 public Color DisplayColor
 {
 set
 {
 myColor = value;
 }
 get
 {
 return myColor;
 }
 }

 /// <summary>
 /// We draw the histogram on the control
 /// </summary>
 /// The values beeing draw
 public void DrawHistogram(long[] Values)
 {
 myValues = new long[Values.Length];
 Values.CopyTo(myValues,0);

 myIsDrawing = true;
 myMaxValue = getMaxim(myValues);

 ComputeXYUnitValues();

 this.Refresh();
 }

 /// <summary>
 /// We get the highest value from the array
 /// </summary>
 /// The array of values in which we look
 /// <returns>The maximum value</returns>
 private long getMaxim(long[] Vals)
 {
 if (myIsDrawing)
 {
 long max = 0;
 for (int i=0;i max)
 max = Vals[i];
 }
 return max;
 }
 return 1;
 }

 private void HistogramaDesenat_Resize(object sender, EventArgs e)
 {
 if (myIsDrawing)
 {
 ComputeXYUnitValues();
 }
 this.Refresh();
 }

 private void ComputeXYUnitValues()
 {
 myYUnit = (float) (this.Height - (2 * myOffset)) / myMaxValue;
 myXUnit = (float) (this.Width - (2 * myOffset)) / (myValues.Length-1);
 }
 }
}


so in the end this is what i want to do :


- 

-
extract the frames from the video file in memory using the ffmpeg.


-
instead using Directory.GetFiles i want to make the histogram manipulation on each frame from the memory that is extracted by the ffmpeg.


-
each extracted frame image to use the histogram to find if there is a lightning(weather lightning) in the image.


-
if there is a lightning save the frame image to the hard disk.












-
Syndiquer tout le site
