Recherche avancée

Sur d’autres sites (11836)

• How to solve Accord.Video.FFMPEG memory leak problem

  26 mai 2021, par mfwoo

  I am developing a digital billboard application that allow customer to click on the touch screen to go back and forth.

  


  Screen 0 -> touch -> Screen 1 -> touch -> Screen 2 -> time out -> Screen 0

  


  If no interaction happens Screen 0 will loop indefinitely. Every Screen is running its own MP4 file.

  


  However, for every running cycle of Screen 1, it gobbled up memory and in no time the application crash.

  


  Is it because of VideoFileSource's video object is not being dispose properly or because of some threading problem in video_NewFrame ?

  


  Because I get this error occasionally - "Invoke or BeginInvoke cannot be called on a control until the windows handle is created"

  


  I am using VS2017 .NET Framework 4.5 with Accord.Video.FFMPEG by Accord.NET version 3.8

  


  Screen 0 MP4 size - 5.5MB
Screen 1 MP4 size - 5.6MB
Screen 2 MP4 size - 7.0MB

  


  Here is my code :-
...

  


  Bitmap image;
VideoFileSource video;
int screenIdx = 0;
bool enableClicking = true;
bool isTimeOut = false;
string VideoPath = @"d:\KioskApp\Bkgrnd\"

public frmMain()
    {
        InitializeComponent(); 
        StartFirstScreen();
        tmrScreen01.Interval = 10000;
        tmrScreen02.Interval = 10000;
    }
    
     private void StartFirstScreen()
    {
        try
        {
            string fileName = VideoPath + Screen00();
            video = new VideoFileSource(fileName);
            video.PlayingFinished += new Accord.Video.PlayingFinishedEventHandler(video_Finished);
            video.NewFrame += new Accord.Video.NewFrameEventHandler(video_NewFrame);
            video.Start();
            screenIdx = 1;
        }
        catch (Exception ex)
        {
            string strErrMsg = strMsg + " - " + ex.Message;
            MessageBox.Show(strErrMsg);
        }
    }
    
    private void video_NewFrame(object sender, Accord.Video.NewFrameEventArgs eventArgs)
    {
        try
        {
            Invoke(new Action(() =>
            {
                System.Drawing.Image OldImage;
                OldImage = pictureBox1.Image;
                pictureBox1.Image = AForge.Imaging.Image.Clone(eventArgs.Frame);
                if (OldImage != null)
                    OldImage.Dispose();
            }));    
        }
        catch (Exception ex)
        {
            var strErrMsg = "video_NewFrame - " + ex.Message;
            MessageBox.Show(strErrMsg);
        }
    }
    
     private void video_Finished(object sender, Accord.Video.ReasonToFinishPlaying reason)
    {
        try
        {
            if (screenIdx == 1)
            {
                video.PlayingFinished -= video_Finished;
                video.NewFrame -= video_NewFrame;
                video = null;                    
                StartFirstScreen();
                return;
            }
            enableClicking = true;

        }
        catch (Exception ex)
        {
            var strErrMsg = "video_Finished - " + ex.Message;
            MessageBox.Show(strErrMsg);

        }
    }
    
    void startLastScreen()
    {
        string fileName = string.Empty;
        video.SignalToStop();
        fileName = VideoPath + Screen02();
        screenIdx = 0;
        if (object.ReferenceEquals(null, video))
        {
            video = new VideoFileSource(fileName);
        }
        else
        {
            video = null;
            video = new VideoFileSource(fileName);
        }

        video.PlayingFinished += new Accord.Video.PlayingFinishedEventHandler(video_Finished);
        video.NewFrame += new Accord.Video.NewFrameEventHandler(video_NewFrame);
        video.Start();
        enableClicking = false;
    }
    
    private void pictureBox1_Click(object sender, EventArgs e)
    {
        if (!enableClicking && screenIdx != 1) return;

        tmrScreen01.Stop();
        tmrScreen02.Stop();
        
        //  Check clickable area before allow to proceed to the next screen     
        string fileName = string.Empty;
        video.SignalToStop();
        video.Stop();

        if (screenIdx == 0)
        {
            fileName = VideoPath + Screen00();
            screenIdx = 1;
        }
        else if (screenIdx == 1)
        {
            fileName = VideoPath + Screen01();
            screenIdx = 2;
            
        }
        else if (screenIdx == 2)
        {
            fileName = VideoPath + Screen02();
            screenIdx = 0;
           
        }

        if (object.ReferenceEquals(null, video))
        {
            video = new VideoFileSource(fileName);
        }
        else
        {
            video = null;
            video = new VideoFileSource(fileName);
        }
        video.PlayingFinished += new Accord.Video.PlayingFinishedEventHandler(video_Finished);
        video.NewFrame += new Accord.Video.NewFrameEventHandler(video_NewFrame);
        enableClicking = false;
        isTimeOut = false;
        video.Start();
    }


  


  ...

  


• How to record/trim/combine audio seamlessly in a React/Node web app

  16 mai 2021, par Rayhan Memon

  I've developed a digital audio workstation for the browser that you can check out here. Essentially it helps authors narrate their own audiobooks themselves at home.

  


  I'm looking to dramatically improve the speed at which audio files are combined or trimmed.

  


  Right now, the user records some variable amount of audio (a line, paragraph, or entire passage). When the user stops recording, this clip is added to the main audio file for the section using ffmpeg.wasm like so :

  


          if (duration === 0) {
            //concatenate the two files (they should already be written to memory)
            await ffmpeg.run('-i', 'concat:fullAudio.mp3|clip.mp3', '-c', 'copy', 'fullAudio.mp3');

        } else {
            //Set the insert time to wherever the user's cursor is positioned
            let insertTime = duration;
            if (selectedObj) {
                insertTime = selectedObj.endTime;
            }

            //split the audio file into two parts at the point we want to insert the audio
            await ffmpeg.run('-i', 'fullAudio.mp3', '-t', `${insertTime}`, '-c', 'copy', 'part1.mp3', '-ss',  `${insertTime}`, '-codec', 'copy', 'part2.mp3');

            //concatenate the three files
            await ffmpeg.run('-i', 'concat:part1.mp3|clip.mp3', '-acodec', 'copy', 'intermediate.mp3');
            await ffmpeg.run('-i', 'concat:intermediate.mp3|part2.mp3', '-acodec', 'copy', 'fullAudio.mp3');
        }

        //Read the result from memory
        const data = ffmpeg.FS('readFile', 'fullAudio.mp3');

        //Create URL so it can be used in the browser
        const url = URL.createObjectURL(new Blob([data.buffer], { type: 'audio/mp3' }));
        globalDispatch({ type: "setFullAudioURL", payload: url });


  


  After every recorded clip, the user is forced to wait a few seconds for this concatenation process to finish up - and the longer the main file or recorded clip gets, the longer the user has to wait. Looking at other browser-based audio editors such as AudioMass, it clearly seems possible to make a seamless recording and editing experience with zero wait time, but I can't seem to figure out how to do the same within my react app.

  


  Is it possible to seamlessly combine or trim audio data within a React app ? Is FFMPEG the best way to go about it, or are there simpler ways using pure javascript ?

  


• How to handle DHAV (.dav) video streams ?

  16 juin 2022, par Mateus Henrique

  I have a WPF app where I need to handle DHAV (.dav) video stream on runtime from a Digital Video Recorder (DVR). I'm using an SDK that can be found here Dahua SDK search

  


  SDK: General_NetSDK_Eng_Win64_IS_V3.052.0000002.0.R.201103


  


  I need to handle every single frame from the video stream, convert it to a BitmapImage and then displays it in a WPF Image control. Something like : MJPEG Decoder

  


  The problem is that I can't find any documentation on how to handle that data and the samples from the SDK doesn't show that either, instead they are built with WinForms and they only pass the Window Handle of the PictureBox's control to an exported DLL function and 'magically' shows the video stream :

  


  [DllImport(LIBRARYNETSDK)]
public static extern IntPtr CLIENT_RealPlayEx(IntPtr lLoginID, int nChannelID, IntPtr hWnd, EM_RealPlayType rType);


  


  


  OBS : 'hWnd' param is the Window Handle to display the video in.

  


  


  The problem with this approach is that I don't have any control over the video stream.

  


  I have tried many FFMPEG wrappers for .NET but they only parse the data if I first write it to disk and only then I can convert it to some type I can handle.

  


  This is the callback function that is called contantly during the application's runtime with the data I need to handle :

  


      private void RealDataCallback(IntPtr lRealHandle, uint dwDataType, IntPtr pBuffer, uint dwBufSize, IntPtr param, IntPtr dwUser)
    {
        switch (dwDataType)
        {
            case 0: // original data
                break;
            case 1: // frame data
                HandleFrameData(lRealHandle, dwDataType, pBuffer, dwBufSize, param, dwUser);
                break;
            case 2: // yuv data
                break;
            case 3: // pcm audio data
                break;
        }
    }

    private void HandleFrameData(IntPtr lRealHandle, uint dwDataType, IntPtr pBuffer, uint dwBufSize, IntPtr param, IntPtr dwUser)
    {
        // The pBuffer parameter format is DHAV (.dav) 
        byte[] buff = new byte[dwBufSize];
        Marshal.Copy(pBuffer, buff, 0, (int)dwBufSize);

        using (var ms = new MemoryStream(buff))
        {
        }
    }


  


  UPDATE

  


  I'm able to convert the YUV data provided in the callback funcion to RGB but that is not the ideal solution. It would be so much better (and faster) if I can convert the original (.dav) data.

  


  The RealDataCallback, in fact, only returns 1 frame per callback, but I don't know how to convert that frame to Bitmap. Any help would be appreciated.