Recherche avancée

Sur d’autres sites (6885)

• Encoding PNG to MP4 using FFMPEG.js

  11 juillet 2024, par Giles Thompson

  I'm attempting to convert a series of base64 encoded PNG frames (captured from a canvas) to an MP4 video using FFMPEG.js but seem to be running into following error on attempting to run the conversion process, FFMPEG reports the following error :

  


  


  frame0000.png : Invalid data found when processing input

  


  


  The specified file is the first in the sequence of circa 25 PNG frames I'm attempting to convert. I've confirmed that the base64 png file is formatted correctly both by checking its signature and actually downloading a single frame and opening it. However, for some reason FFMPEG doesn't appear to recognise the PNG file data. What follows is a snippet of the relevant parts of the conversion routine, some details have been omitted for the sake of brevity :

  


  import ffmpeg from 'ffmpeg.js/ffmpeg-mp4.js';


  


       // map each PNG frame in the array of frames to an object with 
     //with a name and data component. Crucially the data component 
     //will be the result of the conversion of the base64 data to an 
     //equivalent stream of bytes; i've listed the source of the 
     //referenced conversion function below...
        const inputFiles = videoFrames.map((dataUrl, index) => ({
            name: `frame${index.toString().padStart(4, '0')}.png`,
            data: this.#base64ToUint8Array(dataUrl)
        }));

    
        //build up the FFMPEG command to execute.
        let command = `-loglevel debug -framerate 30`;
        inputFiles.forEach((buffer, index) => {
            command += ` -i frame${index.toString().padStart(4, '0')}.png`; // Correctly padded frame names
        });
        command += ` -c:v libx264 -pix_fmt yuv420p ${outputFileName}`;
        

        //validate png frame data by checking its signature.
        inputFiles.forEach(file => {
            if (!this.#isValidPNG(file.data)) {
                throw new Error(`Invalid PNG file: ${file.name}`);
            }
        });

        // Write each input file to MEMFS
        inputFiles.forEach(file => {
            ffmpeg({
                MEMFS: [file],
                arguments: ['-version'],
                stdin: () => { }
            });
        });

        // Run ffmpeg
        const result = ffmpeg({
            MEMFS: inputFiles,
            //arguments: args,
            arguments: command.split(' '),
            stdin: () => { },
        });



  


  Additionally, here is the referenced ancillary, helper function that converts the base64 encoded PNG files to a stream of equivalent bytes :

  


       #base64ToUint8Array(base64) {
        const binaryString = atob(base64.split(',')[1]);
        const len = binaryString.length;
        const bytes = new Uint8Array(len);
        for (let i = 0; i < len; i++) {
            bytes[i] = binaryString.charCodeAt(i);
        }
        return bytes;
    }


  


  Here is the referenced function that validates the PNG file by checking for the existence and validity of its signature i.e the
magic number that all PNG start with :

  


   #isValidPNG(pngArray) {
        const signature = [0x89, 0x50, 0x4E, 0x47, 0x0D, 0x0A, 0x1A, 0x0A];
        for (let i = 0; i < signature.length; i++) {
            if (pngArray[i] !== signature[i]) {
                return false;
            }
        }
        return true;
    }



  


  The most pertinent details of my current environment are as follows :

  


  


  OS : "Ubuntu Linux".
  
VERSION : "21.04 (Hirsute Hippo)".
  
NPM VERSION : "9.5.0".
  
FFMPEG-JS-VERSION : "^2.8.9001".
  
BROWSER : "Chromium 125.0.6422.60 snap"

  


  


  Is there anything obvious that I'm missing that may account for the error I'm seeing in the debug console ? I should mention that I've switched on debug-level logging in FFMPEG and can provide the full console output if necessary.

  


• Is 'Android+FFMpeg' friendship really available ?

  26 janvier 2016, par vold_by

  The question does not mean that I’m interested if ffmpeg code can be used on Andoid. I know that it can. I’m just asking if somebody has the real performance progress with that stuff.
  I’ve created the question after several weeks of experiments with the stuff and I’ve had enough...
  I do not want to write to branches where people even do not say what kind of video they decode (resolution, codec) and talk only about some mystical FPS. I just don’t understand what they want to do. Also I’m not going to develop application only for my phone or for Android 2.2++ phones that have some extended OpenGL features. I have quite popular phone HTC Desire so if the application does not work on it, so what’s next ?

  Well, what do I have ?

  1. FFMpeg source from the latest HEAD branch. Actually I could not buld it with NDK5 so I decided to use stolen one.

  2. Bambuser’s build script (bash) with appropriate ffmpeg source ([web] : http://bambuser.com/r/opensource/ffmpeg-4f7d2fe-android-2011-03-07.tar.gz).
     It builds well after some corrections by using NDK5.

  3. Rockplayer’s gelded ffmpeg source code with huge Android.mk in the capacity of build script ([web] : http://www.rockplayer.com/download/rockplayer_ffmpeg_git_20100418.zip).
     It builds by NDK3 and NDK5 after some corrections. Rockplayer is probably the most cool media player for Android and I supposed that I would have some perks using it’s build.

  I had suitable video for a project (is not big and is not small) : 600x360 H.264.

  Both libraries we got from clauses 2 and 3 provide us possibility to get frames from video (frame-by-frame, seek etc.). I did not try to get an audio track because I did not need one for the project. I’m not publishing my source here because I think that’s traditional and it’s easy to find.

  Well, what’s the results with video ?
  HTC Desire, Android 2.2
  600x360, H.264
  decoding and rendering are in different threads

  1. Bambuser (NDK5 buld for armv5te, RGBA8888) : 33 ms/frame average.
  2. Rockplayer (NDK3 build for neon, RGB565) : 27 ms/frame average.

  It’s not bad for the first look, but just think that these are results only to decode frames.
  If somebody has much better results with decoding time, let me know.

  The most hard thing for a video is rendering. If we have bitmap 600x360 we should scale one somehow before painting because different phones have different screen sizes and we can not expect that our video will be the same size as screen.

  What options do we have to rescale a frame to fit it to screen ?
  I was able to check (the same phone and video source) those cases :

  1. sws_scale() C function in Bambuser’s build : 70 ms/frame. Unacceptable.
  2. Stupid bitmap rescaling in Android (Bitmap.createScaledBitmap) : 65 ms/frame. Unacceptable.
  3. OpenGL rendering in ortho projection on textured quad. In this case I did not need to scale frame. I just needed to prepare texture 1024x512 (in my case it was RGBA8888) containig frame pixels and than load it in GPU (gl.glTexImage2D). Result : 220 ms/frame to render. Unacceptable. I did not expect that glTexImage2D just sucked on Snapdragon CPU.

  That’s all.
  I know that there is some way to use fragment shader to convert YUV pixels using GPU, but we will have the same glTexImage2D and 200 ms just to texture loading.

  But this is not the end. ...my only friend the end... :)
  It’s not hopeless condition.

  Trying to use RockPlayer you definitely will wonder how they do that damn frame scaling so fast. I suppose that they have really good experience in ARM achitecture. They most probably use avcodec_decode_video2 and than img_convert (as I did in RP version), but then they use some tricks (depends of ARM version) for scaling.
  Maybe they also have some "magic" buld configuration for ffmpeg decreasing decoding time but Android.mk that they published is not THE Android.mk they use. Dunno...

  So, now it looks like you can not just buld some easy JNI bridge for ffmpeg and than have real media player for Android platform. You can do this only if you have suitable video that you do not need to scale.

  Any ideas ? I hope for you ;)

• ffmpeg x11grab Fedora 22 cannot open display [on hold]

  4 août 2015, par voor

  Running a fairly recent version of Fedora 22 :

  Linux localhost 4.1.3-200.fc22.x86_64 #1 SMP Wed Jul 22 19:51:58 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

  Dual screen Acer XB270HU and AOC 2369. Also uses NVIDIA Drivers :

  ➜  ~  rpm -qa | grep nvidia
  
  nvidia-driver-libs-352.30-1.fc22.x86_64
  
  nvidia-driver-352.30-1.fc22.x86_64
  
  nvidia-settings-352.30-1.fc22.x86_64
  
  nvidia-libXNVCtrl-352.30-1.fc22.x86_64
  
  dkms-nvidia-352.30-1.fc22.x86_64
  
  nvidia-driver-libs-352.30-1.fc22.i686

  Whenever I attempt to do a x11grab in ffmpeg I get an error :

  ➜  ~  ffmpeg -f x11grab -r 15 -s 1920x1080 -i :0 -vcodec h264 -crf 30 -y output.mp4
  
  ffmpeg version 2.6.3 Copyright (c) 2000-2015 the FFmpeg developers
  
    built with gcc 5.1.1 (GCC) 20150422 (Red Hat 5.1.1-1)
  
    configuration: --prefix=/usr --bindir=/usr/bin --datadir=/usr/share/ffmpeg --incdir=/usr/include/ffmpeg --libdir=/usr/lib64 --mandir=/usr/share/man --arch=x86_64 --optflags='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic' --enable-bzlib --disable-crystalhd --enable-frei0r --enable-gnutls --enable-ladspa --enable-libass --enable-libcdio --enable-libdc1394 --disable-indev=jack --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-openal --enable-libopencv --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libschroedinger --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvorbis --enable-libv4l2 --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-x11grab --enable-avfilter --enable-avresample --enable-postproc --enable-pthreads --disable-static --enable-shared --enable-gpl --disable-debug --disable-stripping --shlibdir=/usr/lib64 --enable-runtime-cpudetect
  
    libavutil      54. 20.100 / 54. 20.100
  
    libavcodec     56. 26.100 / 56. 26.100
  
    libavformat    56. 25.101 / 56. 25.101
  
    libavdevice    56.  4.100 / 56.  4.100
  
    libavfilter     5. 11.102 /  5. 11.102
  
    libavresample   2.  1.  0 /  2.  1.  0
  
    libswscale      3.  1.101 /  3.  1.101
  
    libswresample   1.  1.100 /  1.  1.100
  
    libpostproc    53.  3.100 / 53.  3.100
  
  Invalid MIT-MAGIC-COOKIE-1 key[x11grab @ 0x1fe1d40] Cannot open display :0, error 1.
  
  :0: Input/output error

  I’ve tried a few different commands basically copy/pasta from the web on how to do screengrabs. I don’t need audio, just video, and all of them result in the same error.

  I am able to do xwininfo without any problems :

  ➜  ~  xwininfo    
  
  
  
  xwininfo: Please select the window about which you
  
            would like information by clicking the
  
            mouse in that window.
  
  
  
  xwininfo: Window id: 0x1e0006c "linux - ffmpeg x11grab Fedora 22 cannot open display - Stack Overflow - Google Chrome"
  
  
  
    Absolute upper-left X:  0
  
    Absolute upper-left Y:  27
  
    Relative upper-left X:  0
  
    Relative upper-left Y:  27
  
    Width: 2560
  
    Height: 1413
  
    Depth: 24
  
    Visual: 0x21
  
    Visual Class: TrueColor
  
    Border width: 0
  
    Class: InputOutput
  
    Colormap: 0x20 (installed)
  
    Bit Gravity State: ForgetGravity
  
    Window Gravity State: NorthWestGravity
  
    Backing Store State: NotUseful
  
    Save Under State: no
  
    Map State: IsViewable
  
    Override Redirect State: no
  
    Corners:  +0+27  -1920+27  -1920-0  0
  
    -geometry 2560x14130