Recherche avancée

Sur d’autres sites (14989)

• Workflow for creating animated hand-drawn videos - encoding difficulties

  8 décembre 2017, par Mircode

  I want to create YouTube videos, kind of in the style of a white-board animation.

  Tldr question : How can I encode into a lossless rgb video format with ffmpeg, including alpha channel ?

  More detailed :
  My current workflow looks like this :

  I draw the slides in Inkscape, I group all paths that are to be drawn in one go (one scene so to say) and store the slide as svg. Then I run a custom python script over that. It animates the slides as described here https://codepen.io/MyXoToD/post/howto-self-drawing-svg-animation. Each frame is exported as svg, converted to png and fed to ffmpeg for making a video from it.

  For every scene (a couple of paths being drawn, there are several scenes per slide) I create an own video file and then I also store a png file that contains the last frame of that video.

  I then use kdenlive to join it all together : A video containing the drawing of the first scene, then a png which holds the last image of the video while I talk about the drawing, then the next animated drawing, then the next still image where I continue talking and so on. I use these intermediate images because freezing the last frame is tedious in kdenlive and I have around 600 scenes. Here I do the editing, adjust the duration of the still images and render the final video.

  The background of the video is a photo of a blackboard which never changes, the strokes are paths with a filter to make it look like chalk.

  So far so good, everything almost works.

  My problem is : Whenever there is a transition between an animation and a still image, it is visible in the final result. I have tried several approaches to make this work but nothing is without flaws.

  My first approach was to encode the animations as mp4 like this :

  p = Popen(['ffmpeg', '-y', '-f', 'image2pipe', '-vcodec', 'png', '-r', str(fps), '-i', '-', '-vcodec', 'libx264', '-crf', '21', '-bf', '2', '-flags', '+cgop', '-pix_fmt', 'yuv420p', '-movflags', 'faststart', '-r', str(fps), videofile], stdin=PIPE)

  which is recommended for YouTube. But then there is a little brightness difference between video and still image.

  Then I tried mov with png codec :

  p = Popen(['ffmpeg', '-y', '-f', 'image2pipe', '-vcodec', 'png', '-r', str(fps), '-i', '-', '-vcodec', 'png', '-r', str(fps), videofile], stdin=PIPE)

  I think this encodes every frame as png in the video. It creates way bigger files since every frame is encoded separately. But it’s ok since I can use transparency for the background and just store the chalk strokes. However, sometimes I want to swipe parts of the chalk on a slide away, which I do by drawing background over it. Which would work if those overlaid, animated background chunks which are stored in the video looked exactly like the underlying png in the background. But it doesn’t. It’s slightly more blurry and I believe the color changes a tiny bit as well. Which I don’t understand since I thought the video just stores a sequence of pngs... Is there some quality setting that I miss here ?

  Then I read about ProRes4444 and tried that :

  p = Popen(['ffmpeg', '-y', '-f', 'image2pipe', '-vcodec', 'png', '-r', str(fps), '-i', '-', '-c:v', 'prores_ks', '-pix_fmt', 'yuva444p10le', '-alpha_bits', '8', '-profile:v', '4444', '-r', str(fps), videofile], stdin=PIPE)

  and this actually seems to work. However, the animation files become larger than the bunch of png files they contain, probably because this format stores 12 bit per channel. This is not thaat horrible since only intermediate videos grow big, the final result is still ok.

  But ideally there would be a lossless codec which stores in rgb colorspace with 8 bit per channel, 8 bit for alpha and considers only the difference to the previous frame (because all that changes from frame to frame is a tiny bit of chalk drawing). Is there such a thing ? Alternatively, I’d also be ok without transparency but then I have to store the background in every scene. But if only changes are stored from frame to frame within one scene, that should be manageable.

  Or should I make some fundamental changes in my workflow altogether ?

  Sorry that this is rather lengthy, I appreciate any help.

  Cheers !

• How to get the thumbnail of base64 encoded video file in Nodejs ?

  3 octobre 2018, par Wai Yan Hein

  I am developing a web application using Nodejs. I am using Amazon S3 bucket to store files. What I am doing now is that when I upload a video file (mp4) to the S3 bucket, I will get the thumbnail photo of the video file from the lambda function. For fetching the thumbnail photo of the video file, I am using this package - https://www.npmjs.com/package/ffmpeg. I tested the package locally on my laptop and it is working.

  Here is my code tested on my laptop

  var ffmpeg = require('ffmpeg');
  
  
  
  module.exports.createVideoThumbnail = function(req, res)
  
  {
  
      try {
  
          var process = new ffmpeg('public/lalaland.mp4');
  
          process.then(function (video) {
  
  
  
              video.fnExtractFrameToJPG('public', {
  
                  frame_rate : 1,
  
                  number : 5,
  
                  file_name : 'my_frame_%t_%s'
  
              }, function (error, files) {
  
                  if (!error)
  
                      console.log('Frames: ' + files);
  
                  else
  
                      console.log(error)
  
              });
  
  
  
          }, function (err) {
  
              console.log('Error: ' + err);
  
          });
  
      } catch (e) {
  
          console.log(e.code);
  
          console.log(e.msg);
  
      }
  
      res.json({ status : true , message: "Video thumbnail created." });
  
  }

  The above code works well. It gave me the thumbnail photos of the video file (mp4). Now, I am trying to use that code in the AWS lambda function. The issue is the above code is using video file path as the parameter to fetch the thumbnails. In the lambda function, I can only fetch the base 64 encoded format of the file. I can get id (s3 path) of the file, but I cannot use it as the parameter (file path) to fetch the thumbnails as my s3 bucket does not allow public access.

  So, what I tried to do was that I tried to save the base 64 encoded video file locally in the lambda function project itself and then passed the file path as the parameter for fetching the thumbnails. But the issue was that AWS lamda function file system is read-only. So I cannot write any file to the file system. So what I am trying to do right now is to retrieve the thumbnails directly from the base 64 encoded video file. How can I do it ?

• Open VideoStream using OpenCV 4.5.1 works on Windows but fails on Docker python:3.9.2-slim-buster for specific IP cam

  18 mai 2021, par Qua285

  I have 2 ip cameras - one of Hikvision and another of Provision ISR. Both use Onvif and work on VLC.
I've written a simple python script to record images every 5 sec from their video stream.
On Windows 10, using VSCode they both work as expected. Once deployed to a Docker container, my script works as expected with the Hikvision but fails with the Provision ISR - it doesn't open the stream.

  


  Running python -c "import cv2; print(cv2.getBuildInformation())" on windows (venv 3.9.2) and on docker machine bring slightly different results but it's beyond my understanding to take something out of it...
Here is the Windows one :

  


  General configuration for OpenCV 4.5.1 =====================================
  Version control:               4.5.1

  Platform:
    Timestamp:                   2021-01-02T14:30:58Z
    Host:                        Windows 6.3.9600 AMD64
    CMake:                       3.18.4
    CMake generator:             Visual Studio 14 2015 Win64
    CMake build tool:            C:/Program Files (x86)/MSBuild/14.0/bin/MSBuild.exe
    MSVC:                        1900

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (15 files):         + SSSE3 SSE4_1
      SSE4_2 (1 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (0 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (4 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (29 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2

  C/C++:
    Built as dynamic libs?:      NO
    C++ standard:                11
    C++ Compiler:                C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe  (ver 19.0.24241.7)
    C++ flags (Release):         /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 
/MP  /MT /O2 /Ob2 /DNDEBUG
    C++ flags (Debug):           /DWIN32 /D_WINDOWS /W4 /GR  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /EHa /wd4127 /wd4251 /wd4324 /wd4275 /wd4512 /wd4589 
/MP  /MTd /Zi /Ob0 /Od /RTC1
    C Compiler:                  C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/bin/x86_amd64/cl.exe
    C flags (Release):           /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP   /MT /O2 /Ob2 /DNDEBUG
    C flags (Debug):             /DWIN32 /D_WINDOWS /W3  /D _CRT_SECURE_NO_DEPRECATE /D _CRT_NONSTDC_NO_DEPRECATE /D _SCL_SECURE_NO_WARNINGS /Gy /bigobj /Oi  /fp:precise     /MP /MTd /Zi /Ob0 /Od /RTC1
    Linker flags (Release):      /machine:x64  /NODEFAULTLIB:atlthunk.lib /INCREMENTAL:NO  /NODEFAULTLIB:libcmtd.lib /NODEFAULTLIB:libcpmtd.lib /NODEFAULTLIB:msvcrtd.lib
    Linker flags (Debug):        /machine:x64  /NODEFAULTLIB:atlthunk.lib /debug /INCREMENTAL  /NODEFAULTLIB:libcmt.lib /NODEFAULTLIB:libcpmt.lib /NODEFAULTLIB:msvcrt.lib
    ccache:                      NO
    Precompiled headers:         YES
    Extra dependencies:          ade wsock32 comctl32 gdi32 ole32 setupapi ws2_32
    3rdparty dependencies:       ittnotify libprotobuf zlib libjpeg-turbo libwebp libpng libtiff libopenjp2 IlmImf quirc ippiw ippicv

  OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java python2 ts
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         NO

  Windows RT support:            NO

  GUI:
    Win32 UI:                    YES
    VTK support:                 NO

  Media I/O:
    ZLib:                        build (ver 1.2.11)
    JPEG:                        build-libjpeg-turbo (ver 2.0.6-62)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         build (ver 1.6.37)
    TIFF:                        build (ver 42 - 4.0.10)
    JPEG 2000:                   build (ver 2.3.1)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES (prebuilt binaries)
      avcodec:                   YES (58.91.100)
      avformat:                  YES (58.45.100)
      avutil:                    YES (56.51.100)
      swscale:                   YES (5.7.100)
      avresample:                YES (4.0.0)
    GStreamer:                   NO
    DirectShow:                  YES
    Media Foundation:            YES
      DXVA:                      NO

  Parallel framework:            Concurrency

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2020.0.0 Gold [2020.0.0]
           at:                   C:/Users/appveyor/AppData/Local/Temp/1/pip-req-build-wvn_it83/_skbuild/win-amd64-3.9/cmake-build/3rdparty/ippicv/ippicv_win/icv
    Intel IPP IW:                sources (2020.0.0)
              at:                C:/Users/appveyor/AppData/Local/Temp/1/pip-req-build-wvn_it83/_skbuild/win-amd64-3.9/cmake-build/3rdparty/ippicv/ippicv_win/iw
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.5.1)

  OpenCL:                        YES (NVD3D11)
    Include path:                C:/Users/appveyor/AppData/Local/Temp/1/pip-req-build-wvn_it83/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 C:/Python39-x64/python.exe (ver 3.9)
    Libraries:                   C:/Python39-x64/libs/python39.lib (ver 3.9.0)
    numpy:                       C:/Users/appveyor/AppData/Local/Temp/1/pip-build-env-sk7r7w_5/overlay/Lib/site-packages/numpy/core/include (ver 1.19.3)
    install path:                python

  Python (for build):            C:/Python27-x64/python.exe

  Java:
    ant:                         NO
    JNI:                         C:/Program Files/Java/jdk1.8.0/include C:/Program Files/Java/jdk1.8.0/include/win32 C:/Program Files/Java/jdk1.8.0/include
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    C:/Users/appveyor/AppData/Local/Temp/1/pip-req-build-wvn_it83/_skbuild/win-amd64-3.9/cmake-install
-----------------------------------------------------------------


  


  this is the Docker one (python:3.9.2-slim-buster) :

  


  General configuration for OpenCV 4.5.1 =====================================
  Version control:               4.5.1-dirty

  Platform:
    Timestamp:                   2021-01-02T13:04:10Z
    Host:                        Linux 4.15.0-1077-gcp x86_64
    CMake:                       3.18.4
    CMake generator:             Unix Makefiles
    CMake build tool:            /bin/gmake
    Configuration:               Release

  CPU/HW features:
    Baseline:                    SSE SSE2 SSE3
      requested:                 SSE3
    Dispatched code generation:  SSE4_1 SSE4_2 FP16 AVX AVX2 AVX512_SKX
      requested:                 SSE4_1 SSE4_2 AVX FP16 AVX2 AVX512_SKX
      SSE4_1 (15 files):         + SSSE3 SSE4_1
      SSE4_2 (1 files):          + SSSE3 SSE4_1 POPCNT SSE4_2
      FP16 (0 files):            + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 AVX
      AVX (4 files):             + SSSE3 SSE4_1 POPCNT SSE4_2 AVX
      AVX2 (29 files):           + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2
      AVX512_SKX (4 files):      + SSSE3 SSE4_1 POPCNT SSE4_2 FP16 FMA3 AVX AVX2 AVX_512F AVX512_COMMON AVX512_SKX

  C/C++:
    Built as dynamic libs?:      NO
    C++ standard:                11
    C++ Compiler:                /usr/lib/ccache/compilers/c++  (ver 9.3.1)
    C++ flags (Release):         -Wl,-strip-all   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -Wl,-strip-all   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/lib/ccache/compilers/cc
    C flags (Release):           -Wl,-strip-all   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -Wl,-strip-all   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Wno-comment -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):      -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a -L/root/ffmpeg_build/lib  -Wl,--gc-sections -Wl,--as-needed
    Linker flags (Debug):        -Wl,--exclude-libs,libippicv.a -Wl,--exclude-libs,libippiw.a -L/root/ffmpeg_build/lib  -Wl,--gc-sections -Wl,--as-needed
    ccache:                      YES
    Precompiled headers:         NO
    Extra dependencies:          ade Qt5::Core Qt5::Gui Qt5::Widgets Qt5::Test Qt5::Concurrent /lib64/libpng.so /lib64/libz.so dl m pthread rt
    3rdparty dependencies:       ittnotify libprotobuf libjpeg-turbo libwebp libtiff libopenjp2 IlmImf quirc ippiw ippicv

  OpenCV modules:
    To be built:                 calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo python3 stitching video videoio
    Disabled:                    world
    Disabled by dependency:      -
    Unavailable:                 java python2 ts
    Applications:                -
    Documentation:               NO
    Non-free algorithms:         NO

  GUI:
    QT:                          YES (ver 5.15.0)
      QT OpenGL support:         NO
    GTK+:                        NO
    VTK support:                 NO

  Media I/O:
    ZLib:                        /lib64/libz.so (ver 1.2.7)
    JPEG:                        libjpeg-turbo (ver 2.0.6-62)
    WEBP:                        build (ver encoder: 0x020f)
    PNG:                         /lib64/libpng.so (ver 1.5.13)
    TIFF:                        build (ver 42 - 4.0.10)
    JPEG 2000:                   build (ver 2.3.1)
    OpenEXR:                     build (ver 2.3.0)
    HDR:                         YES
    SUNRASTER:                   YES
    PXM:                         YES
    PFM:                         YES

  Video I/O:
    DC1394:                      NO
    FFMPEG:                      YES
      avcodec:                   YES (58.109.100)
      avformat:                  YES (58.61.100)
      avutil:                    YES (56.60.100)
      swscale:                   YES (5.8.100)
      avresample:                NO
    GStreamer:                   NO
    v4l/v4l2:                    YES (linux/videodev2.h)

  Parallel framework:            pthreads

  Trace:                         YES (with Intel ITT)

  Other third-party libraries:
    Intel IPP:                   2020.0.0 Gold [2020.0.0]
           at:                   /tmp/pip-req-build-ddpkm6fn/_skbuild/linux-x86_64-3.9/cmake-build/3rdparty/ippicv/ippicv_lnx/icv
    Intel IPP IW:                sources (2020.0.0)
              at:                /tmp/pip-req-build-ddpkm6fn/_skbuild/linux-x86_64-3.9/cmake-build/3rdparty/ippicv/ippicv_lnx/iw
    Lapack:                      NO
    Eigen:                       NO
    Custom HAL:                  NO
    Protobuf:                    build (3.5.1)

  OpenCL:                        YES (no extra features)
    Include path:                /tmp/pip-req-build-ddpkm6fn/opencv/3rdparty/include/opencl/1.2
    Link libraries:              Dynamic load

  Python 3:
    Interpreter:                 /opt/python/cp39-cp39/bin/python (ver 3.9)
    Libraries:                   libpython3.9.a (ver 3.9.0)
    numpy:                       /tmp/pip-build-env-jqrfyj0w/overlay/lib/python3.9/site-packages/numpy/core/include (ver 1.19.3)
    install path:                python

  Python (for build):            /bin/python2.7

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Install to:                    /tmp/pip-req-build-ddpkm6fn/_skbuild/linux-x86_64-3.9/cmake-install
-----------------------------------------------------------------


  


  If relevant, the docker is installed on an Intel NUC with Ubuntu Desktop 20.04

  


  If relevant, this is the dockerfile I've used to build the image :

  


  FROM python:3.9.2-slim-buster as builder

# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE=1
# Without this setting, Python never prints anything out.
ENV PYTHONUNBUFFERED=1

RUN pip install --upgrade pip
COPY ./Cam/requirements.txt .
RUN pip install -r requirements.txt
RUN apt-get update
RUN apt-get install ffmpeg libsm6 libxext6 -y

WORKDIR /app

FROM builder
COPY ./Cam .
CMD ["python", "camStreamer.py"]


  


  and last, this is the script code (simplified) :

  


  import os, logging, threading
from os.path import join
import sys, inspect, datetime, time
from pathlib import Path
import cv2
import imutils
from imutils.video import VideoStream

def StartRecording(showVideoWindow, interval, imagePath):
    key = None
    cam = VideoStream(os.getenv("CAM_RTSP")).start()
    counter = 0
    try:
        while True:
            ## 2 min retry to connect if frame is None
            if counter > 60/interval*2: break

            ts = time.time()
            ## Wait for [interval] seconds
            while ts + interval > time.time():
                continue
            print(f"Counter: {counter}, ts: {str(ts)}")

            frame = cam.read()
            if frame is None:
                counter += 1
                continue
            counter = 0

            print("frame is valid")
            if showVideoWindow:
                frame = imutils.resize(frame, width=1200)
                cv2.imshow('VIDEO', frame)

            imageName = f"{datetime.datetime.fromtimestamp(ts).strftime('%Y-%m-%dT%H_%M_%S')}.jpg"
            cv2.imwrite(join(imagePath, imageName), frame)
            print("saved image to disk")

            key = cv2.waitKey(1) & 0xFF
            if key == ord('q') or key == ord('r'):
                break

    except Exception as e:
        exc_tb = sys.exc_info()[2]
        extra = ""
        print(f"{inspect.stack()[0][3]}: {e} (lineno: {exc_tb.tb_lineno}) {extra}")
    finally:
        if showVideoWindow: cv2.destroyAllWindows()
        cam.stop()
        return key


while True:
    log.warning(f"Starting {Name}")
    key = StartRecording(
                showVideoWindow=(Env.startswith("development") and os.getenv("SHOW_VIDEO") == "True"),
                interval=int(os.getenv("SAVE_IMAGE_INTERVAL")),
                imagePath=os.getenv('CAPTURE_FOLDER')
                )
    if key == ord('q'):
        break


  


  I apologize for the very long post. Hopefully someone can put me on the right direction...