Recherche avancée

Sur d’autres sites (12031)

• lavu/x86 : add FFT assembly

  10 avril 2021, par Lynne

  lavu/x86 : add FFT assembly
  This commit adds a pure x86 assembly SIMD version of the FFT in libavutil/tx.
  
  The design of this pure assembly FFT is pretty unconventional.
  On the lowest level, instead of splitting the complex numbers into
  
  real and imaginary parts, we keep complex numbers together but split
  
  them in terms of parity. This saves a number of shuffles in each transform,
  
  but more importantly, it splits each transform into two independent
  
  paths, which we process using separate registers in parallel.
  
  This allows us to keep all units saturated and lets us use all available
  
  registers to avoid dependencies.
  
  Moreover, it allows us to double the granularity of our per-load permutation,
  
  skipping many expensive lookups and allowing us to use just 4 loads per register,
  
  rather than 8, or in case FMA3 (and by extension, AVX2), use the vgatherdpd
  
  instruction, which is at least as fast as 4 separate loads on old hardware,
  
  and quite a bit faster on modern CPUs).
  Higher up, we go for a bottom-up construction of large transforms, foregoing
  
  the traditional per-transform call-return recursion chains. Instead, we always
  
  start at the bottom-most basis transform (in this case, a 32-point transform),
  
  and continue constructing larger and larger transforms until we return to the
  
  top-most transform.
  
  This way, we only touch the stack 3 times per a complete target transform :
  
  once for the 1/2 length transform and two times for the 1/4 length transform.
  The combination algorithm we use is a standard Split-Radix algorithm,
  
  as used in our C code. Although a version with less operations exists
  
  (Steven G. Johnson and Matteo Frigo's "A modified split-radix FFT with fewer
  
  arithmetic operations", IEEE Trans. Signal Process. 55 (1), 111–119 (2007),
  
  which is the one FFTW uses), it only has 2% less operations and requires at least 4x
  
  the binary code (due to it needing 4 different paths to do a single transform).
  
  That version also has other issues which prevent it from being implemented
  
  with SIMD code as efficiently, which makes it lose the marginal gains it offered,
  
  and cannot be performed bottom-up, requiring many recursive call-return chains,
  
  whose overhead adds up.
  We go through a lot of effort to minimize load/stores by keeping as much in
  
  registers in between construcring transforms. This saves us around 32 cycles,
  
  on paper, but in reality a lot more due to load/store aliasing (a load from a
  
  memory location cannot be issued while there's a store pending, and there are
  
  only so many (2 for Zen 3) load/store units in a CPU).
  
  Also, we interleave coefficients during the last stage to save on a store+load
  
  per register.
  Each of the smallest, basis transforms (4, 8 and 16-point in our case)
  
  has been extremely optimized. Our 8-point transform is barely 20 instructions
  
  in total, beating our old implementation 8-point transform by 1 instruction.
  
  Our 2x8-point transform is 23 instructions, beating our old implementation by
  
  6 instruction and needing 50% less cycles. Our 16-point transform's combination
  
  code takes slightly more instructions than our old implementation, but makes up
  
  for it by requiring a lot less arithmetic operations.
  Overall, the transform was optimized for the timings of Zen 3, which at the
  
  time of writing has the most IPC from all documented CPUs. Shuffles were
  
  preferred over arithmetic operations due to their 1/0.5 latency/throughput.
  On average, this code is 30% faster than our old libavcodec implementation.
  
  It's able to trade blows with the previously-untouchable FFTW on small transforms,
  
  and due to its tiny size and better prediction, outdoes FFTW on larger transforms
  
  by 11% on the largest currently supported size.
  

  • [DH] libavutil/tx.c
  • [DH] libavutil/tx_priv.h
  • [DH] libavutil/x86/Makefile
  • [DH] libavutil/x86/tx_float.asm
  • [DH] libavutil/x86/tx_float_init.c

• How to record/trim/combine audio seamlessly in a React/Node web app

  16 mai 2021, par Rayhan Memon

  I've developed a digital audio workstation for the browser that you can check out here. Essentially it helps authors narrate their own audiobooks themselves at home.

  


  I'm looking to dramatically improve the speed at which audio files are combined or trimmed.

  


  Right now, the user records some variable amount of audio (a line, paragraph, or entire passage). When the user stops recording, this clip is added to the main audio file for the section using ffmpeg.wasm like so :

  


          if (duration === 0) {
            //concatenate the two files (they should already be written to memory)
            await ffmpeg.run('-i', 'concat:fullAudio.mp3|clip.mp3', '-c', 'copy', 'fullAudio.mp3');

        } else {
            //Set the insert time to wherever the user's cursor is positioned
            let insertTime = duration;
            if (selectedObj) {
                insertTime = selectedObj.endTime;
            }

            //split the audio file into two parts at the point we want to insert the audio
            await ffmpeg.run('-i', 'fullAudio.mp3', '-t', `${insertTime}`, '-c', 'copy', 'part1.mp3', '-ss',  `${insertTime}`, '-codec', 'copy', 'part2.mp3');

            //concatenate the three files
            await ffmpeg.run('-i', 'concat:part1.mp3|clip.mp3', '-acodec', 'copy', 'intermediate.mp3');
            await ffmpeg.run('-i', 'concat:intermediate.mp3|part2.mp3', '-acodec', 'copy', 'fullAudio.mp3');
        }

        //Read the result from memory
        const data = ffmpeg.FS('readFile', 'fullAudio.mp3');

        //Create URL so it can be used in the browser
        const url = URL.createObjectURL(new Blob([data.buffer], { type: 'audio/mp3' }));
        globalDispatch({ type: "setFullAudioURL", payload: url });


  


  After every recorded clip, the user is forced to wait a few seconds for this concatenation process to finish up - and the longer the main file or recorded clip gets, the longer the user has to wait. Looking at other browser-based audio editors such as AudioMass, it clearly seems possible to make a seamless recording and editing experience with zero wait time, but I can't seem to figure out how to do the same within my react app.

  


  Is it possible to seamlessly combine or trim audio data within a React app ? Is FFMPEG the best way to go about it, or are there simpler ways using pure javascript ?

  


• Unable to write textclip with moviepy due to errors with Imagemagick

  5 février, par Sato

  I am trying to write TextClip into a video with moviepy. It has always been working, yet after I reinstalled ffmpeg, it doesn't work anymore (I am not sure whether it has caused it, but I mentioned it in case it does). Some of the TextClip objects are pure spaces.

  


  I got the following error message :

  


  Traceback (most recent call last):&#xA;  File "C:\Users\USER\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\moviepy\video\VideoClip.py", line 1137, in __init__&#xA;    subprocess_call(cmd, logger=None)&#xA;  File "C:\Users\USER\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\moviepy\tools.py", line 54, in subprocess_call&#xA;    raise IOError(err.decode(&#x27;utf8&#x27;))&#xA;OSError: magick.exe: no images for write &#x27;-write&#x27; &#x27;PNG32:C:\Users\USER\AppData\Local\Temp\tmpmnf0fkb5.png&#x27; at CLI arg 14 @ error/operation.c/CLINoImageOperator/4893.&#xA;&#xA;&#xA;During handling of the above exception, another exception occurred:&#xA;&#xA;Traceback (most recent call last):&#xA;  File "media_main_user.py", line 79, in <module>&#xA;    insert_audio_and_subtitles(input_clip,&#x27;output.mp4&#x27;,&#x27;text.mp3&#x27;,subtitles,fontsize=subtitles_font_size,&#xA;  File "D:\UserData\Desktop\Project\影片剪輯\關鍵字版本\make_media.py", line 318, in insert_audio_and_subtitles&#xA;    annotated_clips = [annotate(video.subclip(from_t, to_t), txt) for (from_t, to_t), txt in subtitles]&#xA;  File "D:\UserData\Desktop\Project\影片剪輯\關鍵字版本\make_media.py", line 318, in <listcomp>&#xA;    annotated_clips = [annotate(video.subclip(from_t, to_t), txt) for (from_t, to_t), txt in subtitles]&#xA;  File "D:\UserData\Desktop\Project\影片剪輯\關鍵字版本\make_media.py", line 311, in annotate&#xA;    txtclip = TextClip(txt, fontsize=fontsize, font=font, color=txt_color)&#xA;  File "C:\Users\USER\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.8_qbz5n2kfra8p0\LocalCache\local-packages\Python38\site-packages\moviepy\video\VideoClip.py", line 1146, in __init__&#xA;    raise IOError(error)&#xA;OSError: MoviePy Error: creation of None failed because of the following error:&#xA;&#xA;magick.exe: no images for write &#x27;-write&#x27; &#x27;PNG32:C:\Users\USER\AppData\Local\Temp\tmpmnf0fkb5.png&#x27; at CLI arg 14 @ error/operation.c/CLINoImageOperator/4893.&#xA;.&#xA;&#xA;.This error can be due to the fact that ImageMagick is not installed on your computer, or (for Windows users) that you didn&#x27;t specify the path to the ImageMagick binary in file conf.py, or that the path you specified is incorrect&#xA;</listcomp></module>

  


  I have tried to solve the problems multiple ways, like checking the config-default.py file in moviepy installation directory, and the path for ImageMagick it's pointing to is not wrong. I tried to reinstall ImageMagick, it is also of no use. And I tried to edit the policy.xml file in ImageMagick directory from none to read|write, but it also doesn't work. Can anyone suggest other possible solutions ?