Recherche avancée

Sur d’autres sites (9333)

• How do I use FFMPEG/libav to access the data in individual audio samples ?

  15 octobre 2022, par Breadsnshreds

  The end result is I'm trying to visualise the audio waveform to use in a DAW-like software. So I want to get each sample's value and draw it. With that in mind, I'm currently stumped by trying to gain access to the values stored in each sample. For the time being, I'm just trying to access the value in the first sample - I'll build it into a loop once I have some working code.

  


  I started off by following the code in this example. However, LibAV/FFMPEG has been updated since then, so a lot of the code is deprecated or straight up doesn't work the same anymore.

  


  Based on the example above, I believe the logic is as follows :

  


  

  1. get the formatting info of the audio file

  


  2. get audio stream info from the format

  


  3. check that the codec required for the stream is an audio codec

  


  4. get the codec context (I think this is info about the codec) - This is where it gets kinda confusing for me

  


  5. create an empty packet and frame to use - packets are for holding compressed data and frames are for holding uncompressed data

  


  6. the format reads the first frame from the audio file into our packet

  


  7. pass that packet into the codec context to be decoded

  


  8. pass our frame to the codec context to receive the uncompressed audio data of the first frame

  


  9. create a buffer to hold the values and try allocating samples to it from our frame

  


  


  From debugging my code, I can see that step 7 succeeds and the packet that was empty receives some data. In step 8, the frame doesn't receive any data. This is what I need help with. I get that if I get the frame, assuming a stereo audio file, I should have two samples per frame, so really I just need your help to get uncompressed data into the frame.

  


  I've scoured through the documentation for loads of different classes and I'm pretty sure I'm using the right classes and functions to achieve my goal, but evidently not (I'm also using Qt, so I'm using qDebug throughout, and QString to hold the URL for the audio file as path). So without further ado, here's my code :

  


  // Step 1 - get the formatting info of the audio file
    AVFormatContext* format = avformat_alloc_context();
    if (avformat_open_input(&format, path.toStdString().c_str(), NULL, NULL) != 0) {
        qDebug() << "Could not open file " << path;
        return -1;
    }

// Step 2 - get audio stream info from the format
    if (avformat_find_stream_info(format, NULL) < 0) {
        qDebug() << "Could not retrieve stream info from file " << path;
        return -1;
    }

// Step 3 - check that the codec required for the stream is an audio codec
    int stream_index =- 1;
    for (unsigned int i=0; inb_streams; i++) {
        if (format->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) {
            stream_index = i;
            break;
        }
    }

    if (stream_index == -1) {
        qDebug() << "Could not retrieve audio stream from file " << path;
        return -1;
    }

// Step 4 -get the codec context
    const AVCodec *codec = avcodec_find_decoder(format->streams[stream_index]->codecpar->codec_id);
    AVCodecContext *codecContext = avcodec_alloc_context3(codec);
    avcodec_open2(codecContext, codec, NULL);

// Step 5 - create an empty packet and frame to use
    AVPacket *packet = av_packet_alloc();
    AVFrame *frame = av_frame_alloc();

// Step 6 - the format reads the first frame from the audio file into our packet
    av_read_frame(format, packet);
// Step 7 - pass that packet into the codec context to be decoded
    avcodec_send_packet(codecContext, packet);
//Step 8 - pass our frame to the codec context to receive the uncompressed audio data of the first frame
    avcodec_receive_frame(codecContext, frame);

// Step 9 - create a buffer to hold the values and try allocating samples to it from our frame
    double *buffer;
    av_samples_alloc((uint8_t**) &buffer, NULL, 1, frame->nb_samples, AV_SAMPLE_FMT_DBL, 0);
    qDebug () << "packet: " << &packet;
    qDebug() << "frame: " <<  frame;
    qDebug () << "buffer: " << buffer;


  


  For the time being, step 9 is incomplete as you can probably tell. But for now, I need help with step 8. Am I missing a step, using the wrong function, wrong class ? Cheers.

  


• Cortex-A7 instruction cycle timings

  15 mai 2014, par Mans — ARM

  The Cortex-A7 ARM core is a popular choice in low-power and low-cost designs. Unfortunately, the public TRM does not include instruction timing information. It does reveal that execution is in-order which makes measuring the throughput and latency for individual instructions relatively straight-forward.

  The table below lists the measured issue cycles (inverse throughput) and result latency of some commonly used instructions.

  It should be noted that in some cases, the perceived latency depends on the instruction consuming the result. Most of the values were measured with the result used as input to the same instruction. For instructions with multiple outputs, the latencies of the result registers may also differ.

  Finally, although instruction issue is in-order, completion is out of order, allowing independent instructions to issue and complete unimpeded while a multi-cycle instruction is executing in another unit. For example, a 3-cycle MUL instruction does not block ADD instructions following it in program order.

  ALU instructions	Issue cycles	Result latency
  MOV Rd, Rm	1/2	1
  ADD Rd, Rn, #imm	1/2	1
  ADD Rd, Rn, Rm	1	1
  ADD Rd, Rn, Rm, LSL #imm	1	1
  ADD Rd, Rn, Rm, LSL Rs	1	1
  LSL Rd, Rn, #imm	1	2
  LSL Rd, Rn, Rs	1	2
  QADD Rd, Rn, Rm	1	2
  QADD8 Rd, Rn, Rm	1	2
  QADD16 Rd, Rn, Rm	1	2
  CLZ Rd, Rm	1	1
  RBIT Rd, Rm	1	2
  REV Rd, Rm	1	2
  SBFX Rd, Rn	1	2
  BFC Rd, #lsb, #width	1	2
  BFI Rd, Rn, #lsb, #width	1	2
  NOTE : Shifted operands and shift amounts needed one cycle early.
  Multiply instructions	Issue cycles	Result latency
  MUL Rd, Rn, Rm	1	3
  MLA Rd, Rn, Rm, Ra	1	31
  SMULL Rd, RdHi, Rn, Rm	1	3
  SMLAL Rd, RdHi, Rn, Rm	1	31
  SMMUL Rd, Rn, Rm	1	3
  SMMLA Rd, Rn, Rm, Ra	1	31
  SMULBB Rd, Rn, Rm	1	3
  SMLABB Rd, Rn, Rm, Ra	1	31
  SMULWB Rd, Rn, Rm	1	3
  SMLAWB Rd, Rn, Rm, Ra	1	31
  SMUAD Rd, Rn, Rm	1	3
  1 Accumulator forwarding allows back to back MLA instructions without delay.
  Divide instructions	Issue cycles	Result latency
  SDIV Rd, Rn, Rm	4-20	6-22
  UDIV Rd, Rn, Rm	3-19	5-21
  Load/store instructions	Issue cycles	Result latency
  LDR Rt, [Rn]	1	3
  LDR Rt, [Rn, #imm]	1	3
  LDR Rt, [Rn, Rm]	1	3
  LDR Rt, [Rn, Rm, lsl #imm]	1	3
  LDRD Rt, Rt2, [Rn]	1	3-4
  LDM Rn, regs	1-8	3-10
  STR Rt, [Rn]	1	2
  STRD Rt, Rt2, [Rn]	1	2
  STM Rn, regs	1-10	2-12
  NOTE : Load results are forwarded to dependent stores without delay.
  VFP instructions	Issue cycles	Result latency
  VMOV.F32 Sd, Sm	1	4
  VMOV.F64 Dd, Dm	1	4
  VNEG.F32 Sd, Sm	1	4
  VNEG.F64 Dd, Dm	1	4
  VABS.F32 Sd, Sm	1	4
  VABS.F64 Dd, Dm	1	4
  VADD.F32 Sd, Sn, Sm	1	4
  VADD.F64 Dd, Dn, Dm	1	4
  VMUL.F32 Sd, Sn, Sm	1	4
  VMUL.F64 Dd, Dn, Dm	4	7
  VMLA.F32 Sd, Sn, Sm	1	81
  VMLA.F64 Dd, Dn, Dm	4	112
  VFMA.F32 Sd, Sn, Sm	1	81
  VFMA.F64 Dd, Dn, Dm	5	82
  VDIV.F32 Sd, Sn, Sm	15	18
  VDIV.F64 Dd, Dn, Dm	29	32
  VSQRT.F32 Sd, Sm	14	17
  VSQRT.F64 Dd, Dm	28	31
  VCVT.F32.F64 Sd, Dm	1	4
  VCVT.F64.F32 Dd, Sm	1	4
  VCVT.F32.S32 Sd, Sm	1	4
  VCVT.F64.S32 Dd, Sm	1	4
  VCVT.S32.F32 Sd, Sm	1	4
  VCVT.S32.F64 Sd, Dm	1	4
  VCVT.F32.S32 Sd, Sd, #fbits	1	4
  VCVT.F64.S32 Dd, Dd, #fbits	1	4
  VCVT.S32.F32 Sd, Sd, #fbits	1	4
  VCVT.S32.F64 Dd, Dd, #fbits	1	4
  1 5 cycles with dependency only on accumulator. 2 8 cycles with dependency only on accumulator.
  NEON integer instructions	Issue cycles	Result latency
  VADD.I8 Dd, Dn, Dm	1	4
  VADDL.S8 Qd, Dn, Dm	2	4
  VADD.I8 Qd, Qn, Qm	2	4
  VMUL.I8 Dd, Dn, Dm	2	4
  VMULL.S8 Qd, Dn, Dm	2	4
  VMUL.I8 Qd, Qn, Qm	4	4
  VMLA.I8 Dd, Dn, Dm	2	4
  VMLAL.S8 Qd, Dn, Dm	2	4
  VMLA.I8 Qd, Qn, Qm	4	4
  VADD.I16 Dd, Dn, Dm	1	4
  VADDL.S16 Qd, Dn, Dm	2	4
  VADD.I16 Qd, Qn, Qm	2	4
  VMUL.I16 Dd, Dn, Dm	1	4
  VMULL.S16 Qd, Dn, Dm	2	4
  VMUL.I16 Qd, Qn, Qm	2	4
  VMLA.I16 Dd, Dn, Dm	1	4
  VMLAL.S16 Qd, Dn, Dm	2	4
  VMLA.I16 Qd, Qn, Qm	2	4
  VADD.I32 Dd, Dn, Dm	1	4
  VADDL.S32 Qd, Dn, Dm	2	4
  VADD.I32 Qd, Qn, Qm	2	4
  VMUL.I32 Dd, Dn, Dm	2	4
  VMULL.S32 Qd, Dn, Dm	2	4
  VMUL.I32 Qd, Qn, Qm	4	4
  VMLA.I32 Dd, Dn, Dm	2	4
  VMLAL.S32 Qd, Dn, Dm	2	4
  VMLA.I32 Qd, Qn, Qm	4	4
  NEON floating-point instructions	Issue cycles	Result latency
  VADD.F32 Dd, Dn, Dm	2	4
  VADD.F32 Qd, Qn, Qm	4	4
  VMUL.F32 Dd, Dn, Dm	2	4
  VMUL.F32 Qd, Qn, Qm	4	4
  VMLA.F32 Dd, Dn, Dm	2	81
  VMLA.F32 Qd, Qn, Qm	4	81
  1 5 cycles with dependency only on accumulator.
  NEON permute instructions	Issue cycles	Result latency
  VEXT.n Dd, Dn, Dm, #imm	1	4
  VEXT.n Qd, Qn, Qm, #imm	2	5
  VTRN.n Dd, Dn, Dm	2	5
  VTRN.n Qd, Qn, Qm	4	5
  VUZP.n Dd, Dn, Dm	2	5
  VUZP.n Qd, Qn, Qm	4	6
  VZIP.n Dd, Dn, Dm	2	5
  VZIP.n Qd, Qn, Qm	4	6
  VTBL.8 Dd, Dn, Dm	1	4
  VTBL.8 Dd, Dn-Dn+1, Dm	1	4
  VTBL.8 Dd, Dn-Dn+2, Dm	2	5
  VTBL.8 Dd, Dn-Dn+3, Dm	2	5

• ffmpeg background worker runs in debug but not in application

  2 novembre 2019, par Purgitoria

  My application has a function of taking captured images and using an FFmpeg background worker to stitch these into a time-lapse video. The GUI has some simple options for video quality and for the source folder and output file. I had an older version of my application written in VB.NET and that worked without issue but I am rewriting in C# as it supports additional capture and filter capability in the image processing but am having real trouble figuring out what is wrong with this function.

  I have tried relocating FFmpeg to different locations just in case it was a permissions issue but that had no effect and I also tried to put the function in a "try" with a message box to output any exceptions but I got different errors that prevented me from compiling the code. When I run the application from within VS 2015 in the debugging tool the function works just fine and it will create a video from a collection of still images but when I build and install the application it does not work at all and I cannot see what is causing it to fail. In the options for ffmpeg I used the -report to output a log of what happens in the background worker and in debug, it creates this log but from the application, it does not so I presume it is not even running ffmpeg and going straight to the completed stage of the function.

  Function startConversion()
  
  
  
      CheckForIllegalCrossThreadCalls = False
  
      Dim quality As Integer = trbQuality.Value
  
      Dim input As String = tbFolderOpen.Text
  
      Dim output As String = tbFolderSave.Text
  
      Dim exepath As String = Application.StartupPath + "\\bin\ffmpeg.exe"
  
      input = input & "\SCAImg_%1d.bmp"
  
      input = Chr(34) & input & Chr(34)
  
      output = Chr(34) & output & Chr(34)
  
  
  
      Dim sr As StreamReader
  
      Dim ffmpegOutput As String
  
  
  
      ' all parameters required to run the process
  
      proc.StartInfo.UseShellExecute = False
  
      proc.StartInfo.CreateNoWindow = True
  
      proc.StartInfo.RedirectStandardError = True
  
      proc.StartInfo.FileName = exepath
  
      proc.StartInfo.Arguments = "-framerate 25 -start_number 0 -pattern_type sequence -framerate 10 -i " & input & " -r 10 -c:v libx264 -preset slow -crf " & quality & " " & output
  
      proc.Start()
  
  
  
      lblInfo.Text = "Conversion in progress... Please wait..."
  
      sr = proc.StandardError 'standard error is used by ffmpeg
  
      btnMakeVideo.Enabled = False
  
      Do
  
          ffmpegOutput = sr.ReadLine
  
          tbProgress.Text = ffmpegOutput
  
      Loop Until proc.HasExited And ffmpegOutput = Nothing Or ffmpegOutput = ""
  
  
  
      tbProgress.Text = "Finished !"
  
      lblInfo.Text = "Completed!"
  
      MsgBox("Completed!", MsgBoxStyle.Exclamation)
  
      btnMakeVideo.Enabled = True
  
      Return 0
  
  
  
  End Function

  I checked the application folder and it does contain a subfolder \bin withe the ffmpeg.exe located within the folder so I then used cmd to run an instance of the installed ffmpeg from the application folder and it seemed to be throwing out permissions errors :

  Failed to open report "ffmpeg-20191101-191452.log" : Permission denied
  Failed to set value ’1’ for option ’report’ : Permission denied
  Error parsing global options : Permission denied

  This seems then like it is certainly a permissions problem but where I am not sure. I did not run into this error when using VB.NET so I am wondering where I am going wrong now. I thought perhaps it would just be a write permission in the application folder so I the removed the -report and ran ffmpeg again using cmd from my application folder and it then gave the error

  C :\Users\CEAstro\Pictures\AnytimeCap : Permission denied

  Am I missing something really obvious in my code or is there something more fundamental I have wrong in my setup ?

  I should also add that I tried running ffmpeg via cmd from a copy that was manually placed elsewhere (i used the same file) and that actually worked. For some reason, it seems like it just will not work from wherever my application installs it.