Recherche avancée

Sur d’autres sites (9991)

• Android + ffmpeg + AudioTrack produces bad audio output

  12 septembre 2014, par Goddchen

  here is what I am trying to do : use an AudioRecord and "pipe" the output of AudioRecord.read(byte[],...) to an ffmpeg process’ stdin that will convert to a 3gp (AAC) file.

  The ffmpeg call is as follows :

          ProcessBuilder processBuilder = new ProcessBuilder(BINARY.getAbsolutePath(),
  
                  "-y",
  
                  "-ar", "44100", "-c:a", "pcm_s16le", "-ac", "1","-f","s16le",
  
                  "-i", "-",
  
                  "-strict", "-2", "-c:a", "aac",
  
                  outFile.getAbsolutePath());

  The AudioRecord is setup as follows :

   AudioRecord record = new AudioRecord(/*AudioSource.VOICE_RECOGNITION,*/ AudioSource.MIC,
  
              SAMPLING_RATE,
  
              AudioFormat.CHANNEL_IN_MONO,
  
              AudioFormat.ENCODING_PCM_16BIT,
  
              bufferSize);

  SAMPLING_RATE = 44100 and bufferSize is the one returned by AudioRecord.getMinBufferSize(...)

  I am writing the data to ffmpeg like this :

  try {
  
                          IOUtils.write(data, getFFmpegHelper().getCurrentProcessOutputStream());
  
                      } catch (Exception e) {
  
                          Log.e(Application.LOG_TAG, "Error writing data to ffmpeg process", e);
  
                          //TODO notify user, stop the recording, etc...
  
                      }

  So far so good, the ffmpeg runs and created a proper 3gp file. But the audio in the file is totally off. It seems "choppy" (not sure if this is the correct english word ;) ) and also the pace is wrong, is plays too fast.

  Check out this sample : http://goddchen.de/android/tmp/tmp.3gp

  This is the output of the ffmpeg process :

      [s16le @ 0x23634d0] Estimating duration from bitrate, this may be inaccurate
  
      Guessed Channel Layout for  Input Stream #0.0 : mono
  
      Input #0, s16le, from 'pipe:':
  
      Duration: N/A, start: 0.000000, bitrate: 705 kb/s
  
      Stream #0:0: Audio: pcm_s16le, 44100 Hz, mono, s16, 705 kb/s
  
      [aformat @ 0x2363100] auto-inserting filter 'auto-inserted resampler 0' between the filter 'src' and the filter 'aformat'
  
      [aresample @ 0x235b0a0] chl:mono fmt:s16 r:44100Hz -> chl:mono fmt:flt r:44100Hz
  
      Output #0, 3gp, to '/data/data/com.test.audio/files/tmp.3gp':
  
      Metadata:
  
      encoder         : Lavf54.6.100
  
      Stream #0:0: Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, flt, 128 kb/s
  
      Stream mapping:
  
      Stream #0:0 -> #0:0 (pcm_s16le -> aac)
  
      size=       3kB time=00:00:00.18 bitrate= 132.5kbits/s    
  
  size=       8kB time=00:00:00.55 bitrate= 120.9kbits/s    
  
  size=      12kB time=00:00:00.83 bitrate= 121.8kbits/s    
  
  size=      16kB time=00:00:01.04 bitrate= 122.8kbits/s    
  
  size=      20kB time=00:00:01.32 bitrate= 122.5kbits/s    
  
  size=      23kB time=00:00:01.53 bitrate= 121.6kbits/s    
  
  size=      27kB time=00:00:01.81 bitrate= 121.0kbits/s    
  
  size=      31kB time=00:00:02.11 bitrate= 120.7kbits/s    
  
  size=      35kB time=00:00:02.32 bitrate= 123.4kbits/s
  
      video:0kB audio:34kB global headers:0kB muxing overhead 3.031610%

• How to set pts, dts and duration in ffmpeg library ?

  24 mars, par hslee

  I want to pack some compressed video packets(h.264) to ".mp4" container.
One word, Muxing, no decoding and no encoding.
And I have no idea how to set pts, dts and duration.

  



  

  1. I get the packets with "pcap" library.

  


  2. I removed headers before compressed video data show up. e.g. Ethernet, VLAN.

  


  3. I collected data until one frame and decoded it for getting information of data. e.g. width, height. (I am not sure that it is necessary)

  


  4. I initialized output context, stream and codec context.

  


  5. I started to receive packets with "pcap" library again. (now for muxing)

  


  6. I made one frame and put that data in AVPacket structure.

  


  7. I try to set PTS, DTS and duration. (I think here is wrong part, not sure though)

  


  



  *7-1. At the first frame, I saved time(msec) with packet header structure.

  



  *7-2. whenever I made one frame, I set parameters like this : PTS(current time - start time), DTS(same PTS value), duration(current PTS - before PTS)

  



  I think it has some error because :

  



  


  1. I don't know how far is suitable long for dts from pts.

  


  2. At least, I think duration means how long time show this frame from now to next frame, so It should have value(next PTS - current PTS), but I can not know the value next PTS at that time.

  


  



  It has I-frame only.

  



  // make input context for decoding

AVFormatContext *&ic = gInputContext;

ic = avformat_alloc_context();

AVCodec *cd = avcodec_find_decoder(AV_CODEC_ID_H264);

AVStream *st = avformat_new_stream(ic, cd);

AVCodecContext *cc = st->codec;

avcodec_open2(cc, cd, NULL);

// make packet and decode it after collect packets is be one frame

gPacket.stream_index = 0;

gPacket.size    = gPacketLength[0];

gPacket.data    = gPacketData[0];

gPacket.pts     = AV_NOPTS_VALUE;

gPacket.dts     = AV_NOPTS_VALUE;

gPacket.flags   = AV_PKT_FLAG_KEY;

avcodec_decode_video2(cc, gFrame, &got_picture, &gPacket);

// I checked automatically it initialized after "avcodec_decode_video2"

// put some info that I know that not initialized

cc->time_base.den   = 90000;

cc->time_base.num   = 1;

cc->bit_rate    = 2500000;

cc->gop_size    = 1;

// make output context with input context

AVFormatContext *&oc = gOutputContext;

avformat_alloc_output_context2(&oc, NULL, NULL, filename);

AVFormatContext *&ic = gInputContext;

AVStream *ist = ic->streams[0];

AVCodecContext *&icc = ist->codec;

AVStream *ost = avformat_new_stream(oc, icc->codec);

AVCodecContext *occ = ost->codec;

avcodec_copy_context(occ, icc);

occ->flags |= CODEC_FLAG_GLOBAL_HEADER;

avio_open(&(oc->pb), filename, AVIO_FLAG_WRITE);

// repeated part for muxing

AVRational Millisecond = { 1, 1000 };

gPacket.stream_index = 0;

gPacket.data = gPacketData[0];

gPacket.size = gPacketLength[0];

gPacket.pts = av_rescale_rnd(pkthdr->ts.tv_sec * 1000 /

    + pkthdr->ts.tv_usec / 1000 /

    - gStartTime, Millisecond.den, ost->time_base.den, /

    (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));

gPacket.dts = gPacket.pts;

gPacket.duration = gPacket.pts - gPrev;

gPacket.flags = AV_PKT_FLAG_KEY;

gPrev = gPacket.pts;

av_interleaved_write_frame(gOutputContext, &gPacket);


  



  Expected and actual results is a .mp4 video file that can play.

  


• Bit-field badness

  30 janvier 2010, par Mans — Compilers, Optimisation

  Consider the following C code which is based on an real-world situation.

  struct bf1_31 
      unsigned a:1 ;
      unsigned b:31 ;
   ;
  void func(struct bf1_31 *p, int n, int a)
  
  
  
      int i = 0 ;
  
      do 
  
          if (p[i].a)
  
              p[i].b += a ;
  
       while (++i < n) ;
  
  
  

  How would we best write this in ARM assembler ? This is how I would do it :
  

  func :
          ldr     r3,  [r0], #4
          tst     r3,  #1
          add     r3,  r3,  r2,  lsl #1
          strne   r3,  [r0, #-4]
          subs    r1,  r1,  #1
          bgt     func
          bx      lr
  

  The add instruction is unconditional to avoid a dependency on the comparison. Unrolling the loop would mask the latency of the ldr instruction as well, but that is outside the scope of this experiment.

  Now compile this code with gcc -march=armv5te -O3 and watch in horror :

  func :
          push    r4
          mov     ip, #0
          mov     r4, r2
  loop :
          ldrb    r3, [r0]
          add     ip, ip, #1
          tst     r3, #1
          ldrne   r3, [r0]
          andne   r2, r3, #1
          addne   r3, r4, r3, lsr #1
          orrne   r2, r2, r3, lsl #1
          strne   r2, [r0]
          cmp     ip, r1
          add     r0, r0, #4
          blt     loop
          pop     r4
          bx      lr
  

  This is nothing short of awful :

  • The same value is loaded from memory twice.
  • A complicated mask/shift/or operation is used where a simple shifted add would suffice.
  • Write-back addressing is not used.
  • The loop control counts up and compares instead of counting down.
  • Useless mov in the prologue ; swapping the roles or r2 and r4 would avoid this.
  • Using lr in place of r4 would allow the return to be done with pop {pc}, saving one instruction (ignoring for the moment that no callee-saved registers are needed at all).

  Even for this trivial function the gcc-generated code is more than twice the optimal size and slower by approximately the same factor.

  The main issue I wanted to illustrate is the poor handling of bit-fields by gcc. When accessing bitfields from memory, gcc issues a separate load for each field even when they are contained in the same aligned memory word. Although each load after the first will most likely hit L1 cache, this is still bad for several reasons :

  • Loads have typically two or three cycles result latency compared to one cycle for data processing instructions. Any bit-field can be extracted from a register with two shifts, and on ARM the second of these can generally be achieved using a shifted second operand to a following instruction. The ARMv6T2 instruction set also adds the SBFX and UBFX instructions for extracting any signed or unsigned bit-field in one cycle.
  • Most CPUs have more data processing units than load/store units. It is thus more likely for an ALU instruction than a load/store to issue without delay on a superscalar processor.
  • Redundant memory accesses can trigger early flushing of store buffers rendering these less efficient.

  No gcc bashing is complete without a comparison with another compiler, so without further ado, here is the ARM RVCT output (armcc --cpu 5te -O3) :

  func :
          mov     r3, #0
          push    r4, lr
  loop :
          ldr     ip, [r0, r3, lsl #2]
          tst     ip, #1
          addne   ip, ip, r2, lsl #1
          strne   ip, [r0, r3, lsl #2]
          add     r3, r3, #1
          cmp     r3, r1
          blt     loop
          pop     r4, pc
  

  This is much better, the core loop using only one instruction more than my version. The loop control is counting up, but at least this register is reused as offset for the memory accesses. More remarkable is the push/pop of two registers that are never used. I had not expected to see this from RVCT.

  Even the best compilers are still no match for a human.