Sur d’autres sites (8804)

• FFmpeg : avconv and ffserver - connection reset by peer

  5 mars 2016, par einsA

  I’m trying to stream a video via ffserver. My dev environment is a Raspberry Pi 2.

  I’m getting an av_interleaved_write_frame(): Connection reset by peer with the following approach :

  pi@raspberrypi:/opt/livefeed $ avconv -f mjpeg -i movie.mjpg -r 20 -b 2000k -vcodec libx264 -preset ultrafast -f flv -metadata streamName="mjpeg_to_flv" -metadata title="converting mjpeg to flv via avconv" http://localhost:8090/feed1.ffm
  
  avconv version 11.4-6:11.4-1~deb8u1+rpi1, Copyright (c) 2000-2014 the Libav developers
  
    built on Jun 16 2015 05:32:34 with gcc 4.9.2 (Raspbian 4.9.2-10)
  
  [mjpeg @ 0x20461e0] Estimating duration from bitrate, this may be inaccurate
  
  Input #0, mjpeg, from 'movie.mjpg':
  
    Duration: N/A, bitrate: N/A
  
      Stream #0.0: Video: mjpeg, yuvj422p, 768x512, 25 fps, 25 tbn
  
  [tcp @ 0x2050bc0] Connection to tcp://localhost:8090 failed (Connection refused), trying next address
  
  [libx264 @ 0x20501a0] using cpu capabilities: none!
  
  [libx264 @ 0x20501a0] profile Constrained Baseline, level 3.0
  
  [libx264 @ 0x20501a0] 264 - core 142 r2431 a5831aa - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=20 scenecut=0 intra_refresh=0 rc=abr mbtree=0 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
  
  Output #0, flv, to 'http://localhost:8090/feed1.ffm':
  
    Metadata:
  
      streamName      : mjpeg_to_flv
  
      title           : converting mjpeg to flv via avconv
  
      encoder         : Lavf56.1.0
  
      Stream #0.0: Video: libx264, yuv420p, 768x512, q=-1--1, 2000 kb/s, 20 fps, 1k tbn, 20 tbc
  
      Metadata:
  
        encoder         : Lavc56.1.0 libx264
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
  
  Press ctrl-c to stop encoding
  
  av_interleaved_write_frame(): Connection reset by peer0000.00 bitrate=   0.0kbits/s

  ---

  I am able to stream the file doing it step by step with the following settings and commands :

  ffserver

  ffserver config :

  pi@raspberrypi:/opt/livefeed $ cat ffserver_flv.conf 
  
  # FLV live feed
  
  
  
  Port 8090
  
  BindAddress 0.0.0.0
  
  MaxHTTPConnections 2000
  
  MaxClients 1000
  
  MaxBandwidth 1000
  
  CustomLog -
  
  
  
  <stream>
  
    Format status
  
    ACL allow localhost
  
    ACL allow 192.168.0.0 192.168.255.255
  
  </stream>
  
  
  
  <feed>
  
    File /tmp/feed_flv.ffm
  
    FileMaxSize 1G
  
    ACL allow 127.0.0.1
  
  </feed>
  
  
  
  <stream>
  
    Format flv
  
    Feed feed1.ffm
  
  
  
    VideoCodec libx264
  
    VideoFrameRate 30
  
    VideoBitRate 800
  
    VideoSize 720x576
  
    AVOptionVideo crf 23
  
    AVOptionVideo preset medium
  
    AVOptionVideo me_range 16
  
    AVOptionVideo qdiff 4
  
    AVOptionVideo qmin 10
  
    AVOptionVideo qmax 51
  
    AVOptionVideo flags +global_header
  
  
  
    NoAudio
  
  </stream>

  starting the server with :

  pi@raspberrypi:/opt/livefeed $ ffserver -v debug -f ffserver_flv.conf
  
  ffserver version N-78909-gfb2f164 Copyright (c) 2000-2016 the FFmpeg developers
  
    built with gcc 4.9.2 (Raspbian 4.9.2-10)
  
    configuration: --arch=armel --target-os=linux --enable-gpl --enable-libx264 --enable-nonfree --enable-libass --enable-libfaac --enable-libmp3lame --enable-libfreetype
  
    libavutil      55. 19.100 / 55. 19.100
  
    libavcodec     57. 27.101 / 57. 27.101
  
    libavformat    57. 28.100 / 57. 28.100
  
    libavdevice    57.  0.101 / 57.  0.101
  
    libavfilter     6. 39.100 /  6. 39.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  ffserver_flv.conf:6: Port option is deprecated. Use HTTPPort instead.
  
  ffserver_flv.conf:7: BindAddress option is deprecated. Use HTTPBindAddress instead.
  
  ffserver_flv.conf:48: Setting default value for video bit rate tolerance = 200000. Use NoDefaults to disable it.
  
  ffserver_flv.conf:48: Setting default value for video rate control equation = tex^qComp. Use NoDefaults to disable it.
  
  ffserver_flv.conf:48: Setting default value for video max rate = 49537952. Use NoDefaults to disable it.
  
  ffserver_flv.conf:48: Setting default value for video buffer size = 1600000. Use NoDefaults to disable it.
  
  Sat Mar  5 13:29:46 2016 [file @ 0x2f3d0d0]Setting default whitelist 'file'
  
  Sat Mar  5 13:29:46 2016 [ffm @ 0x2f40bc0]Format ffm probed with size=2048 and score=101
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'time_base' to value '1/30'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'b' to value '800000'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'video_size' to value '720x576'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'me_range' to value '16'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'qdiff' to value '4'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'qmin' to value '10'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'qmax' to value '51'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'flags' to value '+global_header'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'bt' to value '200000'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'rc_eq' to value 'tex^qComp'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'maxrate' to value '1600000'
  
  Sat Mar  5 13:29:46 2016 [NULL @ 0x2f43760]Setting entry with key 'bufsize' to value '1600000'
  
  Sat Mar  5 13:29:46 2016 [AVIOContext @ 0x2f41450]Statistics: 32768 bytes read, 0 seeks
  
  Sat Mar  5 13:29:46 2016 FFserver started.

  avconv

  I’m converting a mjpeg movie (movie.mjpeg) via avconv to flv (movie.flv) :

  pi@raspberrypi:/opt/livefeed $ avconv -f mjpeg -i movie.mjpg -r 20 -b 2000k -vcodec libx264 -preset ultrafast -f flv -metadata streamName="mjpeg_to_flv" -metadata title="converting mjpeg to flv via avconv" movie.flv
  
  avconv version 11.4-6:11.4-1~deb8u1+rpi1, Copyright (c) 2000-2014 the Libav developers
  
    built on Jun 16 2015 05:32:34 with gcc 4.9.2 (Raspbian 4.9.2-10)
  
  [mjpeg @ 0x34f1e0] Estimating duration from bitrate, this may be inaccurate
  
  Input #0, mjpeg, from 'movie.mjpg':
  
    Duration: N/A, bitrate: N/A
  
      Stream #0.0: Video: mjpeg, yuvj422p, 768x512, 25 fps, 25 tbn
  
  [libx264 @ 0x3591a0] using cpu capabilities: none!
  
  [libx264 @ 0x3591a0] profile Constrained Baseline, level 3.0
  
  [libx264 @ 0x3591a0] 264 - core 142 r2431 a5831aa - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=0 ref=1 deblock=0:0:0 analyse=0:0 me=dia subme=0 psy=1 psy_rd=1.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=20 scenecut=0 intra_refresh=0 rc=abr mbtree=0 bitrate=2000 ratetol=1.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=0
  
  Output #0, flv, to 'movie.flv':
  
    Metadata:
  
      streamName      : mjpeg_to_flv
  
      title           : converting mjpeg to flv via avconv
  
      encoder         : Lavf56.1.0
  
      Stream #0.0: Video: libx264, yuv420p, 768x512, q=-1--1, 2000 kb/s, 20 fps, 1k tbn, 20 tbc
  
      Metadata:
  
        encoder         : Lavc56.1.0 libx264
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264))
  
  Press ctrl-c to stop encoding
  
  frame=   93 fps=  9 q=-1.0 Lsize=    1078kB time=4.60 bitrate=1919.3kbits/s    ts/s    
  
  video:1076kB audio:0kB other streams:0kB global headers:0kB muxing overhead: 0.201471%
  
  [libx264 @ 0x3591a0] frame I:1     Avg QP:19.00  size: 35978
  
  [libx264 @ 0x3591a0] frame P:92    Avg QP:21.43  size: 11574
  
  [libx264 @ 0x3591a0] mb I  I16..4: 100.0%  0.0%  0.0%
  
  [libx264 @ 0x3591a0] mb P  I16..4: 25.9%  0.0%  0.0%  P16..4: 53.6%  0.0%  0.0%  0.0%  0.0%    skip:20.5%
  
  [libx264 @ 0x3591a0] final ratefactor: 20.82
  
  [libx264 @ 0x3591a0] coded y,uvDC,uvAC intra: 28.9% 46.9% 18.2% inter: 33.7% 39.8% 2.8%
  
  [libx264 @ 0x3591a0] i16 v,h,dc,p: 29% 30% 28% 14%
  
  [libx264 @ 0x3591a0] i8c dc,h,v,p: 53% 26% 15%  7%
  
  [libx264 @ 0x3591a0] kb/s:1893.82

  send flv to ffserver

  pi@raspberrypi:/opt/livefeed $ ffmpeg -stream_loop -1 -i movie.flv http://localhost:8090/feed1.ffm
  
  ffmpeg version N-78909-gfb2f164 Copyright (c) 2000-2016 the FFmpeg developers
  
    built with gcc 4.9.2 (Raspbian 4.9.2-10)
  
    configuration: --arch=armel --target-os=linux --enable-gpl --enable-libx264 --enable-nonfree --enable-libass --enable-libfaac --enable-libmp3lame --enable-libfreetype
  
    libavutil      55. 19.100 / 55. 19.100
  
    libavcodec     57. 27.101 / 57. 27.101
  
    libavformat    57. 28.100 / 57. 28.100
  
    libavdevice    57.  0.101 / 57.  0.101
  
    libavfilter     6. 39.100 /  6. 39.100
  
    libswscale      4.  0.100 /  4.  0.100
  
    libswresample   2.  0.101 /  2.  0.101
  
    libpostproc    54.  0.100 / 54.  0.100
  
  Input #0, flv, from 'movie.flv':
  
    Metadata:
  
      streamName      : mjpeg_to_flv
  
      title           : converting mjpeg to flv via avconv
  
      encoder         : Lavf56.1.0
  
    Duration: 00:00:04.65, start: 0.000000, bitrate: 1898 kb/s
  
      Stream #0:0: Video: h264 (Constrained Baseline), yuv420p, 768x512, 2000 kb/s, 20 fps, 20 tbr, 1k tbn, 40 tbc
  
  [tcp @ 0x30d2920] Connection to tcp://localhost:8090 failed (Connection refused), trying next address
  
  [tcp @ 0x329a080] Connection to tcp://localhost:8090 failed (Connection refused), trying next address
  
  [libx264 @ 0x325fb30] using cpu capabilities: ARMv6 NEON
  
  [libx264 @ 0x325fb30] profile High, level 3.1
  
  [libx264 @ 0x325fb30] 264 - core 148 r2665 a01e339 - H.264/MPEG-4 AVC codec - Copyleft 2003-2016 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 vbv_maxrate=1600 vbv_bufsize=1600 crf_max=0.0 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00
  
  Output #0, ffm, to 'http://localhost:8090/feed1.ffm':
  
    Metadata:
  
      streamName      : mjpeg_to_flv
  
      title           : converting mjpeg to flv via avconv
  
      creation_time   : now
  
      encoder         : Lavf57.28.100
  
      Stream #0:0: Video: h264 (libx264), yuv420p, 720x576, q=10-51, 800 kb/s, 20 fps, 1000k tbn, 30 tbc
  
      Metadata:
  
        encoder         : Lavc57.27.101 libx264
  
      Side data:
  
        cpb: bitrate max/min/avg: 1600000/0/800000 buffer size: 1600000 vbv_delay: -1
  
  Stream mapping:
  
    Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  
  Press [q] to stop, [?] for help
  
  Past duration 0.999992 too large
  
      Last message repeated 2 times
  
  Past duration 0.999992 too large      4kB time=00:00:00.00 bitrate=N/A dup=5 drop=0 speed=   0x    
  
      Last message repeated 3 times
  
  Past duration 0.999992 too large      4kB time=00:00:00.00 bitrate=N/A dup=9 drop=0 speed=   
  
  [croped]    
  
  video:968kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 3.719636%
  
  [libx264 @ 0x325fb30] frame I:11    Avg QP:22.49  size: 13888
  
  [libx264 @ 0x325fb30] frame P:487   Avg QP:16.10  size:  1382
  
  [libx264 @ 0x325fb30] frame B:980   Avg QP:21.31  size:   168
  
  [libx264 @ 0x325fb30] consecutive B-frames: 11.0%  1.8%  0.4% 86.9%
  
  [libx264 @ 0x325fb30] mb I  I16..4: 11.5% 81.2%  7.3%
  
  [libx264 @ 0x325fb30] mb P  I16..4:  1.6%  5.0%  0.2%  P16..4: 12.0%  1.6%  0.7%  0.0%  0.0%    skip:79.0%
  
  [libx264 @ 0x325fb30] mb B  I16..4:  0.0%  0.0%  0.0%  B16..8:  5.8%  0.1%  0.0%  direct: 0.0%  skip:94.1%  L0:55.3% L1:44.4% BI: 0.3%
  
  [libx264 @ 0x325fb30] 8x8 transform intra:75.4% inter:88.7%
  
  [libx264 @ 0x325fb30] coded y,uvDC,uvAC intra: 38.6% 48.2% 14.9% inter: 1.5% 2.3% 0.0%
  
  [libx264 @ 0x325fb30] i16 v,h,dc,p: 28% 32% 11% 28%
  
  [libx264 @ 0x325fb30] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 21% 31%  3%  4%  4%  4%  4%  4%
  
  [libx264 @ 0x325fb30] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 25% 23%  4%  7%  5%  7%  4%  3%
  
  [libx264 @ 0x325fb30] i8c dc,h,v,p: 63% 20% 14%  4%
  
  [libx264 @ 0x325fb30] Weighted P-Frames: Y:1.6% UV:1.2%
  
  [libx264 @ 0x325fb30] ref P L0: 67.5% 11.6% 13.8%  6.8%  0.3%
  
  [libx264 @ 0x325fb30] ref B L0: 58.7% 40.4%  0.9%
  
  [libx264 @ 0x325fb30] ref B L1: 96.2%  3.8%
  
  [libx264 @ 0x325fb30] kb/s:160.83
  
  Exiting normally, received signal 2.

  On another machine I can open the flv-stream via VLC by opening the address http://ip.running.ffserver:8090/live.flv.

  So why does it works step by step via the movie.flv file and not directly ?

  What is the difference between

  • avconv -f mjpeg -i movie.mjpg -r 20 -b 2000k -vcodec libx264 -preset ultrafast -f flv -metadata streamName="mjpeg_to_flv" -metadata title="converting mjpeg to flv via avconv" movie.flv
    followed by
    ffmpeg -stream_loop -1 -i movie.flv http://localhost:8090/feed1.ffm and

  • avconv -f mjpeg -i movie.mjpg -r 20 -b 2000k -vcodec libx264 -preset ultrafast -f flv -metadata streamName="mjpeg_to_flv" -metadata title="converting mjpeg to flv via avconv" http://localhost:8090/feed1.ffm ? Why ends this command with a av_interleaved_write_frame(): Connection reset by peer ?

  ---

  Informations and logs

  • ffserver log (both times) :

    Sat Mar  5 13:54:08 2016 127.0.0.1 - - [POST] "/feed1.ffm HTTP/1.1" 200 4096

  • ffserver version

    pi@raspberrypi:/opt/livefeed $ ffserver -version
    
    ffserver version N-78909-gfb2f164 Copyright (c) 2000-2016 the FFmpeg developers
    
    built with gcc 4.9.2 (Raspbian 4.9.2-10)
    
    configuration: --arch=armel --target-os=linux --enable-gpl --enable-libx264 --enable-nonfree --enable-libass --enable-libfaac --enable-libmp3lame --enable-libfreetype
    
    libavutil      55. 19.100 / 55. 19.100
    
    libavcodec     57. 27.101 / 57. 27.101
    
    libavformat    57. 28.100 / 57. 28.100
    
    libavdevice    57.  0.101 / 57.  0.101
    
    libavfilter     6. 39.100 /  6. 39.100
    
    libswscale      4.  0.100 /  4.  0.100
    
    libswresample   2.  0.101 /  2.  0.101
    
    libpostproc    54.  0.100 / 54.  0.100

  • ffmpeg version

    pi@raspberrypi:/opt/livefeed $ ffmpeg -version
    
    ffmpeg version N-78909-gfb2f164 Copyright (c) 2000-2016 the FFmpeg developers
    
    built with gcc 4.9.2 (Raspbian 4.9.2-10)
    
    configuration: --arch=armel --target-os=linux --enable-gpl --enable-libx264 --enable-nonfree --enable-libass --enable-libfaac --enable-libmp3lame --enable-libfreetype
    
    libavutil      55. 19.100 / 55. 19.100
    
    libavcodec     57. 27.101 / 57. 27.101
    
    libavformat    57. 28.100 / 57. 28.100
    
    libavdevice    57.  0.101 / 57.  0.101
    
    libavfilter     6. 39.100 /  6. 39.100
    
    libswscale      4.  0.100 /  4.  0.100
    
    libswresample   2.  0.101 /  2.  0.101
    
    libpostproc    54.  0.100 / 54.  0.100

  • avconv version :

    pi@raspberrypi:/opt/livefeed $ avconv -version
    
    avconv version 11.4-6:11.4-1~deb8u1+rpi1, Copyright (c) 2000-2014 the Libav developers
    
    built on Jun 16 2015 05:32:34 with gcc 4.9.2 (Raspbian 4.9.2-10)
    
    avconv 11.4-6:11.4-1~deb8u1+rpi1
    
    libavutil     54.  3. 0 / 54.  3. 0
    
    libavcodec    56.  1. 0 / 56.  1. 0
    
    libavformat   56.  1. 0 / 56.  1. 0
    
    libavdevice   55.  0. 0 / 55.  0. 0
    
    libavfilter    5.  0. 0 /  5.  0. 0
    
    libavresample  2.  1. 0 /  2.  1. 0
    
    libswscale     3.  0. 0 /  3.  0. 0

  Thanks and cheers

  Clemens

• DASH package (mpd) I made freezes after raising the onratechange event

  7 mars 2016, par Nick Birke

  I am trying to created an adaptive stream from an mp4 file. I have a decent amount of experience at doing streaming, and have successfully implemented smooth streaming via Silverlight. We want a solution for everything that works in HTML5. I am pursuing DASH. So here are the steps I take...

  1. Resample the mp4 to different sizes using ffmpeg.
  2. Ensure the files are properly fragmented using Bento4s mp4Fragment utility.
  3. Create the DASH mpd file and associated segments using Bento4s mp4dash utility.

  I have done a lot of work to learn each of these utilities and create a process that gets to output that I think will work. To create reliable DASH artifacts. But it seems I am doing something wrong. Here are examples of command lines I run.

  For step one I create three different files... for three different rates and sizes... (these may not be the exact command lines, they are from old notes... but the parameter values are the ones I am using for sure)

  ffmpeg -y -i C:\_SIMULATED_SHARE_C\ff\big_buck_bunny.mp4 -c:v libx264 -x264opts keyint=24:min-keyint=24:no-scenecut -b:v 1500k -maxrate 1500k -bufsize 1000k -vf "scale=-1:720" C:\_SIMULATED_SHARE_C\ff\output\outputfile720.mp4
  
  
  
  ffmpeg -y -i C:\_SIMULATED_SHARE_C\ff\big_buck_bunny.mp4 -c:v libx264 -x264opts keyint=24:min-keyint=24:no-scenecut -b:v 800k -maxrate 800k -bufsize 500k -vf "scale=-1:540" C:\_SIMULATED_SHARE_C\ff\output\outputfile540.mp4
  
  
  
  ffmpeg -y -i C:\_SIMULATED_SHARE_C\ff\big_buck_bunny.mp4 -c:v libx264 -x264opts keyint=24:min-keyint=24:no-scenecut -b:v 400k -maxrate 400k -bufsize 400k -vf "scale=-1:360" C:\_SIMULATED_SHARE_C\ff\output\outputfile360.mp4

  So that gives you the three files that need to be fragmented. So I fragment them using the mp4fragment utility from Bento4. That command line is simple so I will not offer an example.

  The I use Bento4s mp4dash to create the dash. Something like this...

  "E:\\_INSTALL\\Bento4\\Bento4-SDK-1-4-3-607.x86-microsoft-win32-vs2010\\Bento4-SDK-1-4-3-607.x86-microsoft-win32-vs2010\\bin\\mp4dash.exe --mpd-name=\"\\\\d9fqt52\\_SIMULATED_SHARE_C\\ff\\output\\dash\\something.mpd\" -f -o \"\\\\d9fqt52\\_SIMULATED_SHARE_C\\ff\\output\\dash\" \"\\\\d9fqt52\\_SIMULATED_SHARE_C\\ff\\output\\bunny360f.mp4\" \"\\\\d9fqt52\\_SIMULATED_SHARE_C\\ff\\output\\bunny540f.mp4\" \"\\\\d9fqt52\\_SIMULATED_SHARE_C\\ff\\output\\bunny720f.mp4\" "

  This runs ok. And creates the mpd file, an audio folder with segment files, an a video folder with 3 folders each with the segment files.

  I am able to configure IIS to play the mpd ok. As well as the segement files. The video plays fine... and then it stops. It freezes between 50 and 60 seconds. Right before it freezes I can confirm that the onratechange event of the HTML5 video tag is raised. This is an event that is raised with the rate of play is changed. I do nothing to change this it just happens.

  The actual video in question is about 1hr long. Also I am using the dash.js library as a player.

  If you can identify any components or specifics I should change please let me know !

  UPDATE ON MAR 7, 2016

  As requested here is the MPD content...

  <?xml version="1.0" ?>
  
  <mpd mediapresentationduration="PT51M51.909S" minbuffertime="PT0.80S" profiles="urn:mpeg:dash:profile:isoff-live:2011" type="static" xmlns="urn:mpeg:dash:schema:mpd:2011">
  
    
  
    <period>
  
      
  
      <adaptationset lang="en" mimetype="audio/mp4" segmentalignment="true" startwithsap="1">
  
        <segmenttemplate duration="801" initialization="$RepresentationID$/init.mp4" media="$RepresentationID$/seg-$Number$.m4f" startnumber="1" timescale="1000"></segmenttemplate>
  
        <representation audiosamplingrate="48000" bandwidth="132491" codecs="mp4a.40.2">
  
          <audiochannelconfiguration schemeiduri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011" value="2"></audiochannelconfiguration>
  
        </representation>
  
      </adaptationset>
  
      
  
      <adaptationset maxheight="720" maxwidth="1280" mimetype="video/mp4" minheight="540" minwidth="640" segmentalignment="true" startwithsap="1">
  
        <segmenttemplate duration="801" initialization="$RepresentationID$/init.mp4" media="$RepresentationID$/seg-$Number$.m4f" startnumber="1" timescale="1000"></segmenttemplate>
  
        <representation bandwidth="785536" codecs="avc1.64001F" framerate="30000/1001" height="540" scantype="progressive" width="960"></representation>
  
        <representation bandwidth="403537" codecs="avc1.64001E" framerate="30000/1001" height="360" scantype="progressive" width="640"></representation>
  
        <representation bandwidth="1496514" codecs="avc1.64001F" framerate="30000/1001" height="720" scantype="progressive" width="1280"></representation>
  
      </adaptationset>
  
    </period>
  
  </mpd>

  UPDATE ON MAR 7, 2016

  Here is the console output for two different videos that just stop for some unknown reason between 50 and 60s

  ///ABS VIDEO 
  
  
  
  58393][rulesController] [RULES]: playbackTimeRule [Fragment:download,82,MediaSegment,0video] Default 
  
  dash.all.js:11 [58395][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,82,MediaSegment,0,video] Default 
  
  dash.all.js:11 [58401][indexHandler][audio] Getting the request for time: 82.082 
  
  dash.all.js:11 [58403][indexHandler][audio] Index for time 82.082 is 81 
  
  dash.all.js:11 [58405][indexHandler][audio] SegmentTemplate: 81.08099999999999 / 1361.235 
  
  dash.all.js:11 [58406][indexHandler][audio] SegmentTemplate: 82.082 / 1361.235 
  
  dash.all.js:11 [58407][rulesController] [RULES]: playbackTimeRule [Fragment:download,82,MediaSegment,0audio] Default 
  
  dash.all.js:11 [58417][fragmentLoader] loaded video:MediaSegment:82.082 (200, 18ms, 2ms) 
  
  dash.all.js:11 [58419][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,82,MediaSegment,0,audio] Default 
  
  dash.all.js:11 [58435][fragmentLoader] loaded audio:MediaSegment:82.082 (200, 14ms, 0ms) 
  
  dash.all.js:11 [59390][indexHandler][video] Getting the request for time: 83.083 
  
  dash.all.js:11 [59392][indexHandler][video] Index for time 83.083 is 82 
  
  dash.all.js:11 [59393][indexHandler][video] SegmentTemplate: 82.082 / 1361.235 
  
  dash.all.js:11 [59394][indexHandler][video] SegmentTemplate: 83.08299999999998 / 1361.235 
  
  dash.all.js:11 [59395][rulesController] [RULES]: playbackTimeRule [Fragment:download,83,MediaSegment,0video] Default 
  
  dash.all.js:11 [59396][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,83,MediaSegment,0,video] Default 
  
  dash.all.js:11 [59401][indexHandler][audio] Getting the request for time: 83.083 
  
  dash.all.js:11 [59403][indexHandler][audio] Index for time 83.083 is 82 
  
  dash.all.js:11 [59404][indexHandler][audio] SegmentTemplate: 82.082 / 1361.235 
  
  dash.all.js:11 [59406][indexHandler][audio] SegmentTemplate: 83.08299999999998 / 1361.235 
  
  dash.all.js:11 [59408][rulesController] [RULES]: playbackTimeRule [Fragment:download,83,MediaSegment,0audio] Default 
  
  dash.all.js:11 [59416][fragmentLoader] loaded video:MediaSegment:83.08299999999998 (200, 17ms, 1ms) 
  
  dash.all.js:11 [59418][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,83,MediaSegment,0,audio] Default 
  
  dash.all.js:11 [59437][fragmentLoader] loaded audio:MediaSegment:83.08299999999998 (200, 14ms, 1ms) 
  
  dash.all.js:11 [59902][scheduleController][video] Stalling Buffer 
  
  dash.all.js:11 [59903][bufferController][video] Waiting for more buffer before starting playback. 
  
  dash.all.js:11 [59906][scheduleController][audio] Stalling Buffer 
  
  dash.all.js:11 [59907][bufferController][audio] Waiting for more buffer before starting playback. 
  
  dash.all.js:11 [63684][playbackController] <video> ratechange:  0 
  
  
  
  
  
  
  
  
  
  ///HOG VIDEO
  
  
  
  [58250][rulesController] [RULES]: playbackTimeRule [Fragment:download,105,MediaSegment,0audio] Default 
  
  dash.all.js:11 [58266][fragmentLoader] loaded video:MediaSegment:84.105 (200, 28ms, 1ms) 
  
  dash.all.js:11 [58269][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,105,MediaSegment,0,audio] Default 
  
  dash.all.js:11 [58287][fragmentLoader] loaded audio:MediaSegment:84.105 (200, 14ms, 1ms) 
  
  dash.all.js:11 [58982][indexHandler][video] Getting the request for time: 84.906 
  
  dash.all.js:11 [58983][indexHandler][video] Index for time 84.906 is 105 
  
  dash.all.js:11 [58984][indexHandler][video] SegmentTemplate: 84.105 / 3111.909 
  
  dash.all.js:11 [58984][indexHandler][video] SegmentTemplate: 84.906 / 3111.909 
  
  dash.all.js:11 [58985][rulesController] [RULES]: playbackTimeRule [Fragment:download,106,MediaSegment,0video] Default 
  
  dash.all.js:11 [58986][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,106,MediaSegment,0,video] Default 
  
  dash.all.js:11 [58989][indexHandler][audio] Getting the request for time: 84.906 
  
  dash.all.js:11 [58990][indexHandler][audio] Index for time 84.906 is 105 
  
  dash.all.js:11 [58991][indexHandler][audio] SegmentTemplate: 84.105 / 3111.909 
  
  dash.all.js:11 [58992][indexHandler][audio] SegmentTemplate: 84.906 / 3111.909 
  
  dash.all.js:11 [58993][rulesController] [RULES]: playbackTimeRule [Fragment:download,106,MediaSegment,0audio] Default 
  
  dash.all.js:11 [59002][fragmentLoader] loaded video:MediaSegment:84.906 (200, 14ms, 1ms) 
  
  dash.all.js:11 [59003][rulesController] [RULES]: sameTimeRequestRule [Fragment:download,106,MediaSegment,0,audio] Default 
  
  dash.all.js:11 [59014][fragmentLoader] loaded audio:MediaSegment:84.906 (200, 8ms, 1ms) 
  
  dash.all.js:11 [59735][scheduleController][video] Stalling Buffer 
  
  dash.all.js:11 [59736][bufferController][video] Waiting for more buffer before starting playback. 
  
  dash.all.js:11 [59738][scheduleController][audio] Stalling Buffer 
  
  dash.all.js:11 [59739][bufferController][audio] Waiting for more buffer before starting playback. 
  
  dash.all.js:11 [61695][playbackController] <video> ratechange:  0 
  
  </video></video>

• Android recording video with overlay view [way 2]

  2 mars 2016, par t0m

  I am trying app in android which have functionality to capture video with overlay views. I tried two ways (1. and 2.).
  1. Via SurfaceView and JavaCV with FFmpeg.
  2. Via OpenCV and JavaCV with FFmpeg.
  3. For API21+ maybe with MediaProjection.

  (Question is divided to two questions, due to stackoverflow length limit.)

  ad 1. Via SurfaceView and JavaCV with FFmpeg :

  Here

  ad 2. Via OpenCV and JavaCV with FFmpeg :

  OpenCVCameraActivity.java :

  import android.app.Activity;
  
  import android.hardware.Camera;
  
  import android.media.AudioFormat;
  
  import android.media.AudioRecord;
  
  import android.media.MediaRecorder;
  
  import android.os.Bundle;
  
  import android.os.Environment;
  
  import android.util.Log;
  
  import android.view.Menu;
  
  import android.view.MenuItem;
  
  import android.view.MotionEvent;
  
  import android.view.SubMenu;
  
  import android.view.SurfaceView;
  
  import android.view.View;
  
  import android.view.WindowManager;
  
  import android.widget.Toast;
  
  
  
  import org.bytedeco.javacv.FFmpegFrameRecorder;
  
  import org.bytedeco.javacv.Frame;
  
  import org.opencv.android.BaseLoaderCallback;
  
  import org.opencv.android.CameraBridgeViewBase;
  
  import org.opencv.android.LoaderCallbackInterface;
  
  import org.opencv.android.OpenCVLoader;
  
  import org.opencv.core.Mat;
  
  
  
  import java.io.File;
  
  import java.nio.ByteBuffer;
  
  import java.nio.ShortBuffer;
  
  import java.text.SimpleDateFormat;
  
  import java.util.Date;
  
  import java.util.List;
  
  import java.util.ListIterator;
  
  
  
  @SuppressWarnings("ALL")
  
  public class OpenCVCameraActivity extends Activity implements
  
          CameraBridgeViewBase.CvCameraViewListener2,
  
          View.OnTouchListener {
  
  
  
      //name of activity, for DEBUGGING
  
      private static final String TAG = OpenCVCameraActivity.class.getSimpleName();
  
  
  
      private OpenCVCameraPreview mOpenCvCameraView;
  
      private List mResolutionList;
  
      private MenuItem[] mEffectMenuItems;
  
      private SubMenu mColorEffectsMenu;
  
      private MenuItem[] mResolutionMenuItems;
  
      private SubMenu mResolutionMenu;
  
  
  
      private static long frameCounter = 0;
  
  
  
      long startTime = 0;
  
      private Mat edgesMat;
  
      boolean recording = false;
  
      private int sampleAudioRateInHz = 44100;
  
      private int imageWidth = 1280;
  
      private int imageHeight = 720;
  
      private int frameRate = 30;
  
      private Frame yuvImage = null;
  
      private File ffmpeg_link;
  
      private FFmpegFrameRecorder recorder;
  
  
  
      /*audio data getting thread */
  
      private AudioRecord audioRecord;
  
      private AudioRecordRunnable audioRecordRunnable;
  
      private Thread audioThread;
  
      volatile boolean runAudioThread = true;
  
      ShortBuffer[] samples;
  
  
  
  
  
      private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
  
          @Override
  
          public void onManagerConnected(int status) {
  
              switch (status) {
  
                  case LoaderCallbackInterface.SUCCESS:
  
                      Log.i(TAG, "OpenCV loaded successfully");
  
                      mOpenCvCameraView.enableView();
  
                      mOpenCvCameraView.setOnTouchListener(OpenCVCameraActivity.this);
  
                  break;
  
                  default:
  
                      super.onManagerConnected(status);
  
                  break;
  
              }
  
          }
  
      };
  
  
  
      @Override
  
      protected void onCreate(Bundle savedInstanceState) {
  
          super.onCreate(savedInstanceState);
  
          if(Static.DEBUG) Log.i(TAG, "onCreate()");
  
  
  
          getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
  
  
  
          Thread.setDefaultUncaughtExceptionHandler(uncaughtExceptionHandler);
  
  
  
          try {
  
              setContentView(R.layout.activity_opencv);
  
  
  
              mOpenCvCameraView = (OpenCVCameraPreview) findViewById(R.id.openCVCameraPreview);
  
              mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
  
              mOpenCvCameraView.setCvCameraViewListener(this);
  
  
  
              mOpenCvCameraView.enableFpsMeter();
  
  
  
              ffmpeg_link = new File(Environment.getExternalStorageDirectory(), "stream.mp4");
  
          } catch (Exception e){
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      private Thread.UncaughtExceptionHandler uncaughtExceptionHandler =
  
              new Thread.UncaughtExceptionHandler() {
  
                  public void uncaughtException(Thread thread, Throwable ex) {
  
                      if(Static.DEBUG) Log.e(TAG, "Uncaught exception", ex);
  
                  }
  
              };
  
  
  
      @Override
  
      protected void onRestart() {
  
          if (Static.DEBUG) Log.i(TAG, "onRestart()");
  
          super.onRestart();
  
      }
  
  
  
      @Override
  
      protected void onStart() {
  
          if (Static.DEBUG) Log.i(TAG, "onStart()");
  
          super.onStart();
  
      }
  
  
  
      @Override
  
      protected void onResume() {
  
          if (Static.DEBUG) Log.i(TAG, "onResume()");
  
          super.onResume();
  
  
  
          if (!OpenCVLoader.initDebug()) {
  
              Log.i(TAG, "Internal OpenCV library not found. Using OpenCV Manager for initialization");
  
              OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_3_1_0, this, mLoaderCallback);
  
          } else {
  
              Log.i(TAG, "OpenCV library found inside package. Using it!");
  
              mLoaderCallback.onManagerConnected(LoaderCallbackInterface.SUCCESS);
  
          }
  
  
  
      }
  
  
  
      @Override
  
      public boolean onCreateOptionsMenu(Menu menu) {
  
          if (Static.DEBUG) Log.i(TAG, "onCreateOptionsMenu()");
  
          super.onCreateOptionsMenu(menu);
  
  
  
          List<string> effects = mOpenCvCameraView.getEffectList();
  
  
  
          if (effects == null) {
  
              Log.e(TAG, "Color effects are not supported by device!");
  
              return true;
  
          }
  
  
  
          mColorEffectsMenu = menu.addSubMenu("Color Effect");
  
          mEffectMenuItems = new MenuItem[effects.size()];
  
  
  
          int idx = 0;
  
          ListIterator<string> effectItr = effects.listIterator();
  
          while(effectItr.hasNext()) {
  
              String element = effectItr.next();
  
              mEffectMenuItems[idx] = mColorEffectsMenu.add(1, idx, Menu.NONE, element);
  
              idx++;
  
          }
  
  
  
          mResolutionMenu = menu.addSubMenu("Resolution");
  
          mResolutionList = mOpenCvCameraView.getResolutionList();
  
          mResolutionMenuItems = new MenuItem[mResolutionList.size()];
  
  
  
          ListIterator resolutionItr = mResolutionList.listIterator();
  
          idx = 0;
  
          while(resolutionItr.hasNext()) {
  
              Camera.Size element = resolutionItr.next();
  
              mResolutionMenuItems[idx] = mResolutionMenu.add(2, idx, Menu.NONE,
  
                      Integer.valueOf(element.width).toString() + "x" + Integer.valueOf(element.height).toString());
  
              idx++;
  
          }
  
  
  
          return true;
  
      }
  
  
  
      @Override
  
      protected void onPause() {
  
          if (Static.DEBUG) Log.i(TAG, "onPause()");
  
          super.onPause();
  
  
  
          if (mOpenCvCameraView != null)
  
              mOpenCvCameraView.disableView();
  
  
  
      }
  
  
  
      @Override
  
      protected void onStop() {
  
          if (Static.DEBUG) Log.i(TAG, "onStop()");
  
          super.onStop();
  
      }
  
  
  
      @Override
  
      protected void onDestroy() {
  
          if (Static.DEBUG) Log.i(TAG, "onDestroy()");
  
          super.onDestroy();
  
  
  
          if (mOpenCvCameraView != null)
  
              mOpenCvCameraView.disableView();
  
      }
  
  
  
      public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
  
  
  
          ++frameCounter;
  
          //Log.i(TAG, "Frame number: "+frameCounter);
  
  
  
          final Mat rgba = inputFrame.rgba();
  
          //Core.flip(rgba, rgba, 1);
  
  
  
          /*if(Static.DEBUG) Log.i(TAG,"rgba.total(): "+rgba.total());
  
          if(Static.DEBUG) Log.i(TAG,"rgba.channels(): " +rgba.channels());*/
  
          byte[] data = new byte[(int) (rgba.total() * rgba.channels())];
  
          rgba.get(0, 0, data);
  
          //if(Static.DEBUG) Log.i(TAG,"return_buff: "+return_buff.length);
  
  
  
          if (audioRecord == null || audioRecord.getRecordingState() != AudioRecord.RECORDSTATE_RECORDING) {
  
              startTime = System.currentTimeMillis();
  
              return rgba;
  
          }
  
  
  
          // get video data
  
          if (yuvImage != null &amp;&amp; recording) {
  
              ByteBuffer b = (ByteBuffer)yuvImage.image[0].position(0);
  
              b.put(data);
  
  
  
              try {
  
                  long t = 1000 * (System.currentTimeMillis() - startTime);
  
                  if(Static.DEBUG) Log.i(TAG,"Writing Frame on timestamp: "+t);
  
                  if (t > recorder.getTimestamp()) {
  
                      recorder.setTimestamp(t);
  
                  }
  
                  recorder.record(yuvImage);
  
              } catch (FFmpegFrameRecorder.Exception e) {
  
                  if(Static.DEBUG) Log.i(TAG,e.getMessage());
  
                  e.printStackTrace();
  
              }
  
          }
  
  
  
          return rgba;
  
      }
  
  
  
      @Override
  
      public void onCameraViewStarted(int width, int height) {
  
          edgesMat = new Mat();
  
      }
  
  
  
      @Override
  
      public void onCameraViewStopped() {
  
          if (edgesMat != null)
  
              edgesMat.release();
  
  
  
          edgesMat = null;
  
      }
  
  
  
      public boolean onOptionsItemSelected(MenuItem item) {
  
          Log.i(TAG, "called onOptionsItemSelected; selected item: " + item);
  
          if (item.getGroupId() == 1)
  
          {
  
              mOpenCvCameraView.setEffect((String) item.getTitle());
  
              Toast.makeText(this, mOpenCvCameraView.getEffect(), Toast.LENGTH_SHORT).show();
  
          } else if (item.getGroupId() == 2) {
  
              int id = item.getItemId();
  
              Camera.Size resolution = mResolutionList.get(id);
  
              mOpenCvCameraView.setResolution(resolution);
  
              resolution = mOpenCvCameraView.getResolution();
  
              String caption = Integer.valueOf(resolution.width).toString() + "x" + Integer.valueOf(resolution.height).toString();
  
              Toast.makeText(this, caption, Toast.LENGTH_SHORT).show();
  
          }
  
  
  
          return true;
  
      }
  
  
  
      @Override
  
      public boolean onTouch(View v, MotionEvent event) {
  
          Log.i(TAG,"onTouch event");
  
          SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd_HH-mm-ss");
  
          String currentDateandTime = sdf.format(new Date());
  
          String fileName = Environment.getExternalStorageDirectory().getPath() +
  
                  "/sample_picture_" + currentDateandTime + ".jpg";
  
          mOpenCvCameraView.takePicture(fileName);
  
          Toast.makeText(this, fileName + " saved", Toast.LENGTH_SHORT).show();
  
          return false;
  
      }
  
  
  
      /**
  
       * Click to ImageButton to start recording.
  
       */
  
      public void onClickBtnStartRecord2(View v) {
  
          if (Static.DEBUG) Log.i(TAG, "onClickBtnStartRecord()");
  
  
  
          if(!recording)
  
              startRecording();
  
          else
  
              stopRecording();
  
      }
  
  
  
      private void startRecording() {
  
          if (Static.DEBUG) Log.i(TAG, "startRecording()");
  
          initRecorder();
  
  
  
          try {
  
              recorder.start();
  
              startTime = System.currentTimeMillis();
  
              recording = true;
  
              audioThread.start();
  
              if (Static.DEBUG) Log.i(TAG, "startRecording() success");
  
          } catch(FFmpegFrameRecorder.Exception e) {
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      private void stopRecording() {
  
          if (Static.DEBUG) Log.i(TAG, "stopRecording()");
  
  
  
          runAudioThread = false;
  
          try {
  
              audioThread.join();
  
          } catch(InterruptedException e) {
  
              e.printStackTrace();
  
          }
  
          audioRecordRunnable = null;
  
          audioThread = null;
  
          if (Static.DEBUG) Log.i(TAG, "stopRecording() 2");
  
          if(recorder != null &amp;&amp; recording) {
  
  
  
              recording = false;
  
              try {
  
                  recorder.stop();
  
                  recorder.release();
  
                  Log.i(TAG, "Finishing recording, calling stop and release on recorder");
  
              } catch(FFmpegFrameRecorder.Exception e) {
  
                  e.printStackTrace();
  
              }
  
              recorder = null;
  
          }
  
      }
  
  
  
  
  
      //---------------------------------------
  
      // initialize ffmpeg_recorder
  
      //---------------------------------------
  
      private void initRecorder() {
  
  
  
          Log.i(TAG, "init recorder");
  
          try {
  
  
  
              if (yuvImage == null) {
  
                  yuvImage = new Frame(imageWidth, imageHeight, Frame.DEPTH_UBYTE, 4);
  
                  Log.i(TAG, "create yuvImage");
  
              }
  
  
  
              Log.i(TAG, "ffmpeg_url: " + ffmpeg_link.getAbsolutePath());
  
              //Log.i(TAG, "ffmpeg_url: " + ffmpeg_link.exists());
  
              recorder = new FFmpegFrameRecorder(ffmpeg_link, imageWidth, imageHeight, 1);
  
              recorder.setFormat("mp4");
  
              recorder.setSampleRate(sampleAudioRateInHz);
  
              // Set in the surface changed method
  
              recorder.setFrameRate(frameRate);
  
  
  
              audioRecordRunnable = new AudioRecordRunnable();
  
              audioThread = new Thread(audioRecordRunnable);
  
              runAudioThread = true;
  
              Log.i(TAG, "recorder initialize success");
  
          } catch (Exception e){
  
              e.printStackTrace();
  
          }
  
      }
  
  
  
      //---------------------------------------------
  
      // audio thread, gets and encodes audio data
  
      //---------------------------------------------
  
      class AudioRecordRunnable implements Runnable {
  
  
  
          @Override
  
          public void run() {
  
              android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
  
  
  
              // Audio
  
              int bufferSize;
  
              ShortBuffer audioData;
  
              int bufferReadResult;
  
  
  
              bufferSize = AudioRecord.getMinBufferSize(sampleAudioRateInHz,
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT);
  
              audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC, sampleAudioRateInHz,
  
                      AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSize);
  
  
  
              audioData = ShortBuffer.allocate(bufferSize);
  
  
  
              Log.d(TAG, "audioRecord.startRecording()");
  
              audioRecord.startRecording();
  
  
  
              // ffmpeg_audio encoding loop
  
              while(runAudioThread) {
  
                  //Log.v(TAG,"recording? " + recording);
  
                  bufferReadResult = audioRecord.read(audioData.array(), 0, audioData.capacity());
  
                  audioData.limit(bufferReadResult);
  
                  if(bufferReadResult > 0) {
  
                      Log.v(TAG, "bufferReadResult: " + bufferReadResult);
  
                      // If "recording" isn't true when start this thread, it never get's set according to this if statement...!!!
  
                      // Why?  Good question...
  
                      if(recording) {
  
                              try {
  
                                  recorder.recordSamples(audioData);
  
                                  //Log.v(TAG,"recording " + 1024*i + " to " + 1024*i+1024);
  
                              } catch(FFmpegFrameRecorder.Exception e) {
  
                                  Log.v(TAG, e.getMessage());
  
                                  e.printStackTrace();
  
                              }
  
                      }
  
                  }
  
              }
  
              Log.v(TAG, "AudioThread Finished, release audioRecord");
  
  
  
              // encoding finish, release recorder
  
              if(audioRecord != null) {
  
                  audioRecord.stop();
  
                  audioRecord.release();
  
                  audioRecord = null;
  
                  Log.v(TAG, "audioRecord released");
  
              }
  
          }
  
      }
  
  }
  
  </string></string>

  OpenCVCameraPreview.java :

  import android.content.Context;
  
  import android.hardware.Camera;
  
  import android.util.AttributeSet;
  
  import android.util.Log;
  
  
  
  import org.opencv.android.JavaCameraView;
  
  
  
  import java.io.FileOutputStream;
  
  import java.util.List;
  
  
  
  public class OpenCVCameraPreview extends JavaCameraView implements Camera.PictureCallback {
  
  
  
      private static final String TAG =  OpenCVCameraPreview.class.getSimpleName();
  
      private String mPictureFileName;
  
  
  
      public OpenCVCameraPreview(Context context, AttributeSet attrs) {
  
          super(context, attrs);
  
      }
  
  
  
      public List<string> getEffectList() {
  
          return mCamera.getParameters().getSupportedColorEffects();
  
      }
  
  
  
      public boolean isEffectSupported() {
  
          return (mCamera.getParameters().getColorEffect() != null);
  
      }
  
  
  
      public String getEffect() {
  
          return mCamera.getParameters().getColorEffect();
  
      }
  
  
  
      public void setEffect(String effect) {
  
          Camera.Parameters params = mCamera.getParameters();
  
          params.setColorEffect(effect);
  
          mCamera.setParameters(params);
  
      }
  
  
  
      public List getResolutionList() {
  
          return mCamera.getParameters().getSupportedPreviewSizes();
  
      }
  
  
  
      public void setResolution(Camera.Size resolution) {
  
          disconnectCamera();
  
          mMaxHeight = resolution.height;
  
          mMaxWidth = resolution.width;
  
          connectCamera(getWidth(), getHeight());
  
      }
  
  
  
      public Camera.Size getResolution() {
  
          return mCamera.getParameters().getPreviewSize();
  
      }
  
  
  
      public void takePicture(final String fileName) {
  
          Log.i(TAG, "Taking picture");
  
          this.mPictureFileName = fileName;
  
          // Postview and jpeg are sent in the same buffers if the queue is not empty when performing a capture.
  
          // Clear up buffers to avoid mCamera.takePicture to be stuck because of a memory issue
  
          mCamera.setPreviewCallback(null);
  
  
  
          // PictureCallback is implemented by the current class
  
          mCamera.takePicture(null, null, this);
  
      }
  
  
  
      @Override
  
      public void onPictureTaken(byte[] data, Camera camera) {
  
          Log.i(TAG, "Saving a bitmap to file");
  
          // The camera preview was automatically stopped. Start it again.
  
          mCamera.startPreview();
  
          mCamera.setPreviewCallback(this);
  
  
  
          // Write the image in a file (in jpeg format)
  
          try {
  
              FileOutputStream fos = new FileOutputStream(mPictureFileName);
  
  
  
              fos.write(data);
  
              fos.close();
  
  
  
          } catch (java.io.IOException e) {
  
              Log.e("PictureDemo", "Exception in photoCallback", e);
  
          }
  
  
  
      }
  
  }
  
  </string>

  activity_opencv.xml :

  <?xml version="1.0" encoding="utf-8"?>
  
  <relativelayout>
  
  
  
      
  
  
  
      &lt;ImageButton<br />
  
          android:id=&quot;@+id/btnStartRecord2&quot;<br />
  
          android:layout_width=&quot;70dp&quot;<br />
  
          android:layout_height=&quot;70dp&quot;<br />
  
          android:scaleType=&quot;fitXY&quot;<br />
  
          android:src=&quot;@drawable/record_icon&quot;<br />
  
          android:background=&quot;@null&quot;<br />
  
          android:text=&quot;@string/btnStartRecord&quot;<br />
  
          android:onClick=&quot;onClickBtnStartRecord2&quot;<br />
  
          android:layout_centerVertical=&quot;true&quot;<br />
  
          android:layout_alignParentRight=&quot;true&quot;<br />
  
          android:layout_alignParentEnd=&quot;true&quot;/&gt;
  
  
  
  
  
  </relativelayout>

  Overlay views working, but recorded video is without overlay views, and recording with onCameraFrame method is very slow.