Recherche avancée

Sur d’autres sites (9717)

• WebRTC : unsync audio after processing using ffmpeg (audio length is less than that of video)

  22 novembre 2013, par QuickSilver

  I am recording a video and using RecordRTC : WebRTC . After receiving the webm video and wav audio at server, I'm encoding it to a mp4 file using ffmpeg(executing shell command via php). But after encoding process, the audio is unsync with video (audio ends before video). How can I fix this ?

  I have noticed that the recorded audio is 1 sec less in length with video.

  js code is here

  record.onclick = function() {
  
      record.disabled = true;
  
      var video_constraints = {
  
          mandatory: {
  
              "minWidth": "320",
  
              "minHeight": "240",
  
              "minFrameRate": "24",
  
              "maxWidth": "320",
  
              "maxHeight": "240",
  
              "maxFrameRate": "24"
  
          },
  
          optional: []
  
      };
  
      navigator.getUserMedia({
  
          audio: true,
  
          video: video_constraints
  
      }, function(stream) {
  
          preview.src = window.URL.createObjectURL(stream);
  
          preview.play();
  
  
  
          // var legalBufferValues = [256, 512, 1024, 2048, 4096, 8192, 16384];
  
          // sample-rates in at least the range 22050 to 96000.
  
          recordAudio = RecordRTC(stream, {
  
              /* extra important, we need to set a big buffer when capturing audio and video at the same time*/
  
              bufferSize: 16384
  
              //sampleRate: 45000
  
          });
  
  
  
          recordVideo = RecordRTC(stream, {
  
              type: 'video'
  
          });
  
  
  
          recordVideo.startRecording();
  
          recordAudio.startRecording();
  
  
  
          stop.disabled = false;
  
          recording_flag = true;
  
          $("#divcounter").show();
  
          $("#second-step-title").text('Record your video');
  
          initCountdown();
  
          uploadStatus.video = false;
  
          uploadStatus.audio = false;
  
      });
  
  };

  ffmpeg command used is :

  ffmpeg -y -i 166890589.wav -i 166890589.webm -vcodec libx264 166890589.mp4

  Currently I'm adding an offset of -1 to ffmpeg, but i don't think it's right.

  ffmpeg -y -itsoffset -1 -i 166890589.wav -i 166890589.webm -vcodec libx264 166890589.mp4

• Use ffmpeg to extract picture from m4v file

  31 octobre 2017, par Brian

  I used a program called MetaZ on my mac to tag all my video files (m4v). I am now trying to use these m4v files in Kodi which requires .nfo files and separate picture files for movie posters, etc. I want to extract the picture that is already in the m4v file.

  When I use ffprobe -show_streams, I can see that index4 is a png file (codec_name=png). How do I extract it ? I believe ffmpeg can do it, but can’t figure out how.

  Here is the output from ffprobe :

  Brians-Mac-mini:PythonScript brianjhille$ ffprobe -show_streams badwords.m4v
  
  ffprobe version N-88046-g0cb8369-tessus Copyright (c) 2007-2017 the FFmpeg developers
  
    built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
  
    configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libass --enable-libbluray --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay
  
    libavutil      56.  0.100 / 56.  0.100
  
    libavcodec     58.  0.100 / 58.  0.100
  
    libavformat    58.  0.100 / 58.  0.100
  
    libavdevice    58.  0.100 / 58.  0.100
  
    libavfilter     7.  0.100 /  7.  0.100
  
    libswscale      5.  0.100 /  5.  0.100
  
    libswresample   3.  0.100 /  3.  0.100
  
    libpostproc    55.  0.100 / 55.  0.100
  
  [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd67b002a00] stream 0, timescale not set
  
  Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'badwords.m4v':
  
    Metadata:
  
      major_brand     : mp42
  
      minor_version   : 0
  
      compatible_brands: mp42isomavc1
  
      creation_time   : 2014-10-20T13:01:06.000000Z
  
      iTunEXTC        : mpaa|R|400|
  
      title           : Bad Words
  
      artist          : Jason Bateman, Kathryn Hahn, Allison Janney, Philip Baker Hall, Rohan Chand, Ben Falcone, Patricia Belcher, Beth Grant, Rachel Harris, Steve Witting, Greg Cromer
  
      date            : 2013-09-06T11:00:00Z
  
      track           : 0
  
      disc            : 0
  
      season_number   : 0
  
      episode_sort    : 0
  
      description     : A spelling bee loser sets out to exact revenge by finding a loophole and attempting to win as an adult.
  
      synopsis        : A spelling bee loser sets out to exact revenge by finding a loophole and attempting to win as an adult.
  
      encoder         : HandBrake 0.9.9 2013052900
  
      hd_video        : 0
  
      media_type      : 9
  
      genre           : Comedy
  
      iTunMOVI        : <?xml version="1.0" encoding="UTF-8"?>
  
                      : 
  
                      : <plist version="1.0">
  
                      : <dict>
  
                      :   <key>cast</key>
  
                      :   <array>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Jason Bateman</string>
  
                      :       </dict>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Kathryn Hahn</string>
  
                      :       </dict>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Allison Janney</string>
  
                      :       </dict>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Philip Baker Hall</string>
  
                      :       </dict>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Rohan Chand</string>
  
                      :       </dict>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Ben Falcone</string>
  
                      :       </dict>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Patricia Belcher</string>
  
                      :       </dict>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Beth Grant</string>
  
                      :       </dict>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Rachel Harris</string>
  
                      :       </dict>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Steve Witting</string>
  
                      :       </dict>
  
                      :       <dict>
  
                      :           <key>name</key>
  
                      :           <string>Greg Cromer</string>
  
                      :       </dict>
  
                      :   </array>
  
                      : </dict>
  
                      : </plist>
  
                      : 
  
    Duration: 01:29:02.84, start: 0.000000, bitrate: 1339 kb/s
  
      Chapter #0:0: start 0.000000, end 348.214000
  
      Metadata:
  
        title           : Chapter 1
  
      Chapter #0:1: start 348.214000, end 676.542000
  
      Metadata:
  
        title           : Chapter 2
  
      Chapter #0:2: start 676.542000, end 860.058000
  
      Metadata:
  
        title           : Chapter 3
  
      Chapter #0:3: start 860.058000, end 1171.836000
  
      Metadata:
  
        title           : Chapter 4
  
      Chapter #0:4: start 1171.836000, end 1441.839000
  
      Metadata:
  
        title           : Chapter 5
  
      Chapter #0:5: start 1441.839000, end 1632.129000
  
      Metadata:
  
        title           : Chapter 6
  
      Chapter #0:6: start 1632.129000, end 1925.422000
  
      Metadata:
  
        title           : Chapter 7
  
      Chapter #0:7: start 1925.422000, end 2167.030000
  
      Metadata:
  
        title           : Chapter 8
  
      Chapter #0:8: start 2167.030000, end 2409.605000
  
      Metadata:
  
        title           : Chapter 9
  
      Chapter #0:9: start 2409.605000, end 2748.276000
  
      Metadata:
  
        title           : Chapter 10
  
      Chapter #0:10: start 2748.276000, end 2917.945000
  
      Metadata:
  
        title           : Chapter 11
  
      Chapter #0:11: start 2917.945000, end 3309.502000
  
      Metadata:
  
        title           : Chapter 12
  
      Chapter #0:12: start 3309.502000, end 3634.660000
  
      Metadata:
  
        title           : Chapter 13
  
      Chapter #0:13: start 3634.660000, end 3942.434000
  
      Metadata:
  
        title           : Chapter 14
  
      Chapter #0:14: start 3942.434000, end 4101.626000
  
      Metadata:
  
        title           : Chapter 15
  
      Chapter #0:15: start 4101.626000, end 4336.193000
  
      Metadata:
  
        title           : Chapter 16
  
      Chapter #0:16: start 4336.193000, end 4620.643000
  
      Metadata:
  
        title           : Chapter 17
  
      Chapter #0:17: start 4620.643000, end 4873.729000
  
      Metadata:
  
        title           : Chapter 18
  
      Chapter #0:18: start 4873.729000, end 5153.341000
  
      Metadata:
  
        title           : Chapter 19
  
      Chapter #0:19: start 5153.341000, end 5342.796000
  
      Metadata:
  
        title           : Chapter 20
  
      Stream #0:0(und): Video: h264 (Constrained Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m/smpte170m/bt709), 720x356 [SAR 32:27 DAR 640:267], 716 kb/s, 23.98 fps, 59.94 tbr, 90k tbn, 180k tbc (default)
  
      Metadata:
  
        creation_time   : 2014-10-20T13:01:06.000000Z
  
        encoder         : JVT/AVC Coding
  
      Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 159 kb/s (default)
  
      Metadata:
  
        creation_time   : 2014-10-20T13:01:06.000000Z
  
      Stream #0:2(eng): Audio: ac3 (ac-3 / 0x332D6361), 48000 Hz, 5.1(side), fltp, 448 kb/s
  
      Metadata:
  
        creation_time   : 2014-10-20T13:01:06.000000Z
  
      Side data:
  
        audio service type: main
  
      Stream #0:3(und): Data: bin_data (text / 0x74786574)
  
      Metadata:
  
        creation_time   : 2014-10-21T13:42:00.000000Z
  
      Stream #0:4: Video: png, rgb24(pc), 1400x2100, 90k tbr, 90k tbn, 90k tbc
  
  Unsupported codec with id 100359 for input stream 3
  
  [STREAM]
  
  index=0
  
  codec_name=h264
  
  codec_long_name=H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10
  
  profile=Constrained Baseline
  
  codec_type=video
  
  codec_time_base=40071281/1921695000
  
  codec_tag_string=avc1
  
  codec_tag=0x31637661
  
  width=720
  
  height=356
  
  coded_width=720
  
  coded_height=356
  
  has_b_frames=0
  
  sample_aspect_ratio=32:27
  
  display_aspect_ratio=640:267
  
  pix_fmt=yuv420p
  
  level=30
  
  color_range=tv
  
  color_space=smpte170m
  
  color_transfer=bt709
  
  color_primaries=smpte170m
  
  chroma_location=left
  
  field_order=unknown
  
  timecode=N/A
  
  refs=1
  
  is_avc=true
  
  nal_length_size=4
  
  id=N/A
  
  r_frame_rate=60000/1001
  
  avg_frame_rate=960847500/40071281
  
  time_base=1/90000
  
  start_pts=0
  
  start_time=0.000000
  
  duration_ts=480855372
  
  duration=5342.837467
  
  bit_rate=716167
  
  max_bit_rate=N/A
  
  bits_per_raw_sample=8
  
  nb_frames=128113
  
  nb_read_frames=N/A
  
  nb_read_packets=N/A
  
  DISPOSITION:default=1
  
  DISPOSITION:dub=0
  
  DISPOSITION:original=0
  
  DISPOSITION:comment=0
  
  DISPOSITION:lyrics=0
  
  DISPOSITION:karaoke=0
  
  DISPOSITION:forced=0
  
  DISPOSITION:hearing_impaired=0
  
  DISPOSITION:visual_impaired=0
  
  DISPOSITION:clean_effects=0
  
  DISPOSITION:attached_pic=0
  
  DISPOSITION:timed_thumbnails=0
  
  TAG:creation_time=2014-10-20T13:01:06.000000Z
  
  TAG:language=und
  
  TAG:encoder=JVT/AVC Coding
  
  [/STREAM]
  
  [STREAM]
  
  index=1
  
  codec_name=aac
  
  codec_long_name=AAC (Advanced Audio Coding)
  
  profile=LC
  
  codec_type=audio
  
  codec_time_base=1/48000
  
  codec_tag_string=mp4a
  
  codec_tag=0x6134706d
  
  sample_fmt=fltp
  
  sample_rate=48000
  
  channels=2
  
  channel_layout=stereo
  
  bits_per_sample=0
  
  id=N/A
  
  r_frame_rate=0/0
  
  avg_frame_rate=0/0
  
  time_base=1/48000
  
  start_pts=0
  
  start_time=0.000000
  
  duration_ts=256454656
  
  duration=5342.805333
  
  bit_rate=159788
  
  max_bit_rate=321176
  
  bits_per_raw_sample=N/A
  
  nb_frames=250444
  
  nb_read_frames=N/A
  
  nb_read_packets=N/A
  
  DISPOSITION:default=1
  
  DISPOSITION:dub=0
  
  DISPOSITION:original=0
  
  DISPOSITION:comment=0
  
  DISPOSITION:lyrics=0
  
  DISPOSITION:karaoke=0
  
  DISPOSITION:forced=0
  
  DISPOSITION:hearing_impaired=0
  
  DISPOSITION:visual_impaired=0
  
  DISPOSITION:clean_effects=0
  
  DISPOSITION:attached_pic=0
  
  DISPOSITION:timed_thumbnails=0
  
  TAG:creation_time=2014-10-20T13:01:06.000000Z
  
  TAG:language=eng
  
  [/STREAM]
  
  [STREAM]
  
  index=2
  
  codec_name=ac3
  
  codec_long_name=ATSC A/52A (AC-3)
  
  profile=unknown
  
  codec_type=audio
  
  codec_time_base=1/48000
  
  codec_tag_string=ac-3
  
  codec_tag=0x332d6361
  
  sample_fmt=fltp
  
  sample_rate=48000
  
  channels=6
  
  channel_layout=5.1(side)
  
  bits_per_sample=0
  
  dmix_mode=-1
  
  ltrt_cmixlev=-1.000000
  
  ltrt_surmixlev=-1.000000
  
  loro_cmixlev=-1.000000
  
  loro_surmixlev=-1.000000
  
  id=N/A
  
  r_frame_rate=0/0
  
  avg_frame_rate=0/0
  
  time_base=1/48000
  
  start_pts=0
  
  start_time=0.000000
  
  duration_ts=256453632
  
  duration=5342.784000
  
  bit_rate=448000
  
  max_bit_rate=N/A
  
  bits_per_raw_sample=N/A
  
  nb_frames=166962
  
  nb_read_frames=N/A
  
  nb_read_packets=N/A
  
  DISPOSITION:default=0
  
  DISPOSITION:dub=0
  
  DISPOSITION:original=0
  
  DISPOSITION:comment=0
  
  DISPOSITION:lyrics=0
  
  DISPOSITION:karaoke=0
  
  DISPOSITION:forced=0
  
  DISPOSITION:hearing_impaired=0
  
  DISPOSITION:visual_impaired=0
  
  DISPOSITION:clean_effects=0
  
  DISPOSITION:attached_pic=0
  
  DISPOSITION:timed_thumbnails=0
  
  TAG:creation_time=2014-10-20T13:01:06.000000Z
  
  TAG:language=eng
  
  [SIDE_DATA]
  
  side_data_type=Audio Service Type
  
  [/SIDE_DATA]
  
  [/STREAM]
  
  [STREAM]
  
  index=3
  
  codec_name=bin_data
  
  codec_long_name=binary data
  
  profile=unknown
  
  codec_type=data
  
  codec_tag_string=text
  
  codec_tag=0x74786574
  
  id=N/A
  
  r_frame_rate=0/0
  
  avg_frame_rate=0/0
  
  time_base=1/1000
  
  start_pts=0
  
  start_time=0.000000
  
  duration_ts=5342837
  
  duration=5342.837000
  
  bit_rate=N/A
  
  max_bit_rate=N/A
  
  bits_per_raw_sample=N/A
  
  nb_frames=20
  
  nb_read_frames=N/A
  
  nb_read_packets=N/A
  
  DISPOSITION:default=0
  
  DISPOSITION:dub=0
  
  DISPOSITION:original=0
  
  DISPOSITION:comment=0
  
  DISPOSITION:lyrics=0
  
  DISPOSITION:karaoke=0
  
  DISPOSITION:forced=0
  
  DISPOSITION:hearing_impaired=0
  
  DISPOSITION:visual_impaired=0
  
  DISPOSITION:clean_effects=0
  
  DISPOSITION:attached_pic=0
  
  DISPOSITION:timed_thumbnails=0
  
  TAG:creation_time=2014-10-21T13:42:00.000000Z
  
  TAG:language=und
  
  [/STREAM]
  
  [STREAM]
  
  index=4
  
  codec_name=png
  
  codec_long_name=PNG (Portable Network Graphics) image
  
  profile=unknown
  
  codec_type=video
  
  codec_time_base=0/1
  
  codec_tag_string=[0][0][0][0]
  
  codec_tag=0x0000
  
  width=1400
  
  height=2100
  
  coded_width=1400
  
  coded_height=2100
  
  has_b_frames=0
  
  sample_aspect_ratio=0:1
  
  display_aspect_ratio=0:1
  
  pix_fmt=rgb24
  
  level=-99
  
  color_range=pc
  
  color_space=unknown
  
  color_transfer=unknown
  
  color_primaries=unknown
  
  chroma_location=unspecified
  
  field_order=unknown
  
  timecode=N/A
  
  refs=1
  
  id=N/A
  
  r_frame_rate=90000/1
  
  avg_frame_rate=0/0
  
  time_base=1/90000
  
  start_pts=0
  
  start_time=0.000000
  
  duration_ts=480855372
  
  duration=5342.837467
  
  bit_rate=N/A
  
  max_bit_rate=N/A
  
  bits_per_raw_sample=N/A
  
  nb_frames=N/A
  
  nb_read_frames=N/A
  
  nb_read_packets=N/A
  
  DISPOSITION:default=0
  
  DISPOSITION:dub=0
  
  DISPOSITION:original=0
  
  DISPOSITION:comment=0
  
  DISPOSITION:lyrics=0
  
  DISPOSITION:karaoke=0
  
  DISPOSITION:forced=0
  
  DISPOSITION:hearing_impaired=0
  
  DISPOSITION:visual_impaired=0
  
  DISPOSITION:clean_effects=0
  
  DISPOSITION:attached_pic=1
  
  DISPOSITION:timed_thumbnails=0
  
  [/STREAM]

  Thanks. Brian

• Trying to sync audio/visual using FFMpeg and openAL

  22 août 2013, par user1379811

  hI have been studying dranger ffmpeg tutorial which explains how to sync audio and visual once you have the frames displayed and audio playing which is where im at.

  Unfortunately, the tutorial is out of date (Stephen Dranger explaained that himself to me) and also uses sdl which im not doing - this is for Blackberry 10 application.

  I just cannot make the video frames display at the correct speed (they are just playing very fast) and I have been trying for over a week now - seriously !

  I have 3 threads happening - one to read from stream into audio and video queues and then 2 threads for audio and video.

  If somebody could explain whats happening after scanning my relevent code you would be a lifesaver.

  The delay (what I pass to usleep(testDelay) seems to be going up (incrementing) which doesn't seem right to me.

  count = 1;
  
      MyApp* inst = worker->app;//(VideoUploadFacebook*)arg;
  
      qDebug() << "\n start loadstream";
  
      w = new QWaitCondition();
  
      w2 = new QWaitCondition();
  
      context = avformat_alloc_context();
  
      inst->threadStarted = true;
  
      cout << "start of decoding thread";
  
      cout.flush();
  
  
  
  
  
      av_register_all();
  
      avcodec_register_all();
  
      avformat_network_init();
  
      av_log_set_callback(&log_callback);
  
      AVInputFormat   *pFormat;
  
      //const char      device[]     = "/dev/video0";
  
      const char      formatName[] = "mp4";
  
      cout << "2start of decoding thread";
  
      cout.flush();
  
  
  
  
  
  
  
      if (!(pFormat = av_find_input_format(formatName))) {
  
          printf("can't find input format %s\n", formatName);
  
          //return void*;
  
      }
  
      //open rtsp
  
      if(avformat_open_input(&context, inst->capturedUrl.data(), pFormat,NULL) != 0){
  
          // return ;
  
          cout << "error opening of decoding thread: " << inst->capturedUrl.data();
  
          cout.flush();
  
      }
  
  
  
      cout << "3start of decoding thread";
  
      cout.flush();
  
      // av_dump_format(context, 0, inst->capturedUrl.data(), 0);
  
      /*   if(avformat_find_stream_info(context,NULL) < 0){
  
          return EXIT_FAILURE;
  
      }
  
       */
  
      //search video stream
  
      for(int i =0;inb_streams;i++){
  
          if(context->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
  
              inst->video_stream_index = i;
  
      }
  
      cout << "3z start of decoding thread";
  
      cout.flush();
  
      AVFormatContext* oc = avformat_alloc_context();
  
      av_read_play(context);//play RTSP
  
      AVDictionary *optionsDict = NULL;
  
      ccontext = context->streams[inst->video_stream_index]->codec;
  
  
  
      inst->audioc = context->streams[1]->codec;
  
  
  
      cout << "4start of decoding thread";
  
      cout.flush();
  
      codec = avcodec_find_decoder(ccontext->codec_id);
  
      ccontext->pix_fmt = PIX_FMT_YUV420P;
  
  
  
      AVCodec* audio_codec = avcodec_find_decoder(inst->audioc->codec_id);
  
      inst->packet = new AVPacket();
  
      if (!audio_codec) {
  
          cout << "audio codec not found\n"; //fflush( stdout );
  
          exit(1);
  
      }
  
  
  
      if (avcodec_open2(inst->audioc, audio_codec, NULL) < 0) {
  
          cout << "could not open codec\n"; //fflush( stdout );
  
          exit(1);
  
      }
  
  
  
      if (avcodec_open2(ccontext, codec, &optionsDict) < 0) exit(1);
  
  
  
      cout << "5start of decoding thread";
  
      cout.flush();
  
      inst->pic = avcodec_alloc_frame();
  
  
  
      av_init_packet(inst->packet);
  
  
  
      while(av_read_frame(context,inst->packet) >= 0 && &inst->keepGoing)
  
      {
  
  
  
          if(inst->packet->stream_index == 0){//packet is video
  
  
  
              int check = 0;
  
  
  
  
  
  
  
              // av_init_packet(inst->packet);
  
              int result = avcodec_decode_video2(ccontext, inst->pic, &check, inst->packet);
  
  
  
              if(check)
  
                  break;
  
          }
  
      }
  
  
  
  
  
  
  
      inst->originalVideoWidth = inst->pic->width;
  
      inst->originalVideoHeight = inst->pic->height;
  
      float aspect = (float)inst->originalVideoHeight / (float)inst->originalVideoWidth;
  
      inst->newVideoWidth = inst->originalVideoWidth;
  
      int newHeight = (int)(inst->newVideoWidth * aspect);
  
      inst->newVideoHeight = newHeight;//(int)inst->originalVideoHeight / inst->originalVideoWidth * inst->newVideoWidth;// = new height
  
      int size = avpicture_get_size(PIX_FMT_YUV420P, inst->originalVideoWidth, inst->originalVideoHeight);
  
      uint8_t* picture_buf = (uint8_t*)(av_malloc(size));
  
      avpicture_fill((AVPicture *) inst->pic, picture_buf, PIX_FMT_YUV420P, inst->originalVideoWidth, inst->originalVideoHeight);
  
  
  
      picrgb = avcodec_alloc_frame();
  
      int size2 = avpicture_get_size(PIX_FMT_YUV420P, inst->newVideoWidth, inst->newVideoHeight);
  
      uint8_t* picture_buf2 = (uint8_t*)(av_malloc(size2));
  
      avpicture_fill((AVPicture *) picrgb, picture_buf2, PIX_FMT_YUV420P, inst->newVideoWidth, inst->newVideoHeight);
  
  
  
  
  
  
  
      if(ccontext->pix_fmt != PIX_FMT_YUV420P)
  
      {
  
          std::cout << "fmt != 420!!!: " << ccontext->pix_fmt << std::endl;//
  
          // return (EXIT_SUCCESS);//-1;
  
  
  
      }
  
  
  
  
  
      if (inst->createForeignWindow(inst->myForeignWindow->windowGroup(),
  
              "HelloForeignWindowAppIDqq", 0,
  
              0, inst->newVideoWidth,
  
              inst->newVideoHeight)) {
  
  
  
      } else {
  
          qDebug() << "The ForeginWindow was not properly initialized";
  
      }
  
  
  
  
  
  
  
  
  
      inst->keepGoing = true;
  
  
  
      inst->img_convert_ctx = sws_getContext(inst->originalVideoWidth, inst->originalVideoHeight, PIX_FMT_YUV420P, inst->newVideoWidth, inst->newVideoHeight,
  
              PIX_FMT_YUV420P, SWS_BILINEAR, NULL, NULL, NULL);
  
  
  
      is = (VideoState*)av_mallocz(sizeof(VideoState));
  
      if (!is)
  
          return NULL;
  
  
  
      is->audioStream = 1;
  
      is->audio_st = context->streams[1];
  
      is->audio_buf_size = 0;
  
      is->audio_buf_index = 0;
  
      is->videoStream = 0;
  
      is->video_st = context->streams[0];
  
  
  
      is->frame_timer = (double)av_gettime() / 1000000.0;
  
      is->frame_last_delay = 40e-3;
  
  
  
      is->av_sync_type = DEFAULT_AV_SYNC_TYPE;
  
      //av_strlcpy(is->filename, filename, sizeof(is->filename));
  
      is->iformat = pFormat;
  
      is->ytop    = 0;
  
      is->xleft   = 0;
  
  
  
      /* start video display */
  
      is->pictq_mutex = new QMutex();
  
      is->pictq_cond  = new QWaitCondition();
  
  
  
      is->subpq_mutex = new QMutex();
  
      is->subpq_cond  = new QWaitCondition();
  
  
  
      is->video_current_pts_time = av_gettime();
  
  
  
  
  
      packet_queue_init(&audioq);
  
  
  
      packet_queue_init(&videoq);
  
      is->audioq = audioq;
  
      is->videoq = videoq;
  
      AVPacket* packet2  = new AVPacket();
  
  
  
      ccontext->get_buffer = our_get_buffer;
  
      ccontext->release_buffer = our_release_buffer;
  
  
  
  
  
      av_init_packet(packet2);
  
      while(inst->keepGoing)
  
      {
  
  
  
  
  
          if(av_read_frame(context,packet2) < 0 && keepGoing)
  
          {
  
              printf("bufferframe Could not read a frame from stream.\n");
  
              fflush( stdout );
  
  
  
  
  
          }else {
  
  
  
  
  
  
  
              if(packet2->stream_index == 0) {
  
                  packet_queue_put(&videoq, packet2);
  
              } else if(packet2->stream_index == 1) {
  
                  packet_queue_put(&audioq, packet2);
  
              } else {
  
                  av_free_packet(packet2);
  
              }
  
  
  
  
  
              if(!videoThreadStarted)
  
              {
  
                  videoThreadStarted = true;
  
                  QThread* thread = new QThread;
  
                  videoThread = new VideoStreamWorker(this);
  
  
  
                  // Give QThread ownership of Worker Object
  
                  videoThread->moveToThread(thread);
  
                  connect(videoThread, SIGNAL(error(QString)), this, SLOT(errorHandler(QString)));
  
                  QObject::connect(videoThread, SIGNAL(refreshNeeded()), this, SLOT(refreshNeededSlot()));
  
                  connect(thread, SIGNAL(started()), videoThread, SLOT(doWork()));
  
                  connect(videoThread, SIGNAL(finished()), thread, SLOT(quit()));
  
                  connect(videoThread, SIGNAL(finished()), videoThread, SLOT(deleteLater()));
  
                  connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
  
  
  
                  thread->start();
  
              }
  
  
  
              if(!audioThreadStarted)
  
              {
  
                  audioThreadStarted = true;
  
                  QThread* thread = new QThread;
  
                  AudioStreamWorker* videoThread = new AudioStreamWorker(this);
  
  
  
                  // Give QThread ownership of Worker Object
  
                  videoThread->moveToThread(thread);
  
  
  
                  // Connect videoThread error signal to this errorHandler SLOT.
  
                  connect(videoThread, SIGNAL(error(QString)), this, SLOT(errorHandler(QString)));
  
  
  
                  // Connects the thread’s started() signal to the process() slot in the videoThread, causing it to start.
  
                  connect(thread, SIGNAL(started()), videoThread, SLOT(doWork()));
  
                  connect(videoThread, SIGNAL(finished()), thread, SLOT(quit()));
  
                  connect(videoThread, SIGNAL(finished()), videoThread, SLOT(deleteLater()));
  
  
  
                  // Make sure the thread object is deleted after execution has finished.
  
                  connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
  
  
  
                  thread->start();
  
              }
  
  
  
          }
  
  
  
      } //finished main loop
  
  
  
      int MyApp::video_thread() {
  
      //VideoState *is = (VideoState *)arg;
  
      AVPacket pkt1, *packet = &pkt1;
  
      int len1, frameFinished;
  
  
  
      double pts;
  
      pic = avcodec_alloc_frame();
  
  
  
      for(;;) {
  
          if(packet_queue_get(&videoq, packet, 1) < 0) {
  
              // means we quit getting packets
  
              break;
  
          }
  
  
  
          pts = 0;
  
  
  
          global_video_pkt_pts2 = packet->pts;
  
          // Decode video frame
  
          len1 =  avcodec_decode_video2(ccontext, pic, &frameFinished, packet);
  
          if(packet->dts == AV_NOPTS_VALUE
  
                  && pic->opaque && *(uint64_t*)pic->opaque != AV_NOPTS_VALUE) {
  
              pts = *(uint64_t *)pic->opaque;
  
          } else if(packet->dts != AV_NOPTS_VALUE) {
  
              pts = packet->dts;
  
          } else {
  
              pts = 0;
  
          }
  
          pts *= av_q2d(is->video_st->time_base);
  
          // Did we get a video frame?
  
  
  
                  if(frameFinished) {
  
                      pts = synchronize_video(is, pic, pts);
  
                      actualPts = pts;
  
                      refreshSlot();
  
                  }
  
                  av_free_packet(packet);
  
      }
  
      av_free(pic);
  
      return 0;
  
  }
  
  
  
  
  
  int MyApp::audio_thread() {
  
      //VideoState *is = (VideoState *)arg;
  
      AVPacket pkt1, *packet = &pkt1;
  
      int len1, frameFinished;
  
      ALuint source;
  
      ALenum format = 0;
  
      //   ALuint frequency;
  
      ALenum alError;
  
      ALint val2;
  
      ALuint buffers[NUM_BUFFERS];
  
      int dataSize;
  
  
  
  
  
      ALCcontext *aContext;
  
      ALCdevice *device;
  
      if (!alutInit(NULL, NULL)) {
  
          // printf(stderr, "init alut error\n");
  
      }
  
      device = alcOpenDevice(NULL);
  
      if (device == NULL) {
  
          // printf(stderr, "device error\n");
  
      }
  
  
  
      //Create a context
  
      aContext = alcCreateContext(device, NULL);
  
      alcMakeContextCurrent(aContext);
  
      if(!(aContext)) {
  
          printf("Could not create the OpenAL context!\n");
  
          return 0;
  
      }
  
  
  
      alListener3f(AL_POSITION, 0.0f, 0.0f, 0.0f);
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
  
      //ALenum alError;
  
      if(alGetError() != AL_NO_ERROR) {
  
          cout << "could not create buffers";
  
          cout.flush();
  
          fflush( stdout );
  
          return 0;
  
      }
  
      alGenBuffers(NUM_BUFFERS, buffers);
  
      alGenSources(1, &source);
  
      if(alGetError() != AL_NO_ERROR) {
  
          cout << "after Could not create buffers or the source.\n";
  
          cout.flush(  );
  
          return 0;
  
      }
  
  
  
      int i;
  
      int indexOfPacket;
  
      double pts;
  
      //double pts;
  
      int n;
  
  
  
  
  
      for(i = 0; i < NUM_BUFFERS; i++)
  
      {
  
          if(packet_queue_get(&audioq, packet, 1) < 0) {
  
              // means we quit getting packets
  
              break;
  
          }
  
          cout << "streamindex=audio \n";
  
          cout.flush(  );
  
          //printf("before decode  audio\n");
  
          //fflush( stdout );
  
          // AVPacket *packet = new AVPacket();//malloc(sizeof(AVPacket*));
  
          AVFrame *decodedFrame = NULL;
  
          int gotFrame = 0;
  
          // AVFrame* decodedFrame;
  
  
  
          if(!decodedFrame) {
  
              if(!(decodedFrame = avcodec_alloc_frame())) {
  
                  cout << "Run out of memory, stop the streaming...\n";
  
                  fflush( stdout );
  
                  cout.flush();
  
  
  
  
  
                  return -2;
  
              }
  
          } else {
  
              avcodec_get_frame_defaults(decodedFrame);
  
          }
  
  
  
          int  len = avcodec_decode_audio4(audioc, decodedFrame, &gotFrame, packet);
  
          if(len < 0) {
  
              cout << "Error while decoding.\n";
  
              cout.flush(  );
  
  
  
              return -3;
  
          }
  
          if(len < 0) {
  
              /* if error, skip frame */
  
              is->audio_pkt_size = 0;
  
              //break;
  
          }
  
          is->audio_pkt_data += len;
  
          is->audio_pkt_size -= len;
  
  
  
          pts = is->audio_clock;
  
          // *pts_ptr = pts;
  
          n = 2 * is->audio_st->codec->channels;
  
          is->audio_clock += (double)packet->size/
  
                  (double)(n * is->audio_st->codec->sample_rate);
  
          if(gotFrame) {
  
              cout << "got audio frame.\n";
  
              cout.flush(  );
  
              // We have a buffer ready, send it
  
              dataSize = av_samples_get_buffer_size(NULL, audioc->channels,
  
                      decodedFrame->nb_samples, audioc->sample_fmt, 1);
  
  
  
              if(!format) {
  
                  if(audioc->sample_fmt == AV_SAMPLE_FMT_U8 ||
  
                          audioc->sample_fmt == AV_SAMPLE_FMT_U8P) {
  
                      if(audioc->channels == 1) {
  
                          format = AL_FORMAT_MONO8;
  
                      } else if(audioc->channels == 2) {
  
                          format = AL_FORMAT_STEREO8;
  
                      }
  
                  } else if(audioc->sample_fmt == AV_SAMPLE_FMT_S16 ||
  
                          audioc->sample_fmt == AV_SAMPLE_FMT_S16P) {
  
                      if(audioc->channels == 1) {
  
                          format = AL_FORMAT_MONO16;
  
                      } else if(audioc->channels == 2) {
  
                          format = AL_FORMAT_STEREO16;
  
                      }
  
                  }
  
  
  
                  if(!format) {
  
                      cout << "OpenAL can't open this format of sound.\n";
  
                      cout.flush(  );
  
  
  
                      return -4;
  
                  }
  
              }
  
              printf("albufferdata audio b4.\n");
  
              fflush( stdout );
  
              alBufferData(buffers[i], format, *decodedFrame->data, dataSize, decodedFrame->sample_rate);
  
              cout << "after albufferdata all buffers \n";
  
              cout.flush(  );
  
              av_free_packet(packet);
  
              //=av_free(packet);
  
              av_free(decodedFrame);
  
  
  
              if((alError = alGetError()) != AL_NO_ERROR) {
  
                  printf("Error while buffering.\n");
  
  
  
                  printAlError(alError);
  
                  return -6;
  
              }
  
          }
  
      }
  
  
  
  
  
      cout << "before quoe buffers \n";
  
      cout.flush();
  
      alSourceQueueBuffers(source, NUM_BUFFERS, buffers);
  
      cout << "before play.\n";
  
      cout.flush();
  
      alSourcePlay(source);
  
      cout << "after play.\n";
  
      cout.flush();
  
      if((alError = alGetError()) != AL_NO_ERROR) {
  
          cout << "error strating stream.\n";
  
          cout.flush();
  
          printAlError(alError);
  
          return 0;
  
      }
  
  
  
  
  
      // AVPacket *pkt = &is->audio_pkt;
  
  
  
      while(keepGoing)
  
      {
  
          while(packet_queue_get(&audioq, packet, 1)  >= 0) {
  
              // means we quit getting packets
  
  
  
              do {
  
                  alGetSourcei(source, AL_BUFFERS_PROCESSED, &val2);
  
                  usleep(SLEEP_BUFFERING);
  
              } while(val2 <= 0);
  
              if(alGetError() != AL_NO_ERROR)
  
              {
  
                  fprintf(stderr, "Error gettingsource :(\n");
  
                  return 1;
  
              }
  
  
  
              while(val2--)
  
              {
  
  
  
  
  
  
  
                  ALuint buffer;
  
                  alSourceUnqueueBuffers(source, 1, &buffer);
  
                  if(alGetError() != AL_NO_ERROR)
  
                  {
  
                      fprintf(stderr, "Error unqueue buffers :(\n");
  
                      //  return 1;
  
                  }
  
                  AVFrame *decodedFrame = NULL;
  
                  int gotFrame = 0;
  
                  // AVFrame* decodedFrame;
  
  
  
                  if(!decodedFrame) {
  
                      if(!(decodedFrame = avcodec_alloc_frame())) {
  
                          cout << "Run out of memory, stop the streaming...\n";
  
                          //fflush( stdout );
  
                          cout.flush();
  
  
  
  
  
                          return -2;
  
                      }
  
                  } else {
  
                      avcodec_get_frame_defaults(decodedFrame);
  
                  }
  
  
  
                  int  len = avcodec_decode_audio4(audioc, decodedFrame, &gotFrame, packet);
  
                  if(len < 0) {
  
                      cout << "Error while decoding.\n";
  
                      cout.flush(  );
  
                      is->audio_pkt_size = 0;
  
                      return -3;
  
                  }
  
  
  
                  is->audio_pkt_data += len;
  
                  is->audio_pkt_size -= len;
  
                  if(packet->size <= 0) {
  
                      /* No data yet, get more frames */
  
                      //continue;
  
                  }
  
  
  
  
  
                  if(gotFrame) {
  
                      pts = is->audio_clock;
  
                      len = synchronize_audio(is, (int16_t *)is->audio_buf,
  
                              packet->size, pts);
  
                      is->audio_buf_size = packet->size;
  
                      pts = is->audio_clock;
  
                      // *pts_ptr = pts;
  
                      n = 2 * is->audio_st->codec->channels;
  
                      is->audio_clock += (double)packet->size /
  
                              (double)(n * is->audio_st->codec->sample_rate);
  
                      if(packet->pts != AV_NOPTS_VALUE) {
  
                          is->audio_clock = av_q2d(is->audio_st->time_base)*packet->pts;
  
                      }
  
                      len = av_samples_get_buffer_size(NULL, audioc->channels,
  
                              decodedFrame->nb_samples, audioc->sample_fmt, 1);
  
                      alBufferData(buffer, format, *decodedFrame->data, len, decodedFrame->sample_rate);
  
                      if(alGetError() != AL_NO_ERROR)
  
                      {
  
                          fprintf(stderr, "Error buffering :(\n");
  
                          return 1;
  
                      }
  
                      alSourceQueueBuffers(source, 1, &buffer);
  
                      if(alGetError() != AL_NO_ERROR)
  
                      {
  
                          fprintf(stderr, "Error queueing buffers :(\n");
  
                          return 1;
  
                      }
  
                  }
  
  
  
  
  
  
  
  
  
  
  
              }
  
  
  
              alGetSourcei(source, AL_SOURCE_STATE, &val2);
  
              if(val2 != AL_PLAYING)
  
                  alSourcePlay(source);
  
  
  
          }
  
  
  
  
  
          //pic = avcodec_alloc_frame();
  
      }
  
      qDebug() << "end audiothread";
  
      return 1;
  
  }
  
  
  
  void MyApp::refreshSlot()
  
  {
  
  
  
  
  
      if(true)
  
      {
  
  
  
          printf("got frame %d, %d\n", pic->width, ccontext->width);
  
          fflush( stdout );
  
  
  
          sws_scale(img_convert_ctx, (const uint8_t **)pic->data, pic->linesize,
  
                  0, originalVideoHeight, &picrgb->data[0], &picrgb->linesize[0]);
  
  
  
          printf("rescaled frame %d, %d\n", newVideoWidth, newVideoHeight);
  
          fflush( stdout );
  
          //av_free_packet(packet);
  
          //av_init_packet(packet);
  
  
  
          qDebug() << "waking audio as video finished";
  
          ////mutex.unlock();
  
          //mutex2.lock();
  
          doingVideoFrame = false;
  
          //doingAudioFrame = false;
  
          ////mutex2.unlock();
  
  
  
  
  
          //mutex2.unlock();
  
          //w2->wakeAll();
  
          //w->wakeAll();
  
          qDebug() << "now woke audio";
  
  
  
          //pic = picrgb;
  
          uint8_t *srcy = picrgb->data[0];
  
          uint8_t *srcu = picrgb->data[1];
  
          uint8_t *srcv = picrgb->data[2];
  
          printf("got src yuv frame %d\n", &srcy);
  
          fflush( stdout );
  
          unsigned char *ptr = NULL;
  
          screen_get_buffer_property_pv(mScreenPixelBuffer, SCREEN_PROPERTY_POINTER, (void**) &ptr);
  
          unsigned char *y = ptr;
  
          unsigned char *u = y + (newVideoHeight * mStride) ;
  
          unsigned char *v = u + (newVideoHeight * mStride) / 4;
  
          int i = 0;
  
          printf("got buffer  picrgbwidth= %d \n", newVideoWidth);
  
          fflush( stdout );
  
          for ( i = 0; i < newVideoHeight; i++)
  
          {
  
              int doff = i * mStride;
  
              int soff = i * picrgb->linesize[0];
  
              memcpy(&y[doff], &srcy[soff], newVideoWidth);
  
          }
  
  
  
          for ( i = 0; i < newVideoHeight / 2; i++)
  
          {
  
              int doff = i * mStride / 2;
  
              int soff = i * picrgb->linesize[1];
  
              memcpy(&u[doff], &srcu[soff], newVideoWidth / 2);
  
          }
  
  
  
          for ( i = 0; i < newVideoHeight / 2; i++)
  
          {
  
              int doff = i * mStride / 2;
  
              int soff = i * picrgb->linesize[2];
  
              memcpy(&v[doff], &srcv[soff], newVideoWidth / 2);
  
          }
  
          printf("before posttoscreen \n");
  
          fflush( stdout );
  
  
  
          video_refresh_timer();
  
          qDebug() << "end refreshslot";
  
  
  
      }
  
      else
  
      {
  
  
  
      }
  
  
  
  
  
  
  
  
  
  
  
  }
  
  
  
  void  MyApp::refreshNeededSlot2()
  
      {
  
          printf("blitting to buffer");
  
          fflush(stdout);
  
  
  
          screen_buffer_t screen_buffer;
  
          screen_get_window_property_pv(mScreenWindow, SCREEN_PROPERTY_RENDER_BUFFERS, (void**) &screen_buffer);
  
          int attribs[] = { SCREEN_BLIT_SOURCE_WIDTH, newVideoWidth, SCREEN_BLIT_SOURCE_HEIGHT, newVideoHeight, SCREEN_BLIT_END };
  
          int res2 = screen_blit(mScreenCtx, screen_buffer, mScreenPixelBuffer, attribs);
  
          printf("dirty rectangles");
  
          fflush(stdout);
  
          int dirty_rects[] = { 0, 0, newVideoWidth, newVideoHeight };
  
          screen_post_window(mScreenWindow, screen_buffer, 1, dirty_rects, 0);
  
          printf("done screneposdtwindow");
  
          fflush(stdout);
  
  
  
      }
  
  
  
  void MyApp::video_refresh_timer() {
  
      testDelay = 0;
  
      //  VideoState *is = ( VideoState* )userdata;
  
      VideoPicture *vp;
  
      //double pts = 0    ;
  
      double actual_delay, delay, sync_threshold, ref_clock, diff;
  
  
  
      if(is->video_st) {
  
          if(false)////is->pictq_size == 0)
  
          {
  
              testDelay = 1;
  
              schedule_refresh(is, 1);
  
          } else {
  
              // vp = &is->pictq[is->pictq_rindex];
  
  
  
              delay = actualPts - is->frame_last_pts; /* the pts from last time */
  
              if(delay <= 0 || delay >= 1.0) {
  
                  /* if incorrect delay, use previous one */
  
                  delay = is->frame_last_delay;
  
              }
  
              /* save for next time */
  
              is->frame_last_delay = delay;
  
              is->frame_last_pts = actualPts;
  
  
  
              is->video_current_pts = actualPts;
  
              is->video_current_pts_time = av_gettime();
  
              /* update delay to sync to audio */
  
              ref_clock = get_audio_clock(is);
  
              diff = actualPts - ref_clock;
  
  
  
              /* Skip or repeat the frame. Take delay into account
  
       FFPlay still doesn't "know if this is the best guess." */
  
              sync_threshold = (delay > AV_SYNC_THRESHOLD) ? delay : AV_SYNC_THRESHOLD;
  
              if(fabs(diff) < AV_NOSYNC_THRESHOLD) {
  
                  if(diff <= -sync_threshold) {
  
                      delay = 0;
  
                  } else if(diff >= sync_threshold) {
  
                      delay = 2 * delay;
  
                  }
  
              }
  
              is->frame_timer += delay;
  
              /* computer the REAL delay */
  
              actual_delay = is->frame_timer - (av_gettime() / 1000000.0);
  
              if(actual_delay < 0.010) {
  
                  /* Really it should skip the picture instead */
  
                  actual_delay = 0.010;
  
              }
  
              testDelay = (int)(actual_delay * 1000 + 0.5);
  
              schedule_refresh(is, (int)(actual_delay * 1000 + 0.5));
  
              /* show the picture! */
  
              //video_display(is);
  
  
  
  
  
              // SDL_CondSignal(is->pictq_cond);
  
              // SDL_UnlockMutex(is->pictq_mutex);
  
          }
  
      } else {
  
          testDelay = 100;
  
          schedule_refresh(is, 100);
  
  
  
      }
  
  }
  
  
  
  void MyApp::schedule_refresh(VideoState *is, int delay) {
  
      qDebug() << "start schedule refresh timer" << delay;
  
      typeOfEvent = FF_REFRESH_EVENT2;
  
      w->wakeAll();
  
      //  SDL_AddTimer(delay,
  
  
  
  
  
  }

  I am currently waiting on data in a loop in the following way

  QMutex mutex;
  
      mutex.lock();
  
      while(keepGoing)
  
      {
  
  
  
  
  
  
  
          qDebug() << "MAINTHREAD" << testDelay;
  
  
  
  
  
          w->wait(&mutex);
  
          mutex.unlock();
  
          qDebug() << "MAINTHREAD past wait";
  
  
  
          if(!keepGoing)
  
          {
  
              break;
  
          }
  
          if(testDelay > 0 && typeOfEvent == FF_REFRESH_EVENT2)
  
          {
  
              usleep(testDelay);
  
              refreshNeededSlot2();
  
          }
  
          else   if(testDelay > 0 && typeOfEvent == FF_QUIT_EVENT2)
  
          {
  
              keepGoing = false;
  
              exit(0);
  
              break;
  
              // usleep(testDelay);
  
              // refreshNeededSlot2();
  
          }
  
          qDebug() << "MAINTHREADend";
  
          mutex.lock();
  
  
  
      }
  
      mutex.unlock();

  Please let me know if I need to provide any more relevent code. I'm sorry my code is untidy - I still learning c++ and have been modifying this code for over a week now as previously mentioned.

  Just added a sample of output I'm seeing from print outs I do to console - I can't get my head around it (it's almost too complicated for my level of expertise) but when you see the frames being played and audio playing it's very difficult to give up especially when it took me a couple of weeks to get to this stage.

  Please someone give me a hand if they spot the problem.

  MAINTHREAD past wait
  pts after syncvideo= 1073394046
  got frame 640, 640
  start video_refresh_timer
  actualpts = 1.66833
  frame lastpts = 1.63497
  start schedule refresh timer need to delay for 123

  pts after syncvideo= 1073429033
  got frame 640, 640
  MAINTHREAD loop delay before refresh = 123
  start video_refresh_timer
  actualpts = 1.7017
  frame lastpts = 1.66833
  start schedule refresh timer need to delay for 115

  MAINTHREAD past wait
  pts after syncvideo= 1073464021
  got frame 640, 640
  start video_refresh_timer
  actualpts = 1.73507
  frame lastpts = 1.7017
  start schedule refresh timer need to delay for 140

  MAINTHREAD loop delay before refresh = 140
  pts after syncvideo= 1073499008
  got frame 640, 640
  start video_refresh_timer
  actualpts = 1.76843
  frame lastpts = 1.73507
  start schedule refresh timer need to delay for 163

  MAINTHREAD past wait
  pts after syncvideo= 1073533996
  got frame 640, 640
  start video_refresh_timer
  actualpts = 1.8018
  frame lastpts = 1.76843
  start schedule refresh timer need to delay for 188

  MAINTHREAD loop delay before refresh = 188
  pts after syncvideo= 1073568983
  got frame 640, 640
  start video_refresh_timer
  actualpts = 1.83517
  frame lastpts = 1.8018
  start schedule refresh timer need to delay for 246

  MAINTHREAD past wait
  pts after syncvideo= 1073603971
  got frame 640, 640
  start video_refresh_timer
  actualpts = 1.86853
  frame lastpts = 1.83517
  start schedule refresh timer need to delay for 299

  MAINTHREAD loop delay before refresh = 299
  pts after syncvideo= 1073638958
  got frame 640, 640
  start video_refresh_timer
  actualpts = 1.9019
  frame lastpts = 1.86853
  start schedule refresh timer need to delay for 358

  MAINTHREAD past wait
  pts after syncvideo= 1073673946
  got frame 640, 640
  start video_refresh_timer
  actualpts = 1.93527
  frame lastpts = 1.9019
  start schedule refresh timer need to delay for 416

  MAINTHREAD loop delay before refresh = 416
  pts after syncvideo= 1073708933
  got frame 640, 640
  start video_refresh_timer
  actualpts = 1.96863
  frame lastpts = 1.93527
  start schedule refresh timer need to delay for 474

  MAINTHREAD past wait
  pts after syncvideo= 1073742872
  got frame 640, 640
  MAINTHREAD loop delay before refresh = 474
  start video_refresh_timer
  actualpts = 2.002
  frame lastpts = 1.96863
  start schedule refresh timer need to delay for 518

  MAINTHREAD past wait
  pts after syncvideo= 1073760366
  got frame 640, 640
  start video_refresh_timer
  actualpts = 2.03537
  frame lastpts = 2.002
  start schedule refresh timer need to delay for 575