Recherche avancée

Sur d’autres sites (9407)

• Muxing raw h264 + aac into mp4 file with av_interleaved_write_frame() returning 0 but the video is not playable

  3 avril 2020, par Jaw109

  I have a program [1] that muxing audio and video into mp4 file(in idividual worker thread, retrieving audio/video frame from a streaming daemon). The audio is perfectly played in VLC, but the video is not playable, VLC debug logs show the start-code of video frame is not found.

  



  I have another demuxing program [2] to retrieve all the frame to see what had happened. I found the video frame is modified

  



  00000001 674D0029... was modified into 00000019 674D0029... (framesize is 29)
00000001 68EE3C80... was modified into 00000004 68EE3C80... (framesize is 8)
00000001 65888010... was modified into 0002D56F 65888010... (framesize is 185715)
00000001 619A0101... was modified into 00003E1E 619A0101... (framesize is 15906)
00000001 619A0202... was modified into 00003E3C 619A0202... (framesize is 15936)
00000001 619A0303... was modified into 00003E1E 619A0303... (framesize is 15581)


  



  It seems like the h264 start-code was replaced with something like... frame-size. but why ? Is there anything I did wrongly ? (Any idea ? something flags ? AVPacket initialization ? AVPacket's data copy wrongly ?)

  



  [1] muxing program

  



  int go_on = 1;
std::mutex g_mutex;
AVStream* g_AudioStream = NULL;
AVStream* g_VideoStream = NULL;

int polling_ringbuffer(int stream_type);

int main(int argc, char** argv)
{

  AVFormatContext* pFmtCntx = avformat_alloc_context();
  avio_open(&pFmtCntx->pb, argv[1], AVIO_FLAG_WRITE);
  pFmtCntx->oformat = av_guess_format(NULL, argv[1], NULL);
  g_AudioStream = avformat_new_stream( pFmtCntx, NULL );
  g_VideoStream = avformat_new_stream( pFmtCntx, NULL );
  initAudioStream(g_AudioStream->codecpar);
  initVideoStream(g_VideoStream->codecpar);
  avformat_write_header(pFmtCntx, NULL);

  std::thread audio(polling_ringbuffer, AUDIO_RINGBUFFER);
  std::thread video(polling_ringbuffer, VIDEO_RINGBUFFER);

  audio.join();
  video.join();

  av_write_trailer(pFmtCntx);
  if ( pFmtCntx->oformat && !( pFmtCntx->oformat->flags & AVFMT_NOFILE ) && pFmtCntx->pb )
    avio_close( pFmtCntx->pb );
  avformat_free_context( g_FmtCntx );

  return 0;
}

int polling_ringbuffer(int stream_type)
{
  uint8_t* data = new uint8_t[1024*1024];
  int64_t timestamp = 0;
  int data_len = 0;
  while(go_on)
  {
    const std::lock_guard lock(g_mutex);
    data_len = ReadRingbuffer(stream_type, data, 1024*1024, &timestamp);

    AVPacket pkt = {0};
    av_init_packet(&pkt);
    pkt.data = data;
    pkt.size = data_len;

    static AVRational r = {1,1000};
    switch(stream_type)
    {
      case STREAMTYPE_AUDIO:
        pkt.stream_index = g_AudioStream->index;
        pkt.flags = 0;
        pkt.pts = av_rescale_q(timestamp, r, g_AudioStream->time_base);
        break;
      case STREAMTYPE_VIDEO:
        pkt.stream_index = g_VIDEOStream->index;
        pkt.flags = isKeyFrame(data, data_len)?AV_PKT_FLAG_KEY:0;
        pkt.pts = av_rescale_q(timestamp, r, g_VideoStream->time_base);
        break;
    }
    static int64_t lastPTS = 0;
    pkt.dts = pkt.pts;
    pkt.duration = (lastPTS==0)? 0 : (pkt.pts-lastPTS);
    lastPTS = pkt.pts;

    int ret = av_interleaved_write_frame(g_FmtCntx, &pkt);
    if(0!=ret)
      printf("[%s:%d] av_interleaved_write_frame():%d\n", __FILE__, __LINE__, ret);
  }

  return 0;
}


  



  [2] demuxing program

  



  int main(int argc, char** argv)
{
  AVFormatContext* pFormatCtx = avformat_alloc_context();
  AVPacket pkt;
  av_init_packet(&pkt);
  avformat_open_input(&pFormatCtx, argv[1], NULL, NULL);
  for(;;)
  {
    if (av_read_frame(pFormatCtx, &pkt) >= 0)
    {
      printf("[%d] %s (len:%d)\n", pkt.stream_index, BinToHex(pkt.data, MIN(64, pkt.size)), pkt.size );
    }
    else
      break;
  }

  avformat_close_input(&pFormatCtx);
  return 0;
}


  



  [3] Here are my environment

  



  Linux MY-RASP-4 4.14.98 #1 SMP Mon Jun 24 12:34:42 UTC 2019 armv7l GNU/Linux
ffmpeg version 4.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 8.2.0 (GCC)

libavutil      56. 22.100 / 56. 22.100
libavcodec     58. 35.100 / 58. 35.100
libavformat    58. 20.100 / 58. 20.100
libavdevice    58.  5.100 / 58.  5.100
libavfilter     7. 40.101 /  7. 40.101
libswscale      5.  3.100 /  5.  3.100
libswresample   3.  3.100 /  3.  3.100
libpostproc    55.  3.100 / 55.  3.100


  


• Running pulseaudio inside docker container to record system audio

  20 mars 2023, par XXLuigiMario

  I'm trying to set up a Docker container with Selenium that takes a recording of the browser with system audio using ffmpeg. I've got video working using Xvfb. Unfortunately, on the audio side, it seems to be more tricky.

  


  I thought I would set up a virtual pulseaudio sink inside the container, which would allow me to record its monitor :

  


  pacmd load-module module-null-sink sink_name=loopback
pacmd set-default-sink loopback
ffmpeg -f pulse -i loopback.monitor test.wav


  


  This works on my host operating system, but when trying to start the pulseaudio daemon in a container, it fails with the following message :

  


  E: [pulseaudio] module-console-kit.c: Unable to contact D-Bus system bus: org.freedesktop.DBus.Error.FileNotFound: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory


  


  This would seem to be related to a freedesktop service called dbus. I've tried installing it and starting its daemon, but I couldn't seem to get it to work properly.
I couldn't find much information on how to proceed from here. What am I missing for pulseaudio ? Perhaps there's an easier way to record the system audio inside a container ?

  


  My goal is not to record it from the host operating system, but to play the audio inside the browser and record it all inside the same container.

  


• how to make cv2.videoCapture.read() faster ?

  9 novembre 2022, par Yu-Long Tsai

  My question :

  



  I was working on my computer vision project. I use opencv(4.1.2) and python to implement it.

  



  I need a faster way to pass the reading frame into image processing on my Computer(Ubuntu 18.04 8 cores i7 3.00GHz Memory 32GB). the cv2.VideoCapture.read() read frame (frame size : 720x1280) will take about 120 140ms. which is too slow. my processing module take about 40ms per run. And we desire 25 30 FPS.

  



  here is my demo code so far :

  



  import cv2
from collections import deque
from time import sleep, time
import threading


class camCapture:
    def __init__(self, camID, buffer_size):
        self.Frame = deque(maxlen=buffer_size)
        self.status = False
        self.isstop = False
        self.capture = cv2.VideoCapture(camID)


    def start(self):
        print('camera started!')
        t1 = threading.Thread(target=self.queryframe, daemon=True, args=())
        t1.start()

    def stop(self):
        self.isstop = True
        print('camera stopped!')

    def getframe(self):
        print('current buffers : ', len(self.Frame))
        return self.Frame.popleft()

    def queryframe(self):
        while (not self.isstop):
            start = time()
            self.status, tmp = self.capture.read()
            print('read frame processed : ', (time() - start) *1000, 'ms')
            self.Frame.append(tmp)

        self.capture.release()

cam = camCapture(camID=0, buffer_size=50)
W, H = 1280, 720
cam.capture.set(cv2.CAP_PROP_FRAME_WIDTH, W)
cam.capture.set(cv2.CAP_PROP_FRAME_HEIGHT, H)


# start the reading frame thread
cam.start()

# filling frames
sleep(5)

while True:
  frame = cam.getframe() # numpy array shape (720, 1280, 3)

  cv2.imshow('video',frame)
  sleep( 40 / 1000) # mimic the processing time

  if cv2.waitKey(1) == 27:
        cv2.destroyAllWindows()
        cam.stop()
        break



  



  What I tried :

  



  


  1. multiThread - one thread just reading the frame, the other do the image processing things.
It's NOT what I want. because I could set a buffer deque saving 50 frames for example. but the frame-reading thread worked with the speed frame/130ms. my image processing thread worked with the speed frame/40ms. then the deque just running out. so I've been tried the solution. but not what I need.

  


  2. this topic is the discussion I found out which is most closest to my question. but unfortunately, I tried the accepted solutions (both of two below the discussion).

  


  



  One of the solution (6 six thumbs up) point out that he could reading and saving 100 frames at 1 sec intervals on his mac. why my machine cannot handle the frame reading work ? Do I missing something ? my installation used conda and pip conda install -c conda-forge opencv, pip install opencv-python(yes, I tried both.)

  



  The other of the solution(1 thumb up) using ffmpeg solution. but it seem's work with video file but not camera device ?

  



  

  3. adjust c2.waitKey() : 
the parameter just controls the frequency when video display. not a solution.

  


  



  Then, I know I just need some keywords to follow.

  



  code above is my demo code so far, I want some method or guide to make me videoCapture.read() faster. maybe a way to use multithread inside videoCapture object or other camera reading module.

  



  Any suggestions ?