Recherche avancée

Sur d’autres sites (11178)

• RTP and H.264 (Packetization Mode 1)... Decoding RAW Data... Help understanding the audio and STAP-A packets

  12 février 2014, par Lane

  I am attempting to re-create a video from a Wireshark capture. I have researched extensively and the following links provided me with the most useful information...

  How to convert H.264 UDP packets to playable media stream or file (defragmentation) (and the 2 sub-links)
  H.264 over RTP - Identify SPS and PPS Frames

  ...I understand from these links and RFC (RTP Payload Format for H.264 Video) that...

  • The Wireshark capture shows a client communicating with a server via RTSP/RTP by making the following calls... OPTIONS, DESCRIBE, SETUP, SETUP, then PLAY (both audio and video tracks exist)

  • The RTSP response from PLAY (that contains the Sequence and Picture Parameter Sets) contains the following (some lines excluded)...

  Media Description, name and address (m) : audio 0 RTP/AVP 0
  Media Attribute (a) : rtpmap:0 PCMU/8000/1
  Media Attribute (a) : control:trackID=1
  Media Attribute (a) : x-bufferdelay:0

  Media Description, name and address (m) : video 0 RTP/AVP 98
  Media Attribute (a) : rtpmap:98 H264/90000
  Media Attribute (a) : control:trackID=2
  Media Attribute (a) : fmtp:98 packetization-mode=1 ;profile-level-id=4D0028 ;sprop-parameter-sets=J00AKI2NYCgC3YC1AQEBQAAA+kAAOpg6GAC3IAAzgC7y40MAFuQABnAF3lwWNF3A,KO48gA==

  Media Description, name and address (m) : metadata 0 RTP/AVP 100
  Media Attribute (a) : rtpmap:100 IQ-METADATA/90000
  Media Attribute (a) : control:trackID=3

  ...the packetization-mode=1 means that only NAL Units, STAP-A and FU-A are accepted

  • The streaming RTP packets (video only, DynamicRTP-Type-98) arrive in the following order...

  1x
  [RTP Header]
  0x78 0x00 (Type is 24, meaning STAP-A)
  [Remaining Payload]

   36x
  [RTP Header]
  0x7c (Type is 28, meaning FU-A) then either 0x85 (first) 0x05 (middle) or 0x45 (last)
  [Remaining Payload]

  1x
  [RTP Header]
  0x18 0x00 (Type is 24, meaning STAP-A)
  [Remaining Payload]

  8x
  [RTP Header]
  0x5c (Type is 28, meaning FU-A) then either 0x81 (first) 0x01 (middle) or 0x41 (last)
  [Remaining Payload]

  ...the cycle then repeats... typically there are 29 0x18/0x5c RTP packets for each 0x78/0x7c packet

  • Approximately every 100 packets, there is an audio RTP packet, all have their Marker set to true and their sequence numbers ascend as expected. Sometimes there is an individual RTP audio packet and sometimes there are three, see a sample one here...

  RTP 1042 PT=ITU-T G.711 PCMU, SSRC=0x238E1F29, Seq=31957, Time=1025208762, Mark

  ...also, the type of each audio RTP packet is different (as far as first bytes go... I see 0x4e, 0x55, 0xc5, 0xc1, 0xbc, 0x3c, 0x4d, 0x5f, 0xcc, 0xce, 0xdc, 0x3e, 0xbf, 0x43, 0xc9, and more)

  • From what I gather... to re-create the video, I first need to create a file of the format

  0x000001 [SPS Payload]
  0x000001 [PPS Payload]
  0x000001 [Complete H.264 Frame (NAL Byte, followed by all fragmented RTP payloads without the first 2 bytes)
  0x000001 [Next Frame]
  Etc...

  I made some progress where I can run "ffmpeg -i file" without it saying a bad input format or unable to find codec. But currently it complains something about MP3. My questions are as follows...

  1. Should I be using the SPS and PPS payload returned by the response to the DESCRIBE RTSP call or use the data sent in the first STAP-A RTP packets (0x78 and 0x18) ?

  2. How does the file format change to incorporate the audio track ?

  3. Why is the audio track payload headers all over the place and how can I make sense / utilize them ?

  4. Is my understanding of anything incorrect ?

  Any help is GREATLY appreciated, thanks !

• Use FFMpeg libraries to read an audio file while it is being generated

  12 mai 2019, par EyAl

  I got an audio engine, which generates aac files. I want to mux this audio with some video. I’m using ffmpeg libraries to do just that - meaning, after the audio file is ready, I read it and mux it.
  Now - for performance reasons, I don’t want to wait until the audio engine completes the audio generation, I want the muxer to start reading the audio while it is being generated.
  Can I achieve that using the FFMpeg libraries ?
  Which approach should I take ?
  Couldn’t find any examples doing that

• perturbation of an audio/video decoding execution trace with gstreamer

  9 mars 2013, par KKc

  I am a new comer in gstreamer community and i have a pipeline to decode and trace a .mp4 file.

  gst-launch --gst debug=filesrc:7,queue:7,audioconvert:7,audioresample:7,qtdemux:7,faad:7,ffmpeg:7,audioresample:7,audioconvert:7,autoaudiosink:7,autovideosink:7,  filesrc location=...! qtdemux name=demuxer demuxer. ! queue ! faad ! audioconvert ! audioresample ! autoaudiosink demuxer. ! queue ! ffdec_h264 ! ffmpegcolorspace ! autovideosink > file1

  I inserted the "identity" component to disturb the decoding, and effectively, i saw images which became very slow and sound disappeared.
  I used this command :

  gst-launch --gst debug=filesrc:7,queue:7,audioconvert:7,audioresample:7,qtdemux:7,faad:7,ffmpeg:7,audioresample:7,audioconvert:7,autoaudiosink:7,autovideosink:7,  filesrc location=...! qtdemux name=demuxer demuxer. ! queue ! faad ! audioconvert ! audioresample ! autoaudiosink demuxer. ! queue ! identity sleep-time=1000000 ! ffdec_h264 ! ffmpegcolorspace ! autovideosink > file2

  The first time i did this execution, two new functions appeared in file2,

  (i) gst_ffmpegdec_chain...'skipping...',

  (ii) gst_ffmpegdec_video_frame...'Dropping..'

  I assumed that the meaning is that some data were dropped or something else

  However, since many days, i use the same pipelines, with the same video to decode ; i obtain the same bad visualization, but any new function in file 2 :(
  the only difference is the number of occurrence of the functions below :

  *gst_ffmpegdec_update_qos :...'update* 558 times in one case

  *gst_ffmpegdec_update_qos :...'update* 4 times in the other case

  I don't know why i am unable to produce again a disturbed trace with 'skipping..' and 'dropping..'

  My questions are :

  1- have you any idea about the meaning of the above functions ?

  2- Do you know any other component useful to disturb a A/V decoding processing ?

  Thank you for any reply