Recherche avancée

Sur d’autres sites (3722)

• Problem decoding h264 over RTP TCP stream

  4 mai 2022, par Kamil.S

  I'm trying to receive RTP stream encoding h264 over TCP from my intercom Hikvision DS-KH8350-WTE1. By reverse engineering I was able to replicate how Hikvision original software Hik-Connect on iPhone and iVMS-4200 on MacOS connects and negotaties streaming. Now I'm getting the very same stream as original apps - verified through Wireshark. Now I need to "make sense" of the stream. I know it's RTP because I inspected how iVMS-4200 uses it using /usr/bin/sample on MacOS. Which yields :

  


    ! :               2 CStreamConvert::InputData(void*, int)  (in libOpenNetStream.dylib) + 52  [0x11ff7c7a6]
+     ! :                 2 SYSTRANS_InputData  (in libSystemTransform.dylib) + 153  [0x114f917f2]
+     ! :                   1 CRTPDemux::ProcessH264(unsigned char*, unsigned int, unsigned int, unsigned int)  (in libSystemTransform.dylib) + 302  [0x114fa2c04]
+     ! :                   | 1 CRTPDemux::AddAVCStartCode()  (in libSystemTransform.dylib) + 47  [0x114fa40f1]
+     ! :                   1 CRTPDemux::ProcessH264(unsigned char*, unsigned int, unsigned int, unsigned int)  (in libSystemTransform.dylib) + 476  [0x114fa2cb2]
+     ! :                     1 CRTPDemux::ProcessVideoFrame(unsigned char*, unsigned int, unsigned int)  (in libSystemTransform.dylib) + 1339  [0x114fa29b3]
+     ! :                       1 CMPEG2PSPack::InputData(unsigned char*, unsigned int, FRAME_INFO*)  (in libSystemTransform.dylib) + 228  [0x114f961d6]
+     ! :                         1 CMPEG2PSPack::PackH264Frame(unsigned char*, unsigned int, FRAME_INFO*)  (in libSystemTransform.dylib) + 238  [0x114f972fe]
+     ! :                           1 CMPEG2PSPack::FindAVCStartCode(unsigned char*, unsigned int)  (in libSystemTransform.dylib) + 23  [0x114f97561]`


  


  I can catch that with lldb and see the arriving packet data making sense as the format I'm describing.

  


  The packet signatures look following :

  


  


  0x24 0x02 0x05 0x85 0x80 0x60 0x01 0x57 0x00 0x00 0x00 0x02 0x00 0x00 0x27 0xde
0x0d 0x80 0x60 0x37 0x94 0x71 0xe3 0x97 0x10 0x77 0x20 0x2c 0x51 | 0x7c
0x85 0xb8 0x00 00 00 00 01 65 0xb8 0x0 0x0 0xa 0x35 ...

  


  0x24 0x02 0x05 0x85 0x80 0x60 0x01 0x58 0x00 0x00 0x00 0x02 0x00 0x00 0x27 0xde
0xd 0x80 0x60 0x37 0x95 0x71 0xe3 0x97 0x10 0x77 0x20 0x2c 0x51 | 0x7c 0x05 0x15 0xac ...

  


  0x24 0x02 0x5 0x85 0x80 0x60 0x01 0x59 0x00 0x0 0x00 0x02 0x00 00x0 0x27 0xde
0xd 0x80 0x60 0x37 0x96 0x71 0xe3 0x97 0x10 0x77 0x20 0x2c 0x51 | 0x7c 0x05 0x5d 0x00 ...

  


  


  By the means of reverse engineering the original software I was able to figure out that 0x7c85 indicates a key frame. 0x7c85 bytes in the genuine software processing do get replaced by h264 00 00 00 01 65 Key frame NALU . That's h264 appendix-B format.
The 0x7c05 packets always follow and are the remaining payload of the key frame. No NALU are added during their handling (the 0x7c05 is stripped away and rest of the bytes is copied). None of the bytes preceding 0x7cXX make it to a mp4 recording (that makes sense as it's the RTP protocol , albeit I'm not sure if it's entirely RTP standard or there's something custom from Hikvision).

  


  If you pay close attention in the Header there are 2 separate bytes indicating order which always match, so I'm sure no packet loss is occurring.

  


  I also observed nonkey frames arriving as 0x7c81 and converted to 00 00 00 01 61 NALU but I want to focus solely on the single key frame for now. Mostly because if I record a movie with the original software it will always begin with 00 00 00 01 65 Key frame (that obviously makes sense).

  


  To get a working mp4 I decided to copy paste a mp4 header of a genuine iVMS-4200 recording (in this sense that's every byte preceding 1st frame NALU 00 00 00 01 65 in the mp4 file). I know that the resolution will match the actual camera footage. With the strategy of waiting for a keyframe , replacing 0x7c85 with 00 00 00 01 65 NALU and appending the remaining bytes, or only appending bytes in the 0x7c05 case I do seem to get something that eventually could work.
When I attempt to ffplay the custom crafted mp4 result I do get something (with a little stretch of imagination that's actually the camera fisheye image forming), but clearly there is a problem.

  


  

  


  It seems around 3-4th 0x7c05 packet (as the failing packet differs on every run), when I copy bytes eventually the h264 stream is incorrect. Just by eye-inspecting the bytes I don't see anything unusual.

  


  This is the failing packet around offset 750 decimal, (I know it's around this place because I keep stripping bytes away to see if there's still same amount frame arriving before it breaks).

More over I did dump those bytes from original software using lldb taking out my own python implementation out of equation. And I run into very same problem with the original packets.

  


  The mp4 header I use should work (since it does for original recordings even if I manipulate number of frames and leave just the first keyframe).
Correct me if I'm wrong but the phase of converting this to MPEG2-PS (which iVMS-4200 does and I don't) should be entirely optional and should not be related to my problem.

  


  Update :
I went the path of setting up recording and only then dumping the original iVMS-4200 packets. I edited the recorded movie to only contain keyframe of interest and it works. I found differences but I cannot explain where they are there yet :

Somehow 00 00 01 E0  13 FA 88 00 02 FF FF is inserted in the genuine recording (that's 4th packet), but I have no idea how this byte string was generated and what is its purpose.
When I fixed the first difference the next one is :

The pattern is striking. But what00 00 01 E0  13 FA 88 00 02 FF FF actually is ? And why is it inserted after 18 03 25 10 & 2F F4 80 FC
  
The 00 00 01 E0 signature would suggest those are Packetized Elementary Stream (PES) headers

  


• ffmpeg's concat demuxer parameters to limit segments at outpoint timestamps [closed]

  17 octobre 2024, par code

  In order to concatenate video files of matching fps, codec, resolution, time_base (in fact some are segments from same video), ffmpeg concat demuxer (to prevent re-encoding) approach has been used.
But ffmpeg did not stop concatenating segments at outpoint time specified in input demuxer text file !

  


  (Note : From Python code programmatically via sub-process approach, ffmpeg has been invoked )

  


  input.txt file for concat demuxer contained :

  


  ffconcat version 1.0
file 'numbered.mp4'
inpoint  2.083333
outpoint 2.166667



  


  (the time stamps provided above are of 50th (frame pts 49 in video, this is a keyframe) to 52nd (frame pts 51 in video, non-key frame), total 3 consecutive frames, inpoints given in this query are of a key-frame

  


  (video file referred in it i.e 'numbered.mp4' has each frame displaying its frame number ; it has h264 high coded 480p resolution, 10-seconds long with 240 frames and 24fps , <1MB in size shared at : https://filetransfer.io/data-package/F5caZ0xT#link )

  &#xA;

  ffmpeg command invoked programmatically with parameters :

  &#xA;

  &#xA;

  ffmpeg -f concat -safe 0 -fflags +genpts -i -c copy -video_track_timescale 24

  &#xA;

  &#xA;

  Output snippet contained :

  &#xA;

  &#xA;

  frame= 5 fps=0.0 q=-1.0 Lsize= 28KiB time=00:00:00.12 bitrate=1857.8kbits/s speed= 56x

  &#xA;

  &#xA;

  Problem 1 : It Shows there are 5 frames ! ffmpeg has concatenated 2 more frames beyond outpoint timestamp !

  &#xA;

  Query 1) : What are right parameter values to be given to ffmpeg concat demuxer method(or in concat demuxer input file) to make ffmpeg concatenate segments accurately till frames matching outpoint timestamp without overshooting or concatenating frames beyond outpoint timestamps ?

  &#xA;

  Problem2 : When another segment from same input file is referenced in concat demuxer input file, frame pts and timestamps were messed up in resultant output !

  &#xA;

  concat demuxer input file content(updated) :

  &#xA;

  ffconcat version 1.0
file 'numbered.mp4'
inpoint  0
outpoint 0.125000
file 'numbered.mp4'
inpoint  2.083333
outpoint 2.166667



  &#xA;

  command invoked was :

  &#xA;

  ffmpeg -f **concat **-safe 0 -fflags +genpts   -i \ -c copy 
  -video_track_timescale 24  \


  &#xA;

  ffprobe output (on above output file, edited to reduce size) :

  &#xA;

  
key_frame=1, pts=0
    pts_time=0.000000
key_frame=0, pts=1
    pts_time=0.041667
key_frame=0, pts=2
    pts_time=0.083333
key_frame=0, pts=3
    pts_time=0.125000
key_frame=0, pts=4
    pts_time=0.166667
key_frame=1, pts=3
    pts_time=0.125000
key_frame=0, pts=6
    pts_time=0.250000
key_frame=0, pts=5
    pts_time=0.208333
key_frame=0, pts=7
    pts_time=0.291667
key_frame=0, pts=7
    pts_time=0.291667


  &#xA;

  Confirms both pts and pts_time values are messed up (though segments referred in demuxer input file were several frames away with no overlapping ).

  &#xA;

  Query 2) What are accurate parameters to be given to concatenate segments represented by input demuxer file without causing this pts or pts_time issues ?

  &#xA;

  (In this test, all segments referred by demuxer have same parameters and are different segments of same video file itself ! so mismatch in codec parameters may not be the cause )

  &#xA;

  problem 3 : while input video had bitrate of 412654 (412.654kbps), concat demuxer resulted in output file with bitrate 1318777 (1.318 Mbps) over 3x the input bitrate.

  &#xA;

  Query 3) : What are accurate parameters to be given to retain all (almost) codec parameters same as input video and only perform concatenation without modifying time_base or framerate ?

  &#xA;

  Note : when -video_track_timescale 24 parameter is not provided as input parameter to concat demuxer, time_base in resultant output was different value (1/1000 ) instead of input files' time_base 1/24 !

  &#xA;

  ( when the Pts times are messed up, Non-monotonic DTS .. errors were observed in output : [vost#0:0/copy @ 000002c1b9b41140] Non-monotonic DTS ; previous : 2, current : 1 ; changing to 3. This may result in incorrect timestamps in the output file..)

  &#xA;

  To clarify, reason to use concat demuxer is to prevent re-encoding video, final usage would be to concatenate some segments of input video file with a few more video files all containing same codec parameters like fps,resolution,time_base etc.

  &#xA;

  Query 4) : Is it accurate at frame level to take pts_time values from ffprobe output and use these in ffmpeg concat demuxer input file for inpoint/outpoint values ?

  &#xA;

  (as ffprobe pts_time values possibly might be aligned with ffmpeg expectations, thought of taking pts_time values from ffprobe command output instead of venturing into computing frame start time)

  &#xA;

  the small (<1MB) input video file used in this test has been shared at : https://filetransfer.io/data-package/F5caZ0xT#link

  &#xA;

  input video file's ffprobe output has been pasted(trimmed to save space) :

  &#xA;

  "codec_name": "h264",
"codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
"profile": "High",
"codec_type": "video",
"codec_tag_string": "avc1",
"codec_tag": "0x31637661",
"width": 640,
"height": 480,
"coded_width": 640,
"coded_height": 480,
"closed_captions": 0,
"film_grain": 0,
"has_b_frames": 2,
"sample_aspect_ratio": "1:1",
"display_aspect_ratio": "4:3",
"pix_fmt": "yuv420p",
"level": 41,
"color_range": "tv",
"color_space": "smpte170m",
"chroma_location": "left",
"field_order": "progressive",
"refs": 1,
"is_avc": "true",
"nal_length_size": "4",
"id": "0x1",
"r_frame_rate": "24/1",
"avg_frame_rate": "24/1",
"time_base": "1/24",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 240,
"duration": "10.000000",
"bit_rate": "409628",
"bits_per_raw_sample": "8",
"nb_frames": "240",
"extradata_size": 49,


  &#xA;

  Searched quite a lot online for a solution but searches didnot result in finding a fix for this concat demuxer situation.&#xA;To reiterate again, ffmpeg concat demuxer approach is chosen to prevent re-encoding !&#xA;(End usage would be on 20000+ frames in video files, so concat demuxer approach would be significantly faster than re-encoding alternatives)

  &#xA;

  I seek helpful answers to technical queries presented above,&#xA;Thanks All

  &#xA;

  (while some workarounds like converting to raw h264 and then applying time scale again to each file was suggested in other discussions, in current scenario the input video file is 1 and segments are extracted from it so it appears accurate parameters to ffmpeg concat demuxer method are to be found )

  &#xA;

• Query on ffmpeg's concat demuxer parameters to end segments at outpoint timestamps and retain most properties same as input (fps, time_base etc)

  17 octobre 2024, par code

  In order to concatenate video files of matching fps, codec, resolution, time_base (in fact some are segments from same video), ffmpeg concat demuxer (to prevent re-encoding) approach has been used.&#xA;But ffmpeg did not stop concatenating segments at outpoint time specified in input demuxer text file !

  &#xA;

  (Note : From Python code programmatically via sub-process approach, ffmpeg has been invoked )

  &#xA;

  input.txt file for concat demuxer contained :

  &#xA;

  ffconcat version 1.0
file 'numbered.mp4'
inpoint  2.083333
outpoint 2.166667



  &#xA;

  (the time stamps provided above are of 50th (frame pts 49 in video, this is a keyframe) to 52nd (frame pts 51 in video, non-key frame), total 3 consecutive frames, inpoints given in this query are of a key-frame

  &#xA;

  (video file referred in it i.e 'numbered.mp4' has each frame displaying its frame number ; it has h264 high coded 480p resolution, 10-seconds long with 240 frames and 24fps , <1MB in size shared at : https://filetransfer.io/data-package/F5caZ0xT#link )

  &#xA;

  ffmpeg command invoked programmatically with parameters :

  &#xA;

  &#xA;

  ffmpeg -f concat -safe 0 -fflags +genpts -i -c copy -video_track_timescale 24

  &#xA;

  &#xA;

  Output snippet contained :

  &#xA;

  &#xA;

  frame= 5 fps=0.0 q=-1.0 Lsize= 28KiB time=00:00:00.12 bitrate=1857.8kbits/s speed= 56x

  &#xA;

  &#xA;

  Problem 1 : It Shows there are 5 frames ! ffmpeg has concatenated 2 more frames beyond outpoint timestamp !

  &#xA;

  Query 1) : What are right parameter values to be given to ffmpeg concat demuxer method(or in concat demuxer input file) to make ffmpeg concatenate segments accurately till frames matching outpoint timestamp without overshooting or concatenating frames beyond outpoint timestamps ?

  &#xA;

  Problem2 : When another segment from same input file is referenced in concat demuxer input file, frame pts and timestamps were messed up in resultant output !

  &#xA;

  concat demuxer input file content(updated) :

  &#xA;

  ffconcat version 1.0
file 'numbered.mp4'
inpoint  0
outpoint 0.125000
file 'numbered.mp4'
inpoint  2.083333
outpoint 2.166667



  &#xA;

  command invoked was :

  &#xA;

  ffmpeg -f **concat **-safe 0 -fflags +genpts   -i \ -c copy 
  -video_track_timescale 24  \


  &#xA;

  ffprobe output (on above output file, edited to reduce size) :

  &#xA;

  
key_frame=1, pts=0
    pts_time=0.000000
key_frame=0, pts=1
    pts_time=0.041667
key_frame=0, pts=2
    pts_time=0.083333
key_frame=0, pts=3
    pts_time=0.125000
key_frame=0, pts=4
    pts_time=0.166667
key_frame=1, pts=3
    pts_time=0.125000
key_frame=0, pts=6
    pts_time=0.250000
key_frame=0, pts=5
    pts_time=0.208333
key_frame=0, pts=7
    pts_time=0.291667
key_frame=0, pts=7
    pts_time=0.291667


  &#xA;

  Confirms both pts and pts_time values are messed up (though segments referred in demuxer input file were several frames away with no overlapping ).

  &#xA;

  Query 2) What are accurate parameters to be given to concatenate segments represented by input demuxer file without causing this pts or pts_time issues ?

  &#xA;

  (In this test, all segments referred by demuxer have same parameters and are different segments of same video file itself ! so mismatch in codec parameters may not be the cause )

  &#xA;

  problem 3 : while input video had bitrate of 412654 (412.654kbps), concat demuxer resulted in output file with bitrate 1318777 (1.318 Mbps) over 3x the input bitrate.

  &#xA;

  Query 3) : What are accurate parameters to be given to retain all (almost) codec parameters same as input video and only perform concatenation without modifying time_base or framerate ?

  &#xA;

  Note : when -video_track_timescale 24 parameter is not provided as input parameter to concat demuxer, time_base in resultant output was different value (1/1000 ) instead of input files' time_base 1/24 !

  &#xA;

  ( when the Pts times are messed up, Non-monotonic DTS .. errors were observed in output : [vost#0:0/copy @ 000002c1b9b41140] Non-monotonic DTS ; previous : 2, current : 1 ; changing to 3. This may result in incorrect timestamps in the output file..)

  &#xA;

  To clarify, reason to use concat demuxer is to prevent re-encoding video, final usage would be to concatenate some segments of input video file with a few more video files all containing same codec parameters like fps,resolution,time_base etc.

  &#xA;

  Query 4) : Is it accurate at frame level to take pts_time values from ffprobe output and use these in ffmpeg concat demuxer input file for inpoint/outpoint values ?

  &#xA;

  (as ffprobe pts_time values possibly might be aligned with ffmpeg expectations, thought of taking pts_time values from ffprobe command output instead of venturing into computing frame start time)

  &#xA;

  the small (<1MB) input video file used in this test has been shared at : https://filetransfer.io/data-package/F5caZ0xT#link

  &#xA;

  input video file's ffprobe output has been pasted(trimmed to save space) :

  &#xA;

  "codec_name": "h264",
"codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
"profile": "High",
"codec_type": "video",
"codec_tag_string": "avc1",
"codec_tag": "0x31637661",
"width": 640,
"height": 480,
"coded_width": 640,
"coded_height": 480,
"closed_captions": 0,
"film_grain": 0,
"has_b_frames": 2,
"sample_aspect_ratio": "1:1",
"display_aspect_ratio": "4:3",
"pix_fmt": "yuv420p",
"level": 41,
"color_range": "tv",
"color_space": "smpte170m",
"chroma_location": "left",
"field_order": "progressive",
"refs": 1,
"is_avc": "true",
"nal_length_size": "4",
"id": "0x1",
"r_frame_rate": "24/1",
"avg_frame_rate": "24/1",
"time_base": "1/24",
"start_pts": 0,
"start_time": "0.000000",
"duration_ts": 240,
"duration": "10.000000",
"bit_rate": "409628",
"bits_per_raw_sample": "8",
"nb_frames": "240",
"extradata_size": 49,


  &#xA;

  Searched quite a lot online for a solution but searches didnot result in finding a fix for this concat demuxer situation&#xA;I seek helpful answers to the queries presented above, Thanks All

  &#xA;

  (while some workarounds like converting to raw h264 and then applying time scale again to each segment is suggested in other discussions, in current scenario, the input video file is just 1 so it appears accurate parameters to ffmpeg concat demuxer method are needed and be helpful to others facing similar issue)

  &#xA;