Recherche avancée

Sur d’autres sites (7132)

• Frame-by-frame video decoding and processing on Android

  22 octobre 2016, par Jason M

  I am working on a project to decode a recorded video on Android into yuv420sp frames with API17 for processing. I have implemented it on Win and iOS with ffmpeg and native api for decoding, respectively. Now I want to reuse the same c++ processing code on Android API17, but found difficulty. I saw straight API to do that in API21, but sadly I have to keep that better compatibility.

  At first, I tried to make the best use of APIs of hardware decoding (following MediaCodec get all frames from video and some others) but found that the data in the outputbuffer does not match that described by the format. For example, I have 1920x1080 frames decoded into yuv420sp but found that y channel actually span 1928x1080. Since I had similar headaches working with still image capturing that was never worked out(YUV image taken from takePicture not match defined format), I am not sure if I may ever solve this.

  Later, I tried to go back to ffmpeg but found most building guides either out of date(for ffmpeg 2.x) or a little hard to understand(such as http://writingminds.github.io/ffmpeg-android/, where they do provide nice pre-built binaries but no instructions on using it. Maybe I should download the sources and use the headers there ?).

  Any suggestions ?

• Is there a video processing test set to cover most scenarios ? [closed]

  9 avril 2024, par 砸石头的西西弗

  My application integrates ffmpeg, and there are often some special videos or parameters that I cannot handle correctly, but they cannot be discovered in advance, so I need a test set to help find bugs.I nedd a video processing test set to cover most scenarios.

  


  A test set which has some different kinds type of sample videos.

  


• Processing h264 video/audio stream

  1er mars 2024, par Dinamo

  I have separate audio and video streams from a camera that I'm trying to merge to a single file.

  


  Video processing

  


  I have JVT NAL sequence, H.264 video, main @ L 31 data, I had the frames separated and I appended (concat) the frames to one file.

  


  here is the ffprobe output of the data :

  


  ffprobe version 4.2.7-0ubuntu0.1 Copyright (c) 2007-2022 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, h264, from 'frame_data.h264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1280x720, 25 fps, 25 tbr, 1200k tbn, 50 tbc


  


  the file is being played on vlc, however the video is played like it is on fast forward (every 2 seconds of the video should be about 1 second)

  


  The video should run at 15fps (not sure about that because of the mentioned avi file)

  


  here is ffprobe output of working avi file that contain the video stream :

  


  Input #0, avi, from 't.avi':
  Duration: 00:03:29.00, start: 0.000000, bitrate: 1116 kb/s
    Stream #0:0: Video: h264 (Main) (H264 / 0x34363248), yuvj420p(pc, bt709, progressive), 1280x720, 849 kb/s, 30 fps, 15 tbr, 30 tbn, 60 tbc


  


  I tried this command ffmpeg -i frame_data.h264 -filter:v fps=15 test.h264 but the video is still playing like its on fast forward

  


  vlc data before changing fps to 15

  


  


  


  vlc data after changing fps to 15

  


  


  


  frames info (raw)

  


  [FRAME]
media_type=video
stream_index=0
key_frame=1
pkt_pts=N/A
pkt_pts_time=N/A
-> pkt_dts=N/A
-> pkt_dts_time=N/A
best_effort_timestamp=N/A
best_effort_timestamp_time=N/A
-> pkt_duration=48000
-> pkt_duration_time=0.040000
pkt_pos=1476573
pkt_size=57677
width=1280
height=720
pix_fmt=yuvj420p
sample_aspect_ratio=N/A
pict_type=I
coded_picture_number=189
display_picture_number=0
interlaced_frame=0
top_field_first=0
repeat_pict=0
color_range=pc
color_space=bt709
color_primaries=bt709
color_transfer=bt709
chroma_location=left
[/FRAME]
[FRAME]
media_type=video
stream_index=0
key_frame=0
pkt_pts=N/A
pkt_pts_time=N/A
-> pkt_dts=N/A
-> pkt_dts_time=N/A
best_effort_timestamp=N/A
best_effort_timestamp_time=N/A
-> pkt_duration=48000
-> pkt_duration_time=0.040000
pkt_pos=1534250
pkt_size=3928
width=1280
height=720
pix_fmt=yuvj420p
sample_aspect_ratio=N/A
pict_type=P
coded_picture_number=190
display_picture_number=0
interlaced_frame=0
top_field_first=0
repeat_pict=0
color_range=pc
color_space=bt709
color_primaries=bt709
color_transfer=bt709
chroma_location=left
[/FRAME]


  


  frame info from working (processed) avi

  


  [FRAME]
media_type=video
stream_index=0
key_frame=0
pkt_pts=N/A
pkt_pts_time=N/A
-> pkt_dts=500
-> pkt_dts_time=16.666667
best_effort_timestamp=500
best_effort_timestamp_time=16.666667
-> pkt_duration=1
-> pkt_duration_time=0.033333
pkt_pos=1772182
pkt_size=3070
width=1280
height=720
pix_fmt=yuvj420p
sample_aspect_ratio=N/A
pict_type=P
coded_picture_number=191
display_picture_number=0
interlaced_frame=0
top_field_first=0
repeat_pict=0
color_range=pc
color_space=bt709
color_primaries=bt709
color_transfer=bt709
chroma_location=left
[/FRAME]


  


  pkt_pts and pkt_dts are missing in the raw data, maybe that is the reason the video plays on fast forward ? in the working (processed) avi pkt_dts, pkt_dts_time, pkt_duration_time are not missing.
pkt_duration_time is always 0.033333
pkt_dts always jumps by 2 (there is no odd pkt_dts)

  


  Audio processing

  


  I have raw pcm audio of the video :

  


  here is an output of ffplay -f s16le -ar 16k -ac 1 frame_data2.h264 on the raw data

  


  

  


  The raw audio does not playing well using this command.

  


  I guessed the parameters (incorrectly ?) from working avi file that contains the pcm data :

  


  Stream #0:1: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 16000 Hz, 1 channels, s16, 256 kb/s


  


  here is frame info from working (processed) avi file

  


  [FRAME]
media_type=audio
stream_index=1
key_frame=1
pkt_pts=4320
pkt_pts_time=0.270000
pkt_dts=4320
pkt_dts_time=0.270000
best_effort_timestamp=4320
best_effort_timestamp_time=0.270000
pkt_duration=480
pkt_duration_time=0.030000
pkt_pos=11670
pkt_size=960
sample_fmt=s16
nb_samples=480
channels=1
channel_layout=unknown
[/FRAME]


  


  I'm trying to understand how should I process the video and the audio so that they will play well (merge them myself with the right av processing to avi/mp4 file)

  


  I also find this extra data that was included for each frame :
I suspect the first column is a sequence number maybe related to dts somehow, second column is a timestamp (milliseconds ?) (maybe you can help me parse it)

  


  600 0xd96533 (audio)
601 0xd9e1dd (audio)
602 0xda4f52 (audio)
603 0xda5a63 (video)
604 0xdacc4b (audio)
605 0xdb39a3 (audio)
606 0xdb5ee9 (video)
607 0xdbb6d8 (audio)
608 0xdc23fe (audio)
609 0xdcb255 (audio)
610 0xdd0e69 (audio)
611 0xdd8b96 (audio)
612 0xdd67d0 (video)
613 0xddf8bd (audio)
614 0xde75fe (audio)
615 0xde6c28 (video)
616 0xdee328 (audio)
617 0xdf6012 (audio)
618 0xdfcd9d (audio)
619 0xe04a7d (audio)
620 0xe07523 (video)
621 0xe0b80c (audio)
622 0xe134db (audio)
623 0xe1798b (video)
624 0xe1a251 (audio)
625 0xe21f5c (audio)
626 0xe28ce1 (audio)
627 0xe27df7 (video)
628 0xe309a7 (audio)
629 0xe376fe (audio)
630 0xe3826e (video)
631 0xe3f3fd (audio)
632 0xe46154 (audio)
633 0xe486e9 (video)
634 0xe4de79 (audio)
635 0xe54bbe (audio)
636 0xe58b79 (video)
637 0xe5c8cb (audio)
638 0xe6361e (audio)
639 0xe6b35b (audio)
640 0xe68fb7 (video)
641 0xe72093 (audio)
642 0xe79db1 (audio)
643 0xe7942f (video)
644 0xe80b09 (audio)
645 0xe887ed (audio)
646 0xe8f546 (audio)
647 0xe97256 (audio)
648 0xe99d2c (video)
649 0xe9dfbd (audio)
650 0xea5ca8 (audio)


  


  Thanks.