Recherche avancée

Sur d’autres sites (4386)

• Stream h264 to javafx possibly using javacv/ffmpeg

  4 octobre 2018, par cagney

  I’m really stuck on getting a video stream to play on a java fx project.

  — Short version :

  I’m streaming h264/avcc flavor video from an android phone to a desktop computer. However javafx doesn’t have an easy solution for displaying stream. I’m attempting to use javacv / ffmpeg in an attempt to make this work. However I am getting errors from ffmpeg.

  1) Is there a better way to display streaming video on javafx ?

  2) Do you have a sample project or good tutorial for javacv ffmpegframegrabber ?

  3) I think I may be missing some small detail in mycode but Im not sure what i would be.

  — Longer Version :

  1) On the android end Im getting video using mediarecorder. In order to get the sps/pps info I record and save a small movie to the device and then parse the sps and pps data.

  2) Next, on the android, I split up the nalus to meet MTU req and send them over a udp connection to my desktop

  3)On my desktop I reassmble the nalus( or trash them if they loose data) and feed those to an input stream that I gave to the framegreabber constructor.

  — The Code and Logs :

  The errors are long and numerous depending on the flavor I feed it. Here are two separate examples which are usually repeated at great length

  [h264 @ 0000020225907a40] non-existing PPS 0 referenced
  
  [h264 @ 0000020225907a40] decode_slice_header error
  
  [h264 @ 0000020225907a40] no frame!
  
  
  
  [h264 @ 00000163d8637a40] illegal aspect ratio
  
  [h264 @ 00000163d8637a40] pps_id 3412 out of range
  
  [AVBSFContext @ 00000163e28a0e00] Invalid NAL unit 0, skipping.

   !! One big caveat that I am aware off is that I have not implemented timestamps
  which I created on the android device when feeding ffmpeg. I think it should still show distorted images without this though

  Because I have spent all day guessing and trying I have several "flavors" of data I have shoved through. I am only showing the first section of each nal which I believe if correct would at least show a garbage image as long as my sps and pps are right

  sps: 67 80 80 1E E9 01 68 22 FD C0 36 85 09 A8
  
  pps: 68 06 06 E2

  Below is annex B style.
  These were each prefixed with either 00 00 01 and 00 00 00 01

  Debug transfer 65 B8 40 0B E5 B8 7B 80 5B 85 
  
  Debug transfer 41 E2 20 7A 74 34 3B D6 BE FA 
  
  Debug transfer 41 E4 40 2F 01 E0 0C 06 EE 91 
  
  Debug transfer 41 E6 60 3E A1 20 5A 02 3C 6D 
  
  Debug transfer 41 E8 80 13 B0 B9 82 C3 03 F4 
  
  Debug transfer 41 EC C0 1B A3 0C 28 F1 B0 C8 
  
  Debug transfer 41 EE E0 1F CE 07 30 EE 05 06 
  
  Debug transfer 41 F1 00 08 ED 80 9C 20 09 73 
  
  Debug transfer 41 F3 20 09 E9 00 86 60 21 C3 
  
  VideoDecoderaddPacket type: 24
  
  Debug transfer 67 80 80 1E E9 01 68 22 FD C0 
  
  Debug transfer 68 06 06 E2 
  
  Debug transfer 65 B8 20 00 9F 80 78 00 12 8A 
  
  Debug transfer 41 E2 20 09 F0 1E 40 7B 0C E0 
  
  Debug transfer 41 E4 40 09 F0 29 30 D6 00 AE 
  
  Debug transfer 41 E6 60 09 F1 48 31 80 99 40 
  
  [h264 @ 000001c771617a40] non-existing PPS 0 referenced

  Here I tried Avcc style. You can see the first line is the combination of the sps pps followed by idr and then repeated non idr

  Debug transfer 18 00 0E 67 80 80 1E E9 01 68 
  
  Debug transfer 00 02 4A 8F 65 B8 20 00 9F C5 
  
  Debug transfer 00 02 2F DA 41 E2 20 09 E8 0F 
  
  Debug transfer 00 02 2C 34 41 E4 40 09 F4 20 
  
  Debug transfer 00 02 4D 92 41 E6 60 09 FC 2B 
  
  Debug transfer 00 02 47 02 41 E8 80 09 F0 72 
  
  Debug transfer 00 02 52 50 41 EA A0 09 EC 0F 
  
  Debug transfer 00 02 58 8A 41 EC C0 09 FC 6F 
  
  Debug transfer 00 02 55 F9 41 EE E0 09 FC 6E 
  
  Debug transfer 00 02 4D 79 41 F1 00 09 F0 3E 
  
  Debug transfer 00 02 4D B6 41 F3 20 09 E8 64 

  The following class is where I try to get javacv/ffmpeg to show the video. I dont think its an ideal solution and am researching canvasfram as a replacement to the image view.

      public class ImageDecoder {
  
  
  
      private final static String TAG = "ImageDecoder ";
  
  
  
      private ImageDecoder(){
  
  
  
      }
  
  
  
  
  
      public static void streamImageToImageView(
  
              final ImageView view,
  
              final InputStream inputStream,
  
              final String format,
  
              final int frameRate,
  
              final int bitrate,
  
              final String preset,
  
              final int numBuffers
  
      )
  
      {
  
          System.out.println("Image Decoder Starting...");
  
  
  
  
  
          try(    final FrameGrabber grabber = new 
  
      FFmpegFrameGrabber(inputStream))
  
          {
  
  
  
              final Java2DFrameConverter converter = new Java2DFrameConverter();
  
  
  
              grabber.setFrameNumber(frameRate);
  
              grabber.setFormat(format);
  
              grabber.setVideoBitrate(bitrate);
  
              grabber.setVideoOption("preset", preset);
  
              grabber.setNumBuffers(numBuffers);
  
  
  
              System.out.println("Image Decoder waiting on grabber.start...");
  
              grabber.start();   //---- this call is blocking the loop
  
  
  
              System.out.println("Image Decoder Looping--------------------------- 
  
     -------- hit stop");
  
              while(!Thread.interrupted()){
  
                  //System.out.println("Image Decoder Looping");
  
                  final Frame frame = grabber.grab();
  
                  if (frame != null){
  
                      final BufferedImage bufferedImage = 
  
      converter.convert(frame);
  
                      if (bufferedImage != null){
  
  
  
                          Platform.runLater(() -> 
  
      view.setImage(SwingFXUtils.toFXImage(bufferedImage, null)));
  
                      }else{
  
                          System.out.println("no buf im");
  
                      }
  
                  }else{
  
                      System.out.println("no fr");
  
                      Thread.currentThread().interrupt();
  
                  }
  
  
  
              }
  
  
  
  
  
  
  
          }catch (Exception e){
  
              System.out.print(TAG + e);
  
          }
  
  
  
  
  
      }
  
  
  
  
  
  
  
  
  
  
  
  
  
      }

  Any help is greatly appreciated.

• Revisiting the Belco Alpha-400

  26 août 2010, par Multimedia Mike — General

  Relieved of the primary FATE maintenance duties, I decided to dust off my MIPS-based Belco Alpha-400 and try to get it doing FATE cycles. And just as I was about to get FATE running, I saw that Mans already got his MIPS-based Popcorn Hour device to run FATE. But here are my notes anyway.

  
  
  

  Getting A Prompt
  For my own benefit, I made a PDF to remind me precisely how to get a root prompt on the Alpha-400. The ‘jailbreak’ expression seems a little juvenile to me, but it seems to be in vogue right now.

  alpha-400-jailbreak.pdf

  Toolchain
  When I last tinkered with the Alpha-400, I was trying to build a toolchain that could build binaries to run on the unit’s MIPS chip, to no avail. Sometime last year, MichaelK put together x86_32-hosted toolchains that are able to build mipsel 32-bit binaries for Linux 2.4 and 2.6. The Alpha-400 uses a 2.4 kernel and the corresponding toolchain works famously for building current FFmpeg (--disable-devices is necessary for building).

  FATE Samples
  Next problem : Making the FATE suite available to the Alpha-400. I copied all of the FATE suite samples onto a VFAT-formatted SD card. The filename case is not preserved for all files which confounds me since it is preserved in other cases. I tried formatting the card for ext3 but the Alpha-400 would not mount it, even though /proc/filesystems lists ext3 (supporting an older version of ext3 ?).

  Alternative : Copy all of the FATE samples to the device’s rootfs. Space will be a little tight, though. Then again, there is over 600 MB of space free ; I misread earlier and thought there were only 300 MB free.

  Remote Execution
  To perform FATE cycles on a remote device, it helps to be able to SSH into that remote device. I don’t even want to know how complicated it would be to build OpenSSH for the device. However, the last time I brought up this topic, I learned about a lighter weight SSH replacement called Dropbear. It turns out that Dropbear runs great on this MIPS computer.

  Running FATE Remotely
  I thought all the pieces would be in place to run FATE at this point. However, there is one more issue : Running FATE on a remote system requires that the host and the target are sharing a filesystem somehow. My personal favorite remote filesystem method is sshfs which is supposed to work wherever there is an SSH server. That’s not entirely true, though– sshfs also requires sftp-server to be installed on the server side, a program that Dropbear does not currently provide.

  I’m not even going to think about getting Samba or NFS server software installed on the Alpha-400. According to the unit’s /proc/filesystems file, nfs is a supported filesystem. I hate setting up NFS but may see if I can get that working anyway.

  Residual Weirdness
  The unit comes with the venerable Busybox program (BusyBox v1.4.1 (2007-06-01 20:37:18 CST) multi-call binary) for most of its standard command line utilities. I noticed a quirk where BusyBox’s md5sum gives weird hex characters. This might be a known/fixed issue.

  Another item is that the Alpha-400′s /dev/null file only has rwxr-xr-x per default. This caused trouble when I first tried to scp using Dropbear using a newly-created, unprivileged user.

• avfilter : take_samples : do not directly return frame when samples are skipped

  18 mai 2017, par Muhammad Faiz

  avfilter : take_samples : do not directly return frame when samples are skipped
  Modifying data pointer when skipping samples may make it unaligned.
  
  Workaround for Ticket6349.
  This should fix the crash of ticket's testcase and a crash/regression
  
  with avxsynth (reported by Michael Niedermayer).
  Also change frame->nb_samples < max to frame->nb_samples <= max.
  
  This improves performance. Benchmark :
  
  ./ffmpeg -filter_complex "aevalsrc=0:n=1166,firequalizer=fixed=on" -f null null
  
  old :
  
    25767 decicycles in take_samples,    1023 runs,      1 skips
  
    25422 decicycles in take_samples,    2047 runs,      1 skips
  
    25181 decicycles in take_samples,    4095 runs,      1 skips
  
    24904 decicycles in take_samples,    8191 runs,      1 skips
  new :
  
      550 decicycles in take_samples,    1024 runs,      0 skips
  
      548 decicycles in take_samples,    2048 runs,      0 skips
  
      545 decicycles in take_samples,    4096 runs,      0 skips
  
      544 decicycles in take_samples,    8192 runs,      0 skips
  Reviewed-by : Nicolas George <george@nsup.org>
  
  Reviewed-by : Michael Niedermayer <michael@niedermayer.cc>
  
  Reviewed-by : Paul B Mahol <onemda@gmail.com>
  
  Signed-off-by : Muhammad Faiz <mfcc64@gmail.com>
  

  • [DH] libavfilter/avfilter.c
  • [DH] libavfilter/framequeue.c
  • [DH] libavfilter/framequeue.h