Recherche avancée

Sur d’autres sites (7151)

• Is there any open source solution to display a remote stream inside a Hololens2 UWP Vuforia application ?

  19 avril 2023, par T777

  What do we need ?

  


  We are trying to develop an application for quality management in which we show an hologram on a metal part as an assitance marking. (using Hololen2 + Vuforia + ModleTargets) The employee uses an sensor to follow this assitance marking and the data will be analyzed live by a test device. The results are outputed on a screen / are visible at an closed source application of the manufacturer of the test device.

  


  Capturing of the video output :
The current plan is to capture the video stream of the test device via capture card. Add a via mrtk2 videopanel inside the vuforia app and stream the captured video to the Hololens2 using obs or an OpenCV python script for screen recording.

  


  What we have tried so far

  


  1) Sending Raw udp stream
via RMTP and decoding + converting with gstreamer server and writing an own library in Unity for Receiving
Result : Temporary stopped, because receiving the udp streams needs connection/ session management (signalling) frame syncing and agreement on video size, color format, frame rate etc.. and we have no solution.
An own implementation of any of this would have high complexity is consuming a lot of time.

  


  2) Using available protocols that i can find on the web
Actually there are some protocols already developed for session creation and streaming :

  


  

  • HTTP streaming (HLS) (Transport + Session)

  


  • RTMP (Transport + Session),

  


  • RTP (Transport) + RTPS (Session),

  


  • WebRTC : Is possible with different protocol stacks
RTMP/TCP/UDP (Transport) + SDP (standardized format for video paramaters) + ICE (p2p)/ WHIP (http, client-server) / Websocket(client-server) (signaling protocols) that can be used and some good open source streaming servers (gstreamer, mediamtx and srs)

  


  


  When using these the video will be encoded typcially with xh264 and need to be decoded on the HoloLens 2. There are APIs to C/C++ native (hardware) decoding libraries like unity-vlc and ffmpeg.NET that needing media library ffmpeg. I could figure out (not tested) that there is an hardware h264 decoder on the HoloLens2 but I have no clue how to access it. Since there I couldnt disvocer any information about HoloLens2 media libraries.

  


  3) Using Unity packages

  


  


  • Unit package WebRTC (https://docs.unity3d.com/Packages/com.unity.webrtc@2.4/manual/index.html) supports multiple transport protocols seem to have no signaling mechanism and

    


  


  • Unity package Render streaming (https://docs.unity3d.com/Packages/com.unity.renderstreaming@3.1/manual/index.html) is a fully integrated unity to unity and unit to browser streaming package with integrated stream Server with web GUI. It offers various streaming protocols (TCP, UDP,rmtp) and signaling mechanism over websocket, http (seems custom and not whip) or Furiouus.
BUT it doesnt support UWP as noted into the documentation. Implementing an example application we could demonstrate an working example with Vuforia, but it fails on build with target UWP on missing libraries.
Similar to : https://www.youtube.com/watch?v=nHRC0uGBnn8

    


  


  


  Will be testing other compile options tomorrow..

  


  

  • Mixed Reality WebRTC (https://github.com/microsoft/MixedReality-WebRTC) :
Various protocol support, Microsoft brought Webrtc specifically to HoloLens.
Deprecated, as fas as I can see just support for Hololens1 and ARM32. So i can not evaluate if trying it with this is worth it.

  


  


  What are the next options ?

  


  

  • Developing a raw udp streaming library with untiy directly.

  


  • Rebuilding the application with visionlib (ARM32) compatible and MixedRealityWebRTC (ARM32)

  


  • Porting ffmpeg + API to UWP ?

  


  • Also there seem some affords to make WebRTC in general available to UWP platforms : https://github.com/microsoft/winrtc

  


  


  The questions

  


  

  • Does Vuforia support ARM32 ?

  


  • How to access hardware decoder of Hololens2 via Unity Code ?

  


  


• How do I convert a .wav file to 16bit 44.1kz using ffmpeg or other utility [closed]

  26 mai 2023, par Seth Edwards

  A preface :
I am building an environment for a my own streaming box. Since building the UI. I turned to the now obsolete MSNTV box to find its UI sound effects.

  


  I found the dump on GitHub. I downloaded and located where the sounds where located.

  


  I listened to them one by one. I noticed that they are wave files. But they sound like they were low quality and may have been compressed before being turned into a wave file.

  


  I was using the Apple Files app on an iPhone 6s running iOS 15.7.1.

  


  They play back fine.

  


  I try importing them into GarageBand for iOS and it gives me an error saying that it only allows 16bit 44.1khz files. This confirmed my suspicion of it being low quality.

  


  I then tried playing them on a Dell Chromebook 3100 running ChromeOS. Chrome’s player would also not play the files.

  


  I need to find out how to convert them to 16bit 44.1khz wave files.

  


  My guess is that since the MSNTV had a small amount of storage space that they compressed the audio.

  


  I tried converting the files to mp3. And they are Noticeably worse.

  


  Does anyone know how to convert these files so they can be played back normally.

  


  In the end I plan to use these files and play them using the pygame library.

  


  I have tried changing the metadata

  


  Converting to mp3

  


• What is the Best Approach for Storing and Displaying Video Files as Base64 in HTML for High Performance and Efficiency ? [closed]

  25 août 2023, par Barthez

  I'm currently working on a project where I need to allow users to upload videos, which will then be converted to Base64 and embedded within an HTML file. I'm reaching out to gain a better understanding of the best practices for accomplishing this task while adhering to Stack Overflow guidelines.Here's my plan so far:Video Upload : Users will be able to upload videos through a web interface.Conversion to Base64 : The uploaded video will be converted to Base64 using [specific library/tool].Embedding in HTML : The Base64 encoded video will be embedded within an HTML file using the tag.Before I proceed, I have a few questions:Are there any particular libraries or tools you recommend for converting videos to Base64 efficiently ?What are the potential performance implications of embedding Base64 videos in HTML files, especially considering large video files ?Are there any security concerns I should be aware of when implementing this process ?How can I ensure cross-browser compatibility when embedding these Base64 videos ?Are there any alternatives to this approach that might be more efficient or manageable ?I want to make sure I'm following best practices and avoiding any pitfalls, so any insights, tips would be greatly appreciated. Thank you for your time and assistance !

  


  I attempted to convert an uploaded video to Base64 and embed it within an HTML file.