Recherche avancée

Sur d’autres sites (6454)

• Writing A Dreamcast Media Player

  6 janvier 2017, par Multimedia Mike — Sega Dreamcast

  I know I’m not the only person to have the idea to port a media player to the Sega Dreamcast video game console. But I did make significant progress on an implementation. I’m a little surprised to realize that I haven’t written anything about it on this blog yet, given my propensity for publishing my programming misadventures.

  
  
  

  This old effort had been on my mind lately due to its architectural similarities to something else I was recently brainstorming.

  Early Days
  Porting a multimedia player was one of the earliest endeavors that I embarked upon in the multimedia domain. It’s a bit fuzzy for me now, but I’m pretty sure that my first exposure to the MPlayer project in 2001 arose from looking for a multimedia player to port. I fed it through the Dreamcast development toolchain but encountered roadblocks pretty quickly. However, this got me looking at the MPlayer source code and made me wonder how I could contribute, which is how I finally broke into practical open source multimedia hacking after studying the concepts and technology for more than a year at that point.

  Eventually, I jumped over to the xine project. After hacking on that for awhile, I remembered my DC media player efforts and endeavored to compile xine to the console. The first attempt was to simply compile the codebase using the Dreamcast hobbyist community’s toolchain. This is when I came to fear the multithreaded snake pit in xine’s core. Again, my memories are hazy on the specifics, but I remember the engine having a bunch of threading hacks with comments along the lines of “this code deadlocks sometimes, so on shutdown, monitor this lock and deliberately break it if it has been more than 3 seconds”.

  Something Workable
  Eventually, I settled on a combination of FFmpeg’s libavcodec library for audio and video decoders, xine’s demuxer library, and xine’s input API, combined with my own engine code to tie it all together along with video and output drivers provided by the KallistiOS hobbyist OS for Dreamcast. Here is a simple diagram of the data movement through this player :

  
  
  

  Details and Challenges
  This is a rare occasion when I actually got to write the core of a media player engine. I made some mistakes.

  xine’s internal clock ran at 90000 Hz. At least, its internal timestamps were all in reference to a 90 kHz clock. I got this brilliant idea to trigger timer interrupts at 6000 Hz to drive the engine. Whatever the timer facilities on the Dreamcast, I found that 6 kHz was the greatest common divisor with 90 kHz. This means that if I could have found an even higher GCD frequency, I would have used that instead.

  So the idea was that, for a 30 fps video, the engine would know to render a frame on every 200th timer interrupt. I eventually realized that servicing 6000 timer interrupts every second would incur a ridiculous amount of overhead. After that, my engine’s philosophy was to set a timer to fire for the next frame while beginning to process the current frame. I.e., when rendering a frame, set a timer to call back in 1/30th of a second. That worked a lot better.

  As I was still keen on 8-bit paletted image codecs at the time (especially since they were simple and small for bootstrapping this project), I got to use output palette images directly thanks to the Dreamcast’s paletted textures. So that was exciting. The engine didn’t need to convert the paletted images to a different colorspace before rendering. However, I seem to recall that the Dreamcast’s PowerVR graphics hardware required that 8-bit textures be twiddled/swizzled. Thus, it was still required to manipulate the 8-bit image before rendering.

  I made good progress on this player concept. However, a huge blocker for me was that I didn’t know how to make a proper user interface for the media player. Obviously, programming the Dreamcast occurred at a very low level (at least with the approach I was using), so there were no UI widgets easily available.

  This was circa 2003. I assumed there must have been some embedded UI widget libraries with amenable open source licenses that I could leverage. I remember searching and checking out a library named libSTK. I think STK stood for “set-top toolkit” and was positioned specifically for doing things like media player UIs on low-spec embedded computing devices. The domain hosting the project is no longer useful but this appears to be a backup of the core code.

  It sounded promising, but the libSTK developers had a different definition of “low-spec embedded” device than I did. I seem to recall that they were targeting something along with likes of a Pentium III clocked at 800 MHz with 128 MB RAM. The Dreamcast, by contrast, has a 200 MHz SH-4 CPU and 16 MB RAM. LibSTK was also authored in C++ and leveraged the Boost library (my first exposure to that code), and this all had the effect of making binaries quite large while I was trying to keep the player in lean C.

  Regrettably, I never made any serious progress on a proper user interface. I think that’s when the player effort ran out of steam.

  The Code
  So, that’s another project that I never got around to finishing or publishing. I was able to find the source code so I decided to toss it up on github, along with 2 old architecture outlines that I was able to dig up. It looks like I was starting small, just porting over a few of the demuxers and decoders that I knew well.

  I’m wondering if it would still be as straightforward to separate out such components now, more than 13 years later ?

  The post Writing A Dreamcast Media Player first appeared on Breaking Eggs And Making Omelettes.

• Nginx rtmp module - on_publish fires multiple time instead of once

  29 juillet 2017, par Stephen Wright

  This is copy and pasted from the bug report I created on the rtmp-module by Arut, I am not completely sure if it is a bug or me not understanding how the module works, I have read the whole directives of module as from https://github.com/arut/nginx-rtmp-module/wiki/Directives

  Proper explanation, if code is not displayed properly I will edit and fix

  Hi, been using the module and finding it very very good !

  Think I have found a issue though, although it may be me misunderstanding the directives.

  Essentially I wish to fire a script (/usr/local/bin/make_thumbnail.sh) which creates a thumbnail automatically from a stream (using ffmpeg), the idea is to have this done for every stream as soon as it is published in order to create a function a bit like twitch tv where the streamer will not have to specify any thumbnail image, authenticated users simply start a stream (which will later be authenticated but is not yet) The script does also write data into the database however this stage works fine and I don’t believe the issue is related, if I comment out these lines then the thumbnail creation still works and my issue continues.

  Initially this was done using the "exec" command as I believe I mis-read the documentation and I believe the exec command doesn’t work for my problem as ". When publishing stops the process is terminated." does this mean it will continually execute until stream stops ?

  I have started using the exec_publish command to try and fix this issue however the same issue seems to occur. The entire script repeats approximately every 15-17 seconds, a new thumbnail is created and a new database entry is create with all the correct information.

  Below is the nginx.conf line. Please ignore if indentation is incorrect couldn’t see a way to indent blocks of code and it’s late here, assume all code is indented correctly unless you believe that could be the issue in which case I will post it indented as early as I can.

  application live {
  
  allow play all;
  
  live on;
  
  record all;
  
  record_path /var/stream/video_recordings/;
  
  record_unique on; 
  
  hls on;
  
  hls_nested on;
  
  hls_path /var/stream/HLS/live;
  
  hls_fragment 10s;
  
  
  
  #on publish create thumbnail using first second of stream and save in 
  
  /var/stream/video_recordings/thumbnails
  
  exec_publish usr/local/bin/make_thumbnail.sh $name;

  The rest can be pasted or attached if needed but is working nginx config for rtmp + website

  The most simple version of the make_thumbnail..sh is pasted below, I have omitted the variables that I have used for database entryys obviously but as the script works without fail from terminal I believe this to be an nginx issue (if I run the command manually under the nginx user e.g. sudo -u nginx /usr/local/bin/make_thumbnail.sh with a name the same as any running stream, it works and only executes once as would expect, all permissions in script are ok and tested.

  make_thumbnail.sh

  #!/bin/bash
  
  
  
  TIME=$(date +%s)
  
  NAME=$1 
  
  echo "time: "
  
  FILENAME=${TIME}_${NAME}
  
  
  
  ffmpeg -i rtmp://192.168.0.98:1935/live/$1 -vframes 1 -s 150x150 -ss 10 -
  
  strftime 1 /var/stream/video_recordings/thumbnails/"$FILENAME.jpg";
  
  
  
  #Writes path to video into database
  
  mysql --user=$DB_USER --password=$DB_PASSWD $DB_NAME << EOF
  
  INSERT INTO $TABLE3 (thumbnailfile) VALUES ('$FILENAME');
  
  set @last_id_in_thumbnails = LAST_INSERT_ID();
  
  INSERT INTO $TABLE (created_at, updated_at, thumnailID) VALUES 
  
  (NOW(),NOW(),@last_id_in_thumbnails);
  
  SET @last_id_in_livestreams = LAST_INSERT_ID();
  
  INSERT INTO $TABLE2 (created_at, updated_at, filename,liveID) VALUES 
  
  (NOW(),NOW(),'$FILENAME',@last_id_in_livestreams); 
  
  EOF

  I have not got the nginx rtmp logs installed, I can obviously do this however some of the logs appear in the nginx error.log, strangely the latest stream I tried did not update in the access log, however I think this is because I did not attempt to connect to it via any method. I don’t fully understand the error.log, in my stupidity I decided to use nginx with which I am quite inexperienced and I am finding it very difficult to troubleshoot this issue, it appears to me that as part of the RTMP protocol or my streaming software (OBS) is either directly pinging the rtmp stream or is being pinged by the server to ensure the connection is still there. And this ping is

  I have left a stream running from approx 4 minutes without interacting with the server, streaming software, computer running the stream, I have ensured the internet connection is constant as my first though was the connection dropped, however on inspecting the database the executing is done always after at least 11 seconds however usually this is 16, I can’t seem to figure out how to select the closest dates from the database however there has been at least a few 17 second differences (potentially when
  I am unsure if this is an issue or if it is intended behavior but I do require this to finish a university degree, I’m not asking for answers but if it is a legitimate issue then I would be happy to spend as much time I can commit to it if some insight into what is causing it, or if there is a workaround I believe it should be documented somewhere, I have googled into making any exec commands run only once on publish

  I can’t seem to pinpoint where in the log the issue is happening however think it is something to do with the below exceprts I would attach the file but can’t seem to select all lines after the timestamp upon starting a stream

  2017/07/26 18:17:35 [info] 1451#0: *2229 exec: starting managed child 
  
  'ffmpeg', client: 192.168.0.78, server: 0.0.0.0:1935
  
  2017/07/26 18:17:35 [info] 1451#0: *2412 client connected '192.168.0.98'
  
  2017/07/26 18:17:35 [info] 1451#0: *2412 connect: app='live' args='' 
  
  flashver='LNX 9,0,124,2' swf_url='' tc_url='rtmp://192.168.0.98:1935/live' 
  
  page_url='' acodecs=4071 vcodecs=252 object_encoding=0, client: 
  
  192.168.0.98, server: 0.0.0.0:1935
  
  2017/07/26 18:17:35 [info] 1451#0: *2412 createStream, client: 192.168.0.98, 
  
  server: 0.0.0.0:1935
  
  2017/07/26 18:17:35 [info] 1451#0: *2412 play: name='newname' args='' 
  
  start=-2000 duration=0 reset=0 silent=0, client: 192.168.0.98, server:  
  
  0.0.0.0:1935
  
  2017/07/26 18:17:36 [info] 1451#0: *2410 recv() failed (104: Connection  
  
  reset by peer), client: 192.168.0.98, server: 0.0.0.0:1935
  
  2017/07/26 18:17:36 [info] 1451#0: *2410 disconnect, client: 192.168.0.98, 
  
  server: 0.0.0.0:1935
  
  2017/07/26 18:17:36 [info] 1451#0: *2410 deleteStream, client: 192.168.0.98, 
  
  server: 0.0.0.0:1935
  
  2017/07/26 18:17:36 [notice] 1451#0: signal 17 (SIGCHLD) received
  
  2017/07/26 18:17:36 [notice] 1451#0: unknown process 10487 exited with code 
  
  0
  
  2017/07/26 18:17:36 [info] 1451#0: *2229 exec: child 10487 exited; ignoring, 
  
  client: 192.168.0.78, server: 0.0.0.0:1935
  
  
  
  ver: 0.0.0.0:1935
  
  2017/07/26 18:17:41 [info] 1451#0: *2229 exec: starting managed child 
  
  'usr/local/bin/make_thumbnail.sh', client: 192.168.0.78, server:  
  
  0.0.0.0:1935
  
  2017/07/26 18:17:41 [info] 1451#0: *2413 client connected '192.168.0.98'
  
  2017/07/26 18:17:41 [info] 1451#0: *2413 connect: app='live' args='' 
  
  flashver='LNX 9,0,124,2' swf_url='' tc_url='rtmp://192.168.0.98:1935/live' 
  
  page_url='' acodecs=4071 vcodecs=252 object_encoding=0, client: 
  
  192.168.0.98, 
  
  server: 0.0.0.0:1935
  
  2017/07/26 18:17:41 [info] 1451#0: *2413 createStream, client: 192.168.0.98, 
  
  server: 0.0.0.0:1935
  
  2017/07/26 18:17:41 [info] 1451#0: *2413 play: name='newname' args='' 
  
  start=-2000 duration=0 reset=0 silent=0, client: 192.168.0.98, server: 
  
  0.0.0.0:1935
  
  2017/07/26 18:17:43 [info] 1451#0: *2229 exec: starting managed child 
  
  'ffmpeg', 
  
  client: 192.168.0.78, server: 0.0.0.0:1935
  
  2017/07/26 18:17:43 [info] 1451#0: *2414 client connected '192.168.0.98'
  
  2017/07/26 18:17:43 [info] 1451#0: *2414 connect: app='live' args='' 
  
  flashver='LNX 9,0,124,2' swf_url='' tc_url='rtmp://192.168.0.98:1935/live' 
  
  page_url='' acodecs=4071 vcodecs=252 object_encoding=0, client: 
  
  192.168.0.98, 
  
  server: 0.0.0.0:1935
  
  @

• Winamp and the March of GUI

  1er juillet 2012, par Multimedia Mike — General, ars technica, gui, user interface, winamp

  Ars Technica recently published a 15-year retrospective on the venerable Winamp multimedia player, prompting bouts of nostalgia and revelations of "Huh ? That program is still around ?" from many readers. I was among them.

  
  
  

  I remember first using Winamp in 1997. I remember finding a few of these new files called MP3s online and being able to play the first 20 seconds using the official Fraunhofer Windows player— full playback required the fully licensed version. Then I searched for another player and came up with Winamp. The first version I ever used was v1.05 in the summer of 1997. I remember checking the website often for updates and trying out every single one. I can’t imagine doing that nowadays— programs need to auto-update themselves (which Winamp probably does now ; I can’t recall the last time I used the program).

  Video Underdog
  The last time Winamp came up on my radar was early in 2003 when a new version came with support for a custom, proprietary multimedia audio/video format called Nullsoft Video (NSV). I remember the timeframe because the date is indicated in the earliest revision of my NSV spec document (back when I was maintaining such docs in a series of plaintext files). This was cobbled together from details I and others in the open source multimedia community sorted out from sample files. It was missing quite a few details, though.

  Then, Winamp founder Justin Frankel — introduced through a colleague on the xine team — emailed me his official NSV format and told me I was free to incorporate details into my document just as long as it wasn’t obvious that I had the official spec. This put me in an obnoxious position of trying to incorporate details which would have been very difficult to reverse engineer without the official doc. I think I coped with the situation by never really getting around to updating my doc in any meaningful way. Then, one day, the official spec was released to the world anyway, and it is now mirrored here at multimedia.cx.

  I don’t think the format ever really caught on in any meaningful way, so not a big deal. (Anytime I say that about a format, I always learn it saw huge adoption is some small but vocal community.)

  What’s Wrong With This Picture ?
  What I really wanted to discuss in this post was the matter of graphical user interfaces and how they have changed in the last 15 years.
  
  I still remember when I first downloaded Winamp v1.05 and tried it on my Windows machine at the time. Indignantly, the first thought I had was, "What makes this program think it’s so special that it’s allowed to violate the user interface conventions put forth by the rest of the desktop ?" All of the Windows programs followed a standard set of user interface patterns and had a consistent look and feel... and then Winamp came along and felt it could violate all those conventions.

  I guess I let the program get away with it because it was either that or only play 20-second clips from the unregistered Fraunhofer player. Though incredibly sterile by comparison, the Fraunhofer player, it should be noted, followed Windows UI guidelines to the letter.

  As the summer of 1997 progressed and more Winamp versions were released, eventually one came out (I think it was v1.6 or so) that supported skins. I was excited because there was a skin that made the program look like a proper Windows program— at least if you used the default Windows color scheme, and had all of your fonts a certain type and size.

  Skins were implemented by packaging together a set of BMP images to overlay on various UI elements. I immediately saw a number of shortcomings with this skinning approach. A big one was UI lock-in. Ironically, if you skin an app and wish to maintain backwards compatibility with the thousands of skins selflessly authored by your vibrant community (seriously, I couldn’t believe how prolific these things were), then you were effectively locked into the primary UI. Forget about adding a new button anywhere.

  Another big problem was resolution-independence. Basing your UI on static bitmaps doesn’t scale well with various resolutions. Winamp had its normal mode and it also had double-sized mode.

  Skins proliferated among many types of programs in the late 1990s. I always treasured this Suck.com (remember them ? that’s a whole other nostalgia trip) essay from April, 2000 entitled Skin Cancer. Still, Winamp was basically the standard, and the best, and I put away my righteous nerd rage and even dug through the vast troves of skins. I remember settling on Swankamp for a good part of 1998, probably due to the neo-swing revival at the time.

  
  
  

  Then again, if Winamp irked me, imagine my reaction when I was first exposed to the Sonique Music Player in 1998 :

  
  
  

  The New UI Order
  Upon reflection, I realize now that I had a really myopic view of what a computer GUI should be. I thought the GUIs were necessarily supposed to follow the WIMP (windows, icons, mouse, pointer) paradigm and couldn’t conceive of anything different. For a long time, I couldn’t envision a useful GUI on a small device (like a phone) because WIMP didn’t fit well on such a small interface (even though I saw various ill-fated attempts to make it work). This thinking seriously crippled me when I was trying to craft a GUI for a custom console media player I was developing as a hobby many years ago.

  I’m looking around at what I have open on my Windows 7 desktop right now. Google Chrome browser, Apple iTunes, Adobe Photoshop Elements, and VMware Player are 4 programs which all seem to have their own skins. Maybe Winamp doesn’t look so out of place these days.