Recherche avancée

Médias (3)

Mot : - Tags -/spip

Autres articles (99)

  • Librairies et binaires spécifiques au traitement vidéo et sonore

    31 janvier 2010, par

    Les logiciels et librairies suivantes sont utilisées par SPIPmotion d’une manière ou d’une autre.
    Binaires obligatoires FFMpeg : encodeur principal, permet de transcoder presque tous les types de fichiers vidéo et sonores dans les formats lisibles sur Internet. CF ce tutoriel pour son installation ; Oggz-tools : outils d’inspection de fichiers ogg ; Mediainfo : récupération d’informations depuis la plupart des formats vidéos et sonores ;
    Binaires complémentaires et facultatifs flvtool2 : (...)

  • Support audio et vidéo HTML5

    10 avril 2011

    MediaSPIP utilise les balises HTML5 video et audio pour la lecture de documents multimedia en profitant des dernières innovations du W3C supportées par les navigateurs modernes.
    Pour les navigateurs plus anciens, le lecteur flash Flowplayer est utilisé.
    Le lecteur HTML5 utilisé a été spécifiquement créé pour MediaSPIP : il est complètement modifiable graphiquement pour correspondre à un thème choisi.
    Ces technologies permettent de distribuer vidéo et son à la fois sur des ordinateurs conventionnels (...)

  • De l’upload à la vidéo finale [version standalone]

    31 janvier 2010, par

    Le chemin d’un document audio ou vidéo dans SPIPMotion est divisé en trois étapes distinctes.
    Upload et récupération d’informations de la vidéo source
    Dans un premier temps, il est nécessaire de créer un article SPIP et de lui joindre le document vidéo "source".
    Au moment où ce document est joint à l’article, deux actions supplémentaires au comportement normal sont exécutées : La récupération des informations techniques des flux audio et video du fichier ; La génération d’une vignette : extraction d’une (...)

Sur d’autres sites (5982)

  • A Comprehensive Guide to Robust Digital Marketing Analytics

    30 octobre 2023, par Erin

    First impressions are everything. This is not only true for dating and job interviews but also for your digital marketing strategy. Like a poorly planned resume getting tossed in the “no thank you” pile, 38% of visitors to your website will stop engaging with your content if they find the layout unpleasant. Thankfully, digital marketers can access data that can be harnessed to optimise websites and turn those “no thank you’s” into “absolutely’s.”

    So, how can we transform raw data into valuable insights that pay off ? The key is web analytics tools that can help you make sense of it all while collecting data ethically. In this article, we’ll equip you with ways to take your digital marketing strategy to the next level with the power of web analytics.

    What are the different types of digital marketing analytics ?

    Digital marketing analytics are like a cipher into the complex behaviour of your buyers. Digital marketing analytics help collect, analyse and interpret data from any touchpoint you interact with your buyers online. Whether you’re trying to gauge the effectiveness of a new email marketing campaign or improve your mobile app layout, there’s a way for you to make use of the insights you gain. 

    As we go through the eight commonly known types of digital marketing analytics, please note we’ll primarily focus on what falls under the umbrella of web analytics. 

    1. Web analytics help you better understand how users interact with your website. Good web analytics tools will help you understand user behaviour while securely handling user data. 
    2. Learn more about the effectiveness of your organisation’s social media platforms with social media analytics. Social media analytics include user engagement, post reach and audience demographics. 
    3. Email marketing analytics help you see how email campaigns are being engaged with.
    4. Search engine optimisation (SEO) analytics help you understand your website’s visibility in search engine results pages (SERPs). 
    5. Pay-per-click (PPC) analytics measure the performance of paid advertising campaigns.
    6. Content marketing analytics focus on how your content is performing with your audience. 
    7. Customer analytics helps organisations identify and examine buyer behaviour to retain the biggest spenders. 
    8. Mobile app analytics track user interactions within mobile applications. 

    Choosing which digital marketing analytics tools are the best fit for your organisation is not an easy task. When making these decisions, it’s critical to remember the ethical implications of data collection. Although data insights can be invaluable to your organisation, they won’t be of much use if you lose the trust of your users. 

    Tips and best practices for developing robust digital marketing analytics 

    So, what separates top-notch, robust digital marketing analytics from the rest ? We’ve already touched on it, but a big part involves respecting user privacy and ethically handling data. Data security should be on your list of priorities, alongside conversion rate optimisation when developing a digital marketing strategy. In this section, we will examine best practices for using digital marketing analytics while retaining user trust.

    Lightbulb with a target in the center being struck by arrows

    Clear objectives

    Before comparing digital marketing analytics tools, you should define clear and measurable goals. Try asking yourself what you need your digital marketing analytics strategy to accomplish. Do you want to improve conversion rates while remaining data compliant ? Maybe you’ve noticed users are not engaging with your platform and want to fix that. Save yourself time and energy by focusing on the most relevant pain points and areas of improvement.

    Choose the right tools for the job

    Don’t just base your decision on what other people tell you. Take the tool for a test drive — free trials allow you to test features and user interfaces and learn more about the platform before committing. When choosing digital marketing analytics tools, look for ones that ensure compliance with privacy laws like GDPR.

    Don’t overlook data compliance

    GDPR ensures organisations prioritise data protection and privacy. You could be fined up to €20 million, or 4% of the previous year’s revenue for violations. Without data compliance practices, you can say goodbye to the time and money spent on digital marketing strategies. 

    Don’t sacrifice data quality and accuracy

    Inaccurate and low-quality data can taint your analysis, making it hard to glean valuable insights from your digital marketing analytics efforts. Regularly audit and clean your data to remove inaccuracies and inconsistencies. Address data discrepancies promptly to maintain the integrity of your analytics. Data validation measures also help to filter out inaccurate data.

    Communicate your findings

    Having insights is one thing ; effectively communicating complex data findings is just as important. Customise dashboards to display key metrics aligned with your objectives. Make sure to automate reports, allowing stakeholders to stay updated without manual intervention. 

    Understand the user journey

    To optimise your conversion rates, you need to understand the user journey. Start by analysing visitors interactions with your website — this will help you identify conversion bottlenecks in your sales or lead generation processes. Implement A/B testing for landing page optimisation, refining elements like call-to-action buttons or copy, and leverage Form Analytics to make informed, data-driven improvements to your forms.

    Continuous improvement

    Learn from the data insights you gain, and iterate your marketing strategies based on the findings. Stay updated with evolving web analytics trends and technologies to leverage new growth opportunities.

    Why you need web analytics to support your digital marketing analytics toolbox

    You wouldn’t set out on a roadtrip without a map, right ? Digital marketing analytics without insights into how users interact with your website are just as useless. Used ethically, web analytics tools can be an invaluable addition to your digital marketing analytics toolbox. 

    The data collected via web analytics reveals user interactions with your website. These could include anything from how long visitors stay on your page to their actions while browsing your website. Web analytics tools help you gather and understand this data so you can better understand buyer preferences. It’s like a domino effect : the more you understand your buyers and user behaviour, the better you can assess the effectiveness of your digital content and campaigns. 

    Web analytics reveal user behaviour, highlighting navigation patterns and drop-off points. Understanding these patterns helps you refine website layout and content, improving engagement and conversions for a seamless user experience.

    Magnifying glass examining various screens that contain data

    Concrete CMS harnessed the power of web analytics, specifically Form Analytics, to uncover a crucial insight within their user onboarding process. Their data revealed a significant issue : the “address” input field was causing visitors to drop off and not complete the form, severely impacting the overall onboarding experience and conversion rate.

    Armed with these insights, Concrete CMS made targeted optimisations to the form, resulting in a substantial transformation. By addressing the specific issue identified through Form Analytics, they achieved an impressive outcome – a threefold increase in lead generation.

    This case is a great example of how web analytics can uncover customer needs and preferences and positively impact conversion rates. 

    Ethical implications of digital marketing analytics

    As we’ve touched on, digital marketing analytics are a powerful tool to help better understand online user behaviour. With great power comes great responsibility, however, and it’s a legal and ethical obligation for organisations to protect individual privacy rights. Let’s get into the benefits of practising ethical digital marketing analytics and the potential risks of not respecting user privacy : 

    • If someone uses your digital platform and then opens their email one day to find it filled with random targeted ad campaigns, they won’t be happy. Avoid losing user trust — and facing a potential lawsuit — by informing users what their data will be used for. Give them the option to consent to opt-in or opt-out of letting you use their personal information. If users are also assured you’ll safeguard personal information against unauthorised access, they’ll be more likely to trust you to handle their data securely.
    • Protecting data against breaches means investing in technology that will let you end-to-end encrypt and securely store data. Other important data-security best practices include access control, backing up data regularly and network and physical security of assets.
    • A fine line separates digital marketing analytics and misusing user data — many companies have gotten into big trouble for crossing it. (By big trouble, we mean millions of dollars in fines.) When it comes to digital marketing analytics, you should never cut corners when it comes to user privacy and data security. This balance involves understanding what data can be collected and what should be collected and respecting user boundaries and preferences.

    Learn more 

    We discussed a lot of facets of digital marketing analytics, namely how to develop a robust digital marketing strategy while prioritising data compliance. With Matomo, you can protect user data and respect user privacy while gaining invaluable insights into user behaviour. Save your organisation time and money by investing in a web analytics solution that gives you the best of both worlds. 

    If you’re ready to begin using ethical and robust digital marketing analytics on your website, try Matomo. Start your 21-day free trial now — no credit card required.

  • FFMPEG : RTSP re-stream dies randomly

    14 mai 2018, par stevendesu

    I have a security camera streaming RTSP, and I wish to re-stream this to an RTMP ingest server. For now I’m using my laptop as an ffmpeg proxy, but eventually I’ll use a raspberry pi or something similar (cheap/small)

    Here’s the command I’m using (pretty simple) :

    ffmpeg -i rtsp://@10.0.0.16:554/1/h264major -c:v libx264 -c:a none -f flv rtmp://output/camera_stream

    This works but after a minute or two the stream dies. Here’s the output :

    ffmpeg version N-90057-g7c82e0f Copyright (c) 2000-2018 the FFmpeg developers
     built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.6) 20160609
     configuration: --prefix=/home/sbarnett/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/sbarnett/ffmpeg_build/include --extra-ldflags=-L/home/sbarnett/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/home/sbarnett/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libspeex --enable-nonfree
     libavutil      56.  7.101 / 56.  7.101
     libavcodec     58. 11.101 / 58. 11.101
     libavformat    58.  9.100 / 58.  9.100
     libavdevice    58.  1.100 / 58.  1.100
     libavfilter     7. 12.100 /  7. 12.100
     libswscale      5.  0.101 /  5.  0.101
     libswresample   3.  0.101 /  3.  0.101
     libpostproc    55.  0.100 / 55.  0.100
    Input #0, rtsp, from 'rtsp://@10.0.0.16:554/1/h264major':
     Metadata:
       title           : h264major
       comment         : h264major
     Duration: N/A, start: 0.360000, bitrate: N/A
       Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 720x480, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream mapping:
     Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
    Press [q] to stop, [?] for help
    [libx264 @ 0x38843c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
    [libx264 @ 0x38843c0] profile High, level 3.0
    [libx264 @ 0x38843c0] 264 - core 155 - H.264/MPEG-4 AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
    Output #0, flv, to 'rtmp://output/camera_stream':
     Metadata:
       title           : h264major
       comment         : h264major
       encoder         : Lavf58.9.100
       Stream #0:0: Video: h264 (libx264) ([7][0][0][0] / 0x0007), yuvj420p(pc), 720x480, q=-1--1, 25 fps, 1k tbn, 25 tbc
       Metadata:
         encoder         : Lavc58.11.101 libx264
       Side data:
         cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1
    Past duration 0.999992 too large
       Last message repeated 29 times
    [rtsp @ 0x3847600] max delay reached. need to consume packet
    [rtsp @ 0x3847600] RTP: missed 48 packets
    Past duration 0.999992 too large
       Last message repeated 4 times
    frame=   44 fps=0.0 q=0.0 size=       0kB time=00:00:00.00 bitrate=N/A dup=0 drop=5 speed=   0x    
    frame=   57 fps= 54 q=28.0 size=      43kB time=00:00:00.16 bitrate=2186.4kbits/s dup=0 drop=5 speed=0.153x    
    ... (lots of similar messages) ...  
    frame= 1163 fps= 26 q=28.0 size=    1341kB time=00:00:44.84 bitrate= 245.0kbits/s dup=0 drop=5 speed=0.99x    
    frame= 1177 fps= 26 q=28.0 size=    1353kB time=00:00:45.40 bitrate= 244.2kbits/s dup=0 drop=5 speed=0.99x    
    [rtsp @ 0x3847600] max delay reached. need to consume packet
    [rtsp @ 0x3847600] RTP: missed 2 packets
    frame= 1190 fps= 26 q=28.0 size=    1370kB time=00:00:45.92 bitrate= 244.4kbits/s dup=0 drop=5 speed=0.99x    
    [h264 @ 0x38c08c0] Increasing reorder buffer to 1
    frame= 1201 fps= 26 q=28.0 size=    1381kB time=00:00:46.36 bitrate= 244.0kbits/s dup=0 drop=5 speed=0.989x    
    frame= 1214 fps= 26 q=28.0 size=    1393kB time=00:00:46.88 bitrate= 243.4kbits/s dup=0 drop=5 speed=0.989x    
    ... (lots of similar messages) ...    
    frame= 1761 fps= 25 q=28.0 size=    2030kB time=00:01:08.80 bitrate= 241.7kbits/s dup=0 drop=5 speed=0.993x    
    frame= 1774 fps= 25 q=28.0 size=    2041kB time=00:01:09.32 bitrate= 241.2kbits/s dup=0 drop=5 speed=0.993x    
    [flv @ 0x3884900] Failed to update header with correct duration.
    [flv @ 0x3884900] Failed to update header with correct filesize.
    frame= 1782 fps= 25 q=-1.0 Lsize=    2127kB time=00:01:11.64 bitrate= 243.2kbits/s dup=0 drop=5 speed=1.02x    
    video:2092kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.679417%
    [libx264 @ 0x38843c0] frame I:8     Avg QP:16.89  size: 42446
    [libx264 @ 0x38843c0] frame P:1672  Avg QP:19.54  size:  1065
    [libx264 @ 0x38843c0] frame B:102   Avg QP:23.00  size:   205
    [libx264 @ 0x38843c0] consecutive B-frames: 92.4%  0.0%  0.0%  7.6%
    [libx264 @ 0x38843c0] mb I  I16..4: 12.9% 36.2% 50.9%
    [libx264 @ 0x38843c0] mb P  I16..4:  0.2%  0.2%  0.0%  P16..4: 16.7%  0.7%  1.0%  0.0%  0.0%    skip:81.1%
    [libx264 @ 0x38843c0] mb B  I16..4:  0.1%  0.1%  0.0%  B16..8: 11.7%  0.1%  0.0%  direct: 1.5%  skip:86.5%  L0:62.2% L1:35.3% BI: 2.5%
    [libx264 @ 0x38843c0] 8x8 transform intra:40.8% inter:47.4%
    [libx264 @ 0x38843c0] coded y,uvDC,uvAC intra: 46.5% 53.0% 17.2% inter: 3.9% 8.7% 0.0%
    [libx264 @ 0x38843c0] i16 v,h,dc,p: 21% 56%  8% 15%
    [libx264 @ 0x38843c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 23% 33% 31%  1%  2%  3%  2%  2%  3%
    [libx264 @ 0x38843c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 25% 39%  9%  3%  3%  4%  5%  3%  8%
    [libx264 @ 0x38843c0] i8c dc,h,v,p: 43% 33% 21%  3%
    [libx264 @ 0x38843c0] Weighted P-Frames: Y:0.0% UV:0.0%
    [libx264 @ 0x38843c0] ref P L0: 88.0%  1.4%  6.6%  4.0%
    [libx264 @ 0x38843c0] ref B L0: 99.4%  0.5%  0.1%
    [libx264 @ 0x38843c0] ref B L1: 99.4%  0.6%
    [libx264 @ 0x38843c0] kb/s:238.73

    The camera is pretty cheap (from China) so it’s likely I’m getting bad data from it or it’s cutting out for a few seconds at a time. Ideally I would need ffmpeg to handle this well (ignore bad data, wait as long as necessary for good data to resume encoding)

    Using ffplay to check out the RTSP stream, I get output like the following :

    $> ffplay -i rtsp://@10.0.0.16:554/1/h264major
    ffplay version N-90057-g7c82e0f Copyright (c) 2003-2018 the FFmpeg developers
     built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.6) 20160609
     configuration: --prefix=/home/sbarnett/ffmpeg_build --pkg-config-flags=--static --extra-cflags=-I/home/sbarnett/ffmpeg_build/include --extra-ldflags=-L/home/sbarnett/ffmpeg_build/lib --extra-libs='-lpthread -lm' --bindir=/home/sbarnett/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libspeex --enable-nonfree
     libavutil      56.  7.101 / 56.  7.101
     libavcodec     58. 11.101 / 58. 11.101
     libavformat    58.  9.100 / 58.  9.100
     libavdevice    58.  1.100 / 58.  1.100
     libavfilter     7. 12.100 /  7. 12.100
     libswscale      5.  0.101 /  5.  0.101
     libswresample   3.  0.101 /  3.  0.101
     libpostproc    55.  0.100 / 55.  0.100
    Input #0, rtsp, from 'rtsp://@10.0.0.16:554/1/h264major':0B f=0/0
     Metadata:
       title           : h264major
       comment         : h264major
     Duration: N/A, start: 0.320000, bitrate: N/A
       Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 720x480, 25 fps, 25 tbr, 90k tbn, 50 tbc
    [swscaler @ 0x7f6bbc093180] deprecated pixel format used, make sure you did set range correctly
    [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
    [rtsp @ 0x7f6bc0000940] RTP: missed 2 packets
    [h264 @ 0x7f6bc0041080] error while decoding MB 44 28, bytestream -37
    [h264 @ 0x7f6bc0041080] concealing 95 DC, 95 AC, 95 MV errors in I frame
    [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
    [rtsp @ 0x7f6bc0000940] RTP: missed 1 packets
    [h264 @ 0x7f6bc0041080] error while decoding MB 43 29, bytestream -49
    [h264 @ 0x7f6bc0041080] concealing 51 DC, 51 AC, 51 MV errors in I frame
    [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
    [rtsp @ 0x7f6bc0000940] RTP: missed 2 packets
    [h264 @ 0x7f6bc0041080] Increasing reorder buffer to 1
    [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
    [rtsp @ 0x7f6bc0000940] RTP: missed 3 packets
    [h264 @ 0x7f6bc02c3600] error while decoding MB 27 29, bytestream -24
    [h264 @ 0x7f6bc02c3600] concealing 67 DC, 67 AC, 67 MV errors in I frame
    [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
    [rtsp @ 0x7f6bc0000940] RTP: missed 2 packets
    [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
    [rtsp @ 0x7f6bc0000940] RTP: missed 42 packets
    [rtsp @ 0x7f6bc0000940] max delay reached. need to consume packet
    [rtsp @ 0x7f6bc0000940] RTP: missed 2 packets

    Then eventually the video just freezes. The first time it froze after around 5 minutes, but I wasn’t able to say definitively if it froze the instant 44 packets were dropped or if it froze randomly later. So the second time I stared intently.... for 21 minutes. Then I got bored of it not freezing, turned to pet my cat, and when I looked back 15 seconds later it was frozen. I think it only breaks when no one is watching it.

    What I can say definitively is :

    • While running normally, M-V hovers around 0 (anywhere between -0.01 and +0.01)
    • Once frozen, M-V begins to count down into negative numbers without stopping - although at a rate slower than -1 per second
    • While running normally, aq is 0KB and vq is a positive number (I think it was 30KB or so ?)
    • Once frozen, vq is also 0KB

    It’s a really cheap camera with a crummy power supply that goes out if you breathe on it, so it’s likely the camera is going temporarily offline during this time — but I’d like ffmpeg to wait out a timeout and resume streaming when it sees the camera again.

  • Re-solving My Search Engine Problem

    28 juillet 2012, par Multimedia Mike — General, swish-e

    14 years ago, I created a web database of 8-bit Nintendo Entertainment System games. To make it useful, I developed a very primitive search feature.

    A few months ago, I decided to create a web database of video game music. To make it useful, I knew it would need to have a search feature. I realized I needed to solve the exact same problem again.

    Requirements
    The last time I solved this problem, I came up with an excruciatingly naïve idea. Hey, it worked. I really didn’t want to deploy the same solution again because it felt so silly the first time. Surely there are many better ways to solve it now ? Many different workable software solutions that do all the hard work for me ?

    The first time I attacked this, it was 1998 and hosting resources were scarce. On my primary web host I was able to put static HTML pages, perhaps with server side includes. The web host also offered dynamic scripting capabilities via something called htmlscript (a.k.a. MIVA Script). I had a secondary web host in my ISP which allowed me to host conventional CGI scripts on a Unix host, so that’s where I hosted the search function (Perl CGI script accessing a key/value data store file).

    Nowadays, sky’s the limit. Any type of technology you want to deploy should be tractable. Still, a key requirement was that I didn’t want to pay for additional hosting resources for this silly little side project. That leaves me with options that my current shared web hosting plan allows, which includes such advanced features as PHP, Perl and Python scripts. I can also access MySQL.

    Candidates
    There are a lot of mature software packages out there which can index and search data and be plugged into a website. But a lot of them would be unworkable on my web hosting plan due to language or library package limitations. Further, a lot of them feel like overkill. At the most basic level, all I really want to do is map a series of video game titles to URLs in a website.

    Based on my research, Lucene seems to hold a fair amount of mindshare as an open source indexing and search solution. But I was unsure of my ability to run it on my hosting plan. I think MySQL does some kind of full text search, so I could have probably made a solution around that. Again, it just feels like way more power than I need for this project.

    I used Swish-e once about 3 years ago for a little project. I wasn’t confident of my ability to run that on my server either. It has a Perl API but it requires custom modules.

    My quest for a search solution grew deep enough that I started perusing a textbook on information retrieval techniques in preparation for possibly writing my own solution from scratch. However, in doing so, I figured out how I might subvert an existing solution to do what I want.

    Back to Swish-e
    Again, all I wanted to do was pull data out of a database and map that data to a URL in a website. Reading the Swish-e documentation, I learned that the software supports a mode specifically tailored for this. Rather than asking Swish-e to index a series of document files living on disk, you can specify a script for Swish-e to run and the script will generate what appears to be a set of phantom documents for Swish-e to index.

    When I ’add’ a game music file to the game music website, I have a scripts that scrape the metadata (game title, system, song titles, composers, company, copyright, the original file name on disk, even the ripper/dumper who extracted the chiptune in the first place) and store it all in an SQLite database. When it’s time to update the database, another script systematically generates a series of pseudo-documents that spell out the metadata for each game and prefix each document with a path name. Searching for a term in the index returns a lists of paths that contain the search term. Thus, it makes sense for that path to be a site URL.

    But what about a web script which can search this Swish-e index ? That’s when I noticed Swish-e’s C API and came up with a crazy idea : Write the CGI script directly in C. It feels like sheer madness (or at least the height of software insecurity) to write a CGI script directly in C in this day and age. But it works (with the help of cgic for input processing), just as long as I statically link the search script with libswish-e.a (and libz.a). The web host is an x86 machine, after all.

    I’m not proud of what I did here— I’m proud of how little I had to do here. The searching CGI script is all of about 30 lines of C code. The one annoyance I experienced while writing it is that I had to consult the Swish-e source code to learn how to get my search results (the "swishdocpath" key — or any other key — for SwishResultPropertyStr() is not documented). Also, the C program just does the simplest job possible, only querying the term in the index and returning the results in plaintext, in order of relevance, to the client-side JavaScript code which requested them. JavaScript gets the job of sorting and grouping the results for presentation.

    Tuning the Search
    Almost immediately, I noticed that the search engine could not find one of my favorite SNES games, U.N. Squadron. That’s because all of its associated metadata names Area 88, the game’s original title. Thus, I had to modify the metadata database to allow attaching somewhat free-form tags to games in order to compensate. In this case, an alias title would show up in the game’s pseudo-document.

    Roman numerals are still a thorn in my side, just as they were 14 years ago in my original iteration. I dealt with it back then by converting all numbers to Roman numerals during the index and searching processes. I’m not willing to do that for this case and I’m still looking for a good solution.

    Another annoying problem deals with Mega Man, a popular franchise. The proper spelling is 2 words but it’s common for people to mash it into one word, Megaman (see also : Spider-Man, Spiderman, Spider Man). The index doesn’t gracefully deal with that and I have some hacks in place to cope for the time being.

    Positive Results
    I’m pleased with the results so far, and so are the users I have heard from. I know one user expressed amazement that a search for Castlevania turned up Akumajou Densetsu, the Japanese version of Castlevania III : Dracula’s Curse. This didn’t surprise me because I manually added a hint for that mapping. (BTW, if you are a fan of Castlevania III, definitely check out the Akumajou Densetsu soundtrack which has an upgraded version of the same soundtrack using special audio channels.)

    I was a little more surprised when a user announced that searching for ’probotector’ correctly turned up Contra : Hard Corps. I looked into why this was. It turns out that the original chiptune filename was extremely descriptive : "Contra - Hard Corps [Probotector] (1994-08-08)(Konami)". The filenames themselves often carry a bunch of useful metadata which is why it’s important to index those as well.

    And of course, many rippers, dumpers, and taggers have labored for over a decade to lovingly tag these songs with as much composer information as possible, which all gets indexed. The search engine gets a lot of compliments for its ability to find many songs written by favorite composers.