MRC Monitor™

Okay, so the idea came out of:

• not being able to follow / subscribe to a topic;
• only a limited overview of updated topics on homepage;
• having a means to keep track of updates without need to F5 / reload the frontpage;
• fun / proof of concept.

Note that this only fetches (scrapes) the (HTML of the) homepage, so server tax is at a minimum. (Note @ site admin: how about implementing/enabling HTTP compression?)

To-do (high prio):

• throttle scrape rate (more often when lot of activity & vice versa);
• monitor news as well;
• non-destructive page swap between scrape log & main page;
• some better styling/layout;
• code stability.

To-do (lower prio):

• monitor other sections as displayed on frontpage;
• topic filter;
• remember state;
• subscribe to topics;
• ...

How much time I'm going to spend on this depends on a number of factors (hint?/wink )

I've not discussed this with the MRC staff, I talked about / mentioned it a couple of times, but thought it would be fun to just drop/demo this at some point I hope they welcome/support the idea (at least till the future big update is in place )

Of course I'm monitoring behavior closely, and put it permanently on only when mature/stable enough (and I know for sure staff approves )

Around 23:30 I'll shutdown the scraper, somewhere tomorrow enable it again for some more test running. Then off again, fix some things etc.

Known bug: relative time updates go wonky.

End-user guide

• orange: there's a new post in a known topic;
• blue: there's a new post in a previously unknown (to the Monitor) topic.

Click status icon to reset status. Click elsewhere in row to visit topic.

Ideas / feedback welcome (of course).

We're running again, fixed 14m interval. You'll need to reload the page.
IE11 is unsupported btw.

First quick fix that relative time issue, seems to not update for updated topics.

Some further notes:

• to be (absolutely) clear: it's my server script that fetches & analyzes MRC's homepage at some interval;
• client requests go to my server, and get fed this analyzed/parsed data;
• avatar images *are* directly requested from msx.org;
• when (exceptionally) busy, it might miss one or more updates;
• (so) (disclaimer) it's no replacement for checking MRC('s front page) yourself, there are no guarantees it's 100% correct / up to date;
• when something unexpected happens, or there's a (possible) blank in available data, I'll try to inform the user (this is an @todo);
• I figure displaying the new comment excerpt is nice to have as well(?)

Does your monitor track new downloads as well?

@Robert: yeah it could do that. I believe those show up under 'latest reactions' (even if there are no reactions yet). Good thing you mention it, I forgot about those

@Bengalack: I figure that's the top wish of most
Definitely doable, though I'm not sure it will be an e-mail notification (at first). I'd like to look into using Push Notifications (which would still work when Monitor tab closed) first.

While it would definitely be fun to develop this further, I've also been thinking I could offer my services to MRC to enhance stuff over here, that would kinda make more sense of course. Though I would have to consider how that would fit my schedule etc. (Plus I'm kinda busy now finding me a new (paid) job (any vacancy tips? ))

The monitor will be offline until I fix some things & implement the other categories: news, downloads, T&C & wiki updates. You'll have a full update then each scrape.

Looking at the logs/traffic, it does seem some people made use of it already during the trial, so that's cool