MRC Monitor™

Door ren

Paragon (1947)

afbeelding van ren

11-04-2021, 18:16

MRC Monitor(tm)

Demo. Might be buggy.
Leave window open (/ break out) and you might see some updates show up.
I'll explain more later, diner time :)

Aangemeld of registreer om reacties te plaatsen

Van ren

Paragon (1947)

afbeelding van ren

11-04-2021, 18:29

Oops, forgot some stuff, one moment, will fix..
- upd: try again, styling/js should work now (Safari)

Van ren

Paragon (1947)

afbeelding van ren

11-04-2021, 22:09

Okay, so the idea came out of:

  • not being able to follow / subscribe to a topic;
  • only a limited overview of updated topics on homepage;
  • having a means to keep track of updates without need to F5 / reload the frontpage;
  • fun / proof of concept.

Note that this only fetches (scrapes) the (HTML of the) homepage, so server tax is at a minimum. (Note @ site admin: how about implementing/enabling HTTP compression?)

To-do (high prio):

  • throttle scrape rate (more often when lot of activity & vice versa);
  • monitor news as well;
  • non-destructive page swap between scrape log & main page;
  • some better styling/layout;
  • code stability.

To-do (lower prio):

  • monitor other sections as displayed on frontpage;
  • topic filter;
  • remember state;
  • subscribe to topics;
  • ...

How much time I'm going to spend on this depends on a number of factors (hint?/wink Wink)

I've not discussed this with the MRC staff, I talked about / mentioned it a couple of times, but thought it would be fun to just drop/demo this at some point Smile I hope they welcome/support the idea (at least till the future big update is in place Wink)

Of course I'm monitoring behavior closely, and put it permanently on only when mature/stable enough (and I know for sure staff approves Wink)

Around 23:30 I'll shutdown the scraper, somewhere tomorrow enable it again for some more test running. Then off again, fix some things etc.

Known bug: relative time updates go wonky.

End-user guide

  • orange: there's a new post in a known topic;
  • blue: there's a new post in a previously unknown (to the Monitor) topic.

Click status icon to reset status. Click elsewhere in row to visit topic.

Ideas / feedback welcome (of course).

Van ren

Paragon (1947)

afbeelding van ren

11-04-2021, 22:31

(fixed a false web filter block (sorry))

Van ren

Paragon (1947)

afbeelding van ren

12-04-2021, 12:39

We're running again, fixed 14m interval. You'll need to reload the page.
IE11 is unsupported btw.

First quick fix that relative time issue, seems to not update for updated topics.

Some further notes:

  • to be (absolutely) clear: it's my server script that fetches & analyzes MRC's homepage at some interval;
  • client requests go to my server, and get fed this analyzed/parsed data;
  • avatar images *are* directly requested from;
  • when (exceptionally) busy, it might miss one or more updates;
  • (so) (disclaimer) it's no replacement for checking MRC('s front page) yourself, there are no guarantees it's 100% correct / up to date;
  • when something unexpected happens, or there's a (possible) blank in available data, I'll try to inform the user (this is an @todo);
  • I figure displaying the new comment excerpt is nice to have as well(?)

Van ren

Paragon (1947)

afbeelding van ren

13-04-2021, 08:32

It's live again.
I think after today I'll suspend it, receive feedback, fix some things, add News (and T&C?)

Van RobertVroemisse

Paragon (1336)

afbeelding van RobertVroemisse

14-04-2021, 15:18

Does your monitor track new downloads as well?

Van Bengalack

Paladin (805)

afbeelding van Bengalack

15-04-2021, 09:34

ren wrote:
  • subscribe to topics;

This would be the one feature I would love the most. Ideally get notifications by mail just like github/sourceforge/etc Smile

Van ren

Paragon (1947)

afbeelding van ren

15-04-2021, 12:29

@Robert: yeah it could do that. I believe those show up under 'latest reactions' (even if there are no reactions yet). Good thing you mention it, I forgot about those Wink

@Bengalack: I figure that's the top wish of most Smile
Definitely doable, though I'm not sure it will be an e-mail notification (at first). I'd like to look into using Push Notifications (which would still work when Monitor tab closed) first.

While it would definitely be fun to develop this further, I've also been thinking I could offer my services to MRC to enhance stuff over here, that would kinda make more sense of course. Though I would have to consider how that would fit my schedule etc. (Plus I'm kinda busy now finding me a new (paid) job (any vacancy tips? Wink))

The monitor will be offline until I fix some things & implement the other categories: news, downloads, T&C & wiki updates. You'll have a full update then each scrape.

Looking at the logs/traffic, it does seem some people made use of it already during the trial, so that's cool Cool

Van Sander

Founder (1873)

afbeelding van Sander

16-04-2021, 23:15

The idea of a tracker did come up ages ago, but we were a little bit finished with it all back then. You're welcome to join the team and build this nice thing with us in your own pace. I can only speak for myself but I would disapprove mrc portals outside of This has a few practical reasons that I'll be happy to discus with you ( in private Smile).