MRTG Statistics Plots for Selected Devices at Semi-Detached Bedlam

This page will contain statistics plots for some of the equipment at Semi-Detached Bedlam, as generated by Tobias Oetiker's Multi-Router Traffic Grapher, MRTG.

Wherever possible, they will be regularly updated via a cron job, normally at midnight UK local time.

I'll also use this page as a diary of my experiments with mrtg - filed in reverse time order, that is, newest first.

Offset from UTC for Tux, as reported by the NTP programme

<MRTG plot of NTP offset for Tux>

Note:- the offset shown here has a 10ms bias added, in hopes of keeping the plotted result positive. This keeps mrtg happy.

Here's an mrtg plot of data flow rates on the ADSL port of the Vigor router. There was a definite correlation between increased ntp offset and download datarate, before the last speed upgrade. Now there are seemingly random excursions, not correlated with what I'm doing.

<MRTG plot of ADSL data flow>

The blue line is the uplink datarate, and as you can see it's nearly unnoticeable. I've added another mrtg logging function, to log the uplink data only (see below) That is working fine, and I'll display it from tomorrow.

<MRTG Plot of ADSL Uplink data flow>


For a little fun with different timescales, look at this -

 local 
 UTC 
 GPS 
 Loran 
 TAI 

which I cribbed directly from Tom van Baak's website, on this page. It needs JavaScript.

Tom is even more of a time-nut than I am - he's had longer to attain that status, and he has probably the best-equipped time lab outside government facilities anywhere in the world. Jane would never let me do anything similar.

Regardless of that, what you see here is the current time (as known by your computer) displayed in multiple different timescales. Local time is obvious, UTC (formerly GMT) is the time as measured near Greenwich (one can no longer say "at Greenwich" due to the redefinition of the Prime Meridian) according to the international timescale, using the second of atomic time, with leap seconds added (or removed) as required to keep UTC within 0.9 seconds of the Earth's rotation, GPS time is Atomic Time with a zero epoch at 1980 January 1st at Midnight, TAI is the Atomic Timescale, maintained inviolate (no leap seconds)

We therefore see that UTC, GPS and TAI run at the same rate, with offsets - TAI is 33 seconds ahead of UTC, GPS is 14 seconds ahead. I am currently (2006-04-17) unsure of the significance of the Time Of Coincidence in Loran time. This only applies to a particular chain of LORAN stations - the one with a GRI (Group Repetition Interval) of 9940 microseconds.

Later: ToC is the epoch of time coincidence between a UTC second and the first pulse (of 8) in a group from one particular LORAN station  With appropriate accounting for all delays, it is quite possible to obtain microsecond accuracy from LORAN-C.


Wednesday 10th March, 2008

I've not commented here of late, since I haven't changed anything major on the mrtg front, but it has been noticeable that when bitorrent is running, especially when uploading at my selected maximum, ntp performance has been abysmal - wild variations of ±10 milliseconds or more on short timescales.

But now that bittorrent has quieted down, stability and calm has returned - except when someone outside Semi-Detached Bedlam does something. At which point, I get an error spike.


Friday, 15th November

I've changed the mrtg logging configuration somewhat. I've started a router monitor, which will display the last 30 days of activity. This, of course, won't be useful until next month, so I've adjusted the existing logging function. You do this by appropriate setting of the HWidth parameter, which controls the width of the generated image. I've been using 576 pixels, which gives exactly 2 days in the "Daily" graph, but temporarily I've reduced it to 360 pixels, for exactly 30 days in the "Monthly" graph. I'll keep it at that until the new plot (at the same resolution) has been running for a month.

All this fiddling is needed because mrtg calculates averages across the visible span of each plotted image - so for a rolling 30 day average, I need 360 pixel wide images. And I need that 30 day average to make sure I don't go over Demon's Fair Use cap. Currently, I've just dodged the bullet - download rate peaked at 184.4 kbit/sec on the 30 day average, but is now falling again since I choked off rtorrent on Tux.


Friday 29th June, 2007

I've implemented the extra stanza in mrtg to monitor the ADSL uplink datarate in a more detailed fashion, without the much larger downlink datarate hiding it. Basically, all I did was clone the ADSL stanza in mrtg.conf, change the target name, to make it a unique instance, and add a

Options[target]: noi

line. This makes mrtg monitor the target, but suppress display of the first (by convention input) variable. You can suppress the second (conventionally output) variable by saying noo in the Options line. It makes no sense to use both options.

This option does not suppress logging, it merely stops mrtg displaying the parameter.


Saturday 16th June, 2007

My ADSL connection is asymmetric (of course it is, by definition) - 7616kBit/sec down, 448kBit/sec up. But mrtg plots both downlink and uplink on the same (dynamic) scale, sized for downlink bandwidth. Which means the uplink can be nearly invisible, being within delta of zero.

There is a configuration directive that allows you to specify different downlink and uplink datarates - instead of saying MaxBytes[interface]: nnnn, you say MaxBytes1[interface]: xxxx  for the input (downlink) and MaxBytes2[interface]: yyyy for the output (uplink) So I did this, hoping to see two scales for datarate - one for downlink and one for uplink.

But no, all that happens is that mrtg plots a line showing the maxbytes for the slower interface on the same graph as the downlink, and that graph is still scaled for the faster datarate. It begins to look as though I'm going to need a new stanza in mrtg.conf, which will plot the uplink bandwidth separately.


Tuesday 26th December, 2006

When I got home this evening - remember it was a short day - I decided to implement snmp monitoring of the Breezenet AP-10. So I followed the usual routine - cfgmaker, edit the resulting file, insert it into mrtg.conf, and wait 2 update cycles, while mrtg complains by e-mail that it can't find various log files. And it works. Traffic is negligible - just over 1kbit/sec on a 5 minute average - but I can see it.


Sunday 24th December, 2007

I screwed up the permissions on the Breezenet MIB file - I found an even dozen e-mails from cron this morning, complaining about permissions. The file was in the right place, and owned by root - which is correct - but had mode 700, which is wrong. It should be root-owned, world-readable (744). A quck chmod fixed that, and the complaining e-mails stopped.


Saturday 23rd December, 2006

Postie knocked on the door this morning, with a padded envelope from the USA, marked USPS Global Priority Mail. Inside that was a UPS envelope, and inside that was the Breezenet cable, in a foam sleeve. Can you say "Overpacked"? I can.

So let's see if we can do some configuration... But the Sony Vaio doesn't have HyperTerminal. WTF? Also the D-Link wireless card doesn't do WPA encryption. Drat! But that last can wait...

OK, now let's get at the Breezenet kit. AP first.

After installing the USB serial adaptor (COM12!) I can talk to the AP-10 via serial. No wonder I couldn't see it - an IP address in the Class A non-routable netblock. But I can change that - and then I can ping. But snmpwalk from Tux doesn't work.

OK, let's try the Alvarion (formerly Breezecom) SNMP utility - which swears bitterly, "This unit has old software. I may not work properly." And sure enough it doesn't.

But I can get newer software. The installed version is 5.0.63 - the latest is 5.1.57. Flashing is done via tftp over the ethernet, and the upgrade kit includes the tftp application - command line only, for Windows.

Which worked. The AP-10 now plays nicely with the Alvarion snmp application - at least, I can view settings, but due to a password funny, I can't change anything remotely.

In version 5.0.63 there was no WEP encryption - I suspect it's a paid-for option.

WEP encryption in Breezenet V5.1.57 also seems to be a paid-for option - it's still not available. And I must have mistyped the installer password for the AP-10, because I can no longer get in to change things via the serial port. But snmp works from Tux - I can read the AP-10's status, and change some things. Breezenet/Alvarion claim to offer a private MIB for the box, which includes an option to change the Installer password, but I can't see it.

So I went looking for a MIB file via Google. And found one. It's a bit convoluted getting it installed, but prvmib.txt is now sitting in /usr/share/snmp/mibs, where snmp wants to see it.

Then I set the AP-10 up in the study, plugged in where the D-Link DP-802 used to be, and changed my attention to the SA-10 Station Adaptor. And once I'd played with the settings, I got it to associate with the AP-10. So I called that a result, and went to bed.


Wednesday 18th December, 2007

The BreezeNet AP-10 802.11 access point arrived by post this morning. 7 days from the US - not bad, especially just before Christmas. Next question - when will the out-of-band setup cable arrive? Knowing my luck - after Christmas: they haven't sent e-mail saying they've shipped it.


Tuesday 12th December, 2006

The week-before-last I mentioned the Breezecom 802.11 kit which I picked up at the radio emporium's Open Day. The two boxes I bought are different types of station unit (akin to wireless cards for modern computers, except that these present as 10BaseT Ethernet). Ebay had an auction for the corresponding access point, which I bid on, and won. So that's winging it's way from the US. And transatlantic shipping is horrendously expensive. The AP cost me $16 and change - shipping is $25. Admittedly, the box (which is the size of 2 cigarette packets side-by-side) weighs about a pound.

But now I've found a source of the pukka out-of-band setup cable (custom 3-pin connector to D9 serial) at $12. And the company wanted to ship by Fed Ex at a cost of $70. No way am I paying 6 times the goods value to have it shipped to me. But I managed to convince them to post it - even so shipping works out at $25 again. For a 1 metre (max) cable? Which probably weighs about 2 ounces? Somebody's laughing here, and it ain't me. But no-one else has stock - and people who claim to have it are fairly thin on the ground, anyway.


Saturday 17th June, 2006

I've added another upload to the nightly stats image upload, to show the data flow over the ADSL. The image you will see until midnight tonight is a manual upload, and is out-of-sync with the ntp plot. After tonight, all will match.


Friday 16th June, 2006

With the arrival of the Vigor 2800 ADSL router, I need to get data flow statistics from it.

First, I had to enable the snmp user agent on the router, and set a community name - not the default. Then I pointed the mrtg cfgmaker Perl script at it. This produced a script that I could copy bits from, to get mrtg to see the Vigor router. Then it was a matter of editing the raw script to make the labels more meaningful - by default, mrtg creates stanzas in the config file that are named for the IP address of the device monitored, with suffix numbers if there is more than one monitorable thing. The Vigor returned 4 items, 2 of which mrtg said were unreasonable to monitor - "zero speed". Once I could see reasonably named web pages, I decided to perform the acid test - get quantities of data, and see if the graphs reflected the data flow. And they did! Result.


Sunday 16th April, 2006

I need to arrange a way to mechanise the upload of mrtg graphics to this page. But first, let's see if I can do it manually with the command-line ftp client.

Later: Having proved that manual ftp direct to the appropriate location works, I need an easily scriptable way of doing it. Using the command line ftp client would probably involve my learning the expect scripting language, but Debian provides a Perl programme ftp-upload, which is designed purely for scripted applications - like invocation from cron.

To be able to tell whether this works, I tried to add a couple of options to mrtg.conf, which would allow the images to be timestamped. The documentation lists a TimeStrPos directive, but mrtg fails to run if I put this in. So, at least for the moment, images will not have a time string embedded in them.

Later still: OK, I've tested ftp-upload manually, and it works. The exact string I used is now in my crontab, timed to happen at 1 minute past midnight. So all else being equal, the ntp offset graph from Tux should be posted automatically every night.

For reference, the command string is -

echo <pass> | ftp-upload -s -h homepages.demon.co.uk -u g8kbz -d images /var/www/mrtg/tux_ntp-day.png

Echoing your password into ftp_upload with the -s option is considered safe from anyone snooping in the process list while this is going on. -s tells ftp-upload to take a password from stdin - in this case the piped echo command. The other options to ftp-upload are, in order, remote host, username at that host, destination directory on that host and local filename to copy.


Saturday, 15th April 2006

Tux still isn't plotting the correct time offset, even though his plots are running fine - the ntp plot is a straight line at my selected bias level - 10msec. This is an improvement over earlier this week, when he wasn't plotting at all.

But why that fixed level?

Maybe trying to tell the getntp Perl script which machine to look at via a command-line variable passed in isn't working.

Nope, it's not that. Let's run the getntp script in isolation - and it reports the bias offset only. OK, the problem must be in this script.

Let's make a different version, and put some print commands in for debugging.

This script works by calling the ntpq comand and parsing the resulting data dump, which consists of about 30 name:value pairs dumped to standard out. It's people readable, which is good, because you have some hope of understanding it.

Here's the script, as published by David Taylor , wth my comments to the right -

#! perl usual *NIX invocation, to set the interpreter for this script
$ntp_str = `ntpq -c rv`; query ntp with appropriate options
$val = (split(/\,/,$ntp_str))[20];                       split the reply string into comma-delimited name=value pairs and choose the 20th. This is wrong - we should choose the 19th
$val =~ s/offset=//i; cut the "offset=" off the front of the chosen name=value pair
$val = int ($val + 100); Add in the display offset in milliseconds - David uses 100, I use 10 - and force it integer. mrtg dislikes non-integers
if ($val < 0) {
$val = 0; disallow negative values - mrtg doesn't like them
}
print "0\n"; First return value, normally incoming bytes count - unused here
print "$val\n"; Second return value, normally outgoing bytes count - this is the number mrtg needs. It wll display as a blue line
print "0\n"; string - device uptime
print "0\n"; string - device name

and here's a sample of ntpq output, as invoked with the options above

assID=0 status=06f4 leap_none,
sync_ntp, 15 events, event_peer/strat_chg,
version="ntpd 4.2.0a@1:4.2.0a+stable-2-r Fri Aug 26 22:10:10 UTC 2005 (1)"?,
processor="ppc", system="Linux/2.4.17_mvl21-sandpoint", leap=00,
stratum=3, precision=-19, rootdelay=124.487, rootdispersion=138.648,
peer=36052,refid=158.152.1.76,
reftime=c7ebd719.8ef1ed17  Sat, Apr 15 2006 21:33:29.558, 
poll=10,clock=0xc7ebd9b1.bd359791, state=4, offset=18.884, 
frequency=19.394,noise=28.691, jitter=39.464, stability=64.435

That tightly packed block of info is actually issued on one line - I've put in line breaks to avoid horizontal scrolling.

The split line in the script breaks the data dump into tokens at each comma, and then chooses the 20th item. The next line cuts the 'offset=' text off the front of the result, and then we can do a little maths on it and print it to standard out, where mrtg can use it. The trouble is, if you count the name:value pairs, the 20th is 'frequency'. 'offset' is number 19.

Once I changed that, I started getting significant data. But the short term variation is fierce. That sample data dump is showing an offset of +18.8 msec.

Here's the plot

which was taken at 21:55 tonight. The first dip to zero at about 19:00 was when I started, and getntp.pl failed to run. At about 19:45 I got it right.

Later, I'll try to mechanise the posting of these stats. But it's now 10 p.m., and I've been up since 5 this morning. So it's me for bed.


Sunday, 9th April 2006

I've got mrtg working, and the network traffic plots for Tux look reasonable. I just need to make the labelling more meaningful.

ntp statistics are a different thing altogether. mrtg does not understand negative numbers - after all, you can't have a negative number of bytes flowing across an interface. There are two ways to kluge this - one is to plot positive and negative offsets in different colours, the other is to plot offsets with a fixed bias, so that the numbers are always positive. David Taylor at Satsignal.net does it by applying a bias, so I cribbed his setup. Literally so - at the moment, my plots purport to be graphing data from one of David's computers. They aren't, of course, it's just the labelling, which I will correct once I've confirmed that all is working well. Pretty comes after working.