Track Changing Public IP Addresses (with a Chumby)
I spent the winter break down in Sunnyvale with my parents for a month, which was somewhat unfortunate because their internet is very unstable, but only when it rains... We're talking pretty much unusable for 6-12 hours after any form of precipitation. Of course, isolating it to this correlation took a tremendous amount of work of itself, but to really finally get it fixed, we needed more data. This was because AT&T takes 4-5 days after you call to get a technician out to look at your phone line, and the likelihood of us being able to call AT&T five days before a rainstorm is... marginal...
Once AT&T got here, if they didn't see anything wrong, they were going to hit us with $55 for wasting their time (ohhh the irony). If we actually had some physical evidence to prove our theoretical correlation between rain and poor phone line quality, the likelihood of getting charged the $55 was reduced.
With the Chumby as my most recent hammer [1] [2], this certainly looked like a nail. Write a short shell script on the Chumby, which is the closest thing to a computer which is always on, to periodically log our public IP address to a text file. Then when it rains and the internet goes down, we get several hours of no public IP address, and a different one when the modem comes back online and gets a new DHCP lease from Earthlink. To make monitoring the logs easier, my dad and I worked on a CGI script on the Chumby's web server to display the logs in an easy to read manner (I needed his help with AWK, which I have never done anything more than 'print $3' with before).
This was our result. (The actual collected data can be viewed on the Chumby itself, which is finally back online after this internet connection problem and some DNS issues)
The logging script is not much more than an hourly cron job dumping the current time and the interesting line out of a wget http request to chumby.com's geoip location service.
logip.sh:
The resulting log file from this script ends up looking like this, which is a little challenging to scan quickly:
The CGI script takes this log file, and formats each pair of lines to a single line which is much easier to read. Additionally, it filters the results by eliminating all the lines that have the same ip address as the one before it by using
ipaddrlog.sh:
Note that the meat of this shell script, the awk command, doesn't have the program inline like is common for simple shell scripts, but instead references an external file, since the awk script ended up being several lines. It also uses the argument -F\", which tell awk to use the double quote instead of white space to seperate fields (the backslash is to escape the double quote from the shell). This splits the XML tag returned by geoip into five fields, so picking out the IP address is simply a matter of used regular expressions to determine if the IP address is returned before or after the country code (which is what you're supposed to actually be interested in). The date lines will always start with an M, T, W, F, or S, so if a line starts with that, we want to go to a new line and print that and the % before finding the next IP address in the log.
ipaddrformat.awk:
As you can see, each line of an AWK program consists of two parts. First, there is a regular expression condition, which tests for some specified pattern. If that pattern happens to be matched by the current line being processed, the commands inside the { } are run on that line.
Take the first line, for example. The pattern it looks for is
This is done by the second and third lines, which are effectively the same except they handle the two cases where the XML comes back with either the country code or the IP address first. For the sake of an example, we'll look at the first of the two:
The regex pattern is
Taking the first two lines of the log, you can see how this converts the log into the result shown below.
The first line starts with an S, matching the first AWK line, so it's printed on a new line with a % symbol after it. The second line has an IP address in the second field, and US in the fourth, so it matches the third pattern, and AWK prints only the second field, giving the final result for the two lines:
The final result of this exercise looks like this (which can also be seen live on the Chumby):
Luckily, we happened to get a very good AT&T tech, who didn't see anything wrong with the line tester (of course it wasn't raining that day), but climbed up the phone pole and said he didn't like how one of the wire crimps looked, so he switched us over to the spare phone line into our house and our internet has been stable ever since.
If you want to learn more about programming in AWK, the de facto book for which is The AWK Programming Language written by the three authors of the original program:
Once AT&T got here, if they didn't see anything wrong, they were going to hit us with $55 for wasting their time (ohhh the irony). If we actually had some physical evidence to prove our theoretical correlation between rain and poor phone line quality, the likelihood of getting charged the $55 was reduced.
With the Chumby as my most recent hammer [1] [2], this certainly looked like a nail. Write a short shell script on the Chumby, which is the closest thing to a computer which is always on, to periodically log our public IP address to a text file. Then when it rains and the internet goes down, we get several hours of no public IP address, and a different one when the modem comes back online and gets a new DHCP lease from Earthlink. To make monitoring the logs easier, my dad and I worked on a CGI script on the Chumby's web server to display the logs in an easy to read manner (I needed his help with AWK, which I have never done anything more than 'print $3' with before).
This was our result. (The actual collected data can be viewed on the Chumby itself, which is finally back online after this internet connection problem and some DNS issues)
The logging script is not much more than an hourly cron job dumping the current time and the interesting line out of a wget http request to chumby.com's geoip location service.
logip.sh:
The resulting log file from this script ends up looking like this, which is a little challenging to scan quickly:
Sat Dec 11 12:00:01 PST 2010
<geoip ip="99.39.7.160" iso_country="US">
Sat Dec 11 13:00:02 PST 2010
<geoip ip="99.39.7.160" iso_country="US">
Sat Dec 11 14:00:02 PST 2010
<geoip ip="99.39.7.160" iso_country="US">
Sat Dec 11 15:00:02 PST 2010
<geoip ip="99.39.7.160" iso_country="US">
Sat Dec 11 16:00:01 PST 2010
<geoip ip="99.39.7.160" iso_country="US">The CGI script takes this log file, and formats each pair of lines to a single line which is much easier to read. Additionally, it filters the results by eliminating all the lines that have the same ip address as the one before it by using
uniq -f 7, which filters duplicate lines, ignoring the first six fields.ipaddrlog.sh:
Note that the meat of this shell script, the awk command, doesn't have the program inline like is common for simple shell scripts, but instead references an external file, since the awk script ended up being several lines. It also uses the argument -F\", which tell awk to use the double quote instead of white space to seperate fields (the backslash is to escape the double quote from the shell). This splits the XML tag returned by geoip into five fields, so picking out the IP address is simply a matter of used regular expressions to determine if the IP address is returned before or after the country code (which is what you're supposed to actually be interested in). The date lines will always start with an M, T, W, F, or S, so if a line starts with that, we want to go to a new line and print that and the % before finding the next IP address in the log.
ipaddrformat.awk:
As you can see, each line of an AWK program consists of two parts. First, there is a regular expression condition, which tests for some specified pattern. If that pattern happens to be matched by the current line being processed, the commands inside the { } are run on that line.
Take the first line, for example. The pattern it looks for is
/^[MTWFS]/. The ^ means "starting at the beginning of the line", and the [...] means "one of these characters", so this pattern is looking for lines that start with any of M, T, W, F, or S. If this condition is met, the command is run, which is printf("\n%s %% ", $0). This means to print a new line (\n), print a string (%s), and print a % symbol (Have to escape it with a second one since % has special meaning for printf), and oh yeah, the string to print is $0, which in AWK is the entire line. So this gets us the date and a % sign on a new line. AWK then looks on the next line of the log file to find an IP address to place after the %.This is done by the second and third lines, which are effectively the same except they handle the two cases where the XML comes back with either the country code or the IP address first. For the sake of an example, we'll look at the first of the two:
/.*"US".*"[0-9.]*"/{printf ("%s",$4)}The regex pattern is
/.*"US".*"[0-9.]*"/. The period (.) means any character, and the asterisk (*) means zero or more of the previous character, so ".*" means any number of any character, before the first ", which is the field delimiter for this specific program (via the -F/" argument). Between the first and second ", there must be the string 'US', which obviously will break if this server leaves the country, but I don't plan on doing than, and changing it to match two of any letters would be relatively simple with ".." (match any character, twice). After the second ", there's another .*, so any number of any characters, then the meat of the pattern, which is the IP address. IPv4 addresses only consist of digits and periods, so we can say [0-9.]*, which means any number of (*) the digits 0-9 and periods ([0-9.]), which is what we're interested in. There's then another ", and that's the end of the pattern, if not the line. The command to execute for lines matching this pattern is fairly simple, being only printf("%s", $4)Taking the first two lines of the log, you can see how this converts the log into the result shown below.
Sat Dec 11 12:00:01 PST 2010
<geoip ip="99.39.7.160" iso_country="US">The first line starts with an S, matching the first AWK line, so it's printed on a new line with a % symbol after it. The second line has an IP address in the second field, and US in the fourth, so it matches the third pattern, and AWK prints only the second field, giving the final result for the two lines:
Sat Dec 11 12:00:01 PST 2010 % 99.39.7.160
The final result of this exercise looks like this (which can also be seen live on the Chumby):
Sat Dec 11 12:00:01 PST 2010 % 99.39.7.160 Sun Dec 12 09:00:02 PST 2010 % 99.169.83.64 Sun Dec 12 10:00:01 PST 2010 % Sun Dec 12 12:00:02 PST 2010 % 99.179.45.111 Sun Dec 12 13:00:01 PST 2010 % 99.39.7.106 Sun Dec 12 17:00:02 PST 2010 % 99.155.195.56 Sun Dec 12 18:00:01 PST 2010 % 99.179.45.138 Mon Dec 13 06:00:02 PST 2010 % 76.254.19.123 Mon Dec 13 09:00:02 PST 2010 % 99.35.216.67 Tue Dec 14 09:00:02 PST 2010 % 99.35.216.109 Tue Dec 14 10:00:01 PST 2010 % 99.155.195.140 Tue Dec 14 14:00:02 PST 2010 % Tue Dec 14 15:00:02 PST 2010 % 99.17.204.184
Luckily, we happened to get a very good AT&T tech, who didn't see anything wrong with the line tester (of course it wasn't raining that day), but climbed up the phone pole and said he didn't like how one of the wire crimps looked, so he switched us over to the spare phone line into our house and our internet has been stable ever since.
If you want to learn more about programming in AWK, the de facto book for which is The AWK Programming Language written by the three authors of the original program:
