Monday, November 29, 2010

Chumby Status CGI Script

After writing about my Chumby based web server this weekend, it managed to work its way onto Hacker News, which meant that my internet connection and alarm clock got to be on the sharp end of a good old fashioned slash-dotting, and considering, held up admirably well.
The statistics for the blog post itself are almost as good, but, these statistics are for a web server which I set up and which I have full control over, so I'm afraid I spent a lot of time this weekend watching `watch -n 10 tail /mnt/storage/logs/lighttpd.access.log` seeing the new hits literally roll in, some times reaching into the tens per minute.

By the end of the weekend, having to ssh into my router, then ssh into my Chumby (obviously not publicly open, due to lack of root password), then run the watch tail command got a little tired.  I decided that it was time to learn some shell tools like awk and write my first useful CGI script to allow me to monitor the Chumby's health remotely from a web browser.

A CGI script is a program available at a URL.  When a web visitor accesses this URL in their browser, it causes the CGI script or program to run, and then the resulting web page the user sees is the output of the CGI script.  As for exactly what language the script is written in is only limited by what languages the server can handle, so one could invent an entirely new programming language, and then simply start their scripts with #!/bin/interpreter or compile the code down to an executable binary, and stick it in their web server's cgi-bin folder.

Since I'm mainly handling large text files and formatted output already available through such programs as df, free, and ifconfig, it seemed most reasonable to write this status script in shell.

The simplest shell cgi script would simply use echo to print first a MIME content-type header, and then as much of that content type as one feels fit to pipe through echo, be it text/html, or text/plain, or anything else for that matter.

I decided to use text/html as my content type, with plenty of pre tags sprinkled throughout, to preserve the mono-space type of the shell output, while allowing me to use html tags to set the page title, include analytics javascript, and eventually include graphs.

The last fancy thing about my script you can notice below is the fact that it uses a cache file for all of the status output.  This is because, when I was writing it, the daily access log was nearing 40,000 lines long, so even relatively simple commands such as wc -l < lighttpd.access.log (Count the number of lines in the log file) started taking several seconds to complete.  When a single page load takes several seconds, the likelihood of a second person visiting the site while it's still working on the previous one increases, which then takes ever longer, and pretty soon my Chumby is sitting in a pool of molten lead-free solder crying.

This caching is done by using `stat` to find the age of the current cache file as a number of seconds since epoch, then subtract that from the current time.  The larger this number, the staler the cache.  By the time I finished, traffic had died down, so I decided to set the cache timeout as low as 20 seconds, which means it rarely serves the same stats twice, but does prevent someone with a hot CTRL-R finger from DoSing my alarm clock.  It also means the script can fire off the cached status for the current request, then worry about regenerating the entire output for the next visitor, meaning the results page loads much faster than it should otherwise. (There is a risk of a race condition where many people hit it when the cache is expired but before any of them finish the CGI script to update it, but the timing would be challenging, and would be a very transient CPU spike)

The problem with this caching system is that if no one visits the status page in a long time, it doesn't get run in a long time, and the cached page served to the first visitor could be horribly (several hours, or even days) old.  To prevent this, in addition to all of the visitors, I have cron run the cgi script once an hour into /dev/null to quietly update the cache for the next web visitor.

I also included the Chumby octopus from /etc/motd, which gets displayed every time you ssh into the Chumby, on the page, because it was cute.

A lot of credit has to be given to my Dad, who has been writing shell scripts much fancier than this since before I was born, who nudged me in the right direction for a lot of the scripting I needed to get done this weekend.

Wednesday, November 24, 2010

Chumby Webserver Without the Flash Drive

This summer, I bought a Chumby as my birthday present for myself.  For those who don't know, the Chumby is an internet-connected alarm clock, meaning my desk-side clock can do such fun things as scroll by my most recent email, play Pandora, and display the weather forecast.  I opted for the Chumby One, saving me a little money compared to the Classic and it better matches my style, I feel.
Click here to visit my Chumby!

Most importantly, the Chumby runs Linux.  Much like the games I've been playing with my WRT54G[L] routers, this means I can make the Chumby do so much more than it's originally presented to do.

The one that particularly interested me, for no good reason, was getting a web server setup on the Chumby.  To think that I was server a website from my alarm clock is just the kind of ridiculous idea I revel in.

The (outstanding) Chumby wiki already has excellent documentation as to how to setup lighttpd as the web server on the Chumby, but has the one short-fall of all being contained on a flash drive which has to stay plugged into the back of the Chumby for the web server to work.  What I really wanted was for all of the storage to be internal to the Chumby, so that I can still pick it up and interact with it like I normally do.

Fortunately, the microSD card inside the Chumby used to store the firmware has additional storage capacity, which is formatted and mounted as /mnt/storage/.  Unfortunately, I timed buying my Chumby when they were building them with 1GB SD cards, which leaves about 500MB for /mnt/storage/, to be shared between this web server, and all of my mp3s that I store on it to play when my internet / wifi isn't cooperating enough to listen to Pandora.  This isn't nearly enough for my taste, but I happen to have had an 8GB microSD card laying around with no use, and found instructions to upgrade the SD card inside the Chumby, which I did.

Now that I have 7GB of FLASH storage to play with, it has become only a software issue, which is easy, right?

Not so much.  The directory prefix of the /lighty/lib folder is a compile-time option, which means the binary posted on the wiki didn't work for me (There may be a config option for it, but I didn't find it).  This left compiling the entire lighttpd package from source, which turned out to not be that bad, once you get the dev environment setup, which I've become pretty good at in Linux (AVR, MSP430, WRT54G[L]).

I simply compiled the lighttpd server from source using the configure option:
instead of the
option documented in the wiki.  Once the compile was complete, I scp'd the finished binary to the Chumby to finish the install.

This suffers from the one shortfall of how to start it.  The USB-based web server used the debugchumby script in the root of the flash drive, but with no flash drive, I have no debugchumby script there to use.  Again, the Chumby wiki has an entire page devoted to documenting the different injection points for scripts to be run.  Moving the sample debugchumby script to /psp/rfs1/userhook1 got my web server to start when I rebooted the Chumby.

Prebuilt tarball of lighty to be extracted in /mnt/storage/
If you're not big into the whole wasting an hour of your life trying to get an ARM compiler working, here is my working binaries and scripts.  The easiest way to get the tarball onto the Chumby would be to load it onto a flash drive, then SSH into the Chumby and move it around from there.

At this point, I have an entirely working web server, which also logs every request to the microSD card for later analysis, if I'm interested.  As it stands, I have little more intest than just how many requests a day are handled, so to prevent the log files from growing out of control, I wrote a small log rotation script to be run by cron at the end of every day, which counts the number of lines in the lighty.access.log file, appends that and the current date to another log file, and moves the access log file to lighty.access.log.yesterday, to give me some sort of chance of reviewing what requests are being made (a lot of bots looking for wiki and wordpress admin pages).  This script can be stored anywhere, and only needs to be added to the cron table by manually adding the following to /psp/crontab/root:
59 23 * * * /mnt/storage/lighty/

Note that the crontab shipping with the Chumby doesn't default to the correct folder, so to edit the cron table requires crontab -c /psp/crontabs/ -e.

I am working on adding on to this script to have it cause a growl notification on my desktop with the day's statistics, as well as email me the raw logs, but I'm working remotely from my parent's house for the weekend, so I can't test that until next week.  Feel free to do what you like with it, and let me know in the comments.

You can visit my bedside web site here:

Friday, November 19, 2010

Switching with a Vacuum Tube

WARNING: This is a bad idea. Prototyping with 170VDC is horrible.  Never work on a live circuit, and always have protective fuses.  I blew several fuses while working on this.

This summer, I managed to get my hands on several miniature vacuum tubes at a good price, and have been rolling around the idea of playing with them.  One of my friends and I were hanging out last night with little better to do, so I decided to see if I could wire one up to use it to switch on and off an LED.
It worked!  I applied 170V to the plate of the pentode, and 3V to the heater, and applying 170V to the middle grid allowed me to turn on and off the LED with 1-3mA.  I tried connecting the other grids positively as well, but this caused a significant amount of current to flow through the tube and blew the LED.


For the record, I do not know what I'm doing here.  I managed to destroy two tubes, an LED, and two 2A fuses.  On the bright side, I never shocked myself, which is the important part.
Close-up of the connections on the tube.  Note that everything is either isolated from ground, or fused such that a short on these connections does no real damage.  The tubes were all 6x6 types, with the final one being a 6CF6.  The others had metal shrouds around them, but once I killed one of them (arced grid to cathode), I broke it open for your enjoyment.
Vacuum tube broken open with the shroud removed.  The solid plates along the outside are the plates, which catch the electrons.  Inside of that is three coils of wire to control the electron flow, and finally the heated cathode in the center, which ejects the electrons.
Finding information on the tubes was challenging online, but luckily I bought an older (1981) edition of the ARRL Handbook on eBay, since the older editions tend to be more interesting from a builder's perspective.  The older editions also have significantly more information about vacuum tubes in them.  Where the modern book only has 3 columns of tube listings and half a page of bases, the older edition had 14 pages of vacuum tube listings.

Finding the pinout of a vacuum tube is a two step process.  On the side of the tube is a part number; for example, the tube I randomly picked out of the box and used was a 6CF6.  Once you know what tube you have, you look up the base type for it:
 This goes on for four pages...
As you can see, the 6CF6 has a 7CM base.

Once you know the base type, you then look that up in another table to get the final pinout.
 This goes on for seven pages!  I guess them all having the same pinout would've been too easy...
And here is shown the 7CM base.  Looking from the bottom of the tube, clockwise from the gap is Grid 1, Cathode, Heater, Heater, Plate, Grid 2, and Grid 3.  I left grid 1 and grid 3 unconnected, treating this tube like a really poor triode.  I won't begin to try and explain the difference between the three grids and their uses, since this has been covered quite well online by people who actually know what they're talking about.

I'll say it again: Don't do this.  I did this last night because I was bored, and that's really a bad excuse.  It was fun to see an LED lit by electrons flying through the air (I also lit a Neon tube with it later), but the risk of electrocution from 170V is very high.  I've been collecting tubes and older electronics books recently (Some of the books are quite quaint, having not yet started using such units as picofarads and Hertz, opting instead for the micromicroFarad and the cycle.), so I very well might end up doing something with this in the future, but for now I'll suggest we all stick to 5V and don't die.

Monday, November 15, 2010

Lamp Controller - Phase 1

WARNING: This project uses high (120VAC) voltage, and thus should not be duplicated without a strong understanding of what one is doing, and a handle on one's compulsive need to touch things.  Not only am I in no way liable for what you do with this information, I would be VERY SAD if I found out someone had killed themselves inspired by my project blog.

ALSO NOTE: Since all of the 120VAC equipment I build plugs into the wall and is not hard wired, I in NO WAY violate the NEC, as this is all "temporary" wiring.  This means I do NOT want to read any comments about how this will void my fire insurance or how unsafe this project is, without a specific critique as to how to improve it (A choice place for improvement would be in the TRIAC triggering circuit, for which I could not find a concise application note as to how to use the TRIACs, which is why the schematic is deliberately vague there.  I've seen a lot of crazy stuff, including BJTs, capacitors, and several resistors, with no explanation as to their purpose.  I very carefully watched my circuits for the first few days and severely derate it.)

This is an on-going project involving the lighting situation in my room.  I live in an apartment which doesn't have any overhead lighting in the bedrooms, which means I have to light my room with several lamps scattered throughout the room.  This would be fine, except that the outlet controlled by the light switch is in quite possibly the least useful place based on my room's arrangement.  This means all of my lamps are individually controlled, and the light switch controls none of them.  One of them didn't even have a switch on it, so I had to hack one together from spare parts.

This post covers only the initial, crude version of this project.  At this point it does nothing other than turn the lights on and off at command.  Future enhancements will be documented as they happen.


Currently, there is individual buttons for each of the three lamps, and a circuit to monitor the switched outlet across the room.  Pressing any of the buttons will toggle the respective lamp.  A positive edge on the light switch will turn a select two of the lamps on, regardless of the current state of the lamps.  A negative edge on the light switch will turn off all lamps.  This makes the light switch useful again, as it allows it to turn on a useful set of lights on entrance to the room, and turn off all lights on exit.  If the light switch is in the wrong position (up when a positive edge is needed), all you have to do is switch it down, and back again.
The controller consists of two main parts: The AVR-based controller breadboarded by my bed, and the TRIAC junction box under my desk.

  • The AVR watches for button presses from five push buttons (only three are function as of yet, toggling individual lights), and changing edges on the input from the switched outlet.  The switched outlet powers a 5V power supply, which then feeds a 3.3V linear regulator, since this entire project runs on 3.3V.  An AC wall wart would allow for detecting the light switch being turned off earlier, since the DC power supply has filtering capacitors (thus the 100 ohm pull-down resistor), but I didn't have one handy, and would require the increased complexity of watching for changing edges and then deciding when the switch is turned off based on a lack of edges.
  • The TRIAC junction box has four independent channels to switch low current (<10A) 120VAC, using pairs of optoisolated TRIACs and power TRIACs in TO-220 packages.  TRAICs can be used to dim lights, but my optoisolators have zero-cross detectors in them, so there is no way to phase the output of the TRIACs (Informative GIF from Wikipedia).
Outside view of the junction box:
 Three of the four channels are in use.
 Screw terminals for connecting the low voltage control lines.  The box was designed to reliably run on either 3V or 5V without requiring external current-limiting resistors.

TRIAC control board:
 Each optoisolator 8-DIP contains two channels (thus why the part number ends in -2)
Standard duplex 15A outlets from ACE.  Note that I cut out the bus bar between the top and bottom outlets to separate their hot conductors, allowing me to control them individually.

Source code:

Saturday, November 6, 2010

Maximum Resistor Voltage Chart

I'm constantly trying to find a happy medium between current limiting resistors too large, causing too small of a current, and too small, exceeding their power ratings.

I finally got tired enough of iterating through one of these problems today that I fired up Octave and made this handy Resistor - Voltage chart for 1/4W resistors.
(Click to Enlarge)

Making the chart was fairly simple:

r = logspace(0, 7, 10000); 
emax = sqrt((0.25).*r);
title("Maximum Voltage Chart (1/4W)")
xlabel("Resistor Value")
ylabel("Maximum Voltage")

This is why I love Octave.