Friday, December 19, 2014

Finally Backing Up

I did something today that I should have done a long time ago - set up automatic backups of my amphibian.com database.

As you may or may not know (depending on how many of my blog posts you've read before), all of the data for my web comics is stored in a MySQL database. The words coming out of the frogs mouths, their positions in each cell, and the SVG images of the frogs themselves - all stored in simple tables in MySQL. I needed to be doing backups.

I set it up with cron and mysqldump. I wrote a simple bash script that will call mysqldump to create the complete backup of my amphibian database. It looks something like this:

#!/bin/bash
fn=/path/to/backups/amphibian-$(date +%d-%b-%Y).dump
mysqldump amphibian > "$fn"

To make sure each backup file gets a unique name, I create the fn variable which includes a section made out of the formatted date, in DD-Mon-YYYY format. To the Linux date command, that is %d-%b-%Y. Then I simply call mysqldump giving it a single parameter: the database name. I redirect the output to a file of the name I made above...and that's it.

Wait, why didn't it prompt for a user name and password to dump that database? Good question! That was my initial problem - I didn't want to put the password right in the script file. I found that if I make a file in my home area named .my.cnf and have it contain a section like this,

[mysqldump]
user = username
password = pw12345

...I can use mysqldump with a default user and without being prompted for a password. As long as the .my.cnf file has 0600 (read-write only owner) file permissions, it is reasonably secure.

I set my backup script to be called once per week by cron and I'm feeling much better now. I made an additional script that automatically copies the backup files offsite to a remote server for added safety. Being prepared for catastrophic data loss will really let me sleep better tonight.

Amphibian.com comic for 19 December 2014

Wednesday, December 17, 2014

You Know, That Christmas Poem

Okay, so my comic up until Christmas Eve is clearly a parody of "A Visit from St. Nicholas." You may know it better as "The Night Before Christmas", which is not actually its title but part of the first line.

This classic poem, written by Clement Clarke Moore way back in 1823, is largely responsible for our modern ideas about Santa Claus. My poem will most likely have no cultural impact whatsoever. But it is a chance for us all to think for a few minutes about the level of effort required for Santa's operation to make it on time every year. To meet his requirements, Santa can't be up there in his workshop hand-assembling dolls. He needs to be an expert at managing a production line of impossible size, and use the most advanced techniques for both planning and execution. Knowing the current wakefulness state of every child on earth, their position on the naughty/nice scale, and their desired gift list at any moment is clearly a monumental achievement of data science that is the envy of everyone from Google to the NSA.

And we haven't even gotten to the whole delivery phase of the operation yet. Amazon can experiment with flying delivery drones all they want, but only Santa Claus has a team of flying reindeer out delivering packages at speeds which imply some sort of controlled distortion of space-time.

There's no company on earth that wouldn't like to send someone to Santa for training - he's clearly mastered things that the rest of us don't even know exist yet.

I'm not the only one considering things of this nature. As part of your Christmas preparations this year, be sure to read Are Santa’s Reindeer Used for Propulsion or Navigation?, in which an MIT graduate student delves into the interesting question using Moore's original poem as evidence.

Amphibian.com comic for 17 December 2014

Monday, December 15, 2014

Making Some Pies

Shoofly Pie by Syounan Taji
My favorite kind of pie is Shoofly Pie, a molasses pie which originates among the Pennsylvania Germans. It is very similar to the English Treacle Tart, where golden syrup is used instead of molasses, which is also one of my favorite desserts when I'm in the U.K.

None of that has anything to do with the kinds of pies I made last week (before I got sick and spent several days barely awake). I made pie charts. You know, like doughnut charts, but without the hole in the middle. Both sound delicious but can't be eaten.

Also, don't confuse charts and graphs. It is my understanding that a graph is a type of chart that uses lines. Like a square is a type of rectangle.

Anyway, I wanted to take some of my data I capture from amphibian.com and plot it in a way that makes analyzing it easy. I am most interested in how many comics are accessed per day and which types of browsers people use to access the site. I have web services that generate the raw numbers from the server, so I wanted to find a JavaScript client library that could draw pretty pictures for me.

After experimenting with a few options, I went with Chart.js. Another popular recommendation was Data-Driven Documents, D3.js, but that was really overkill for my needs. I like to keep it simple.

Chart.js lets you take JSON data and render 6 different chart types using HTML5 Canvas. You can extend it to create your own additional chart types if you want, but the ones already included meet most needs: Line, Bar, Radar, Polar Area, Pie, and Doughnut. One of its best features is that it allows the charts to be responsive. When enabled, it resizes the canvas element as the parent element changes and the chart still looks good.

Using it on a web page is easy. First, just download the library and include it on your page:

<script type="text/javascript" src="/js/Chart.min.js"></script>

Now put a Canvas element, or elements, on your page where you want to draw the chart(s). There's a tricky thing about Canvases in that they really need width and height attributes, which is fine except when we want to have responsive charts that size with the browser. When going responsive, the trick is to just set the size very small but maintain the proportions, such as 80x60 instead of 800x600 and let Chart.js size it up for you - you won't even notice.

<div style="width: 50%">
  <!-- I really want this chart to fill the div automatically -->
  <canvas id="chart1" width="80" height="60"></canvas>
</div>

All that's left to do is organize your data in an appropriate way for the type of chart you'd like to draw, and pass it to the appropriate Chart function. Let's look at a simple line chart for example, but the Chart.js documentation has plenty of detail on all the other options plus more customizations.

var chartData = {
        "labels": ["A", "B", "C", "D", "E"],
        datasets: [
            {
                label: "Something",
                fillColor: "rgba(151,187,205,0.2)",
                strokeColor: "rgba(0,51,102,1)",
                pointColor: "rgba(151,187,205,1)",
                pointStrokeColor: "#fff",
                pointHighlightFill: "#fff",
                pointHighlightStroke: "rgba(220,220,220,1)",
                data: [33, 57, 42, 88, 101]
            }
        ]
    };

var opts = {
    "scaleGridLineColor": "rgba(0,0,0,.08)",
    "datasetStrokeWidth": 3,
    "responsive": true
};
 
var ctx = document.getElementById("chart1").getContext("2d");
var myChart = new Chart(ctx).Line(chartData, opts);

Line charts need a set of labels for the X-axis, and at least one set of data for drawing a line. You can actually draw more than one line on the same chart, useful when comparing data, which is why the datasets field is an array (even though in my example it only had one element).

Each dataset object should contain a label, color-related information for the lines and fills and points, and the set of values to plot on the Y-axis. Those values should in in an array given as the data field.

An options object can be used as well. There are a number of "global" chart options but each kind of chart has some specific options you can use. I customized the color of the scale grid lines, the stroke width of the data line, and made sure to set responsive to true.

All you need to do is pass the 2D context from the appropriate Canvas element into Chart and pass the the chart data and options to the chart type (see the final 2 lines above). You get back a reference to the chart, which you can use later to update the data and redraw if you want.

A chart that is always 50% of the page width

I was pleased with how simple it was to create and customize the simple charts I needed. I would recommend you check out Chart.js if you have similar needs on a project.

Sorry if you are disappointed by the lack of actual pie making in this blog. I know, I didn't even show how I made a pie chart! I did actually make some, after I made a line chart - but I'll let that be an exercise for the reader (or you could check out the amphibian.com repo on GitHub).

Amphibian.com comic for 15 December 2014

Friday, December 12, 2014

Sweet Dreams (of the 1980's)

Whenever I hear the word "logarithm" I can't help but think of this song...


I guess it's just because word "logarithmics" sounds so similar to "Eurythmics". You can understand my confusion, right? Plus, growing up in the 80's I must have heard this song 43,000 times - long before I ever knew what a logarithm was.

While I'm at it, here's a short list of more songs that were the soundtrack of my childhood.
Someday, we're going to look at these songs the same way we look at classical music today. You know, you'll take your flying car to the symphony to listen to them perform some Bon Jovi.

Until that day, we'll have to settle for symphonetic renderings of Edvard Grieg. Or, this...


Wednesday, December 10, 2014

The Christmas Song Lyrics You Never Knew

I'm feeling a bit under-the-weather today, so instead of a post about how to use some JavaScript technology or how to make a paper frog I'd like to help you with your Christmas songs.

Jingle Bells


I would have to suspect that you have never seen more than the first verse of this song. Few people know the rest of it.

Here are the complete lyrics:
Dashing through the snow
In a one-horse open sleigh
O'er the fields we go
Laughing all the way
Bells on bobtail ring
Making spirits bright
What fun it is to ride and sing
A sleighing song tonight!
(chorus)
Jingle bells, jingle bells,
Jingle all the way.
Oh! what fun it is to ride
In a one-horse open sleigh.
Jingle bells, jingle bells,
Jingle all the way;
Oh! what fun it is to ride
In a one-horse open sleigh. 
A day or two ago
I thought I'd take a ride
And soon, Miss Fanny Bright
Was seated by my side,
The horse was lean and lank
Misfortune seemed his lot
He got into a drifted bank
And then we got upsot. 
(repeat chorus) 
A day or two ago,
The story I must tell
I went out on the snow,
And on my back I fell;
A gent was riding by
In a one-horse open sleigh,
He laughed as there I sprawling lie,
But quickly drove away. 
(repeat chorus) 
Now the ground is white
Go it while you're young,
Take the girls tonight
and sing this sleighing song;
Just get a bobtailed bay
Two forty as his speed
Hitch him to an open sleigh
And crack! you'll take the lead. 
(repeat chorus)
The rest of the song isn't quite as catchy as that first verse, huh?

Good King Wenceslas


Now I'll share the words to one my favorite Christmas songs that isn't really even a Christmas song. Of course I'm talking about "Good King Wenceslas." You hear the music to this song all the time but rarely does anyone sing the words. It's not technically a Christmas song if you pay attention to the first line: "Good King Wenceslas looked out, on the Feast of Stephen." The Feast of Stephen takes place on December 26th, but I guess that's close enough. Also worthy of note, Wenceslas wasn't ever even a king in his lifetime - the title was bestowed on him after this death. Here are the words:
Good King Wenceslas looked out
On the feast of Stephen
When the snow lay round about
Deep and crisp and even
Brightly shone the moon that night
Though the frost was cruel
When a poor man came in sight
Gath'ring winter fuel

"Hither, page, and stand by me
If thou know'st it, telling
Yonder peasant, who is he?
Where and what his dwelling?"
"Sire, he lives a good league hence
Underneath the mountain
Right against the forest fence
By Saint Agnes' fountain."

"Bring me flesh and bring me wine
Bring me pine logs hither
Thou and I will see him dine
When we bear him thither."
Page and monarch forth they went
Forth they went together
Through the rude wind's wild lament
And the bitter weather

"Sire, the night is darker now
And the wind blows stronger
Fails my heart, I know not how,
I can go no longer."
"Mark my footsteps, my good page
Tread thou in them boldly
Thou shalt find the winter's rage
Freeze thy blood less coldly."

In his master's steps he trod
Where the snow lay dinted
Heat was in the very sod
Which the Saint had printed
Therefore, Christian men, be sure
Wealth or rank possessing
Ye who now will bless the poor
Shall yourselves find blessing
I don't understand why more people don't sing the words. I've always liked them.

Amphibian.com comic for 10 December 2014

Monday, December 8, 2014

Frog Paper Craft

Today, we're not going to build anything out of code. We're going to use a more primitive medium.

You may notice that my frogs are constructed out of nothing more than simple geometric shapes. I used to draw them by hand this way, but now I draw them as Scalable Vector Graphics in Inkscape. Since SVG defines the shapes using mathematical formulas you might say that the frogs are made out of math. Yesterday my daughter wanted to make a craft, and I came up with this idea: making frogs out of paper!

I made up some templates for the parts of one of my frogs and printed them out on colored paper. We cut and glued the paper together with some cardboard and had some fun. Now I'm going to share the fun with you!

Step 0.

You'll need some thick cardboard, scissors, glue, a marker, and colored paper. Print out the templates of the different pieces on colored paper (see links below). Print the body template on green paper, the bellies template on yellow, and the eyes on white. Or use other colors if you want some more interesting frogs!

Frog Body Template
Frog Belly Template
Frog Eyes Template

Frog Body - Print on Green Paper

Frog Bellies - Print on Yellow Paper

Frog Eyes - Print on White Paper

Step 1.

Cut out all the frog parts. Get a fun color paper for your background (I use purple in my example). Glue the back eye, arm, and leg on your paper something like the following picture.


Step 2.

Glue one of the body pieces on top of the eye, arm, and leg. Glue one of the belly pieces on top of that.


Step 3.

Cut out some cardboard pieces and glue them on as pictured below. The thicker the cardboard, the better. It will give the frog some depth.


Step 4.

Glue the other body piece on top of the cardboard. Glue the other belly piece on top of that. In the picture below, I have the body slightly transparent so you can see the placement.


Step 5.

Glue the second eye, arm, and leg on top of the body and exposed cardboard pieces. Again, I use transparency in the picture to help show the placement of the parts.


Step 6.

Almost done! Get your marker and draw in the eyes and mouth. Your frog craft is complete!


Here are the ones I made with my daughters, but that was before I wrote these nice instructions. Yours should turn out even better.


Send me pictures of your complete crafts!

Amphibian.com comic for 8 December 2014

Friday, December 5, 2014

Your Internet Thermometer

Photo by Bernard Gagnon
It's amazing what access to the Internet on phones has done to us. When I wake up in the morning, I look at my phone to check the outside temperature. I remember when I was a kid and we had to use antiques like thermometers stuck on the outside of our windows!

In today's comic, the frog is trying to get the body temperature reading from another frog. He fails to realize that frogs, being ectotherms, will basically have the same body temperature as that of their environment. His phone shows him (and you, the reader of the comic) the actual temperature from my town here in central Pennsylvania. Look at the comic today and you'll see today's temperature. Come back in 6 months and you'll see a warmer temperature (I hope). Whatever temperature it is here when you load the page, that's what you'll see.

Getting weather data for my comic is not difficult. I basically do the same thing that the weather app on your phone is doing - accessing a server somewhere that has been collecting data from weather stations around the country and getting the latest information for a certain location. I have used both Weather Underground and OpenWeatherMap to get data, as both have APIs that anyone can use. In the case of my comic, I wrote a simple module for Node that I use to manage the API calls.

var http = require('http');

module.exports = function(conf) {

    var currentTemp = 50;  // start with some reasonable value

    var apiKey = conf.weatherApiKey || "";
    var location = conf.location || "";

    function pullWeatherData(cb) {

        // OpenWeatherMap
        var url = "http://api.openweathermap.org/data/2.5/weather?id="
                + location + "&units=imperial&APPID=" + apiKey;

        // Weather Underground
        //var url = "http://api.wunderground.com/api/" + apiKey
        //        + "/conditions/q/" + location + ".json"; 

        http.get(url, function(res) {

            var wData = "";
            res.on("data", function(chunk) {
                wData += chunk;
            });
            res.on("end", function() {
                try {
                    var data = JSON.parse(wData);
                    cb(null, data);
                } catch (e) {
                    cb(e);
                }
            });

        }).on("error", function(e) {
            cb(e);
        });

    }

    function weatherTimer() {

        pullWeatherData(function(err, data) {
            if (err) {
                console.log(err);
            } else {
                try {

                    // OpenWeatherMap
                    if (data.main.temp) {
                        currentTemp = data.main.temp;
                    }

                    // Weather Underground
                    //if (data.current_observation.temp_f) {
                    //    currentTemp = data.current_observation.temp_f;
                    //}

                    console.log("Setting current temp to " + currentTemp);

                } catch (e) {
                    console.log("Unable to update temp");
                    console.error(e.stack);
                }
            }
            setTimeout(weatherTimer, 1800000); // call again in 30 minutes
        });

    }

    weatherTimer(); // run now, and then every 30 minutes

    return {
        temperature: function() {
            return currentTemp;
        }
    };

};

The goal of this module is to give me access to the current temperature value and update that value automatically on a timer. Temperature doesn't usually change that quickly so every 30 minutes is fine for my application.

There are really only two functions - one that calls the API to get the data and the timer that asks for new data and updates the temperature. In my code above, I show it using the OpenWeatherMap API with the Weather Underground stuff commented-out, in case you want to try that one as well.

The first function I called pullWeatherData. It simply calls http.get for whichever service you want to use and reads the response stream, in JSON format, into a string which is parsed into an object at the end. I had to make sure error conditions were handled gracefully, because when you call external APIs you sometimes get garbage back. That's the point of the try/catch around the JSON.parse call - the response might not actually be JSON (it could be a plain text HTTP 500 page, for example) and you don't want to crash your whole app. So if there is any kind of error, the callback function given to pullWeatherData is called with the error as the one and only parameter. In the case of a successful call and parse, the callback function is called with a null as the first parameter to indicate no errors and the weather data object as the second parameter. The exact format of this object will depend on which service you called.

The weatherTimer function simply calls pullWeatherData and then does something with the data before setting a timeout for calling itself again in 30 minutes. All it really does with the data is take the temperature value out of it and store it in a variable. If everything works, that variable will be automatically updated every half hour.

After defining these two functions, my module calls weatherTimer in order to get the data for the first time and start the timer for the next call. The return value is an object with a function simply called temperature which will return the current value of the temperature variable. Here's an example of how the module would be used in an app:

var express = require("express");
var app = express();

/*
 * OpenWeatherMap uses their own location ids.
 * Id 5184309 is near Reedsville, PA.
 * 
 * Weather Underground can use ZIP codes.
 * ZIP 17084 is Reedsville, PA.
 */
var weather = require("./weather")({
    weatherApiKey: "your_key_here",
    location: "5184309"
});

app.get("/", function(req, res, next) {
    res.send("weather demo");
});

app.get("/temp", function(req, res, next) {
    res.send("current temperature is: " + weather.temperature() + " deg F");
});

// ------------ start listening
var server = app.listen(3000, function() {
    console.log("listening on port %d", server.address().port);
});

Just like the comic, this sample app will tell you the temperature here in Reedsville if you point your browser at the /temp path. As you can see, it's cold in Reedsville as I write this.


If you want, you could modify this module to get different weather data or more data and store it in a more complex object. You could really just store off the entire response object and access different values each time. You might even apply this pattern to other non-weather APIs as well. Be creative!

Amphibian.com comic for 5 December 2014