Wednesday, September 2, 2015

Refactoring Express Router Middlewares

This past weekend I went back to an old comic to fix some bad code that it used. It had been bothering me for a long time. I knew it was not great when I made it, but since it was a bit of a rush-job (to capitalize on the sudden popularity of a dress color meme) I just lived with it. But no more! It had to be cleaned-up!

The problem with it was that the comic in question stored data about how people interacted with it, but never persisted that data to disk. Every time I re-started the comic application, the previous data was lost. No problem, I just put in a timer that wrote the data as a JSON string to disk every 30 minutes. When the application starts, it reads the file.

But when I was adding this stuff, it bothered me how all the code for the routes for this comic were scattered about in the main app.js file. They really should have been put in a separate file and configured as a router. Again, no problem. I moved the four routes into a separate router module and included it from app.js.

But then I noticed something else that bothered me. I had a bunch of router middlewares now that all used very repetitious blocks of code. Seemed like it was time to make more improvements.

The code originally looked like this.

//------------ set up routes for /data/*

app.use('/data', dataRoutes({
    express: express,
    auth: ensureAuthenticated,
    dataSource: cfact
}));

//------------ set up routes for /memeGen/*

app.use('/memeGen', memeRoutes({
    express: express,
    config: conf
}));

//------------ set up routes for /fb/*

app.use('/fb', fbRoutes({
    express: express,
    config: conf
}));

//------------ set up routes for /colors/*

app.use('/colors', colorRoutes({
    express: express
}));

//------------ set up routes for /images/*

app.use('/images', imgRoutes({
    express: express,
    auth: ensureAuthenticated,
    dataSource: cfact
}));

All the middlewares were passed a very similar object into their constructor, but it was not always identical. Some needed only a single item, some needed three. But they all overlapped. I decided to create the options object only once and pass it to all the middlewares. They could then use whatever they needed and ignore what they didn't. But instead of copying-and-pasting a block of code whenever I want to add another one, I created an array of their names and a function to set everything up. Now I have this:

var express = require('express');

var routers = ["data", "images", "memeGen", "colors", "fb"];

var app = express();

// ... other stuff ...

(function setupRouters() {

    var opts = {
            express: express,
            auth: ensureAuthenticated,
            dataSource: cfact,
            config: conf
    };

    routers.forEach(function(val) {

        try {
            console.log("loading router [" + val + "] ...");
            var r = require("./routers/" + val);
            app.use("/" + val, r(opts));
        } catch (e) {
            console.error("error loading router [" + val + "] - " + e);
        }

    });

})();

With the array of router names configured, I create and call a function which goes over that list and creates each router. It sets each one up with the path to match its name. It's now very easy for me to add new router middlewares.

I always enjoy a good refactoring. Much more than I enjoy taking a picture of myself (see today's comic).

Amphibian.com comic for 2 September 2015

Monday, August 31, 2015

Restoring a MySQL Database

If you don't test your backup and prove that it works, you don't have a backup.

I've been backing up my Amphibian.com database for some time now, but haven't really had to restore it yet. That changed today. Thankfully, it worked.

This process started because I finally decided to get back to work on switching from MySQL to Postgres. I started this months ago but got sidetracked. To really make some progress, I wanted to set up a virtual machine that will run both the MySQL and Postgres versions of my database on Ubuntu 14. Then I can make my code changes and test against the VM until I know it's all working. I had both databases installed on a physical server in my home, but that limited me to only being able to work on it while I was at home. With a virtual server, I can run it on my laptop as well.

What does this have to do with restoring a MySQL database? I'm getting to that.

After installing VirtualBox on my laptop, installing Ubuntu 14 on a virtual machine, and installing MySQL on Ubuntu 14, it was time to test out one of my Amphibian backup files.

I make the backups using mysqldump on the "real" server (see one of my previous posts for more details):

# mysqldump amphibian > amphibian_data.dump

I copied the backup file off my server and onto my new virtual machine. Now I had a fresh install of MySQL and a .dump file of the Amphibian database. All I had to do was create the amphibian database on my new installation and have MySQL read the contents of the .dump file into it.

# mysql -uroot -p<password>

mysql> create database amphibian;
Query OK, 1 row affected (0.02 sec)

# mysql -uroot -p<password> amphibian < amphibian_data.dump

It ran for a few minutes, but when it finished I had a perfect copy of my Amphibian.com database. If I had to restore my actual server, I am now sure that it will work.

And of course, after spending all that time getting ready to work on the MySQL-to-Postgres transition, I didn't have any time left to actually work on it. But I'll be ready next time. Next time.

Amphibian.com comic for 31 August 2015

Friday, August 28, 2015

Comic Formulas

No technical post today, just me getting some thoughts out there about what types of comics seem to work out the best. I've had a lot of time to think about this over the past 13 months.

Four panels seems to be the best length for me. Particularly, I am fond of a 2+1+1 pattern in which the frogs talk back and forth for two frames, a third frame cuts away to another scene, and then in the final frame a punchline is delivered. You'll notice that pattern a lot in my work. I've done some as short as three panels and as many as 8, but four seems to be my favorite.

Obviously, I do a lot of work with puns. I like puns. I think everyone should.

Do you get it?
Over time, I've tried to back away from the really technical jokes. I had a set of comics about the frogs trying to get accepted into the Order of Log N, but I've never published them. There was a log that guided them through a series of tasks related to efficiency of algorithms. You know, O(log n). No, probably not. Unless you are a computer scientist or software engineer you probably have no idea what that means. I'm not saying that I write jokes specifically for an audience. I write the comics for myself, but part of that is my desire to make people laugh. If more people are confused than amused, I don't feel I've accomplished my goal. It's a complicated feeling.

The comics that I like the least are usually the most well-received. I get more compliments about comics that I almost deleted than any others. I'm not sure what's going on. Maybe I over-think things. Maybe I'm too critical of myself. Maybe my sense of humor is shared by no one else in the world.

I have to write what I know. The whole point of Amphibian.com is that the frogs are trying to master and profit from the latest technologies. As a software engineer who was self-trained from age 10 until entering college in 1997, that's what I know. It's what I still do. It's probably what I'll always do. My kids want me to write comics about soccer or gymnastics because that's what they know (I did a game about frog soccer though). I have to explain to them that my work expresses my own experiences and encourage them to write their own comics about their experiences. I am trying to train them to take over writing Amphibian.com someday, but I can't convince any of them to become software engineers.

Today's comic is about something else I know - answering tech support questions. Besides doing this for family members since 1991, I used to work at a small Internet provider back in the days when Windows didn't always come installed with Internet Explorer or even TCP/IP. Helping people over the phone was tough, but there were worse parts. I once had to re-install Windows on a customer's computer because it would no longer boot. After they took the computer home, they called in furious at me because they couldn't find Windows anymore. I asked what was on their screen. Just a little picture that said "My Computer" and another one that said "Recycle Bin." I told them that they were looking at Windows. They called me a liar. I decided that I should get a real job.

Amphibian.com comic for 28 August 2015

Wednesday, August 26, 2015

Pick a Pack of Assets - for Phaser

If you don't care for asset packs,
how about fanny packs? No? Okay.
I performed one more refactor of my Frog Soccer game. In preload, I used to have lots of individual calls to load different types of assets - static images, sprite sheets, audio, and maps. While looking for a way to improve this, I discovered that Phaser supports loading asset packs.

An asset pack is just a JSON-format file that describes sets of game assets. This is then given to the Loader which parses it and then performs all the loads. It is functionally equivalent to putting the individual game.load.image, game.load.spritesheet, game.load.audio, etc. calls in preload. Loading from a pack is just more convenient.

To use it, you first have to make a pack file. It is a JSON file, and that means it must be valid JSON. Unlike in JavaScript where you can leave off the quotes around the field names, a real JSON-format file should always quote every string.

// this is NOT a valid JSON file:
{
    main: [
        {
            type: "image",
            key: "goal",
            url: "assets/images/goal.png"
        }
    ]
}

// but this IS a valid JSON file:
{
    "main": [
        {
            "type": "image",
            "key": "goal",
            "url": "assets/images/goal.png"
        }
    ]
}

The format of the pack files is detailed in the example file located in the Phaser source code: assetPack.json. It supports arbitrary grouping of assets so that you can make sets of assets for different levels of your game, for example. In my game, there is only one "level" so I just put everything in array referenced by the name main.

{
    "main": [
        {
            "type": "image",
            "key": "goal",
            "url": "assets/images/goal.png"
        },
        {
            "type": "image",
            "key": "goal2",
            "url": "assets/images/goal2.png"
        },
        {
            "type": "image",
            "key": "net",
            "url": "assets/images/net.png"
        },
        {
            "type": "image",
            "key": "net2",
            "url": "assets/images/net2.png"
        },
        {
            "type": "image",
            "key": "tiles",
            "url": "assets/images/field-tiles.png"
        },
        {
            "type": "spritesheet",
            "key": "ball",
            "url": "assets/sprites/ball.png",
            "frameWidth": 45,
            "frameHeight": 45
        },
        {
            "type": "spritesheet",
            "key": "frog",
            "url": "assets/sprites/frog.png",
            "frameWidth": 79,
            "frameHeight": 60
        },
        {
            "type": "spritesheet",
            "key": "frog2",
            "url": "assets/sprites/frog2.png",
            "frameWidth": 79,
            "frameHeight": 60
        },
        {
            "type": "audio",
            "key": "whistle",
            "urls": ["assets/audio/whistle.mp3"]
        },
        {
            "type": "tilemap",
            "key": "map",
            "url": "assets/ground.json",
            "data": null,
            "format": "TILED_JSON"
        }
    ]
}

The basic format is that you specify one or more fields consisting of arrays of objects. I have defined one named main. The objects inside the array represent the individual assets that you want to load. Each type has slightly different fields which essentially map to the fields of each type's loader function. They are all listed in the example file I mentioned earlier if you want details . One interesting thing to note is how audio assets use an array of URLs instead of a single one. This is because you often specify a single audio asset in multiple formats to support all web browsers. The rest of the stuff in the file is fairly self-explanatory.

With this in place, my preload method just looks like this:

function preload() {

    game.load.pack("main", "assets/pack.json");

}

Really, that's it. The whole thing. I really like this method. I also really like frogs. Frogs being chased by angry ducks. That's not weird.

Amphibian.com comic for 26 August 2015

Monday, August 24, 2015

Clean Up Your Code

I'm getting towards the end of my work on the Frog Soccer game for my 404 page, so I thought it was about time I cleaned up the code.

I'm (maybe unfortunately) one of those people who make something work first and make it pretty second. In the first cuts of my programs, the code can be very rough. Not optimal at all. Lots of cut-and-paste repetition. Bad stuff. If I don't end up abandoning the project, I usually take another look through it and make it better.

For Frog Soccer, I cleaned up a few things today.

  1. Needless repetition. Creation of both the home team frog and the opponent frog was 99% identical code. I made a createFrog function that replaced both chunks. Same thing for the goals.
  2. Global variables. Okay, so they're not exactly global but there were a number of variables declared outside of the game functions because they were referenced by multiple of them. In some cases, this was unnecessary. Whenever possible, I replaced the code which referenced the variable in global scope with more "correct" code. Consider the following example:
function update() {

    if (ball.body.velocity.x === 0 && ball.body.velocity.y === 0) {
        anim.stop();
    } else {
        var speed = Math.min(1, Math.max(Math.abs(ball.body.velocity.x),
              Math.abs(ball.body.velocity.y)) / 200) * 9;
        if (anim.isPlaying) {
            anim.speed = speed;
        } else {
            anim.play(speed, true);
        }
    }

    // ... other update stuff ...

}
Here, the variable anim was a global reference to the ball's single animation object. The ball variable is already global, so keeping the anim variable around seemed redundant.
function update() {

    if (ball.body.velocity.x === 0 && ball.body.velocity.y === 0) {
        ball.animations.stop();
    } else {
        var speed = Math.min(1, Math.max(Math.abs(ball.body.velocity.x),
                Math.abs(ball.body.velocity.y)) / 200) * 9;
        if (ball.animations.getAnimation("roll").isPlaying) {
            ball.animations.getAnimation("roll").speed = speed;
        } else {
            ball.animations.play("roll", speed, true);
        }
    }

    // ... other update stuff ...

}
The code above is better because it accesses the ball's roll animation via the provided Phaser animation functions. It's also more clear when looking at the code that this section is dealing with ball animation. Who was to know what anim referred to before? 
  1. Timers. Ugh. This one was terrible. When the ball was tossed back in from being out-of-bounds, I had a timer to determine when "regular" play should start again. I had it figured out how long it would take for the ball to re-enter the play area based on its location and speed - and I hard-coded a timer based on that. Well, that was ugly and wrong. When playing on my phone, which ran just a little bit slower, the ball wasn't back in-bounds before play restarted, which immediately resulted in the ball being declared out-of-bounds again. Endless toss-ins ensued. And I knew when I did it originally that it was a hack, but I did it anyway. Shame on me. But I fixed it now, along with the next issue...
  2. Proper encapsulation. Sure, this is functional programming, not OO. But since the ball object couldn't make the out-of-bounds determination itself, fixing the issue of the toss-in timer would have required more duplication of code. Instead, I took the bounds check logic and put it in the ball itself. Now the update function can ask the ball if it is in or out of bounds and, depending on which play mode we're in, decide what to do.
ball.inPlay = function() {
    return (this.position.x > 40 && this.position.y > 30
            && this.position.x < 1640 && this.position.y < 800);
};

function update() {

    // ... other update stuff ...

    if (mode === PLAY_MODE && !ball.inPlay()) {

        // changes mode to TOSS_MODE and then to READY_MODE
        tossBall(ball.position.x, ball.position.y);

    } else if (mode === READY_MODE && ball.inPlay()) {

        mode = PLAY_MODE;

    }

}

I feel much better about the Frog Soccer code now, and anyone who happens to examine it on GitHub might think I'm a little less of a hack. Just a little less. I'm better than the frogs in today's comic.

Amphibian.com comic for 24 August 2015

Friday, August 21, 2015

Using Touch with Phaser

My goal for Amphibian.com was to develop it as a mobile-first webcomic. That's why it reads vertically and uses SVGs for graphics that stretch and shrink to any size. A year's worth of client browser stats, however, informs me that mobile users are still in the minority on the site. Nonetheless, I still want to make sure everything on there works on phones and tablets.

That includes my 404-page frog soccer game.

Up until now, I've been developing it as a way to learn how to use Phaser for game development and have been focused on the basics. It was more common to take input from the keyboard in the examples so that's all I've been doing. But today I found it was very easy to add touch controls - at least in my game.

There are two places where the keyboard is used in my game.
  1. Hit the space bar to start playing
  2. The arrow keys move the frog
Starting the game with either the spacebar or a tap (or mouse click) was simple:

function update() {

    // ... other update stuff ...

    if (mode === WAIT_MODE && (spacebar.isDown || game.input.activePointer.isDown)) {
        mode = PLAY_MODE;
    }

    // ... other update stuff ...

}

In Phaser, game.input.activePointer normalizes input from the touchscreen and the mouse, and its isDown property is an easy way to check if any button has been clicked or the screen has been touched in any way. If you wanted to get more specific with mouse input, there are other properties that represent the individual buttons. With touch input, you have the option of reading location data from multiple pointers as a way of supporting multitouch, but none of that was necessary for my game. I just want to know that you've touched or clicked to start the play.

The other type of input, from the arrow keys, seemed like it might be more difficult to reproduce with touch input. Not at all! I was impressed with how simple it was to direct the frog based on touching the screen.

function update() {

    // ... other update stuff ...

    if (mode === PLAY_MODE) {

        if (game.input.activePointer.isDown) {
            game.physics.arcade.moveToPointer(frog, 180);
        }

    }

    // ... other update stuff ...

}

Basically, the code above just checks to see if the user is touching the screen or holding down a mouse button. If they are, the frog is directed towards wherever the pointer is located. For mouse input, that's the cursor location. For touch input, it's the location of your finger. Couldn't be easier!

With just these few lines of code I was able to touch-enable my game. I'm still struggling with the canvas size and page navigation on mobile, but it is playable. Give it a try: http://amphibian.com/404/. Don't forget that you can see the complete source code on GitHub!

After playing a few rounds, make sure you read today's comic as well.

Wednesday, August 19, 2015

Loop or Map?

If I use map, can I fold it
back up when I'm done?
Here's something that has been coming up a lot lately: when processing an Array in JavaScript, should I use a for loop or the map function?

If you've looked at my code you'll notice a distinct lack of map. I don't use it very much. I honestly don't think I do a lot of Array iteration in general, but when I do, I typically just write a for loop. Good? Bad? Does it matter?

Let's look at a typical scenario in my comic code...taking an array of results from a database query and turning them into an array of data to return.

pool.query('SELECT filename, type FROM comic_img', function(err, rows) {

    var data = [];

    if (rows.length > 0) {
        for (var i = 0; i < rows.length; i++) {
            data.push({
                name: rows[i].filename,
                mimeType: rows[i].type
            });
        }
    }

}

That's the method using a for loop. Pretty straightforward. Make an empty array for the return data, loop over the rows of the query results, pushing new objects to the data array that are made from parts of the objects in the rows array.

Now let's do the same thing with the map function.

pool.query('SELECT filename, type FROM comic_img', function(err, rows) {

    var data = rows.map(function(val) {
        return {
            name: val.filename,
            mimeType: val.type
        };
    });

}

It's a little more compact, without the need for the definition and incrementation of i, comparison with rows.length, or access to the object by index. You could argue that the map version is more readable, which is nice. I do like my code to be very readable.

What you can't say is that the map version is any faster. Internally, map has to be doing a loop plus other stuff, which will undoubtedly make it slightly less performant than the basic for loop. The speed difference won't be that much different, so in most cases you shouldn't be too concerned.

There is another benefit to the map coding style. When you already have your code structured this way, it's easy to add asynchronicity. What if you just want the array processed but don't care about the order? Remember when I talked about the Node async module? It's simple to turn the plain map code into asynchronous map code:

var async = require('async');

var rows = getSomeArray();

async.map(rows, function(val, callback) {
    callback(null, {
        name: val.filename,
        mimeType: val.type
    });
}, function(err, data) {
    console.log(data);
    // do whatever you want with the resulting data
});

Instead of the processing function returning the new data, it passes it as the second argument (first would be any error that occurred) to its supplied callback function. And instead of having map return the new array, async has you specify a third argument which is a callback function. When everything is done, it gets called with the results. Sure, in this example there's not really any point to making the processing asynchronous, but if your processing involves anything with the file system or a web service call you have a perfect candidate.

That would be a lot more difficult to do with a for loop. You'd actually need several for loops. And probably some recursion. And magic. Just forget about it.

The moral of the story is, map is probably better in most cases than for loops. It's more readable and gives you a good foundation for expansion in the future. I'll try to do better at it myself. I'll also try to make better comics, but for today this is all you get:

Amphibian.com comic for 19 August 2015