Friday, October 2, 2015

Tracking Visitors from Ads with Express Middleware

I'm getting ready to run another month of promotion for Amphibian.com. This time I'll be running an ad on TopWebcomics.com, a webcomics list and voting site.

The best list of active web comics, including mine!
Unlike before, when I ran ads on Facebook, I'd like to get a better idea about how people interact with the site after coming from an ad banner. This is sometimes called conversion tracking, but since I'm not really trying to convert any clicks to purchases I'm not sure the term applies. I am trying to convert people to regular readers of the comic, so maybe it does. In any case, I added a piece of middleware to my Express web app in order to track visitors from ads.

Amphibian.com is built using Node with the Express framework. I use various Express middlewares to add features to the site, such as access control, logging, multipart uploads, etc. I've written quite a bit about these in the past (here, here, and here, for example). I wrote a custom middleware and modified my logging middleware in order to create this ad tracking feature.

Here's how it should work... I create my ad banner and give it to a site to display, along with a URL. Instead of just plain http://amphibian.com, I assign each ad its own unique code and append that to the URL as part of the query string. For example, I tell TopWebcomics that I want my ad banner to link to http://amphibian.com/?s=abcdefg. When I see a value for s (I use "s" for "source") on the query string of any comic access, I add the value (abcdefg in this example) to the request object in a special field. Then in my logger, I look for a value in that field and write it to the database along with the rest of the request data. Now I'll be able to query my database later and determine which accesses originated with a user clicking on the banner. If I ran more than one banner at a time, I could just give them each a different code and compare their performances.

My tracker middleware looks like this:

module.exports = function() {

    var tracker = function(req, res, next) {

        var src = null;

        if (req.query.s) {

            src = req.query.s;
            res.cookie("s", src, { maxAge: 2629746000 });

        } else if (req.cookies.s) {

            src = req.cookies.s;

        }

        if (src !== null) {
            req["click-source"] = src;
        }

        next();

    };

    return function(req, res, next) {
        tracker(req, res, next);
    };

};

The code is fairly simple. First I check the query string for an s field (line 7). If present, I set the source code variable to its value and set a cookie with the same value. The cookie has an expiry time one month in the future. If the request does not have a s value in the query string but does have the cookie value, I set the source code variable from the cookie. This is how I know if you come back to the site later without clicking on the ad - you'll have the cookie. As long as you clicked on the ad once, I count the ad as being the source of all your visits for the next month.

The source code gets put on the request object in a field called "click-source" there on line 19. This is the field that I check in the logger middleware and, if found, write to the database.

One final note about using this middleware. Since it relies on the cookie-parser middleware to populate the request's cookies field, I have to make sure that I set my Express app to use the cookie-parser first. Express chains the middlewares together in the order they are declared.

var cookieParser = require('cookie-parser');
var adTrack = require('./sourceTracker');
var logging = require('./logger');

// ... other stuff ...

app.use(cookieParser()); // required before source tracker
app.use(adTrack());
app.use(logging());

// ... other stuff ...

So my plan for tracking conversions is simple, and only took a few minutes to implement. After I run this ad for a month I'll be sure to write up my results and what, if any, improvements I want to make for the next time I try a promotion. Until then, keep reading the comics by clicking on the link below.

Amphibian.com comic for 2 October 2015

Wednesday, September 30, 2015

Spider Attacks

One thing that's been bothering me ever since I started the comics on Amphibian.com is how my visitor stats keep getting skewed by spider attacks.

Not like actual arachnids, I'm talking about the automated systems that are constantly out there reading web pages and cataloging data extracted from them. All the search engines use them to index the web, but spammers use them as well - seeking out web pages with comments forms or other input fields in which they can inject links to Russian pornography sites. I get these things all the time.

While most of my traffic is legitimate, every once in a while I'll have a day where there's a huge spike in "visitors" to the comics, and 90% of them are not real people. I can tell because I record the user agent strings and then check them for legitimacy.

While I can't stop the spiders from crawling, I just want to make sure I don't count them in my stats. I want to know how many real humans viewed comics. Today I tried something new with my stats that maybe I should have done months ago but didn't think of until recently. Instead of counting accesses to one of the comic URLs directly (like http://amphibian.com/227) I'll count access to a small 1x1 pixel image with the comic number as part of the name (like http://amphibian.com/images/v/227). I put this image on every comic. Real browsers operated by humans will load the image while Russian porno spiders won't. I checked the logs, except for the Google Image Search bot, images are largely ignored by spiders.

We'll see if that helps. So if you're a human, please read my comic.

Amphibian.com comic for 30 September 2015

Monday, September 28, 2015

Our 13th Anniversary

Today marks 13 years since my wife Rebecca and I were married. While we've mutually agreed to abandon the practice of gift purchasing every year on this date, I thought it would be nice if I devote my blog today to our lives together.

Really that just means I'm going to share lots of old pictures. So here it is, our relationship over the last 15 years...


One of our earliest photos, this is Rebecca in my Cougar at the All Ford Nationals car show in Carlisle. As you can see from the date imprint (pre-digital camera here), it was June 3, 2000.


Here she is as my fiance in January of 2002.


And here we are shoving cake in each other's faces at our wedding reception. That's a weird tradition. And what's with that light being held by that person on the right? Is that a camera flash?


We used to travel to the U.K. back before we had kids.
Here she is in London back in the Spring of 2003.


And the two of us together in front of a big pile of rocks a week later.
They call the rocks Stonehenge. I don't know what you call that old jacket I was wearing, but the thing hanging around my neck is for the audio tour.


In the fall of that year, we somehow got this picture of us taken at a scenic overlook near our home. I don't know if the camera had a timer or we found someone out in the forest to help us.


We were back in the U.K. again the following year. This picture was taken in Bourton-on-the-Water in 2004. They had a lovely hedge maze which entrapped us for hours, but today could easily be thwarted by the availability of satellite imagery on mobile phones.


There was no international travel for us in 2005, as we brought home our first daughter. Here she is with Rebecca when she was about a month old.


Here is Rebecca showing off baby Alex's first Halloween costume.


And here's a family picture from early 2006. I have no idea where we were in this photo.


In this photo from the summer of 2007, we find Rebecca and a two-year-old Alex sharing an ice cream cone at a local fair.


This was the best picture I could find of the two of us together in 2008. I have a strange expression on my face, but I probably have one of those most of the time.


The following year, we added another daughter to our family. This picture shows Rebecca and Gladrielle at the Juniata County Fair in September.


In 2010, we had this lovely vacation at Walt Disney World in Florida. Above, Rebecca waits for the Big Thunder Mountain Railroad after just getting off Splash Mountain.


Here we have a wonderful family picture from that same 2010 trip. Goofy is not part of our family. But notice the frog shirts. I'm wearing Science Frog years before he showed up in a comic.


In 2011 we took two vacations. Above, you see us back at Disney World for the Star Wars Weekends. That was the Droid we were looking for.


And then later that summer we visited Corbin, Kentucky. In the photo above, we're about to get in a raft for an up-close encounter with Cumberland Falls (which produced a moon-bow for us that evening).


In this picture from the fall of 2012, we have the first appearance of our third daughter, Natasha (on the far right). We were foster parents when this picture was taken, but we would adopt her later. Yes, it is the same scenic overlook where we were photographed in 2003.


By the time this photo was taken in March of 2013, Rebecca had this wonderfully purple hair. This picture was taken at the airport...


...where we were on our way to the big OUYA party in San Francisco on March 28. In the above photo, Rebecca was waiting with me for the fun to start. Unfortunately, OUYA didn't make it (it sold to Razer earlier this year) but she still has dress she bought for the occasion.


Here is a fun family photo from early 2014.


Here we see that Rebecca is not afraid to face the Dark Side, at least when taking the kids to get autographs. We're back at Disney World for Star Wars Weekends 2014.


We also had this great family photo taken there that year. No one was frozen in carbonite.


Here she is with all our girls in April of this year, 2015. It was a little windy that day, so she couldn't keep her hair out of her face. Also, since she's standing behind Gladrielle it is not easy to notice that she is pregnant in this picture.


Finally, here is Rebecca with our fourth daughter, Kayci. This picture was taken on June 6th, 2015, almost exactly 15 years after that picture of her in my old Cougar.

And there is my photographic journey through the last 15 years of our lives together, 13 of which we've spent as husband and wife. Hopefully she doesn't hate any of these pictures, since I didn't tell her I was going to share them here.

She's also been really good about putting up with me writing these blogs and frog comics every night...

Amphibian.com comic for 28 September 2015

Friday, September 25, 2015

Faux Bold is Better Than No Bold

Today I'm going to forego another in-depth discussion of a new ES6 language feature that recently became available in Node 4. Instead, I'd like to make a brief mention of what I learned this last week concerning "faux" font styles and PhantomJS.

First of all, what are faux font styles? As I recently learned, when you use a web font and not all the special styles are available (such as bold and italic), the browser will just alter the normal version of the font to create the faux styles. As a side note, I also learned that faux has only been used in English by itself to mean "fake" since the 1980's. I am older than this usage.

So when a font has its own bold and italic styles available, you should probably use them. Typically, this is accomplished by specifying multiple @font-face directives in your CSS. Each should define font-weight and font-style appropriately.

@font-face {
    font-family: 'Whatnot';
    src: url('/css/Whatnot-Regular.ttf');
    font-weight: normal;
    font-style: normal;
}

@font-face {
    font-family: 'Whatnot';
    src: url('/css/Whatnot-Bold.ttf');
    font-weight: bold;
    font-style: normal;
}

@font-face {
    font-family: 'Whatnot';
    src: url('/css/Whatnot-Italic.ttf');
    font-weight: normal;
    font-style: italic;
}

Like the above example, if you have multiple versions of the font you should use them. They'll probably look better than the faux versions made by the browser.

I use Sniglet as my standard font for the comics and it doesn't have an italic version. I have no choice but to settle for the faux version. It's not the end of the world, but I noticed a problem. When I made a comic that actually relied on bold italic text, the text was showing up as neither bold nor italic in the .png image generated by PhantomJS. Why?

After a bit of investigation, it would appear that PhantomJS can't make faux font styles out of SVG fonts. Any attempt to show bold or italic Sniglet was being ignored for as long as I've been making these comics. The solution? Switch to the True Type version of the font. It doesn't look that great when PhantomJS renders it but it's better than nothing.

Amphibian.com comic for 25 September 2015

Wednesday, September 23, 2015

Template Strings - New in Node 4

Today on Amphibian.com, the frogs try to get past ad blockers by interleaving the text of ads with the rest of the text on their website. But since their website is their comic about their website (see Monday's topic - meta), there are problems.

Similarly, when you want to put the values of variables or other expressions in your JavaScript strings you probably do something like this:

var str1 = "some static text and " + someVar + " and some more text";

JavaScript Strings, circa 1905
A cool new ES6 feature now available in Node 4 is the Template String, which can make the above code a little nicer.

Template Strings are a new way of creating strings with embedded expressions. These expressions can be simply variables, formulas, or function calls. To make a Template String, use the back-tick ` (the thing that shares the key with ~ on an American English keyboard) instead of the regular tick ' or double-quote ". Inside, put expressions inside curly braces { } prefixed with a dollar sign $.

Here's an example.

var frogType = "Science";
var numFlies = 42;

console.log("The " + frogType + " frog caught " + numFlies + " flies");
console.log(`The ${frogType} frog caught ${numFlies} flies`);

Both messages print out exactly the same, but the second one, using a Template String, is shorter and looks nicer. I think it helps keep your code cleaner.

Another feature of Template Strings is that they can include literal new-lines. Look at this:

var str1 = "this is string 1\n" +
           "with two lines";

var str2 = `this is string 1
with two lines`;

console.log(str1);
console.log(str2);

console.log(str1 === str2); // prints true

The str2 variable is created by simply starting a new line in the middle of the string instead of embedding the \n. Also note that the two strings are equal when compared, despite being created in different ways. Personally, I'm not sold on this particular attribute of Template Strings. I think putting literal new-lines in there hurts the readability of the code, but you might not have a problem with it. Just throwing it out there!

Obviously, using this technique to include variable values in strings is useful. But you can put more complex expressions inside the braces.

var a = 2;
var b = 4;
console.log(`the product is ${a * b}`);

The code above prints the product of a and b by performing the multiplication in the Template String. But please note that the expression is only evaluated once, when the string is created. The following code exemplifies that:

var c = 3;
var d = 7;
var str3 = `the sum is ${c + d}`;

console.log(str3); // the sum is 10

d = 14;

console.log(str3); // the sum is 10

Changing the value of d after str3 has been created does not change the value of str3. Be careful, however, because the process of creating a Template String can change the values of your variables - just like if you did things the old-fashioned way.

var e = 1;
console.log(`the variable e = ${e++}`);

console.log(e); // prints 2

In the above code, printing out the Template String changed the value of e. Maybe this is what you want...maybe it's not. Be mindful.

But wait! There's more!

Template Strings also support tagging, meaning that they can be passed to a function which can modify their output. This feature is a bit unusual.

To use tagging, simply define a string processor function which should expect to receive as arguments
  1. a string array containing the "normal" parts of the Template String
  2. zero or more additional arguments, each representing an expression value
The tagging function should return a string, which is probably going to be some version of the data passed to it, but doesn't have to be related at all! Here's an example.

var test1 = 33;
var test2 = 44;

function tag(strings) {

    // print the static parts, in order
    strings.forEach(function(s) {
        console.log(s);
    });

    // print the expression values, if any
    if (arguments.length > 1) {
        for (var i = 1; i < arguments.length; i++) {
            console.log(arguments[i]);
        }
    } else {
        console.log("no expressions!");
    }

    // this will be the tagged string value
    return "not the string you're looking for";

}

console.log(tag `Part 1 ${test1} part 2 ${test2}`);
console.log(tag `what? no expressions?`);

I've called my tagging function tag, but it can be anything you want. This first example just prints out the parts of the Template String so you can see how it works. It totally ignores the real contents of the Template String and returns nonsense. Those two calls to console.log both print the return value from tag. Probably not useful, but shows you what can be done. I honestly haven't thought of a legitimate use for tagging yet, but I might someday. Maybe you already know of one?

And there you have it. Template Strings, the good and the weird. I think the hardest part will be hitting that back-tick key. Not used to typing that one...

Amphibian.com comic for 23 September 2015

Monday, September 21, 2015

Symbols - New in Node 4

In continuing my discussions of new ES6 language features now available in Node 4, today I'm turning my attention to Symbols.

I'm not as sold on Symbols as I am on some of the other new features. What is a Symbol? A technical explanation is that Symbols are unique and immutable data types. Their purpose is to provide another way of making private properties on objects.

Ecmascript 6: The Future is Now!

Making Symbols is easy.

var sym1 = Symbol();
var sym2 = Symbol("whatnot");
var sym3 = Symbol("whatnot");

console.log(sym2 == sym3);  // false
console.log(sym2 === sym3); // false

The string parameter passed to Symbol() is optional and simply serves as a label. It doesn't mean anything to the Symbol itself. Making another Symbol with the same label doesn't mean the two Symbols are equal. Each Symbol created is unique.

So what does one do with these Symbols? I said that they're used for making private properties on objects, but how is that accomplished?

Let's start by defining the problem. When you have an object, JavaScript provides no protection of its properties. They can be redefined, or even removed! Consider this example.

var Frog = (function() {

    function Frog(type) {
        this.type = type;
    }

    Frog.prototype.getType = function() {
        return this.type;
    };

    return Frog;

}());

var f = new Frog("Science");
console.log(f.getType()); // prints Science

f.type = 27;
console.log(f.getType()); // prints 27

delete f.type;
console.log(f.getType()); // prints undefined

How can the frog type be made immutable outside of its own functions? Symbols are one possibility. Look at this code, slightly modified to use Symbols.

var Frog = (function() {

    var typeSymbol = Symbol("type");

    function Frog(type) {
        this[typeSymbol] = type;
    }

    Frog.prototype.getType = function() {
        return this[typeSymbol];
    };

    return Frog;

}());

var f = new Frog("CEO");
console.log(f.getType()); // prints CEO

console.log(f.type); // prints undefined

console.log(f["type"]); // prints undefined

var s = Symbol("type");
console.log(f[s]); // prints undefined

delete f.type;
console.log(f.getType()); // prints CEO

Instead of setting and accessing the frog type property internally using the dot notation (this.type), this code uses the bracket notation (this["type"]). BUT it uses the Symbol instead of the string "type" to make it inaccessible from outside. All attempts to read or delete the private type property will be unsuccessful. Even trying to do it with another Symbol fails, because Symbols made with the same label are not equivalent.

So that's Symbols. There are of course other ways to make fields private in JavaScript, such as using more closures or with Object.defineProperty. I haven't experienced any scenario where Symbols are really that much better than the previously available alternatives, but it's always nice to have more options.

I can't think of any way to tie this to the Amphibian.com comic for today, so here's the link to it without further comment.

Amphibian.com comic for 21 September 2015

Friday, September 18, 2015

Block Scoping - New in Node 4.0.0

Today wraps up "games week" on Amphibian.com. I had the frogs playing the lottery on Monday, Let's Make a Deal on Wednesday, and poker today. Since there's not much to discuss concerning poker being a game of skill or a game of chance, I'm back to discussing new JavaScript language features available in the recently released Node 4.0.0. (they've actually released 4.1.0 already but it's mostly just bug fixes).

So let's talk about Block Scoping!


This new ES6 feature gives you a new option for limiting the scope of declared variables to a block, statement, or expression. Just declare a variable with let instead of var and it won't be accessible to the entire function!

Here's an example that illustrates the point.

var x = 37;
if (x > 30) {
    let x = -1;
    let y = 900;
    console.log(x); // prints -1
    console.log(y); // prints 900
}
console.log(x); // prints 37
console.log(y); // prints undefined

if (x > 10) {
    var x = 5;
    var y = 777;
    console.log(x); // prints 5
    console.log(y); // prints 777
}
console.log(x); // prints 5
console.log(y); // prints 777

Inside the first if block, let x = -1 declares a block scoped variable x which is distinct from the global x variable declared on the first line. Changing it inside the block has no effect outside the block. By contrast, the second if block declares var x = 5, which isn't really much of a declaration at all. The x variable already in scope simply has its value reassigned. Once outside the block, it still has the value give it inside the block.

Also note the behavior of the y variable. Declared using let inside the first if block, it is undefined when used outside. But declaring it using var inside the second if block makes it global.

Declaring variables with let also works in for loops.

var i = 50;
for (let i = 0; i < 5; i++) {
    console.log(i); // prints 0 through 4
}
console.log(i); // prints 50

for (var j = 0; j < 5; j++) {
    console.log(j); // prints 0 through 4
}
console.log(j); // prints 5

The first loop, using let, prints the value of the variable i which is scoped to just the for loop. Outside of the loop, i still refers to the global variable. The second loop declares j with var and so j becomes available outside of the loop.

Making good use of let for variable declaration can make your code cleaner and lead to fewer bugs.

One final thing to note, however. While block scoping is available in Node 4.0.0+ it only works in strict mode. That means either launch node with the --use_strict flag or put
"use strict";
at the top of your source file. But you're probably using strict mode already, right?

Good. Now enjoy today's comic.

Amphibian.com comic for 18 September 2015