Make Memes via API Calls

In today's comic the frogs are recycling pictures into memes, and by clicking on the meme in the last frame you can create your own frog meme. You just type in your own text and your personalized meme appears. You can even share it.

I don't want to get into a debate about how the sharing of memes has destroyed what remains of our humanity - it's just a comic about frogs. The interesting thing is how I was able to find a meme generation API that I could work with to make this comic possible.

My first attempt at this used memegenerator.net. It's a very popular meme site. They have a JSON API and there are even some modules for Node.js that interface with it. That all sounded good. I tried out the node-meme module and it worked - for all the read methods. Once I tried to create a meme with it I was informed by an error code that I needed valid login credentials.

No problem! I'll just register for an account.

Nope.

Or not. Apparently, account registration has been unavailable for quite some time. All attempts at it result in a rather unintelligible error message. There goes that plan!

After some more searching I found Imgflip.com. They have both a meme-generation API and functioning account registration! Yeah!

To make the comic work the way I intended, I needed to make all the memes from my server and then just update the browser with the Imgflip URL. This prevents everyone who uses the comic from having to get their own account, but it makes things a little more complicated for me. I essentially had to set up a meme-generation proxy. Your browser POSTs data to my server, my server POSTs data to Imgflip, Imgflip generates a meme and tells my server the URL, and finally my server tells your browser the URL.

This also required me to do something I haven't done a lot of before - use the Node http client. Sure, I've done simple things with it in the past, but the out-of-the-box features make it a bit difficult to use when you want to access a REST web service.

So I also started using the request module. It greatly simplifies working with HTTP(S) requests in Node. Here is an example of how I use it to interact with Imgflip's API:

var request = require("request");

var formData = {
    template_id : "41754803",
    username : "user",
    password : "passwd",
    text0 : "i look displeased...",
    text1 : "but i have no idea what's going on"
};

request.post("https://api.imgflip.com/caption_image", {
    form : formData
}, function(error, response, body) {

    var meme = JSON.parse(body);

    if (!error && response.statusCode == 200) {
        console.log(meme.data.url);
    }

});

Simple, right? The first parameter to request.post is the URL.

The second is what to POST. In this case, I want to POST a form, specifying the form data as an object. Imgflip only accepts form data as input, and every POST must include your username and password. You get those by registering on their site. The template_id form field is the id number for the meme image you want to use. If you upload your own meme template images, you can find the id numbers on template details page linked from your account. The text0 field is the text for the top of the image, and text1 is for the bottom.

The third parameter is the callback function for when the POST completes. This function gets an error (if there was one), the response object for checking the status code (and other stuff if you want), and the body of the response. In the case of the Imgflip web service, I know the body will be JSON. In this example I just log the meme image URL.

That is much simpler than doing it the purely native way. I recommend the request module if you need to do any kind of REST API access and there's no custom client available.

Now the fun begins: frog memes for everyone!

Amphibian.com comic for 31 July 2015

MIME Type is Important for REST

Today's comic makes fun of MIME (or Internet Media) types, but they can a very useful part of web applications that make use of REST.

MIME started out as a way to tell email clients what type of file you'd attached to a message, but have become much more useful. Every time your web browser makes a request for content, it sends along a header called "Accept" which is a list of data types it can process. Typically, this includes HTML, JavaScript, and images. The server, as part of its response, includes a header called "Content-Type" which specifies what is actually being returned. The two sides usually find some common ground.

When making REST web services, being a little more particular about your MIME types can open up new possibilities. REST stands for Representational State Transfer. When a human being is driving a web browser and asks for a resource (a.k.a. a web page or URL), they expect the server to transfer a representation of the current state of that resource. That representation is often HTML or an image because that's what humans can understand (well, assuming the browser turns the HTML into something pretty first). But the real power of REST web services is how it opens things up to the possibilities of machine-to-machine communication. Let's say a computer program requests the same resource as the human. The machine would have an easier time dealing with it if it could be in a different representation other than HTML, such as JSON. Two representations, one resource.

And that's where the MIME type comes in. When making a web service call, specifying exactly what you can deal with (your desired representation of that resource) lets the server know if it should give you HTML, an image, JSON, or something else. When serving requests, explicitly setting the content type of the response lets the client know if it should even try to process it.

To help visualize it, I've created a demo. The following Node/Express server has a single route, /pizza. Depending on what the client wants, it will return either a picture of a pizza, a JSON string describing a pizza, or an HTML snippet describing a pizza.

var express = require("express");
var fs = require("fs");
var app = express();

app.get("/pizza", function(req, res, next) {

    if (req.accepts("image/jpg") === "image/jpg") {

        fs.readFile("pizza.jpg", function(err, data) {

            if (err) {
                res.sendStatus(500).send(err);
            } else {
                res.setHeader("Content-Type", "image/jpg");
                res.send(data);
            }

        });

    } else if (req.accepts("application/json") === "application/json") {

        res.setHeader("Content-Type", "application/json");
        res.send({
            size: "large",
            toppings: ["pepperoni", "mushrooms"]
        });

    } else if (req.accepts("text/html") === "text/html") {

        res.setHeader("Content-Type", "text/html");
        res.send("<p>pizza with<ul><li>pepperoni</li><li>mushrooms</li></ul></p>");

    } else {
        res.sendStatus(406);
    }

});

//------------ static content
app.use(express.static("public"));

// ------------ start listening
var server = app.listen(3000, function() {
    console.log("listening on port %d", server.address().port);
});

To make this happen, the route has to check what the client can accept, which is sent as part of the "accept" header. Express gives us the method req.accepts() to check that header. Unfortunately, I found it a little awkward to use. You pass a string or an array of strings to req.accepts() and it returns either a string of the best match or undefined if there is no match. This is because browsers typically send a whole list of things as part of the accept header - after all, a browser can deal with HTML, plain text, JavaScript, and a variety of images to say the least. Express tries to help you by telling you what the browser prefers for each request if there is more than one option. That's why I did things like put the image check first (browsers accept image/* first when requesting the src of an img tag but also accept anything else!) and always check to see if the best choice exactly equals my test string.

Look at this jQuery client code:

$.ajax("/pizza", {
    headers: { Accept: "text/html" },
    success: function(data) {
        $("#d1").html(data);
    }
});

$.ajax("/pizza", {
    headers: { Accept: "application/json" },
    success: function(data) {

        var h = "<h3>got a " + data.size + " pizza with ";
        for (var i = 0; i < data.toppings.length; i++) {
            if (i > 0) h += " and ";
            h += data.toppings[i];
        }
        h += "</h3>";
        $("#d2").html(h);

    }
});

$.ajax("/pizza", {
    headers: { Accept: "application/xml" },
    success: function(data) {
        console.log("success!");
    },
    error: function() {
        console.log("error!");
    }
});

$("#d3").html("<img src=\"/pizza\" />");

Using jQuery to make Ajax calls to a single URL three different times, I specifically set the "accept" header to just one thing in each of the requests. The first request says it can accept only text/html, so the server gives me exactly that. The second request wants JSON, so the server responds accordingly and the JSON is turned into HTML on the client side. The third request wants XML, but the server doesn't support that and responds with a 406 - Not Acceptable. That response sounds backwards, since the server is telling the client that it doesn't have a way of giving it something that it can accept not that the server got something unacceptable from the client. Finally, an image tag is generated which specifies that same URL as the source and it does in fact get the JPG from the server.

Running that JavaScript code on this web page

<!doctype html>

<html lang="en">
<head>
  <meta charset="utf-8">
  <title>MIME Types</title>
</head>

<body>

  <div id="d1"></div>
  
  <div id="d2"></div>
  
  <div id="d3"></div>

</body>

<script src="https://code.jquery.com/jquery-1.11.1.min.js"></script>

</html>

produces results that look like this (supply your own pizza image):

And that's how you can help usher in the future of machine-to-machine communications via the Internet. Make web services that serve machines as well as people. Build the Internet of Things. Now go get some REST.

Amphibian.com comic for 13 May 2015

Your Internet Thermometer

Photo by Bernard Gagnon

It's amazing what access to the Internet on phones has done to us. When I wake up in the morning, I look at my phone to check the outside temperature. I remember when I was a kid and we had to use antiques like thermometers stuck on the outside of our windows!

In today's comic, the frog is trying to get the body temperature reading from another frog. He fails to realize that frogs, being ectotherms, will basically have the same body temperature as that of their environment. His phone shows him (and you, the reader of the comic) the actual temperature from my town here in central Pennsylvania. Look at the comic today and you'll see today's temperature. Come back in 6 months and you'll see a warmer temperature (I hope). Whatever temperature it is here when you load the page, that's what you'll see.

Getting weather data for my comic is not difficult. I basically do the same thing that the weather app on your phone is doing - accessing a server somewhere that has been collecting data from weather stations around the country and getting the latest information for a certain location. I have used both Weather Underground and OpenWeatherMap to get data, as both have APIs that anyone can use. In the case of my comic, I wrote a simple module for Node that I use to manage the API calls.

var http = require('http');

module.exports = function(conf) {

    var currentTemp = 50;  // start with some reasonable value

    var apiKey = conf.weatherApiKey || "";
    var location = conf.location || "";

    function pullWeatherData(cb) {

        // OpenWeatherMap
        var url = "http://api.openweathermap.org/data/2.5/weather?id="
                + location + "&units=imperial&APPID=" + apiKey;

        // Weather Underground
        //var url = "http://api.wunderground.com/api/" + apiKey
        //        + "/conditions/q/" + location + ".json"; 

        http.get(url, function(res) {

            var wData = "";
            res.on("data", function(chunk) {
                wData += chunk;
            });
            res.on("end", function() {
                try {
                    var data = JSON.parse(wData);
                    cb(null, data);
                } catch (e) {
                    cb(e);
                }
            });

        }).on("error", function(e) {
            cb(e);
        });

    }

    function weatherTimer() {

        pullWeatherData(function(err, data) {
            if (err) {
                console.log(err);
            } else {
                try {

                    // OpenWeatherMap
                    if (data.main.temp) {
                        currentTemp = data.main.temp;
                    }

                    // Weather Underground
                    //if (data.current_observation.temp_f) {
                    //    currentTemp = data.current_observation.temp_f;
                    //}

                    console.log("Setting current temp to " + currentTemp);

                } catch (e) {
                    console.log("Unable to update temp");
                    console.error(e.stack);
                }
            }
            setTimeout(weatherTimer, 1800000); // call again in 30 minutes
        });

    }

    weatherTimer(); // run now, and then every 30 minutes

    return {
        temperature: function() {
            return currentTemp;
        }
    };

};

The goal of this module is to give me access to the current temperature value and update that value automatically on a timer. Temperature doesn't usually change that quickly so every 30 minutes is fine for my application.

There are really only two functions - one that calls the API to get the data and the timer that asks for new data and updates the temperature. In my code above, I show it using the OpenWeatherMap API with the Weather Underground stuff commented-out, in case you want to try that one as well.

The first function I called pullWeatherData. It simply calls http.get for whichever service you want to use and reads the response stream, in JSON format, into a string which is parsed into an object at the end. I had to make sure error conditions were handled gracefully, because when you call external APIs you sometimes get garbage back. That's the point of the try/catch around the JSON.parse call - the response might not actually be JSON (it could be a plain text HTTP 500 page, for example) and you don't want to crash your whole app. So if there is any kind of error, the callback function given to pullWeatherData is called with the error as the one and only parameter. In the case of a successful call and parse, the callback function is called with a null as the first parameter to indicate no errors and the weather data object as the second parameter. The exact format of this object will depend on which service you called.

The weatherTimer function simply calls pullWeatherData and then does something with the data before setting a timeout for calling itself again in 30 minutes. All it really does with the data is take the temperature value out of it and store it in a variable. If everything works, that variable will be automatically updated every half hour.

After defining these two functions, my module calls weatherTimer in order to get the data for the first time and start the timer for the next call. The return value is an object with a function simply called temperature which will return the current value of the temperature variable. Here's an example of how the module would be used in an app:

var express = require("express");
var app = express();

/*
 * OpenWeatherMap uses their own location ids.
 * Id 5184309 is near Reedsville, PA.
 * 
 * Weather Underground can use ZIP codes.
 * ZIP 17084 is Reedsville, PA.
 */
var weather = require("./weather")({
    weatherApiKey: "your_key_here",
    location: "5184309"
});

app.get("/", function(req, res, next) {
    res.send("weather demo");
});

app.get("/temp", function(req, res, next) {
    res.send("current temperature is: " + weather.temperature() + " deg F");
});

// ------------ start listening
var server = app.listen(3000, function() {
    console.log("listening on port %d", server.address().port);
});

Just like the comic, this sample app will tell you the temperature here in Reedsville if you point your browser at the /temp path. As you can see, it's cold in Reedsville as I write this.

If you want, you could modify this module to get different weather data or more data and store it in a more complex object. You could really just store off the entire response object and access different values each time. You might even apply this pattern to other non-weather APIs as well. Be creative!

Amphibian.com comic for 5 December 2014