Monday, June 15, 2015

Using File Drop in Web Pages

Don't Litter - Drop Files in the Right Place!
When I'm making some of the more elaborate comics (such as the fire alarm from Friday or the agile dodgeball game) I like to work out the JavaScript on my test server here on my local network. But sharing the actual comic data (positions of frogs, text bubbles, etc) was always a pain. I would copy and paste JSON from the production server into a SQL statement for my local server or vice-versa. I decided that I should make an "import data" feature directly in the editor.

It is certainly easy enough to put a text area on the screen and let me copy-and-paste in a big JSON string. But while I was doing it, I thought, "Hey, I should just be able to drop a text file in here and have it auto-populate the text area from the file contents."

And so that's what I did.

It's not really that difficult thanks to the File API stuff that's been in JavaScript for a while now. Here is a sample web page that has a single text area on it.

<!doctype html>

<html lang="en">
<head>
  <meta charset="utf-8">
  <title>File Drop</title>
</head>

<body>

  <textarea id="drophere" style="width: 200px; height: 100px;">drop a file here</textarea>

</body>

<script src="https://code.jquery.com/jquery-1.11.2.min.js"></script>

</html>

To allow dropping text files in the text area, the following JavaScript is used.

$(function() {

    $("#drophere").on(
        "dragover",
        function(e) {
            e.preventDefault();
            e.stopPropagation();
        }
    );

    $("#drophere").on(
        "dragenter",
        function(e) {
            e.preventDefault();
            e.stopPropagation();
        }
    );

    $('#drophere').on("drop", function (evt) {

        var e = evt.originalEvent;

        if (e.dataTransfer) {

            if (e.dataTransfer.files.length) {

                evt.preventDefault();
                evt.stopPropagation();

                var file = e.dataTransfer.files[0];

                if (file.type != "text/plain") {
                    console.log("wrong file type");
                } else {

                    var reader = new FileReader();
                    reader.onload = function(fevent) {
                        var txt = fevent.target.result;
                        $('#drophere').val(txt);
                    }
                    reader.readAsText(file);

                }

            }

        }

    });

});

Since I use jQuery, everything is wrapped in a function that will be called as soon as the document is fully ready. Before setting up the actual drop handler, there are two other event handlers that should be registered to prevent undesirable browser behavior.

The first, on line 3, is the ondragover event handler. This event fires constantly when an element is being drug over a drop target. All the event handler does here is prevent the default behaviors of the browser, which in the case of a text area is to move the cursor around where the drop will take place. That isn't needed in my case because I plan on replacing the entire contents of the text area when the drop occurs.

The second event handler (line 11) is for the dragenter event. This event fires once when the element being drug first enters the drop zone. Again, I am just turning off the browser default behavior in here.

The next and final event handler that I register is for the drop event. This is where the good stuff happens. Because jQuery's event object wrapper doesn't really have direct support for the dataTransfer element, the first thing I do here is get the original event object from it. That's the object I will be using for most of the subsequent processing. First I check to make sure that there is a data transfer associated with this event and that the list of files in that transfer is at least one. If those two checks pass, I once again turn off event propagation and the default browser behavior. Remember, the browser will typically load any file you drop on a page as a new document - definitely not what I want to happen!

The next step is to get the file from the list of files in the data transfer and check the type. For my purposes, I only want to accept files that are plain text. It wouldn't make sense to drop an image or something in a text area! Assuming that the file type checks out, I can finally read the contents of the file. On line 36 I create a FileReader and then set the onload event handler. This is the function that will be called with the file data (or possibly an error) once the read is complete. It will be passed an event object, in which the text can be found in the target.result field. Once this function is set up, I just call readAsText and pass in the file (line 41).

Inside the onload function, line 39, is where I set the value of the text area to the contents of the text file. You could just as easily send the file contents directly to the server at this point or do some other processing, This technique will work on other kinds of files as well - instead of reading as text you could read as a binary string or an array buffer or a data URL. See the documentation for more info!

Give my demo a try for yourself and see how convenient it is to drop text file contents in text areas. You'll probably want to add this feature anywhere you have a text area on your own web pages.

And now the obligatory link to today's comic!

Amphibian.com comic for 15 June 2015

Friday, June 12, 2015

Free SSL Certificates

On Wednesday I talked about how I added SSL (but actually TLS) support to Amphibian.com in response to user requests. One thing that is always a problem when trying to secure a public web site is the high cost of certificates signed by "real" certificate authorities.

In order for you to not scare away your users with dire browser warnings, any secure certificate served by your web site must be signed by an authority that ships with the browser. While it is possible to install new authorities, it is not something that 99% of people would ever do. That means you're stuck paying a yearly fee to someone for keeping your site secure.

Unless you use StartSSL.

Yes! StartSSL provides FREE server certificates with 1-year terms and is a trusted authority in all common browsers. The catch is that it can be a little tricky to request and use them.

There are three steps to the process. First, you need a certificate installed in your browser that StartSSL can use to verify that you are who you claim to be. Go to https://www.startssl.com and click on the button in the upper right that looks like an ID card and some keys.

Click on the Keys to Begin
 Then click on the link to sign-up for an account. After filling out your name, address, phone number, and email address, click "Continue" and they will email you a verification code that you need to type in on the web page.

Fill out this form with your contact information.
Don't navigate off the page until you get the code and enter it! After that, they'll generate a key and install it in your browser. Now you can authenticate yourself with their system and move on to the next step.

The second step in this process is to use the Validations Wizard to validate your domain. Click on the Validations Wizard tab and then select "Domain Name Validation" from the drop-down box.

You want Domain Name Validation
On the next screen, enter the domain name for which you desire validation. StartSSL will read the WhoIs record for that domain and offer to send a verification code to one of the email addresses listed as contacts.

Enter your domain name. No www nothing, just the domain.
If you really control this domain, you should be able to get the email for at least one of them. Assuming that is true, get the validation code from your email and enter it on the next screen. Congratulations, you just verified your domain and you can move on to step 3.

Now go to the Certificates Wizard tab and select "Web Server SSL/TLS Certificate" from the drop-down list.


The next step will be to create the private key. This is the the one you want to keep to yourself! Enter a password for it and continue to the next step. It will show you the key in a text area. Copy and paste it into a file called ssl.key and click Continue.


The next screen has you select the domain for which you are generating this certificate. A drop-down box will show you the list of all the domains that you have verified (that was the second part of this 3-part process). After that, you have to supply a single subdomain. The certificate will be good for both. Most people use "www" for this. After one final confirmation screen, they are ready to generate your certificate.

If everything went well, you should be given another text area with your certificate in it. Copy and save it to a file named ssl.cert or something with the domain name in it so you remember which one it is for.

Now you should have a private key and a certificate signed by a legitimate authority. You're all set, right? Well, almost. I know I said there are only three steps but there is maybe kinda one more. You see, the certificate is actually signed by a StartCom intermediate authority, not the ultimate root. Some platforms, such as Android devices, don't trust the intermediate already and will reject the certificate when served from your website. The solution is to make a complete key chain using the certificate from the intermediate signer combined with your server's certificate.

First, go to the Toolbox tab and then click on "StartCom CA Certificates" on the left side. The certificates for the intermediate servers will then be available for download.


My certificate, since it is one of the free ones, is signed by the Class 1 Intermediate Server. Download that certificate and then concatenate it together with your ssl.cert file.

cat ssl.cert sub.class1.server.sha2.ca.pem > combined.cert

Now use the file combined.cert instead of ssl.cert for the server certificate in your web serving application of choice (Apache, Node, Nginx, etc) and even Androids will be happy.

I've done this a couple of times now for a few different domains and it works great. It is a maybe a little involved but it is free. In this case you get even more than what you pay for! Now have a look at today's comic - view it using transport layer security if you want!

Amphibian.com comic for 12 June 2015

Wednesday, June 10, 2015

Serving Your Express App With Encryption

While today's comic has nothing to do with encryption, you can actually view it (or any of my other content) using encrypted web communications. Most people refer to this as SSL, which stands for Secure Socket Layer, but SSL is an older and now non-recommended way of securing web traffic. The current method is called TLS and stands for Transport Layer Security. Doesn't make a whole lot of difference what you call it, it's the "s" in "https" when you're viewing a page with the reasonable assurance that no one is able to spy on your network traffic.

Don't mess with my network traffic!
Before last week, Amphibian.com had no way of delivering web pages in a secure manner. Why would it need to? It's just a web comic. But after my Bitcoin Paywall comic got so insanely popular I started to get people asking why I didn't have encryption enabled. Without it, there is a small chance that someone could inject their own Bitcoin address into the response from my server and take the money you are trying to send to me. So I decided to get a server certificate signed by a real authority (more on that Friday) and enable encryption for my comics.

This of course led me to create Monday's comic in which the frogs are surrounded by danger when viewed via http://amphibian.com/177 but are much safer when viewed via https://amphibian.com/177.

Now for the technical part. In order to get my Node/Express app to serve web pages over both secure and insecure web ports, 443 and 80 respectively, I had to make a few changes.

For simple apps, you are probably familiar with listening on a port this way:

var express = require("express");
var app = express();

// ... set up routes ...

var server = app.listen(3000, function() {
    console.log("listening on port %d", server.address().port);
});

But if you want the same Express instance to handle the traffic on multiple ports, you have to use the Node http module directly to create servers and pass in the Express instance as a parameter:

var express = require("express");
var http = require("http");
var app = express();

// ... set up routes ...

var server = http.createServer(app).listen(3000, function() {
    console.log("listening on port %d", server.address().port);
});

You could of course use this technique to listen on multiple insecure ports, like 3000 and 4000 as in this example:

var express = require("express");
var http = require("http");
var app = express();

// ... set up routes ...

var server1 = http.createServer(app).listen(3000, function() {
    console.log("listening on port %d", server1.address().port);
});

var server2 = http.createServer(app).listen(4000, function() {
    console.log("listening on port %d", server2.address().port);
});

However, if you want one of the listening ports to use encrypted communications you need to use the Node https module instead of http for one of them. In the simplest possible configuration it takes just one extra parameter - an options object which contains at a minimum the private key and public certificate for the server.

var express = require("express");
var http = require("http");
var https = require("https");
var fs = require("fs");
var app = express();

// ... set up routes ...

var server1 = http.createServer(app).listen(3000, function() {
    console.log("listening on port %d", server1.address().port);
});

var sslOptions = {
    key: fs.readFileSync("/path/to/private.key"),
    cert: fs.readFileSync("/path/to/server.cert")
};

var server2 = https.createServer(sslOptions, app).listen(4000, function() {
    console.log("listening securely on port %d", server2.address().port);
});

And just like that you can enable encrypted communications in your Express web app. Now read today's comic where the frogs try to avoid upsetting the apple cart...

Amphibian.com comic for 10 June 2015

Monday, June 8, 2015

Daughters++

While I was planning on writing about using Transport Layer Security in a Node + Express application to go along with today's comic, something else came up.

I went to the hospital this past Thursday with my wife for the birth of our fourth daughter. Since I was there until late afternoon on Saturday and sleep-deprived the whole time, blogging about encryption for web communications will have to wait until later in the week.

But here I am with the newest member of my family:


So enjoy today's comic without a technical discussion, and be sure to look at both secure and insecure versions!

Amphibian.com comic for 8 June 2015

Friday, June 5, 2015

Using Noise to Generate Game Maps

There is absolutely no technical tie-in to the comic today. It's just tree puns. Sorry. But since trees are part of the landscape, I thought this might be a good time to share some of what I'm working on with noise and map generation.

Yes, noise. Noise can generate random game maps. Not noise like that new Google Chrome plugin thing that sends URLs to people nearby using your computer speakers (interesting, but odd), noise as in Perlin noise.

Perlin noise is a procedural way to generate pseudo-random textures that appear more natural than those created with alternative methods. It was invented by Ken Perlin in 1983 while working on the CGI for the movie Tron.

I learned about this just over a month ago at a local Meetup group. If you consider that the map in a computer game is like a texture, Perlin noise can be used to easily generate terrain which seems realistic. I want to use this is a side project I'm working on for Amphibian.com but I haven't had much time to experiment with it until this week. I now have a JavaScript utility that generates the noise so I can generate randomized maps both in the client browser and in my Node.js server.

I am using a noise generation library, perlin.js, by Joseph Gentle. It is based on Stefan Gustavson's public domain implementation and can be found here on GitHub: https://github.com/josephg/noisejs

Here is a simple example of it in action.

<!doctype html>

<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Perlin Noise</title>
  <style>
    body {
      margin: 0px;
    }

    div {
      height: 40px;
    }

  </style>
</head>

<body>

  <div id="map"></div>

</body>

<script src="http://code.jquery.com/jquery-1.11.3.min.js"></script>
<script src="perlin.js"></script>
<script>

var width = 33;
var height = 17;

noise.seed(Math.random());

for (var y = 0; y < height; y++) {

  var row = "<div>";

  for (var x = 0; x < width; x++) {

    var value = noise.simplex2( y / height, x / width );

    var e = "grass";
    if (value > 0.5) {
        e = "grass2";
    } else if (value < -0.5) {
        e = "water";
    }

    row += "<span><img src='" + e + ".png'/></span>";

  }

  row += "</div>";
  $("#map").append(row);

}

</script>

</html>

I'm building a simple map out three possible tile images: grass, rough grass, and water. They are just 40x40 squares which are either green, green with darker green lines, or blue.

grass.png
grass2.png
water.png
I am looping over the height and width values and generating a noise value for each cell of the grid. The noise value will always be between -1 and 1. The "default" cell is grass. If the noise value is over 0.5 then that cell becomes rough grass. If it is under -0.5 it becomes water. I am just throwing the images in span tags wrapped in div rows for the purposes of this example. Try it yourself and see what it looks like. Hit refresh on your browser to generate a different random landscape each time. They always look fairly natural with the way the water and rough grass group together and intermingle with the plain grass tiles.

Look at a few of the ones I generated.




Now if I could just randomly generate some comics. Fifty percent of the time, they'd be funny all of the time.

Amphibian.com comic for 5 June 2015

Wednesday, June 3, 2015

The Amphibian.com Software Stack

The work of a full-stack engineer. Delicious.
The comic today referenced a number of popular software solutions stacks, before realizing that the CEO frog really just wants someone to make stacks of pancakes. Instead of hiring a short-order cook, try listing the job as "full stack engineer" and see who applies.

If you're not familiar with them, here they are:

LAMP
  • Linux Operating System
  • Apache HTTP Server
  • MySQL Database
  • Perl / PHP / Python Programming Language
MEAN
  • MongoDB
  • Express.js
  • AngularJS
  • Node.js
ELK
  • Elasticsearch
  • Logstash
  • Kibana
These "stacks" get catchy acronym names because they are so often used together, but there's nothing really binding about them. They're also not consistent in which parts of the system they name. For example, the MEAN stack still has to run on an operating system (probably Linux) but that part is left out of the name.

To make Amphibian.com, I didn't use one of these named stacks.

Here is my Frog Stack:
  • Linux
  • Express.js
  • Node.js
  • MySQL
It just doesn't have a nice acronym. LENM? I could re-arrange the words. MELN? ELMN? I could leave the operating system out, I guess. MEN? I could be more specific with my operating system and say Ubuntu instead of just Linux. Then I could have MUEN, NUME, MUNE, EMUN, or ENUM. I started the process of switching from MySQL to Postgres, so maybe someday I could run a LENP stack. Or a PELN? NELP?

Those are all terrible.

Why did I select Ubuntu Linux? Mainly because it's free and I like it. I initially selected MySQL because I've used it in a lot of projects over the years and I was familiar with it. I want to switch to Postgres now because I store mostly JSON and want to use the new features related to JSON storage in Postgres 9.4. I chose Node and Express because I wanted to start building applications with Node to learn about it. Express, I learned, was just the most popular web framework for Node. So I learned that as well. And it's been a good experience. I really like building applications with this set of components, even if it doesn't have a cool name.

I would like a full stack of pancakes now.

Amphibian.com comic for 3 June 2015

Monday, June 1, 2015

Easy Node Debugging

Today's comic is just silly. A frog's debugger is his tongue, since it is a very effective tool for removing bugs. Or maybe it just relocates them - directly to the frog's stomach. If you want to use a debugger with your Node.js applications, you'll want something better than a tongue.

I will admit that I was slow to adopt the use of a debugger for runtime code inspection. Early on in my software development career, when a program wasn't working the way I thought that it should, I just put in some code to print stuff to the console and restarted. That's when my applications were small and didn't take long to recompile and restart. By 2003, I was working on large Java EE systems where the process of rebuilding and redeploying took too long for System.out.println(...) to be an acceptable method for finding bugs. I actually got pretty good with the Java command-line debugger, but after seeing the Java debugger in the Eclipse IDE I gave up Emacs as my primary development environment and switched. I have no regrets.

Today I also do a lot of JavaScript development for Node in addition to my work with Java, but have been so far unimpressed with the debugging capabilities for Node integrated into Eclipse. I use Nodeclipse and for development purposes I am fairly happy, but the debugger left me wanting something else.

I believe I found it with Node Inspector. The idea behind it is pretty cool - debug your JavaScript applications with the JavaScript debugger with which you are probably most familiar: the Chrome Dev Tools!

Here's how it works. first install Node Inspector as a global module.
$ npm install -g node-inspector
Then instead of launching your Node app the "normal" way, do this instead:
$ node-debug app.js
As long as Chrome is your default browser (come on, why shouldn't it be?), the debugger window will launch as a new browser tab which will look like the Developer Tools window. Awesome!

Node Inspector debugging one of my demo apps. Using the Chrome Dev Tools in Chrome!

There are a few things to note here. First, you'll want to be using a version of node greater than or equal to v0.11.13. Older versions work but the debugging capabilities are limited. Second, since the debugger is also running in the browser, if you are debugging a web app you might find yourself needing to switch tabs all the time. You can pop the tab out into a separate window, but it's still a third thing you need to deal with in addition to your IDE and the application you are testing. Not a huge problem, but it is something to consider. Maybe it's enough to make you finally add that third monitor to your development workstation...

Or if you want to get rid of bugs the old-fashioned way, try a frog.

Amphibian.com comic for 1 June 2015