Saturday, April 16, 2011

Building by the Byte - the HTML5 File API

One of the major features needed in JavaScript to make it truly useful as an application language is file processing. I'm talking about handling the contents of a file totally in your web browser. No server needed. Even non-text format files. Now with HTML5 we have this capability in the File API! I've been thinking about the awesome new possibilities opened up by this development, and put together an example of what it can be used for.

First, let's talk about the browser support. The latest Chrome, Safari, and Firefox browsers support the new JavaScript File API. Internet Explorer? Nope. Get a real browser.

The first thing to understand is the FileReader object. It's a new built-in object, sort of like the XMLHttpRequest object. Like the familiar XHR, FileReader is designed to work asynchronously. That means you'll need to specify your own onload function to the object, which will be called when the browser is done with the file. Think about it - it could take a while to process a file and you don't necessarily need your app tied up waiting for it. Look at this simple example...
var reader = new FileReader();
reader.onload = function(event) {
// file is loaded, contents are in event.target.result
// do something with it!
}
reader.readAsBinaryString(file);
Now you're probably asking a few questions at this point. Where did you get the file object? How does JavaScript handle binary data? What if there's an error? How do they get the peanut butter inside the peanut butter cups? I can answer all but that last one.

First, there are a few ways to get a file object. My favorite is to grab one simply by dragging it into the browser window. This is accomplished via the dataTransfer property of the event object. For example, let's say you have the following div in your page...

<div id="drophere" style="text-align: center; width: 200px; height: 100px;">drop a file here</div>
And then you had some JavaScript like this...
document.getElementById('drophere').ondrop = function (evt) {
evt.preventDefault();
var file = evt.dataTransfer.files[0];
// now you've got a file object, which is the file you dropped
return false; // don't let the browser navigate away
}
Now just drag a file into your browser window and drop it on your div. Awesome! You've got a file. Now just combine this function with the previous one and you're all set to process anything you can drag in. Well, almost. There's still that binary data issue. JavaScript doesn't really have a data structure designed for binary data.

This is where you break out the FileReader and pass in that file object. Add the code from the first example into the second example....

function handleDrop(evt) {
evt.preventDefault();
var file = evt.dataTransfer.files[0];
var reader = new FileReader();
reader.onload = function(event) {
// file is loaded, contents are in event.target.result
// do something with it!
}
reader.readAsBinaryString(file);
return false; // don't let the browser navigate away
}
So when you get the event.target.result object (in the reader's onload function), what will it be? It's actually going to be a String where each character code is between 0 and 255. To read the "bytes" of the file, just loop through all the characters calling charCodeAt on each one. I made an object to help with all the functions you might want to do with the "byte array"...

function DataReader(a) {
 
 this.bytes = a;
 this.index = 0;
 this.byteRead = 0;
 this.bitIndex = 0;
 this.endian = "big";
 
}

DataReader.prototype.readByte = function() {
 if (this.eof()) return;
 var ret = this.bytes.charCodeAt(this.index);
 this.index++;
 return ret;
}

DataReader.prototype.readBytes = function(howMany) {
 if (this.eof()) return;
 var ret = new Array();
 for (var i = 0; i < howMany; i++) {
  ret.push(this.readByte());
 }
 return ret;
}


DataReader.prototype.readInteger = function(numBytes) {
 
 if (this.eof()) return;
 
 var howMany = 4; // default to a 4-byte integer
 if (numBytes) {
  howMany = numBytes;
 }
 
 var ret = 0;
 if (this.endian == "little") {
  var origIndex = this.index;
  for (var n = this.index + howMany - 1; n >= origIndex; n--) {
   ret = ((ret << 8) | this.bytes.charCodeAt(n));
   this.index++;
  }
 } else {
  for (var n = 0; n < howMany; n++) {
   ret = ((ret << 8) | this.bytes.charCodeAt(this.index));
   this.index++;
  }
 }
 return ret;
 
}

DataReader.prototype.readString = function(len) {
 if (!len || this.eof()) return;
 var ret = this.bytes.substring(this.index, this.index + len);
 this.index += len;
 return ret;
}

DataReader.prototype.readNullTerminatedString = function() {
 if (this.eof()) return;
 var slen = 0;
 var n = this.index;
 var finished = false;
 while (!finished && n <= this.bytes.length) {
  var c = this.bytes.charCodeAt(n);
  if (c == 0) {
   finished = true;
  }
  slen++;
  n++;
 }
 var ret = this.bytes.substring(this.index, this.index + (slen - 1));
 this.index += slen;
 return ret;
 
}

DataReader.prototype.skip = function(num) {
 if (this.eof()) return;
 this.index += num;
}

DataReader.prototype.eof = function() {
 return (this.index >= this.bytes.length - 1);
}

I should mention that there are other options for processing the file. If you used readAsText instead of readAsBinaryString, you'd just get a normal string containing the contents of the file. That's only really useful if you know the file will only contain text data. A third option is readAsDataURL, which returns a data: URL instead of a string. You can use this to directly set the src attribute of an img tag with the dropped file. Again, this will have limited usefulness. Getting the binary string is the most powerful.

This is a good time to talk about the onerror function. If you tried the above example in Chrome using a local HTML file, it won't work. You'll get an error. You'll only know that if you specify an onerror function as well as an onload. Don't expect a whole lot of details in the error, however.
reader.onerror = function (event) {
console.log(this.error.code);
}
You'll see a "4" in the console. That's helpful... It actually means that Chrome, by default, does not allow local files (your test HTML file) access other local files (the file you drop in). Firefox does. It's not a real big deal, you can either test using a local server instead of just loading the file or add the "--allow-file-access-from-files" flag to Chrome when you start it. Security thing.

Okay, okay, okay...now what can you build with this? Well, some really amazing things. I put together this nifty example that will read PNG files and display them in the browser not as images, but as a bunch of DIVs (one for each pixel). To accomplish this, I just needed two things. One, the PNG specification which can be found here. And two, a way to inflate compressed data blocks inside the files. For that part, I used my pure JavaScript Inflater that I talked about in my last post.


If you don't have your own PNG file handy, use this one: http://www.amphibian.com/blogstuff/small_dr_frog.png

Make sure you check out the page source to see how it all works. It turns out that PNG files are fairly easy to work with once you have the data inflated.

I know my example is not particularly practical, but I hope it can at least inspire you to make something of your own that uses these splendid new HTML5 features. Use your imagination and let me know what you come up with!

Saturday, January 1, 2011

Inflate in JavaScript

Here's a little something I've been working with for the last few weeks...the Inflate algorithm implemented in JavaScript.

Just what is Inflate? Well, it's the opposite of Deflate. Obviously. In addition, it's also the algorithm used by gzip, WinZip, zlib, etc. to uncompress data. You can read all about it here or read the full RFC. It's been around a long time and has been implemented in lots of different languages, but I really wanted a pure JavaScript version. I'm that crazy.

I needed this because I've been working with the new HTML5 File API. With it, you can process file data in the client browser before uploading it to the server. This is great, until you try to work with a type of file (PNG, for example) that uses the Deflate algorithm to compress its data. I basically converted the simplest possible implementation of the process, Mark Adler's puff.c, to JavaScript and it works pretty well.

I'll have more to say on the HTML5 File API later, but now I release the JavaScript Inflate to the world! Typically, the algorithm works by processing streams of bytes. JavaScript, however, does not have such a data structure. Instead we just use arrays of numbers between 0 and 255 as input and output.

And a word of caution, I wouldn't try running this on a 3-years-out-of-date web browser or some old version of Netscape Navigator you've got running somewhere. It works great in the latest version of Chrome.


UPDATE! Here's a link to a page that you can use to see this thing in action. It uses a little HTML5 File API, which I'll discuss in a later post. I didn't explain much how to use the inflater in my original post so I hope this helps. It is really simple. Once you have your array of "bytes" representing deflated data, just pass it to the puff function along with an empty array you want to get filled up with the inflated "bytes". It will look like this:
var deflated = new Array();
// fill deflated with "bytes"
var inflated = new Array();
puff(inflated, deflated);
Your inflated array will contain the inflated data. It's that easy.

Saturday, November 6, 2010

Break the Request-Response Cycle! WebSockets are Coming!

Since the dawn of time, or at least since 1994, web developers have been forced to work within the confines of the request-response cycle. I've seen good developers be totally confused by the strange disconnected nature of the client and server. For over a decade now, we've been looking for ways to overcome this limitation to deliver web applications that are more like "regular" applications. Around the turn of the century, AJAX started to appear. Now the client could send and receive data from the server without having to interrupt the user with a page load. Great, but we still lacked a way for the server to initiate communication with the client. Comet, sometimes called Reverse-AJAX, seemed to be the answer in 2006. But long-polling HTTP requests are not perfect by any means, and the search continued for a better way. Could WebSockets be the ultimate solution to this problem? Let's give them a try and see how it works.

WebSockets (read the official spec here) are HTML5's new method of connecting the server with the client, via an "upgraded" HTTP connection. Basically, the client makes an HTTP request to the server and requests an upgrade to WebSockets. If the server agrees, the socket stays open and both the client and server can send information back and forth any time they want. In a compliant web browser a WebSocket API is available in JavaScript that makes this possible, much like the XMLHttpRequest API does for AJAX. So if you have Chrome, Safari, or Firefox 4, you're all set on the client side. But what about the server?

You'll need a web server that supports WebSockets in order to run an application that makes use of them. I wanted to find a way to use WebSockets with Tomcat, but had no luck. There is a project underway to add WebSockets support to Tomcat, but it is done via an add-on of what is essentially another server. I wanted something a little cleaner, so I went with Jetty 7.2. If you're going to serve a Java application with WebSockets I think Jetty is the way to go. Using Java's NIO, it has supported a highly efficient version of the Bayeux protocol (CometD) for a while now. WebSockets support is the logical next step, and it's nicely integrated. A WebSocketServlet class is available for you to extend which makes life easy. Here's a really simple one I made to test with.
import javax.servlet.http.HttpServletRequest;
import org.eclipse.jetty.websocket.WebSocket;
import org.eclipse.jetty.websocket.WebSocketServlet;

public class WebSocketTestServlet extends WebSocketServlet {

protected WebSocket doWebSocketConnect(HttpServletRequest request,
                                      String protocol) {
 return new TestWebSocket();
}

}
And here's the code for the TestWebSocket class:

import java.io.IOException;
import org.eclipse.jetty.websocket.WebSocket;

public class TestWebSocket implements WebSocket {

private Outbound ob;

public void onConnect(Outbound outbound) {
 this.ob = outbound;
 SocketStorage.getInstance().addSocket(this);
}

public void onDisconnect() {
 SocketStorage.getInstance().removeSocket(this);
}

public void onFragment(boolean arg0, byte arg1, byte[] arg2, int arg3,
  int arg4) {
 // ignore fragments for now
}

public void onMessage(byte arg0, String arg1) {

 System.out.println("--- received string \"" + arg1 + "\"");
 try {
  ob.sendMessage("thanks for the " + arg1);
 } catch (IOException e) {
  e.printStackTrace();
 }

}

public void onMessage(byte arg0, byte[] arg1, int arg2, int arg3) {
 // ignore byte arrays for now
}


}

That looks really simple, right? All you need to do is add a mapping for this servlet in your web.xml and you'll be able to connect via WebSockets to that URL. So how does this work? Well, if a request comes into the servlet with the right upgrade request headers, the request will be handed off to the doWebSocketConnect method instead of the regular doGet or doPost methods. Then it's up to you to return an instance of a class that implements the WebSocket interface. In the WebSocketServlet base class, Jetty will take care of holding on to that instance and making sure it gets it's data from the client. But remember, if you want to be able to send data asynchronously to those clients you need to keep track of that instance yourself somewhere. That's why you'll see the SocketStorage calls in onConnect and onDisconnect. SocketStorage is just a singleton I made that has a set of all the open WebSockets.

It should be obvious what most of the methods in TestWebSocket do. The Outbound instance that is passed in to the onConnect method is what you'll use to send data outbound to the client, so you need to hold on to it. The onMessage method is invoked whenver the client sends some data. In my example, I basically just echo back to the client whatever it sends me. If you were to add another method to TestWebSocket, you could easily expose the ability to send data to the client unrelated to what the client is sending to the server. Just call sendMessage on the Outbound instance and the data will find its way.

What about the client-side code? It's just as simple! Look at this JavaScript for the client:

     var ws;

  function connect() {

      ws = new WebSocket("ws://yourwebsite.com/path/to/websocket/servlet");
      ws.onopen = function() {
          console.log("connection openned");
      }
      ws.onmessage = function(m) {
          console.log(m.data);
      }
      ws.onclose = function() {
          console.log("connection closed");
      }

  }

  function sendMessage(msg) {
   ws.send(msg);
  }
Note the new protocol, "ws://", in the URL given to the WebSocket constructor. If you want secure WebSockets, "wss://" is also available. By default, the WebSocket ports are 80 and 443 (non-secure and secure, respectively) the same as HTTP.

So obviously my example here doesn't do much of anything. It was just a little test I put together to see how WebSockets work in real life. But now that we have this capability, what will we do with it? The common examples people always give are chat applications, stock alerts, things like that. But I'm thinking that there's a really unique idea out there just waiting to be built using new HTML5 technologies like this. I, for one, intend to use it to make some multi-player frog games.

2011 should be the year of the WebSocket. With Safari and Chrome already supporting them and Firefox 4 due out the beginning of the year, they'll be available in a huge percentage of clients. It looks like IE9 won't support them, but IE's market share will hopefully continue to decline. What I'd really like to see is some kind of Java standard approach for the server side. Jetty's implementation seems pretty good, so hopefully sometime next year Tomcat adds support for WebSockets in a similar fashion. I'd be happy to work on adding that if anyone asked me to...

Friday, August 13, 2010

Bring Your Web Apps Alive on the iPhone with Sound!

Not long ago, we had few options as developers of web applications when it came to playing sounds on the iPhone. We could link to a sound file, but it would navigate to a QuickTime screen to play it. Since Flash is not an option and JavaScript has no built-in sound capability, we were without hope. Our JavaScript applications were doomed to be forever silent.

But now, with the HTML5 audio capabilities present in Safari on the iPhone with the latest iOS 4.0 update there is new hope! Yes, your iPhone can now play sounds inline with the web page. I'm going to talk now about how to best take advantage of this new capability while still supporting other browsers with not as much HTML5 support.

I've been using a great JavaScript audio library called SoundManager 2 for years to make the frogs on my website ribbit on command. Up until recently, it accomplished the miraculous feat of adding audio capabilities to JavaScript using small Flash files. While this was great for regular web browsers of the past (nearly every computer that can browse the web has at least Flash 8 support), with the advent of the mobile browser my website became quiet for many of my visitors. That all changed in the last few months when SoundManager updated to include support for the HTML5 Audio capabilities and the iPhone OS updated to 4.0.

There are a couple of good reasons for using SoundManager 2 to bring sound to your website. First, it has an extremely easy to use API. Second, it supports so many different browser configurations (now including mobile Safari!) in a totally transparent manner. You can be playing sounds from your web application in just a few lines of code. Let's see how simple it can be.

Step 1: Download the latest version here.

Step 2: Unzip and copy the following files to a directory in your web application (I use /soundmanager2 off the root for this).
From the script directory: soundmanager2-jsmin.js, soundmanager2-nodebug-jsmin.js, and soundmanager2.js

From the swf directory: soundmanager2.swf and soundmanager2_flash9.swf
Step 3: Include the js file in your web page.
<script src="/soundmanager2/soundmanager2.js" type="text/javascript"></script>
Step 4: Set up the SoundManager configuration. The most important item in the configuration is the location of the SWF files, but you also have to turn on HTML5 Audio support if you want it. Here's the configuration I use:
<script type="text/javascript">

soundManager.debugMode = false;
soundManager.url = '/soundmanager2/';
soundManager.useHTML5Audio = true;

</script>
Step 5: Initialize the sound objects. You can't do this until SoundManager has completely initialized itself, so it provides an onload property that you just set to an appropriate function. Here's an example.
<script type="text/javascript">

var frogSound;

soundManager.onload = function() {

frogSound = soundManager.createSound({
id: 'frogSound',
url: '/sounds/frog_1.mp3'
});

}

</script>
Step 6: Play the sound! When you want to play the sound, you just call the play() function on the variable you initialized (frogSound in the example above). You might want to put a try/catch around the play call just in case something goes wrong, but in most cases it will work and you'll hear your sound!
frogSound.play();
Give it a try for yourself and see what kinds of HTML5/JavaScript games you can make that play with sound on your iPhone. Be sure to read the documentation for SoundManager2, because the new HTML5 features and limitations are changing as more browsers add and change support for HTML5.

Sunday, July 25, 2010

Decorating for Geeks


The decoration of your home should be a reflection of your personality. How, then, should an individual who loves software engineering decorate? I decorate mainly with old arcade machines, giant inflatable Mountain Dew promotional bottles, an expertly framed poster of the periodic table of the elements, and the green Ikea plush snake.

But if I had pillows on my couch, I would want them to make a statement. Pillows can be more than a place to rest your head for a nap, they can turn your couch into a great conversation piece. Here is a collection of the best pillows and cushions I've found.

Ctrl+Alt+Del Pillows. Reboot your sofa.

RSS Pillow. Really Simple Sleep?

Facebook Pillow. Yes, even your furniture is on Facebook. 3 of your friends like this.

Calendar Icon Pillows. Can be a constant reminder of important dates.

Giant Hamburger Scatter Cushions. I wouldn't eat it, but I would sit on it.

Super Mario Bros. Mario Hat Pillow, and Super Mario Bros. Luigi Hat Pillow. Suitable for resting on after those marathon Wii sessions (or SNES, if you're old-school like me).

Fortune Cookie Pillow. The beginning of wisdom is to desire it. Your lucky numbers are 4, 33, 7, and 12.

Peanut Butter and Jelly Pillow. Put one half on each side of the couch?

Bacon Pillow, and its associate the Egg Pillow. Sleep on them, then wake up and eat them.

Ketchup Packet Pillow. Oddly enough, I could find no french fry pillows.

Cherry Pie Pillow. I'm allergic to apples.

And no cookie lover's home would be complete without the Chocolate Chip Cookie Pillow and the Chocolate Cream Cookie Pillow!

The pillows will add a lot to any room. But I'm still looking for Pac-Man curtains.

Saturday, July 24, 2010

Have a Programming Party

Want to have a completely ridiculous amount of fun this weekend? Plan to have a programming party! Yes, a programming party. In one day you can combine the three guiding principles of Amphibian.com - thinking, building, and playing - together with your friends, pizza, and beer.

Here's the deal. Back some 15 years ago, one of my favorite activities was getting together with some friends in a unused, run-down room off the side of a honey bottling plant with a couple computers and writing software. Yes, I know that sounds a little strange but don't let that distract you. We were writing games for BBS's. That's Bulletin Board Systems for you kids that don't remember a time before the Internet. We'd eat flavored honey, drink RC Cola, and write what was probably terrible code, but it was great fun.

I think back to those days often lately, because if you look at the types of software products that are really making news today you'll find them to be striking similar to the old BBS games we worked on. They were small, turn-based, social games - the same kind you'll find on Facebook or your iPhone today. Except we don't use ANSI graphics anymore. That's a shame. But the point is that if the "programming party" paradigm worked for us back then, think how much better it can work today!

Goals of the Programming Party Paradigm

1. Development of a small game or game framework in one day

This seems like a lofty goal. Does a small game have any chance of really being completed in a single day? Well, keep in mind there are 24 hours in a day. And if you have 5 friends working together, that's like 120 mythical man-hours that can be squeezed into a single box on the calendar! Alright, so I'm not serious. I don't think you could actually have a polished, bug-free, ready-to-go application done in a single day. But you could have a playable, slightly buggy, half-working application done enough to share with other people. That should be enough to convince you to either meet again to finish it (after taking feedback from your testers, a.k.a. other friends) or use what you created as the springboard for your next project.

2. Creativity in design

Don't host the party as an attempt to extract free labor from your friends. If you have an idea for what you'd like to create, great. But the point of inviting people over to the party is to get creativity in the design. You need to all throw your ideas together, bounce them off each other, throw out creative feedback as soon as it jumps into your heads. Even if they deny it, I think most people who write software are creative people. They are creating things every day. They combine different software packages together to come up with new and interesting things. When you put a whole group of creative people together with the goal of coming up with something fun and cool, it will absolutely amaze you what can be produced. Yes, software is math and algorithms and logic and a whole bunch of other dull-sounding things, but to make software products that people are really going to enjoy takes a creative touch. Windows 7 and Farmville are both software products, but people enjoy Farmville (although I don't know why). Do people enjoy Windows 7? Eh, maybe if they used Windows Vista they'll say they enjoy Windows 7 because it doesn't suck as much, but that's a whole different meaning of enjoyment. Windows is a tool. Farmville is entertainment, made by creative people. While I guess you could use a party to make a tool, I really think they are best suited to making fun-to-use software that is like nothing the world has ever seen before.

3. Social interaction and fun

There's a stereotype that programmers are loners. Freaky nerds who like to sit alone in poorly lit basements cranking out code. I think people who like to write software are just as social as the rest of the world, but they have a harder time finding people to socialize with. For example, when I am visiting someone, usually because I've been dragged someplace by my wife, people try to talk to me about the local sports teams, or last night's episode of American Idol, or something along those lines. I have nothing to add to conversations of that nature, because I don't follow any sports teams or television shows. But if someone started talking to me about the new Facebook social graph API, I could go on for hours. So getting together with people who share the common interest of programming will be great fun. Even if you don't actually produce anything that works in the end, you'll have probably learned something and had a great time anyway. If you are tempted to try having your programming party over Skype or something like that, think again! You've actually got to get in the same physical location as your friends. Back in the BBS days, this wasn't optional. We were programming in DOS mostly. There was no multi-tasking OS to let us use Borland Turbo Pascal and dial up to BBS chat at the same time (in fact, many BBS's only allowed one caller at a time anyway!). If we wanted to talk to each other about what we were developing, we had to meet in person. It worked for us then, and it will work today.

Programming Party Plan

This is my outline for having an effective and fun programming party.

1. Set up your location. To use my old BBS programming analogy, set up your own honey plant. Everyone in attendance will need to have a computer to work on and a place to sit. Ideally, everyone should be in the same room or two really close rooms. Set up folding tables and chairs if you need to, and get extra furniture out of the way. If people are bringing their own desktops or laptops, make sure they can all hook up to your Internet connection, either wired or wireless. You should also try to have a whiteboard or similar large drawing area nearby.

2. Do your homework. Depending on what type of application you plan to create, you or some of your friends may need to do some research before the event has to start. You don't want to waste a lot of time the day of the party reading whole books, so make sure you tell people in advance if there is a particular topic they should brush up on. Using the Internet to look things up while working is one thing, but you don't want to spend too much time learning the language you are trying to work in while you're working.

3. Start with a schedule, but be flexible. I would recommend getting started early, like 0900. Plan on going until at least 2300. Make sure everyone has a good breakfast, lots of protein and Mountain Dew. Do a couple hours of brainstorming for ideas, then order some lunch while you get started on the programming. Keep in constant communication throughout the day, so if anyone gets stuck on something they can get immediate help. Take breaks every 1.5 hours and have a snack. And more Mountain Dew. Order some pizza for dinner, then break out the alcohol. You don't want to start drinking too early in the day, or the code will get really messed up later on. See what you end up with at the end of the day!

Addressing Criticisms

Many will read this and think I'm crazy. "This will never work!" they will cry out. "Games take years to develop and millions of dollars!" If you think that, you and me are talking about totally different kinds of games. I'm talking about casual games. The kind you pay $0.99 for on your phone or play against your friends on Facebook. They are no different than the old BBS door games. People like me used to crank those things out in no time, working alone, without the benefit of Google searches to help us out when we get stuck on something. This will work because the resources at our disposal today are infinite compared to what we had 15 years ago, and because when a group of friends get together to accomplish something fun they will never fail.

When you schedule your party, don't forget to invite me.

Monday, July 12, 2010

Cheese Soup and the Semantic Web

If you haven't yet heard of the Semantic Web, don't worry. You will eventually. It's going to take off any day now. Really. Web 3.0. Ok, so the world has been moving very slowing in the direction of the Semantic Web. Sorta like IPv6. We'll get there eventually.

So I've decided that I want to help move this thing along. I see big potential here, and I think there are some practical applications already. Consider searching for a recipe on Google. Let's say you want to make some cheese soup. It's delicious and I highly recommend it. Your search results will probably look something like this...

That first site sounds like a great place for cheese soup recipes, but wait... Alton Brown's recipe has a bit of extra information displayed. A rating, number of reviews, prep time, and cook time. How is this possible? Google's pretty smart, but how did they manage to parse that information out of the web page? Searching for other types of recipes reveals that this information is listed on most recipes from the big recipe sites (Food Network, All Recipes, My Recipes). Here's the trick - they've already embraced the semantic web by adding RDFa to their web pages. Google has the ability to parse this information out and uses it for better search results.

It just so happens that CheeseSoup.com is my site, and today we're going to work through the steps of adding RDFa to my cheese soup recipes. Not only will this enable me to better serve the cheese soup-loving public, but it will teach a lot of practical concepts in the emerging world of machine-readable web pages.

To begin, let's first understand what RDFa is. It stands for Resource Description Framework in Attributes. Basically, it's an extension to XHTML that adds a set of attributes for embedding metadata in web documents. All we'll have to do is add some simple attributes to the existing XHTML tags and we're done.

To make this work, you have to include the namespace declaration for the RDF vocabulary you want to use. You can see how I do that here in the markup from one of the recipes. I've simplified it a little for this example, but it looks pretty much like this:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML+RDFa 1.0//EN"
"http://www.w3.org/MarkUp/DTD/xhtml-rdfa-1.dtd">

<html xmlns="http://www.w3.org/1999/xhtml"
xmlns:v="http://rdf.data-vocabulary.org/#">

<head>
<meta http-equiv="Content-type" content="text/html;charset=UTF-8" />
<title>Cheese Soup</title>
</head>
Make sure you put the # at the end of the URL, it's required. You can use this namespace whenever you want to mark up recipes, and also people, places, events, and reviews. There are lots of other RDF vocabularies for things other than recipes (Dublin Core, Friend of a Friend) you can use, and you can even make your own. We might cover that in a later discussion.

Another thing to note in the code snippet above is the doctype declaration. I'm using the XHTML+RDFa document type because this document will be XHMTL with RDFa and I want it to be completely valid.

Now when it comes to the actual recipe, we just have to add the right attributes in the right places. There is a containing div around the whole recipe, so we'll add a typeof attribute to that indicating that the div represents a recipe. Everything contained inside the div will be associated with the recipe.
<div typeof="v:Recipe">
Inside the recipe div, we'll add property attributes in the appropriate places to mark up the various properties of recipes.
<h2 property="v:name">Canadian Cheddar Cheese Soup</h2>
Basically, all we've done here is set the "name" property of a "Recipe" to be the contents of the h2 tag. These tag contents are now doing double-duty. They're both human-readable and machine-readable. Things get a little more interesting when we get to the ingredients list.
<ul>
<li rel="v:ingredient">
<span typeof="v:Ingredient">
<span property="v:amount">1 lb</span>
<span property="v:name">shredded cheddar cheese</span>
</span>
</li>
<li rel="v:ingredient">
<span typeof="v:Ingredient">
<span property="v:amount">1/2 lb</span>
<span property="v:name">bacon</span>
</span>
</li>
...
</ul>
You'll notice that the li tags contain a rel attribute. This indicates a relationship with another resource, which in this case in an ingredient. Ingredients are types of objects just like Recipes, so inside the li tag we put a span with the typeof attribute indicating that its contents are an Ingredient. Ingredients have names and amounts, so spans inside have property attributes to annotate them. All of this extra information is included for the machines to read, and the humans don't even notice!

There's just one more special thing to consider. What happens when you want the made-for-human content to look different than the made-for-machines content? This happens especially when working with times. RDF wants everything in a particular date/time format, which happens to be ISO 8601 duration format. Fortunately, there is a simple solution. An optional content attribute overrides the contents of the element, so the human sees one thing and the machine sees something else. Here's an example of this for the preparation time.
<p>Preparation time: <span property="v:prepTime" content="PT15M">
15 minutes</span></p>
The rest of the recipe (instructions, rating, etc.) can annotated in a similar fashion. You can see the complete document here. As you can see, the page looks perfectly normal despite having all this great "hidden" information. Be sure to view the source of the page to see all the RDFa markup, and notice the link at the bottom. The page successfully validates as XHTML + RDFa using the W3C validation tool.

If you're interested in everything that the Google bots can use from your RDFa markup, check out this page on Google Webmaster Tools. Hopefully, my cheese soup recipes will get re-indexed soon and the extra info will show up in the search results. But let's not do this just for Google searches. There's amazing potential here if we annotate all of our web pages with as much RDF as possible. Check out the Dublin Core and Friend of a Friend vocabularies. The Semantic Web will only appear if we build it!