I enjoy looking at my age group to see how I rank when compared to others near my age so I decided to write a quick script to generate age groups for the race: http://www.xmech.net/Leopards5kAgeGroups.html

I did much better than my last 5k (4th out of 5th in my age group) and I hope to be in the top 10 next time.

Being able to look back at your collection of photos will allow you to build an idea of how your life unfolded through a given month, year, and beyond.

To start things off I snapped a picture Sunday night:

Jamie and I decided to try and make some Gnocchi for dinner (it’s always nice to spend some time together cooking) and it turned out great.

I’ll be posting my photos to my picasa album and hope to continue this indefinitely.

Zoo World also has a few mechanisms in place that prevent people from earning money while not at their computer, specifically a button appears when a period of inactivity has passed and the zoo day timer stops ticking until you press this button (there’s a handful of other buttons that can appear but this is the main culprit).

Recently I discovered java’s robot library (java.awt.robot) and this seemed like the perfect opportunity to use it and ZooBot is the result. ZooBot is a little application that will scan the screen every few minutes looking for these buttons and presses them if they’re found.

Now you can earn cash while sleeping! I wonder what other facebook games I could use method on…

We’ve all been there, you’re working on a website and you’re constantly testing your changes to make sure you haven’t broken anything unexpectedly. This can really become time consuming when you make a code change that could affect multiple features. This process is also susceptible to human error (sometimes you might forget to test a feature or run slightly different tests each time). If only there was a way to take the human out of the equation and make this process faster… wait a minute, we can write a robot to run our tests for us! Let’s take a look at how we would accomplish this using PHP.

The idea of using a robot to run tests for us is nothing new, take a look at some common unit testing methods and you’ll get an idea. However if we write our robot in PHP we can easily run it from a development environment without needing to download, install, or modify any existing code.

Setup The Example Project

Before we can begin we’ll need to setup a simple application so our robot has something to test. The example we’ll be using is a very rudimentary todo list manager. We want to make sure users can perform the following actions: sign-in, add/remove tasks, and then sign-out. Download this zip file which contains a few php files and unzip it into your document root directory on your local test machine.

The example application is relatively straight forward so you can browse through it if you’d like; however we’ll go over most of it as we go through the tutorial. There are a few steps we need to complete before we can start writing our robot. First open the mysql.php file and provide a valid username/password.

// MySQL database connection settings
$username = "change";
$password = "me";

Now execute the setup file by visiting it (http://localhost/todo/setup.php) in your browser. If everything went as expected you should see the word “Success” and we can continue on (this will create a database, required tables, and some test data for us to work with).

Now you can play around with the todo app (log in with test/password1, create and delete a task, logout).

(Please note, the application is extremely basic since we’re only using it as an example, therefor I’ve ignored some common practices like hashing passwords, cleaning user input, styles, etc)

Building Our Robot

With all of the setup out of the way we can finally get started on the fun part! We’ll be using cURL to talk with our server and regular expressions to decide if our test passed or failed. You can view the complete script here, don’t worry, we’ll walk through it step by step.

Base Robot Class

We’ll start by looking at our base robot class, later we’ll build our own todo robot class to work specifically with our example and we’ll inherit from our base robot class. This way you can easily extend the base robot to build robots for other projects.

Lets start by defining a few variables that will be used to report the status of our tests as well as the variable that our cURL instance will be assigned to:

class Robot {

    private $passTemplate = ': <span style="color: green;">Pass</span>';
    private $failTemplate = ': <span style="color: red;">Fail</span>';
    private $ch;

The class constructor takes care of configuring the bulk of our cURL options however there is one item that needs some attention:

    function __construct() {
        // Sometimes servers are configured with "&" as their url separator.
        // This can break the url strings we construct so we'll change it to use the real "&" character.
        ini_set('arg_separator.output', '&');
        $this->configureCurl();
    }

You can find more details about arg_separator here.

    function getResultTemplate($status) {
        if ($status == "passed") {
            return $this->passTemplate;
        } else {
            return $this->failTemplate;
        }
    }

The above function will retrieve the formatted pass/fail string we defined earlier so we can output it to display the status of a test.

It’s fairly likely that our robot will need to interact with forms to complete some of our tests so we need a way to encode our test strings before they can be sent in a URL as a form post. Lucky for us PHP happens to have a function that will take care of this for us so all we need to do is pass an array containing our test strings. The array key should be the name of the form and the value should be the contents of the form field. You’ll see an example of this later on.

    function buildUrlQuery($data) {
        return http_build_query($data);
    }

Next we have the cURL configuration function which was called by the robot class constructor. The comments should give you a good idea of what each option does however there are a few that need some extra explanation:

• CURLOPT_USERAGENT: Some websites check the user agent so you might need to mascurade as a web browser.
• CURLOPT_COOKIEFILE: cURL doesn’t keep track of cookies by default so if your application uses sessions we need to define a file that cURL can write and load cookies from.
• CURLOPT_FOLLOWLOCATION: This tells cURL to automatically follow any header redirects.
• CURLOPT_RETURNTRANSFER: Normally cURL will respond with a true or false after the request. Setting this option will return the any HTML that the server sent to us as a response to our request.

    function configureCurl() {
        // Open connection
        $this->ch = curl_init();

        // Fake user agent
        //curl_setopt($this->ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1");

        // Handle cookies (this allows sessions to work)
        curl_setopt($this->ch, CURLOPT_COOKIEFILE, 'cookies.txt');

        // Automatically follow 302 HTTP redirects
        curl_setopt($this->ch, CURLOPT_FOLLOWLOCATION, 1);

        // Tells cURL to return any HTML instead of a boolean depending on request status
        curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, 1);

        // Use a Post instead of usual Get
        curl_setopt($this->ch, CURLOPT_POST, true);
    }

We wrap up the robot base class with the function that actually runs the tests. This includes setting the cURL url to our test url and we’ll also set the post data (CURLOPT_POSTFIELDS) that we encoded earlier.

    function runTest($url, $urlFields) {
        // Set the url
        curl_setopt($this->ch, CURLOPT_URL, $url);

        // Assign form post fields from our data set
        curl_setopt($this->ch, CURLOPT_POSTFIELDS, $urlFields);

        // Execute request
        $result = curl_exec($this->ch);

        // Info about the post request
        //$info = curl_getinfo($this->ch);

        return $result;
    }
}

Notice that we return the $result containing HTML thanks to the CURLOPT_RETURNTRANSFER setting which we configured earlier. Later we’ll use the $result data in our todo robot class to check and see if the test passed or failed.

Also notice the $info variable containing a large amount of meta information about our request (times, sizes, counts, etc). It’s commented out because it’s generally not helpful but could come in handy if a request isn’t functioning properly.

That’s all for the base class, now we can move on to our todo robot class where we create the actual tests that will run against our todo application.

Todo Robot Class

First we’ll go ahead and define our class and inherit from the base class:

class TodoRobot extends Robot {

You’ll notice that each test has it’s own function so we can call them independently. Lets start with the login test which takes two arguments, a username and password:

    function testLogin($username, $password) {

Next construct a string (using our buildUrlQuery function from earlier) that contains the data normally sent to the server by the login form:

        $urlArgs = $this->buildUrlQuery(array("username" => $username, "password" => $password));

Now we can build our first test. We need to send the request to the location that the login form is pointing to. Open the index.php page and you’ll notice that the form is pointing to login.php

<form method="post" action="login.php">

So we’ll configure our test to run against login.php with our encoded arguments:

        $result = $this->runTest("http://localhost/todo/login.php", $urlArgs);

Next we need to find out if our test passed or failed. We’ll use regular expressions to look for a unique status identifier in the $result HTML. Before we can continue we need to see what will happen if we login. Fire up the todo app and login using valid credentials. Upon a successful login you’ll notice the form is replaced with the following:

<a href="add.php">Add new item</a> | <a href="logout.php">Logout</a>

Using this we can look for the words “Add new item” to verify we logged in successfully. Now return the successful test template string so we can print out the status. This approach also covers any failed login attempts (if you were to pass an incorrect username/password combination).

        if (preg_match('/Add\snew\sitem/', $result)) {
            return $this->getResultTemplate("passed");
        } else {
            return $this->getResultTemplate("failed");
        }
    }

From here on out the rest of our todo robot consists of different test functions: adding a task, removing a task, and finally logging out. All tests follow the same format as our login test. You can see how easy it is to quickly generate various tests, all you need to do is follow the same steps as we preformed in the login test:

• Perform the action you’re trying to test by hand
• Find unique text to match on and decide if your test passed
• Return a passed/failed status message

    function testAddingTask() {
        $urlArgs = $this->buildUrlQuery(array("item" => "Task added by robot!"));
        $result = $this->runTest("http://localhost/todo/addItem.php", $urlArgs);

        if (preg_match('/Item\sadded/', $result)) {
            return $this->getResultTemplate("passed");
        } else {
            return $this->getResultTemplate("failed");
        }
    }

    function testRemovingTask($itemNumber) {
        $urlArgs = $this->buildUrlQuery(array("item" => $itemNumber));
        $result = $this->runTest("http://localhost/todo/remove.php", $urlArgs);

        if (preg_match('/Item\sremoved/', $result)) {
            return $this->getResultTemplate("passed");
        } else {
            return $this->getResultTemplate("failed");
        }
    }

    function testLogout() {
        $result = $this->runTest("http://localhost/todo/logout.php", "");

        if (preg_match('/Logged\sout/', $result)) {
            return $this->getResultTemplate("passed");
        } else {
            return $this->getResultTemplate("failed");
        }
    }
}

Finally we can run our first test!

Running the robot

Instantiate the robot, print some text to the screen so we know which test is running, and run it!

$todoRobot = new TodoRobot();
echo "<h3>Login</h3>";
echo "Correct credentials"  . $todoRobot->testLogin("test", "password1");
echo "<br /><hr>";

The rest of the tests:

echo "<h3>Add Task</h3>";
echo "Adding task" . $todoRobot->testAddingTask();
echo "<br /><hr>";
echo "<h3>Remove Task</h3>";
echo "Remove task" . $todoRobot->testRemovingTask("1");
echo "<br /><hr>";
echo "<h3>Logout</h3>";
echo "Logout" . $todoRobot->testLogout();

Go ahead and run the robot (http://localhost/todo/todoRobot.php) and you should see all of the tests have run and passed!

Now lets take a look at what a failed test would look like. Replace the login test block with the following code:

echo "<h3>Login</h3>";
echo "Correct credentials"  . $todoRobot->testLogin("test", "password1");
echo "<br />Wrong username/password" . $todoRobot->testLogin("kerry", "password1");
echo "<br />Only username" . $todoRobot->testLogin("kerry", "");
echo "<br /><hr>";

Run the robot again and you’ll see that our recently added tests have failed. This is to be expected because we didn’t pass the correct credentials. You can see how this would be useful because if the tests had passed we’d have a serious issue on our hands.

Conclusion

Hopefully you can see how beneficial having a robot to automate your testing can be. After you make a large code change you can run your robot and make sure the basic functionality of your website hasn’t been changed in any unexpected ways.

Helpful Links

• Complete robot file
• Todo example application files
• cURL library

Intro

If you’re reading this I’m going to assume you’re familiar with XML so we’ll skip the XML introduction and get right into it. We’re using SimpleXML because true to it’s name, it makes working with an XML document simple. However it has a large draw back that might cause you to use a different parsing method, the problem lies with the entire XML document needing to be loaded into memory before parsing can begin. This isn’t a problem with a 100k document however if you’re parsing something in 100+MB range… you should probably use different method such as parsing from a stream.

Let’s use the following scenario: Twitter is popular with the hip kids these days so lets generate a list of people I follow whom have at least 2,000 people following them. We’ll need to add some change data (so any apps that consume our modified data will know how many followers we originally had) and then save our changes to a new XML document.

Lucky for us twitter just so happens to provide the data we need in XML format:

http://twitter.com/statuses/friends.xml?screen_name=dremex

Now we have a list of the last 100 people I’ve followed and we can begin to parse through the document.

Code Walkthrough

Here’s the complete code for this article, go ahead and take a look and then we’ll break it down and walk through it:

http://www.xmech.net/tutorials/twitterFollowersDisplay.php

First we’ll start with the bottom of the file and then walk through each class function call:

$friendList = new TwitterFollowers("http://twitter.com/statuses/friends.xml?screen_name=dremex", 2000);

We’re going to create a TwitterFollowers object and pass it the url of our friends feed followed by the number of followers we want to check to make sure our friends have.

function TwitterFollowers($file, $maxFollowersCount) {
    $this->loadXml($file);
    $this->maxFollowersCount = $maxFollowersCount;
    $this->originalFollowers = $this->followerCount();
}

The class constructor takes care of loading our friends feed, assigns a local variable to keep track of how many followers to check for, and sets up a statistic to keep track of our original follower count. Let’s go through each step in order:

function loadXml($file) {
    $this->xml = simplexml_load_file($file);
}

This is the first step in parsing any XML with SimpleXML. There are two methods to load XML data, simplexml_load_file and simplexml_load_string. Pretty self explanitory, file loads from a file and string loads from a string. For our project we’re going to use the file loading (or in our case a URL) and load the XML data into our local $xml variable.

As an FYI the syntax for string version of our document would be:

$this->xml = simplexml_load_string($fileAsString);

The next function we’re going to look at grabs the number of our current friends:

function followerCount() {
    return count($this->xml->user);
}

This is where SimpleXML starts to shine with its simplicity. SimpleXML assigns elements to array key value pairs so we can easily iterate over them, more on this later.

Next we want to go ahead and check our friends for their followers count and remove any who don’t have at least 2,000:

$friendList->updateFollowerList();

This calls our objects “updateFollowerList” function:

function updateFollowerList() {
    $this->checkFollowers();
    $this->removeFollowers();
}

Let’s take a look at both of these starting with checking follower count:

function checkFollowers() {
    foreach ($this->xml->user as $user) {
        if ($user->followers_count < $this->maxFollowersCount) {
            array_push($this->toRemove, $user);
        }
    }
}

As I mentioned before, since SimpleXML created an array for us, we can easily iterate over each user record and check to see if they have fewer followers than our $maxFollowersCount variable we setup at the beginning.

It’s interesting to include the following code so you can get a feel for how the SimpleXML array is structured. Just above the foreach in the “checkFollowers” function, go ahead and add:

echo "<pre>";
print_r($this->xml);
echo "</pre>";

Now the SimpleXML array will be displayed in a readable manner. It should look something like this:

http://www.xmech.net/tutorials/simplexmlFormatExample.html

Go ahead and remove the debug info and let’s continue on. If the user doesn’t have enough followers we’ll go ahead an add it to and array of “users to remove” so we can remove the users later. We don’t want to remove them now because we would loose our position in the loop and never parse through all of the records.

Next we call the “removeFollowers” function that handles removing friends from the data set:

function removeFollowers() {
    foreach ($this->toRemove as $user) {
        $oNode = dom_import_simplexml($user);
        $oNode->parentNode->removeChild($oNode);
    }
}

Now we’re going to walk through our list of friends to remove and using DOM we’ll remove them from our SimpleXML array. Unfortunately SimpleXML has no way to remove elements. First we’ll call dom_import_simplexml and pass it our user (a SimpleXML element) and then remove it. Now our $followers array will no longer contain this user.

Next we’ll need to add statistics for any changes made to this document:

$friendList->addChanges();

SimpleXML makes adding elements simple:

function addChanges() {
    $this->xml->addChild('changes');
    $this->xml->changes->addAttribute('before', $this->originalFollowers);
    $this->xml->changes->addAttribute('after', $this->followerCount());
}

First we’ll name our new element “changes” and give it a few attributes such as our friend count before and after our changes. The above code will add the following child element to our SimpleXML data at the end of our document:

<changes before="50" after="25"/>

Now we’ll wrap up by saving our modified document:

$friendList->saveFollowerList("updatedFollowers.xml");

Once again SimpleXML makes our life easy:

function saveFollowerList($file) {
    $this->xml->asXML($file);
}

We simply need to call the “asXML” and feed it a file name and it will save our XML to a file. If no file name is passed it will return the data as a string (handy for debugging).

That’s it! However there are a few things that could be done to improve usability.

Error Handling

Lets and add some error checking to make sure our XML document isn’t malformed before we begin parsing. After the opening php tag add the following:

libxml_use_internal_errors(true);

This command will let us handle any errors instead of libxml.

Next replace the “loadXML” function with:

function loadXml($file) {
    $this->xml = simplexml_load_file($file);
    if (!$this->xml) {
        echo "<b>Failed loading XML</b>:<br />";
        foreach(libxml_get_errors() as $error) {
            echo $error->message;
        }
        exit(1);
    }
}

First we’ll check to see if we manage to load any XML data. Since we toggled the internal libxml error to true our $xml variable will be empty if there were errors when SimpleXML tried to load the file. Now we can inform the user that there were errors and then iterate over them using libxml_get_errors().

libxml_get_errors() will provide an error message that contains the following properties: message, line, and column of the error. For our purposes we’re just going to use message since it contains the entire error. After printing all of the errors out we’ll exit so we don’t continue and try to parse through an empty document.

Now modify the followers.xml document so the first three lines are:

<?xml version="1.0" encoding="UTF-8"?>
<users type="array">
<user><user2>

Run the script and you’ll get the following error:

“Failed loading XML:
Opening and ending tag mismatch: user2 line 3 and user expected ‘>’ Premature end of data in tag users line 2″

Conclusion

Using SimpleXML makes a somewhat difficult task (working with XML) simple enough that some massive changes to a document can be done in only a few short lines.

Helpful links

• http://apiwiki.twitter.com/Twitter-REST-API-Method%3A-statuses%C2%A0friends
• http://php.net/manual/en/book.simplexml.php

We’ll be installing the flooring later this week (which means I have to finish putting the cement sealer on the floor… I’m not looking forward to that)

For anyone concerned with the acoustic problems a wood floor may cause, there will be a sound absorbing layer underneath the wood and we plan to add an area rug for the middle of the room that should soak up sound at the proper bounce angles.

We also built a cabinet (from scratch) to house all of the electronics. We still need to fill in some gaps, sand, router the edges, and paint but it’s turning out very nice.

We also cleaned up the speaker wiring and added our face plates

We also managed to order and paint all of the face plates for outlets and speaker boxes.

Now it’s time to put some paint on the walls.

All of our wires should be arriving later today so we can glue the PVC and string the wires through later tonight.

We’re hoping to start dry walling mid week if the budget can hold up to it.