Anyway fast forward two and half years and I find myself in a similar situation. We have only just begun to start writing bits of JavaScript code based around prototype.js to help us create richer user experiences in our products if we detect that JavaScript is enabled in the browser. This now means I want to ensure that we are using the same rigour when writing these bits of code as we do in all other parts of the application – just because its JavaScript and executed inside the browser this doesn’t mean it shouldn’t be tested.

I’ve just spent the morning getting JsUnit installed and figuring out how to get it to run as part of a continuous integration process, as well as thinking about how to write tests for some slightly different scenarios. Here’s what I’ve discovered today:

Installing JsUnit

Couldn’t be easier … go to www.jsunit.net and download the latest distribution, and extract into a folder on your system somewhere, lets say
/jsunit for now. The distribution contains both the standard test runner as well as jsunit server which you will need it if you want to hook it into an ant build.

Writing Tests

In JsUnit we place our tests in a HTML Test Page which is the equivalent of a Test Class, this test page must have a script reference to the jsUnitCore.js so the test runner knows its a test. So lets work through a simple example. Let’s say we want to write a function that returns the result of adding two parameters together. The Test Page for this might look like this:

For now lets save this file to /my-jsunit-tests/addTest.html

To run the test you need to point your browser at the following local url:

file:///jsunit/testRunner.html?testPage=/my-jsunit-tests/addTest.html

The test will not run since we haven’t defined the add function. Let’s do that (very crudely):

Now if you go to that URL it will run the test and report that it passed. Excellent, we’ve written a simple test in JavaScript. Now lets extend this a little, lets say I want to write something more complicated like a piece of JavaScript that uses Prototype.js to update the DOM of a page. Is this possible? Can I do that test first? It turns out that you can …

Lets say we have a div on the page called ‘tableOfContents’ and we want to use Prototype.js to dynamically inject a link onto the page that says [show] and lets say we want to write a function that will toggle this link to say [hide] when the user clicks on it, this link will also set the visible state of the table of contents itself which for now we’ll say is just an ordered list (OL). Our test page is going to be slightly more complex …

There are some differences in this test. Firstly the html contains some markup, that I’m using as the containers for my table of contents. The table of contents has a header and the contents in the form of an empty ordered list. Now I know that I want the javascript to execute when the page is loaded, so I’ve written this test to assume that the script will run and will inject and element called ‘linkToggleToc’ which is the show/hide link next to the heading. Therefore the first line of the test uses prototype.js element selector notation to set a local variable called title to the value of the title of the element that has the id ‘linkToggleToc’. If the script failes to execute then this element will not be present and the subsequent assert will fail. If the assert succeeds, then we call the toggleTableOfContents function and then repeat the same evaluation only now we are checking to see if the link has been changed.

The code for tableOfContents.js is as follows:

Now if we run this test in the same way we executed the previous test it will pass. I accept that this example is a bit contrived since I know it already works and I’ve skimmed over some of the details around it. The point I’m trying to make though is that you can write unit tests for pretty much any kind of JavaScript you need to write, even tests for scripts that do dom manipulation, or make AjaxRequests etc.

Setting up the JsUnit server so you can run it in a build

JsUnit ships with its own ant build file that requires some additional configuration before you can run the server. The top of the build file contains a number of properties that need to be set, here’s what you set them to ( using the paths that I’ve been using in the above example)

You can then type the following command in the root of the jsunit distribution to launch the jsunit server, executes the test, and outputs a test results log file, formatted just like JUnit, and reports that the build was either successful or not if the test fails.

  ant standalone_test

Remember that in this example I’ve used a simple Test Page, however JsUnit, like any XUnit framework allows you to specify Test Suites, which is how you would run multiple Test Pages. Also the parameters in the build file woudn’t be hardcoded in you continuous integration process but would rather be passed in, and you would want to call it from your projects main ant build file … all of which is pretty simple to configure, once you know what is you want to do and what’s possible.

This is a very interesting way of looking at Agile. The talk is a based on an assumption – that the people involved in software development processes deserve more attention. I agree with the many of the points made in the talk, particularly around the intangible nature of the software we build. Which is one of the reasons we are still, as an industry, trying to discover the best practises, and a true sense of engineering discipline.

Its well worth watching this talk, especially to get a different perspective on Agile, focusing on the social and cognitive effects. She used the Prisoners Dilemma to illustrate some of these points and to be honest I had never thought of Software Engineering in those terms, yet as a metaphor it fits so very well,

in software engineering this is reflected in the fact that software is an intangible product so sometimes we cannot verify that our colleagues behave in the same way … so what Agile software development does is it banishes the inability to verify that co-operation is reciprocated, by increasing the process transparency, in such a way it banishes the working assumption of the prisoners dilemma.

Wonderful tech talk by Gerard Meszaros who is a consultant specialising in agile development processes. In this particular presentation Gerard describes a number of common problems encountered when writing and running automated unit and functional tests. He describes these problems as “test smells”, and talks about their root causes. He also suggests possible solutions which he expresses as design patterns for testing. While many of the practices he talks about are directly actionable by developers or testers, it’s important to realise that many also require action from a supportive manager and/or system architect in order to be really achievable.

We use many flavours of xUnit test frameworks in our development group at Talis, and we generally follow a Test First development approach, I found this talk beneficial because many of the issues that Gerard talks about are problems we have encountered and I don’t doubt every development group out there, including ours, can benefit from the insight’s he provides.

The material he uses in his talk and many of the examples are from his book xUnit Test Patterns: Refactoring Test Code, which I’m certainly going to order.

Good programmers manage to get a lot done anyway. But often it requires practically an act of rebellion against the organizations that employ them. Perhaps it will help to understand that the way programmers behave is driven by the demands of the work they do. It’s not because they’re irresponsible that they work in long binges during which they blow off all other obligations, plunge straight into programming instead of writing specs first, and rewrite code that already works. It’s not because they’re unfriendly that they prefer to work alone, or growl at people who pop their head in the door to say hello. This apparently random collection of annoying habits has a single explanation: the power of holding a program in one’s head.


Whether or not understanding this can help large organizations, it can certainly help their competitors. The weakest point in big companies is that they don’t let individual programmers do great work. So if you’re a little startup, this is the place to attack them. Take on the kind of problems that have to be solved in one big brain.

so very true … !

For example a simple test script that goes to Google and verifies that the text “Search:” is present on the screen and the title of the page is “iGoogle” looks like this:

You can choose to export the script in several other languages, including PHP, in which case the test script it produces looks like this:

The Export produces a valid PHPUnit test case that uses the Selenium PHP Client Driver(Selenium.php). Whilst the script is valid and will run you do need add a little more to it before the test will correctly report errors. As it stands all errors captured during the test are added to an array called verificationErrors, by catching the assertion Exceptions that are thrown when an assert fails, in other words if you ran this test as it is, and it did fail you wouldn’t know! To correct this we need to do two things. Firstly, each assert needs to have a message added to it which will printed out in the test report if the assert fails. Secondly we need to modify the tearDown method so that once a test has run, it checks the verificationErrors array, and if any failures have occurred, fails the test. After making these changes the PHP test script looks like this:

Obviously, I have also given the PHP Class and test function slightly more meaningful names. Now you have a PHP Unit Test case that will use the Selenium PHP Client Driver with Selenium Remote Control to launch a browser, go to the specified URL, and test a couple of assertions. If any of those assertions fail, the tearDown method fails the test … pretty cool, right?

Well now it get’s better. Because the Selenium Client Driver has a published api which is pretty easy to follow, there’s no reason why you can’t just write test cases without using Selenium IDE … for those who want to you could even incorporate this into a TDD process. But for all this to hang together we need to be able to run a build, on a Continuous Integration server which checks out the code, runs unit tests and selenium regression tests against that code line, and only if all tests succeed passes the build.

We are currently using ANT, and CruiseControl to handle our CI/Automated build process. When running the automated suite of tests we need to ensure that the Selenium Remote Control server is also running which creates some complications. The Selenium Remote Control server takes several arguments which can also include the location of a test suite of html based selenium tests – which is really nice because the server will start, execute those tests and then end. Unfortunately you can’t invoke the server and pass it the location of a PHP based test suite. This means you need to find a way to start up the server, then run your tests, and once they are complete, shut the selenium server down.

He are the ANT targets that I have written to achieve this, if anyone can think of better ways of doing this I’d welcome any feedback or suggestions, to run this example you’d simply enter the command “ant selenium” :

A couple of notes, the reason I have to use a conditional check at the end of the selenium target is because, if the exec task that runs the PHP tests was set to failonerror=true then the build would never reach the next line which shuts the Selenium RC server down. To ensure that always happens I have to set the exec to failonerror=false, but this means I have to check what the result was from the exec. Which if successful will return 0, if test failures exist will return 1, and if there were any errors (preventing a test to be exectuted ) will return 2. Hence the conditional check sets regressionTest.err if either of these latter two conditions are true.

Also in order to start up the server, which could take up to a second, but can’t be sure precisely how long. I have to use the Ant Parallel task, which calls the task to start the server and the task to run the tests at the same time. The task to run the tests has a 2 second sleep in it, which should be more than enough time to allow the server to start. This all kind of feels a little clunky, but at the moment it does work very well.

In a nutshell, thats how you integrate PHP based Automated Selenium Regression tests into a continuous build.

• Cruise Control – For setting up a Continuous Build process.
• Ant – A Java based build tool.
• JUnit – A Java Based xunit testing framework
• PMD – A tool that checks for common coding standards violations.
• Corbetura – A Java based code coverage too that calculates the percentage of code accessed by unit tests

I’ve managed to set up an analogous process for our PHP project using the following tools:

• Cruise Control – For setting up a Continuous Build process.
• Ant – A Java based build tool.
• PHPUnit – An Xunit testing framework for PHP
• PHP_CodeSniffer – A PHP tool that checks for common coding standards violations.
• Xdebug – Debugging tool, which when combined with PHPUnit, can provide Code Coverage Metrics

It seems to work quite well, here’s the relatively simple ant build script that controls it all.

I’m currently running this on a windows machine although it’s trivial to change it work in an *ix based environment which I’ll probably configure in the next day or so. I had a couple of problems installing PHP_CodeSniffer although it was because I hadn’t installed PEAR properly. If you have any problems installing PHP_CodeSniffer under Windows then follow these instructions:

To install PEAR under windows do the following, which assumes you have PHP5.2x installed in c:\php :

  cd c:\\php
  go-pear.bat

The interactive installer presents you with some options, if you follow the defaults you should be fine.
Once PEAR has installed you can install PHP_CodeSniffer like this:

  cd c:\\php
  pear install PHP_CodeSniffer-beta

This will download the PHP_CodeSniffer package and install into into your php/PEAR folder.

Once this is done you can check to see if it has installed by calling phpcs with -h flag which will produce the following:

C:\\php>phpcs -h
Usage: phpcs [-nlvi] [--report=] [--standard=]
    [--generator=] [--extensions=]  ...
        -n           Do not print warnings
        -l           Local directory only, no recursion
        -v[v][v]     Print verbose output
        -i           Show a list of installed coding standards
        --help       Print this help message
        --version    Print version information
               One or more files and/or directories to check
         A comma separated list of file extensions to check
                     (only valid if checking a directory)
           The name of the coding standard to use
          The name of a doc genertor to use
                     (forces doc generation instead of checking)
             Print either the "full" or "summary" report

Now try it out …

C:\\php>phpcs C:\\Projects\\dummyproject\\test\\

FILE: C:\\Projects\\dummyproject\\test\\AllTests.php
--------------------------------------------------------------------------------
FOUND 7 ERROR(S) AND 0 WARNING(S) AFFECTING 7 LINE(S)
--------------------------------------------------------------------------------
  1 | ERROR | End of line character is invalid; expected "\n" but found "\r\n"
  2 | ERROR | Missing file doc comment
  3 | ERROR | Constants must be uppercase; expected 'PHPUNIT_MAIN_METHOD' but
    |       | found 'PHPUnit_MAIN_METHOD'
 12 | ERROR | Missing class doc comment
 14 | ERROR | Missing function doc comment
 19 | ERROR | Missing function doc comment
 30 | ERROR | Constants must be uppercase; expected PHPUNIT_MAIN_METHOD but
    |       | found PHPUnit_MAIN_METHOD
--------------------------------------------------------------------------------

Check the documentation for what the various command line switches do, but you can generate summary reports as well on an entire directory tree:

C:\\php>phpcs --report=summary --standard=Squiz C:\\Projects\\dummyproject\\test\\

PHP CODE SNIFFER REPORT SUMMARY
--------------------------------------------------------------------------------
FILE                                                            ERRORS  WARNINGS
--------------------------------------------------------------------------------
C:\\Projects\\dummyproject\\test\\AllTests.php                  24      0
C:\\Projects\\dummyproject\\test\\NodeTest.php                  10      0
C:\\Projects\\dummyproject\\test\\NodeTypeTest.php              11      0
--------------------------------------------------------------------------------
A TOTAL OF 45 ERROR(S) AND 0 WARNING(S) WERE FOUND IN 3 FILE(S)
--------------------------------------------------------------------------------

…

Overall I’m quite happy with this set up which for the most part was pretty straight forward. I have no doubt it will evolve over time but I think it’s a good foundation on which to build upon.

“Writing and maintaining software are not engineering activities. So it’s not clear why we call software development software engineering.”

The author of the essay goes onto offer a brief rebuttal of this this based on some comments by Steve McConnell, and pointed out quite rightly in my opinion that:

Software development is slowly and surely moving its way out of the mire of superstition and belief into the realm of empiricism and reason. The transition closely parallels those already made in other disciplines, such as medicine’s evolution from witchcraft into medical science.

What I found really insightful though was how the author likened these two views ( Loka vs McConnell ) to another conflict:

These two represent the age-old conflict between the engineers and the artists, the sciences and the humanities. In the software development domain, some have previously characterized it as the battle between the Greeks and the Romans.

He then applies this same metaphor to the wider issue of adaptive vs the plain-driven approaches to software development and in doing so offers an interesting perspective on the two schools of thought:

By now you will probably have recognized the analogue between the cultural divide separating the Greeks and Romans and the methodological divide between adaptive and plan-driven approaches to software development.

We can think of the Greeks as representative of Agile Methods. The focus is upon loosely coordinated individuals whose talent and passion combine to produce great artefacts of creativity. Any organizational shortcomings the team might experience are overcome by the cleverness and skilful adaptivity of the contributors.

Alternatively, we can consider the Romans as representative of plan driven methods, in which the carefully engineered and executed efforts of competent and well educated practitioners combine to produce works of great size and complexity. The shortcomings of any of the individuals involved are overcome by the systematic methods and peer review structure within which they work.

It’s a wonderful analogy that I think illustrates some of the differences between the two approaches in a way that most practitioners would readily understand and perhaps even agree with. Which is the right approach? I guess it depends on you, your project but most importantly the kind of people you have in your team. I have certainly spent a considerable amount of time extolling what I believe are the virtues of the agile methodology on this blog.

I spent several years working in a very Roman-esque organisation and I’d probably argue that the competent and well educated practitioners in such organisations rarely fit either of those terms. Probably because the very nature of such systems require those working within that confine to accept a certain level of conformity. Consequently individual flair, creativity or even imagination are less important than the uniformity that such organisations require – a sort of fill-in-the-blanks approach to development where your responsible for only those small pieces assigned to you, and whether you necessarily know what the bigger picture is or even understand it isn’t important because some more senior in the team who assigns you your tasks does know. This often results in developers feeling disaffected or perhaps less likely to feel any sense of personal ownership or even responsibility, because they know they aren’t in a position to be responsible for anything.

I do find myself agreeing that part of what makes agile teams successful is this notion of heroics on the part of individuals. It does work well when you have talented individuals who can work together in a reasonably small scale. How well does that scales up? … I can’t personally answer that question?  I’ve read plenty of accounts and listened to the likes of Scott Ambler explain how agile can work on large scale projects. I must admit I’m not sure if I’m convinced of this. … but that’s only because of my earlier point that this decision needs, in part, to be based on the type of people you have your in team.

Our development group at Talis is quite small,at the moment no more than 20 people – and it’s not for a lack of trying, were constantly trying to recruit people but we look specifically for skilled developers who could fit into the culture that we have. Individuals who are self motivated, self learners who take a great deal of personal pride in what they do, take responsibility and also want to have a sense of ownership over what they are building. As a result we tend to be quite jealous of who we let into the team, but again that’s probably something that is unique to our little group and says more about us than the Agile process.

I do find myself partly agreeing with the conclusion the author of the essay makes:

I favour a development method that is predominantly Greek, with sufficient Roman thrown in to keep things under control and prevent needless wheel spinning. The Greeks are a great bunch of guys, but they tend to put too much emphasis on individual heroics, and pay too little attention to the needs of the maintenance programmers that will following in their path. The Romans are a little stuffy and formal, but they really know how to keep things under control and keep your project running smoothly and predictably

 I don’t agree that, in terms of software development,  the Roman approach was ever truly able to keep things under control or necessarily running smoothly even predictably. If it was we wouldn’t have so many examples of failed projects that went over budget, went over time and totally failed to deliver what the customer actually wanted.

I personally do prefer a predominantly Greek approach and I do see methodologies such as SCRUM imposing the right level of Roman-esque control and structure to keep the project moving along smoothly and predictably, with the added bonus that the customer actually gets what they want.

The purpose of the spike was two fold. Firstly to try to better understand how to build a funky new set of features into Cenote, and secondly to allow the members of the team become familiar with and experiment with some technologies that they aren’t familiar with.

In fact it felt quite good leaving work today, having gotten to the point where the little prototype is pretty much feature complete. What’s really impressed me isn’t necessarily what it does (which is cool!), but the speed with which we’ve been able to put it all together. The spike was timeboxed to two weeks, but the reality is that the bulk of the implementation has actually been completed within the last few days. It’s by no means a production system it’s just enough to hopefully facilitate some of the discussions we hope to have both internally and externally … much like the first release of Cenote, which we Open Sourced recently.

The original version of Cenote was a read only application that allowed users to search for books and then mashed up the results with data held in various stores in the Talis Platform as well as external sources such as Amazon in order to provide users with some pretty useful information. This spike extends the original version by allowing users to use that data to create and share some really useful things.

I think there’s some important reasons why we have been able to put this together so quickly. The technology stack has been kept very simple – its just an application built in PHP5, running under Apache 2. Furthermore the application is built upon our Talis Platform which is constantly evolving and becoming more and more powerful. I’m not saying that just as someone who has worked on building that platform, I’m actually saying that as someone who has been using and consuming it’s services primarily to build applications with it.

When Rob and I originally wrote Cenote, we were both impressed at how easily we were able to use the platform, as it was then, to put together a cool looking application in the space of a couple of days. If that impressed me at the time, then I’m doubly impressed at how simple it’s been to create an application that supports creation, deletion and updating of data.

Much of the talk is really an introduction to Agile Development which is beneficial to listen to because Scott dispels some of the myths around agile, and offers his own views on best practises using some examples. It does get a bit heated around the 45 minute mark when he’s discussing Database Refactoring, some of the people in the audience were struggling with the idea he was presenting which I felt was fairly simple. If you really want to skip all that try to forward to the 50 minute mark where he starts talking about sandboxes. What I will say is that if your having difficulty getting Agile accepted into your organisation then this might be a video you want to show your managers since it covers all the major issues and benefits.

Here’s some of the tips he has with regard to testing and improving quality:

• Do Test Driven Development, the unit tests are the detailed design, they force developers to think about the design. Call it Just-in-time design.
• Use Continuous Integration to build and run unit tests on each check-in to trunk.
• Acceptance Tests are primary artefacts. Don’t bother with a requirements document simply maintain the acceptance test since the reality is that all testing teams will do is take that requirement and copy it into an acceptance test, so why introduce a traceability issue when you don’t need it. http://www.agilemodeling.com/essays/singleSourceInformation.htm
• Use Standards and Guidelines to help ensure teams are creating consistent artefacts.
• Code Reviews and Inspections are not a best practise. They are used to compensate for people working alone, not sharing their work, not communicating, poor teamwork, and poor collaboration. Guru checks output is an anti-pattern. Working together, pairing, good communication, teamwork should negate the need for code reviews and inspections.
• Short feedback loop is extremely important. The faster you can get testing results and feedback from stakeholders the better.
• Testers need to be flexible, willing to pick up new skills, need to be able to work with others. They need to be generalising specialists. The trend that is emerging in agile or the emerging belief is that there is no need for traditional testers.

Scott is a passionate speaker and very convincing, some of the points he makes are quite controversial yet hard to ignore – especially his argument that traditional testers are becoming less necessary. I’m not sure I agree with all his views yet he has succeeded in forcing me to challenge my own views which I need to mull over and for that reason alone watching his talk has been invaluable.

I’m fairly new to the SCRUM methodology, in fact this is the first project I’ve worked on that formally uses it. Our development group here at Talis has adopted the SCRUM process across all of our current projects with what I feel has been a great deal of success.

For me personally the transition from traditional waterfall approaches to agile methodologies has been a revelation in many ways. Before joining Talis I’d spent a number of years developing software based on traditional waterfall methodologies. What was frustrating with these traditional approaches was that you’d spend months capturing and documenting requirements, you’d then spend a while analysing these requirements and then designing your software product, before implementing it and then testing it. Any changes to the requirements invariably meant going through a process of change impact analysis and then going through the whole process again (for the changed requirements), which naturally increases the cost to the customer.

A side effect of which, from the perspective of the customer, was that changing requirements during the project was a bad idea, because of the extra costs it would incur. A consequence of this is that customers would often take delivery of systems which after a couple of years of development, don’t actually satisfy the requirements that they now have. These same customers would then have to take out a maintenance contract with the vendor to get the systems updated to satisfy their new requirements.

From a developers point of view, I often found this to be very demoralising, you knew you you were building something the customer didn’t really want, but the software house you work for wants to do that because they have a signed off requirements document and contract that guarantee’s them more money if the customer changes their mind. I often found that when we reached the end of a project, the delivery of that software to the customer was a very and nervous and stressful time. The customer at this point has not necessarily had any visibility of the product so there’s usually a couple of months when your organisation is trying to get them to accept the product – which invariably led to arguments over the interpretation of requirements – and sometimes scary looking meetings between lawyers from both sides.

There was always something very wrong with it.

Since joining Talis, and transitioning to agile methodologies I can finally see why it was so wrong, and why agile, and in this case SCRUM, work so well.

For one thing, I’m not nervous about the end of sprint demonstrations. The customers have been involved all along, they’ve been using the system, constantly providing feedback, constantly letting us know what were doing well, and what we need to improve on.

Our sprints have been four weeks long, which means at the beginning of the sprint we agree which stories we are going to implement based on what the customers have asked us for, these can be new stories that have been identified, or stories from the backlog. The customers have an idea, from previous sprints, what our velocity is – in other words they, and we, know how much work we can get done in a sprint so when we pick the stories for the sprint we ensure we don’t exceed that limit. This keeps things realistic. Any story that doesn’t get scheduled in for the sprint because it was deemed less of a priority than the stories that are selected gets added to a backlog.

This iterative cycle is great! For one thing customer’s are encouraged to change their minds, to change their requirements, because they then have to prioritise whether that change is more important than the other items on the backlog. They are empowered to choose what means the most to them, and that’s what we give them ! The customer doesn’t feel like the enemy, but an integral part of the team, and for me that’s vital.

As a developer it feels great to know that your customers like your product … and why shouldn’t they, they’ve been involved every step of the way, they’ve been using it every step of the way.

I’ve only been here at Talis for ten months, and in that time I’ve had to constantly re-examine and re-evaluate not only what I think it means to be a good software developer but pretty much every facet of the process of building services and products for customers. For me it’s been an amazing journey of discovery and I’m pretty certain it’s going to continue to be for a very long time.

The really wonderful thing though is that in our development group I’m surrounded by a team of people who believe passionately that it’s the journey and how we deal with it that defines us, and not the destination. So we are constantly looking for ways to improve, that in itself can be inspiring.
So yeah … I feel good!

Our development group is always looking for talented developers who share our ethos and could fit into the culture we have here. If you’d like to be a part of this journey then get in touch with us. Some of us, including myself, will be at XTECH 2007 next month, so if your attending the conference come and have a chat with us.

I think this tech talk is very relevant not just to the discussions were having internally, but to any organisation that is serious about wanting to ensure that they are developing great, maintainable software.

It’s safe to say that I was not expecting to be moved quite as much as I was by this account, which you can read online in its entirety over at http://ea-spouse.livejournal.com/. Before I go any further I will say this, I believe that ANYONE working in the software industry or in human resources, in fact everyone should read the essay.

It’s written by the spouse of an Electronic Arts employee who wrote this under the anonymous moniker ea_spouse, she chose to remain anonymous because in her own words she has “no illusions about what the consequences would be for my family if I was explicit“. Her account dramatically made the world aware of the shocking sweatshop-like labor practises at EA. It’s important to point out that this account was originally written in 2004 and since it was first published the controversy it generated has led to class action law suits against EA, as well a shedding light on what appears to be a commen trend within the gaming industry.

She describes what her family has to endure as her spouse is forced to work in excess of 85 hours a week for months on end, or in her own words:

Every step of the way, the project remained on schedule. Crunching neither accelerated this nor slowed it down; its effect on the actual product was not measurable. The extended hours were deliberate and planned; the management knew what they were doing as they did it. The love of my life comes home late at night complaining of a headache that will not go away and a chronically upset stomach, and my happy supportive smile is running out

It’s a heart wrenching expose that both captivates and evokes an extremely emotional response in you as you read it. As she laments the forced hours without any overtime or compensation, or even time off for employees you cant help but feel sickened. I had to put the book down and walk away for a moment when she wrote:

“If they don’t like it, they can work someplace else.” Put up or shut up and leave: this is the core of EA’s Human Resources policy. The concept of ethics or compassion or even intelligence with regard to getting the most out of one’s workforce never enters the equation:

Like anyone in the software industry you accept that you do have to work long hours sometimes as deadlines begin to loom, most of the developers that I have known dont mind this, but commonsense alone should tell us that this should always be the exception – never the norm. Ultimately it’s un-sustainable. We’re all human beings, we have lives outside of our work, other interests to persue, other dreams to achieve. ea_spouse ends her account with this …

…when you keep our husbands and wives and children in the office for ninety hours a week, sending them home exhausted and numb and frustrated with their lives, it’s not just them you’re hurting, but everyone around them, everyone who loves them? When you make your profit calculations and your cost analyses, you know that a great measure of that cost is being paid in raw human dignity, right?

Before joining Talis I used to work for an organisation, where I did clock up close to 70 hours a week for sustained periods. Of course no-one actually forces you, your just left to wander if you want the stigma of being labelled not a team player. I can only comment from my own perspective but I have no doubt that much of the apathy, cynicism and even contempt I had for the industry was a product of just how soul destroying it is to wake up, go to work, come home, sleep for a little and then wake up and go to work again. Your depressed, your constantly tired, your irratable, you become less and less attentive … to the point where you dont even sense someone running up behind you with a lead bar!

But what doesnt kill you, generally makes you stronger … at least thats something I try to believe. As I Read ea_spouse’s account, and thought about my own experiences as a developer working extended hours for sustained periods, I was immediatly able to contrast that with what things are like now.

For me Talis is a very different kind of environment to work in as a developer. I dont know whether its because we’ve embraced agile methodologies that are based around the principle of sustainable iterations of work, or whether its because the people I work with and work for genuinley care about the wellbeing of every member of the team. Or as I suspect its probably a combination of both. Our iterations in Skywalk are weekly, the small team on average completes around 15 units of work per week (our velocity – dont ask me to define what our units represent … I always quote my estimates in donuts! ), but I recall how our programme lead reacted a few months ago when the team over a couple of iterations averaged twice to three times that figure.

Our programme lead on skywalk, Ian Davis, is probably one of the finest programme mangers I have ever worked with. Probably because he doesnt think of himself as a programme manager. He’s extremely goal driven and yet a humanist who puts the well being of his team before anything else. As a team leader he’s a pragmatist, but it’s his charm and his passion that has helped bring together bunch of talented geeks and focused them into a team in every sense of the word. Anyway a few months back our velocity shot up, we were coming to the end of the development on a research prototype we call, Cenote, we wanted to have the piece up and running so Paul could show off some of our achievements at a conference in Canada. There wasnt a real requirement for the prototype to be made available, it was always a nice to have. But the team wanted to showcase its work, we take a great deal of pride in what we do. Ian was on vacation and in his absence we simply plowed on got it all done and delivered. When he came back and checked our velocity, he was appreciative yet told us that he didnt want us to make a habit of that because it wasnt sustainable. He then planned our next iteration to be around half our normal average velocity on the grounds that he wanted to make sure we all got a bit of rest. I’d never known a programme manager to react like that … or for a company to let him.

Anyway afterwards we went to to grab a bite to eat and catch up on what we’ve both been up to. We both work in the IT industry and love development but we tend to have differing views on a lot of things ( Listen J were not starting the Java vs .NET argument on my blog if you do I’ll really kick your ….). Anyway J and I both used to work for the same company and the senior management of that organisation has recently announced that its going to outsource the development of some its long term projects overseas to India, citing that engineering talent over there is cheap and thus far more cost effective … but they intend to run the projects from over here. As a result my friend, like many of his colleagues are obviously worried about their futures and whether the company will want to retain their services as developers for much longer. Anyway this kind of got me thinking …

I’m not going to pay credence to any of those boorish, tired and lamentable arguments about how its morally wrong to take UK jobs and hand them to people overseas. The fact is we live in a global economy and we have to accept that we need to remain competitive in that economy. Besides I believe that kind of xenophobia generally tends to cloud the issue and overshadow far more important reasons as to why outsourcing is a bad idea for our industry.

I remember towards the tail end of the 90’s when venture capitalists where really pushing the idea of outsourcing development to places like China and India it’s cheaper, more cost effective and will help the organisations overall operational effectiveness. I’m of the opinion that this was generally because they looked at other industries where that model worked really well, I guess what they thought was if Matel can manufacture toys abroad cheaper, why cant we get software written abroad cheaper, right? Lots of large firms bought into this thinking Oracle and Hewlett Packard are just two examples of companies that followed this trend … only to slowly distance themselves after encountering the problems I touch upon below.

The problem though, is that writing code isn’t something you can translate into an assembly line. What I think the people pushing this type of outsourcing failed to comprehend, and seemingly still dont understand is that farming out development overseas doesn’t lead to innovation. The idea that you get a a large group of programmers together and they’ll just produce cool code – doesn’t work! I remember at university I did an elective in post-war Japanese history (wish I could tell you that I did this cos it was interesting but the truth was the lecturer was the hottest chick I’d ever …*off daydreaming*), anyway one of the things we were taught was that the japanese rebuilt their economy around their manufacturing industry, using automated methods of production and rigourous quality control. Through all of this they actually revolutionised manufacturing industry globally the effects of which are still being seen today.

Towards the end of the seventies and into the eighties Japanese companies tried to set up software factories where they basically got a shed load of developers together and tried to apply their tried and tested manufacturing experience to writing software … they failed miserably and learnt that putting loads of developers together doesn’t create innovative software. The reason is that writing software isn’t something that translates into a purely mechanical activity – like making a toy or a car. So none of their tried and tested rules applied.

Someone famously once said every line of code is a design decision, I’m struggling to remember who it was [insert clever guys name here]. But that single statement embodies for me what the real problem is with outsourcing projects abroad. You loose sight of the decisions that the developers are making as they piece together the product from your requirements. Farming out to developers overseas successfully means you have to pay meticulous attention to the details of what is being produced, and that’s damn difficult when you factor in communication problems, cultural differences and attitudes, and like it or not glaringly obvious fact that this model makes it difficult to be flexible or be able to react quickly to changes in your market place.

Anyway its well worth reading Joel’s post, I think hes absolutely right in pointing out that a lack of features isnt what made the iPod or BaseCamp such a success but the fact that amongst other attributes both these products were built to correspond to a user model that resulted in a high degree of usability. To my mind the problem we face in this industry is that in order to build useable systems we need to accept that we can only do that by understanding the users mental model, we can no longer afford to build software solutions as simply the delivery of a set of discrete requirements and leave usability as something we can bolt on at the end when we’ve got the rest of the system working. We have to understand our users and put them first.

I want to expand a little on what Coplien has covered so well …

I’ve been fortunate enough to have worked closely with Alan Dix and Russell Beale, the authors of Human Computer Interaction. I do class myself as a developer but I have had a strong interest in HCI, and user interface design and its an area I’ve been interested in and reading around for the best part of a decade. In that time I’ve often had to work in teams where its apparent that other developers dont either understand the need for HCI, or even how to begin to think about it.

In my own experience this comes from the fact that programmers tend to focus on the mechanics of building an interface, getting the right framework, and then designing around the limitations of the toolset they have picked. Invariably they tend to spend tonnes of time just focusing on getting the UI to work and this UI is designed from the perspective of satisfying the flows defined in requirements documents or design docs like Use Cases. You can contrast this with people trying to use the user interface, who dont care how it was built, or what the limitations in the underlying technologies were, these people just want something they can use. To me its this emphasis that seperates a working interface from a “good” user interface. Good User Interface are designed around the user, unfortunatly most user interfaces tend to be designed around building something that satisfies a series of discrete requirements.

I dont think it’s fair to simply say that programmers arent capable and can’t do good UI design, the truth is that on most software projects, and in most teams, it just isnt their emphasis.

In order to improve things developers need to be taught the importance of user interface design. Our development group at Talis is a great example of this. As a team we know we need get better at understanding how to build user interfaces. One of the activities we undertook recently was to get everyone to read Spolsky’s User Interface Design for Programmers as part of our geek book club. It’s an excellent text that introduces programmers to key concepts in user interface design.

However as more and more applications are being delivered over the web developers should take the time to read Jakob Nielsen’s seminal text Designing Web Usability – The practice of Simplicity. Bruce Tognazzini’s TOG on Interface, might be a little dated but its still an incredibly good text and has a great section on conceptual models.

Most importantly though: one of the things our group has realised is that User Interface Design and User Interaction Design are specialised skills, and whilst its important that developers should be encouraged to learn more about this area, its vitally important that organisations recruit people with these specialised skills into their teams. To that end Talis is openly recruiting for individuals who specialise in HCI/User Interface Design, so if you think you fit the bill and its an area your passionate about, and you’d like to work for an organisation that takes this very seriously then check out the job spec here and send your CV along with a covering letter to team.talent@talis.com.

This chapter really resonates with me since parts of it pretty much describe the approach to estimation we follow in our development group at Talis. We estimate how much effort it would take to complete indiviual stories. Our iterations last a week. At the beginning of each week we total up our velocity for the previous week, for example 16, and then select 16 units worth of work for the coming iteration. If we repeatedly hit our target over the course of a couple of iterations then as a group we might pick more than our velocity for the following iteration. All the members of the development team take participate in the estimation, this ensures that everyone is “bought-into” the plan. This approach leads to better communication and can help clarify requirements it also engenders greater trust within the team.

As a metaphor this should sound familiar to anyone working on a large ( even not so large ) software project. Your coders are the equivalent of your kitchen brigade, guys and girls who have to deliver, without them you cant serve anything to your customers. Your ingredients are API’s, frameworks, and technologies. Your complicated menu’s are over analysed, over designed, and over complicated software architectures; and lets not forget your restaurant managers are the same as your project managers, who generally know sod all about writing code, they promise your customers everything under the sun, generally work their brigade to death to deliver to unrealistic time frames and have a penchant for blaming the coders when it all goes tits up!

Worst of all the customer rarely gets what he or she actually ordered, because so many software company’s still persist on following dated project management and development methodologies trying to gather requirements up front, do exhaustive analysis and design then coding, and finally delivering a system 12 months later to a customer who’s requirements have now changed.

Did I sound bitter then? It’s probably because I realise that I’ve spent the better part of a decade working for the kinds of software companies where this kind of thing is considered the norm. Sadly for many company’s it still is the norm!

So lets apply the Ramsay formula to this. How do you turn teams that are building software in the manner above around?

Firstly, you have to empower your brigade. In the past I’ve worked in places where the job of most developers is simply to “fill-in-the-blanks”… in other words just implement method stubs that are generated by a design tool that the architects use. This creates hierarchical divisions within teams, and is generally extremely de-motivating. Like a cook who has given up using his imagination, has no passion, and has given up thinking creatively and has been reduced to doing little more than reheating pre-cooked frozen hash.

In order to empower the team, they need to OWN the code collectively. It doesn’t belong to one person, it belongs to everyone. As a team they’re passionate about it. You have to be break people out of the traditional thinking that I wrote this class so people have to check with me if they want to change it!

We need “fresh” ingredients. Yes we should re-use software but only if its appropriate. How many times have you worked on software projects where the architecture isn’t based on whether its the right technology or tool for the job, but is based on other factors like the company you work for has a cool licensing agreement with a vendor and wants to use their technology because they don’t have to fork out for something more appropriate? This, using a square-peg to fill a round hole approach, in variably leads to difficulties.

Have “simpler menus”! We need to get rid of up-front complicated over-architected designs, and over analysing solutions – which are a product of old-style waterfall approaches. Teams need to be moving towards iterative, agile development methodologies. These processes engage the customers who are able to provide feedback on the product at the end of each iteration so this approach to developing software encourages customers to change their requirements if and when they want to which means that when you finally deliver … your actually giving them what they want! And this squarely puts the customer first!

Keep management out of the kitchen. If your the kind of organisation willing to invest in bringing a smart bunch of technical people together, then trust them to do their job. They don’t need project managers standing over their shoulders asking them for an hourly update on their progress. This kind of Orwellian micro-management is culturally ingrained into some large software companies and in my opinion it is very, very damaging. It fosters resentment, encourages bullying and totally de-motivates developers because of the perpetual monkey on their backs!

I am so glad im not working for an organisation that suffers from these problems, and perhaps if you are then maybe you should consider a change?

Martin Fowler mentions it on his site, obviously hes recommending it because of his involvement with thoughtworks. Nevertheless its a great idea, based around shortening iteration zero.

There’s a good blog by one of the “authors” describing why they made this, which is well worth reading.

I’ve found it easy to use and configure and it gives a great head start to project teams trying to get continuous integration environments set up.