Why software sucks

Over on slashdot theres an excellent little article and debate around the issue of why software sucks. The slashdot article points to this news story on the Fox News Network. that discusses the book by David Platt entitled “Why software sucks …. and what you can do about it“. I haven’t read the book yet but I’ve added it to my things to read list. The debate on slashdot though is actually quite interesting and worth reading in its own right. What interests me is how some of the sentiments echoed in the articles and discussions resonate around my earlier views that programmers arent usability experts, and until we start developing software centred around the user … software will continue to suck.

Code reviews at Google with Mondrian

Another excellent Tech Talk. Our development group at Talis has been experimenting with different ways to carry out code reviews and I think its safe to say were still trying to find what works best for us.

I think this tech talk is very relevant not just to the discussions were having internally, but to any organisation that is serious about wanting to ensure that they are developing great, maintainable software.

ea_spouse : profit at the cost of human dignity

My copy of The Best Software Writing I – selected and introduced by Joel Spolsky arrived the day before yesterday. I finally managed to start reading it last night after getting back from a truly magical evening at the Chinese State Circus. The book is a collection of essays/posts on online blogs that Spolsky has brought together as examples of simple goodle writing that engages the reader and captivates them. Spolsky introduces each essay with his own take on the subject matter. The essay I chose to read first was entitled EA – The Human Story. Anyone who knows me, knows I play several online games (most FPS ones), and I have a great interest in the gaming industry in terms of the products and technologies that they produce.

It’s safe to say that I was not expecting to be moved quite as much as I was by this account, which you can read online in its entirety over at http://ea-spouse.livejournal.com/. Before I go any further I will say this, I believe that ANYONE working in the software industry or in human resources, in fact everyone should read the essay.

It’s written by the spouse of an Electronic Arts employee who wrote this under the anonymous moniker ea_spouse, she chose to remain anonymous because in her own words she has “no illusions about what the consequences would be for my family if I was explicit“. Her account dramatically made the world aware of the shocking sweatshop-like labor practises at EA. It’s important to point out that this account was originally written in 2004 and since it was first published the controversy it generated has led to class action law suits against EA, as well a shedding light on what appears to be a commen trend within the gaming industry.

She describes what her family has to endure as her spouse is forced to work in excess of 85 hours a week for months on end, or in her own words:

Every step of the way, the project remained on schedule. Crunching neither accelerated this nor slowed it down; its effect on the actual product was not measurable. The extended hours were deliberate and planned; the management knew what they were doing as they did it. The love of my life comes home late at night complaining of a headache that will not go away and a chronically upset stomach, and my happy supportive smile is running out

It’s a heart wrenching expose that both captivates and evokes an extremely emotional response in you as you read it. As she laments the forced hours without any overtime or compensation, or even time off for employees you cant help but feel sickened. I had to put the book down and walk away for a moment when she wrote:

“If they don’t like it, they can work someplace else.” Put up or shut up and leave: this is the core of EA’s Human Resources policy. The concept of ethics or compassion or even intelligence with regard to getting the most out of one’s workforce never enters the equation:

Like anyone in the software industry you accept that you do have to work long hours sometimes as deadlines begin to loom, most of the developers that I have known dont mind this, but commonsense alone should tell us that this should always be the exception – never the norm. Ultimately it’s un-sustainable. We’re all human beings, we have lives outside of our work, other interests to persue, other dreams to achieve. ea_spouse ends her account with this …

…when you keep our husbands and wives and children in the office for ninety hours a week, sending them home exhausted and numb and frustrated with their lives, it’s not just them you’re hurting, but everyone around them, everyone who loves them? When you make your profit calculations and your cost analyses, you know that a great measure of that cost is being paid in raw human dignity, right?

Before joining Talis I used to work for an organisation, where I did clock up close to 70 hours a week for sustained periods. Of course no-one actually forces you, your just left to wander if you want the stigma of being labelled not a team player. I can only comment from my own perspective but I have no doubt that much of the apathy, cynicism and even contempt I had for the industry was a product of just how soul destroying it is to wake up, go to work, come home, sleep for a little and then wake up and go to work again. Your depressed, your constantly tired, your irratable, you become less and less attentive … to the point where you dont even sense someone running up behind you with a lead bar!

But what doesnt kill you, generally makes you stronger … at least thats something I try to believe. As I Read ea_spouse’s account, and thought about my own experiences as a developer working extended hours for sustained periods, I was immediatly able to contrast that with what things are like now.

For me Talis is a very different kind of environment to work in as a developer. I dont know whether its because we’ve embraced agile methodologies that are based around the principle of sustainable iterations of work, or whether its because the people I work with and work for genuinley care about the wellbeing of every member of the team. Or as I suspect its probably a combination of both. Our iterations in Skywalk are weekly, the small team on average completes around 15 units of work per week (our velocity – dont ask me to define what our units represent … I always quote my estimates in donuts! 😉 ), but I recall how our programme lead reacted a few months ago when the team over a couple of iterations averaged twice to three times that figure.

Our programme lead on skywalk, Ian Davis, is probably one of the finest programme mangers I have ever worked with. Probably because he doesnt think of himself as a programme manager. He’s extremely goal driven and yet a humanist who puts the well being of his team before anything else. As a team leader he’s a pragmatist, but it’s his charm and his passion that has helped bring together bunch of talented geeks and focused them into a team in every sense of the word. Anyway a few months back our velocity shot up, we were coming to the end of the development on a research prototype we call, Cenote, we wanted to have the piece up and running so Paul could show off some of our achievements at a conference in Canada. There wasnt a real requirement for the prototype to be made available, it was always a nice to have. But the team wanted to showcase its work, we take a great deal of pride in what we do. Ian was on vacation and in his absence we simply plowed on got it all done and delivered. When he came back and checked our velocity, he was appreciative yet told us that he didnt want us to make a habit of that because it wasnt sustainable. He then planned our next iteration to be around half our normal average velocity on the grounds that he wanted to make sure we all got a bit of rest. I’d never known a programme manager to react like that … or for a company to let him.

Outsourcing developers abroad … Do people still really think its a good idea?

Had an interesting evening went to the gym and ran into an old friend I’ve not seen in a few years. We sparred for a bit (its a guy thing … guess we wanted to see which of us had improved the most since our last … encounter … it was me of course not that I’m insanely competitive … honest!)

Anyway afterwards we went to to grab a bite to eat and catch up on what we’ve both been up to. We both work in the IT industry and love development but we tend to have differing views on a lot of things ( Listen J were not starting the Java vs .NET argument on my blog if you do I’ll really kick your ….). Anyway J and I both used to work for the same company and the senior management of that organisation has recently announced that its going to outsource the development of some its long term projects overseas to India, citing that engineering talent over there is cheap and thus far more cost effective … but they intend to run the projects from over here. As a result my friend, like many of his colleagues are obviously worried about their futures and whether the company will want to retain their services as developers for much longer. Anyway this kind of got me thinking …

I’m not going to pay credence to any of those boorish, tired and lamentable arguments about how its morally wrong to take UK jobs and hand them to people overseas. The fact is we live in a global economy and we have to accept that we need to remain competitive in that economy. Besides I believe that kind of xenophobia generally tends to cloud the issue and overshadow far more important reasons as to why outsourcing is a bad idea for our industry.

I remember towards the tail end of the 90’s when venture capitalists where really pushing the idea of outsourcing development to places like China and India it’s cheaper, more cost effective and will help the organisations overall operational effectiveness. I’m of the opinion that this was generally because they looked at other industries where that model worked really well, I guess what they thought was if Matel can manufacture toys abroad cheaper, why cant we get software written abroad cheaper, right? Lots of large firms bought into this thinking Oracle and Hewlett Packard are just two examples of companies that followed this trend … only to slowly distance themselves after encountering the problems I touch upon below.

The problem though, is that writing code isn’t something you can translate into an assembly line. What I think the people pushing this type of outsourcing failed to comprehend, and seemingly still dont understand is that farming out development overseas doesn’t lead to innovation. The idea that you get a a large group of programmers together and they’ll just produce cool code – doesn’t work! I remember at university I did an elective in post-war Japanese history (wish I could tell you that I did this cos it was interesting but the truth was the lecturer was the hottest chick I’d ever …*off daydreaming*), anyway one of the things we were taught was that the japanese rebuilt their economy around their manufacturing industry, using automated methods of production and rigourous quality control. Through all of this they actually revolutionised manufacturing industry globally the effects of which are still being seen today.

Towards the end of the seventies and into the eighties Japanese companies tried to set up software factories where they basically got a shed load of developers together and tried to apply their tried and tested manufacturing experience to writing software … they failed miserably and learnt that putting loads of developers together doesn’t create innovative software. The reason is that writing software isn’t something that translates into a purely mechanical activity – like making a toy or a car. So none of their tried and tested rules applied.

Someone famously once said every line of code is a design decision, I’m struggling to remember who it was [insert clever guys name here]. But that single statement embodies for me what the real problem is with outsourcing projects abroad. You loose sight of the decisions that the developers are making as they piece together the product from your requirements. Farming out to developers overseas successfully means you have to pay meticulous attention to the details of what is being produced, and that’s damn difficult when you factor in communication problems, cultural differences and attitudes, and like it or not glaringly obvious fact that this model makes it difficult to be flexible or be able to react quickly to changes in your market place.

What does Simplicity mean when it comes to software?

Our geek bookclub at Talis has been reading 37Signals Getting Real. It’s interesting text that contains for the most part a common sense advice with which we as a group can relate to. But one of the things thats been grating on my mind for a while now is the notion that simplicity sells, and by simplicity im referring to a lack of features. Unfortunatly whilst I was drafting this entry Joel got there first 😉 ( LiveWriter is excellent for offline editing of blog entries, unfortunatly it doesnt stop you getting pipped to the post!)

Anyway its well worth reading Joel’s post, I think hes absolutely right in pointing out that a lack of features isnt what made the iPod or BaseCamp such a success but the fact that amongst other attributes both these products were built to correspond to a user model that resulted in a high degree of usability. To my mind the problem we face in this industry is that in order to build useable systems we need to accept that we can only do that by understanding the users mental model, we can no longer afford to build software solutions as simply the delivery of a set of discrete requirements and leave usability as something we can bolt on at the end when we’ve got the rest of the system working. We have to understand our users and put them first.

Programmers are generally bad at user interface design?

Was intrigued when I read James O’ Coplien latest blog entry: The interface is the Program, and it ain’t Agile. Coplien discusses, quite correctly I believe, the fact that historically developers have tended to be very bad at building good user interfaces. Coplien discusses this in the context of Agile development groups and how usability is rarely captured as a story because stories in themselves are generally short-term goals that you want to achieve in the space of an iteration or even a sprint. Therefore because Agile teams are rapidly making changes to their application its hard to be able to do the focussed studies and analysis that is required in order to gain an understanding of how useable the application your building is. Fundementally programmers are not User Interface designers – Coplien attributes this to several reasons, which include the fact that university courses on software engineering have often done little more than pay lip service to HCI or User Interface Design, its often a topic that undergraduates are only ever given the most basic of introductions to.

I want to expand a little on what Coplien has covered so well …

I’ve been fortunate enough to have worked closely with Alan Dix and Russell Beale, the authors of Human Computer Interaction. I do class myself as a developer but I have had a strong interest in HCI, and user interface design and its an area I’ve been interested in and reading around for the best part of a decade. In that time I’ve often had to work in teams where its apparent that other developers dont either understand the need for HCI, or even how to begin to think about it.

In my own experience this comes from the fact that programmers tend to focus on the mechanics of building an interface, getting the right framework, and then designing around the limitations of the toolset they have picked. Invariably they tend to spend tonnes of time just focusing on getting the UI to work and this UI is designed from the perspective of satisfying the flows defined in requirements documents or design docs like Use Cases. You can contrast this with people trying to use the user interface, who dont care how it was built, or what the limitations in the underlying technologies were, these people just want something they can use. To me its this emphasis that seperates a working interface from a “good” user interface. Good User Interface are designed around the user, unfortunatly most user interfaces tend to be designed around building something that satisfies a series of discrete requirements.

I dont think it’s fair to simply say that programmers arent capable and can’t do good UI design, the truth is that on most software projects, and in most teams, it just isnt their emphasis.

In order to improve things developers need to be taught the importance of user interface design. Our development group at Talis is a great example of this. As a team we know we need get better at understanding how to build user interfaces. One of the activities we undertook recently was to get everyone to read Spolsky’s User Interface Design for Programmers as part of our geek book club. It’s an excellent text that introduces programmers to key concepts in user interface design.

However as more and more applications are being delivered over the web developers should take the time to read Jakob Nielsen’s seminal text Designing Web Usability – The practice of Simplicity. Bruce Tognazzini’s TOG on Interface, might be a little dated but its still an incredibly good text and has a great section on conceptual models.

Most importantly though: one of the things our group has realised is that User Interface Design and User Interaction Design are specialised skills, and whilst its important that developers should be encouraged to learn more about this area, its vitally important that organisations recruit people with these specialised skills into their teams. To that end Talis is openly recruiting for individuals who specialise in HCI/User Interface Design, so if you think you fit the bill and its an area your passionate about, and you’d like to work for an organisation that takes this very seriously then check out the job spec here and send your CV along with a covering letter to team.talent@talis.com.

Providing estimates for building software

James Shore is currently working on his book – The Art of Agile Development. For a while now Shore has been posting up pre-publication review chapters from this upcoming book. The latest section up for review covers the topic of Estimation, and its well worth reading.

This chapter really resonates with me since parts of it pretty much describe the approach to estimation we follow in our development group at Talis. We estimate how much effort it would take to complete indiviual stories. Our iterations last a week. At the beginning of each week we total up our velocity for the previous week, for example 16, and then select 16 units worth of work for the coming iteration. If we repeatedly hit our target over the course of a couple of iterations then as a group we might pick more than our velocity for the following iteration. All the members of the development team take participate in the estimation, this ensures that everyone is “bought-into” the plan. This approach leads to better communication and can help clarify requirements it also engenders greater trust within the team.

Gordon Ramsay could teach software engineers a thing or two?

Rob and I took a coffee break from refactoring some code yesterday and he asked me if I had watched Ramsay’s Kitchen Nightmares the night before, which I had! We commented on how formulaic the show is – each episode Ramsay turns up at the door of an ailing restaurant, and helps get them back on track and making money by empowering and motivating the cooks, using fresh ingredients, coming up with a simple less complicated menu, keeping management out of the kitchen and in the front where they belong, and above all putting the customer first!! Rob then said something along the lines of “You know when you think about it its not too different to the problems that many other software engineering firms face“. That’s when the penny dropped …

As a metaphor this should sound familiar to anyone working on a large ( even not so large ) software project. Your coders are the equivalent of your kitchen brigade, guys and girls who have to deliver, without them you cant serve anything to your customers. Your ingredients are API’s, frameworks, and technologies. Your complicated menu’s are over analysed, over designed, and over complicated software architectures; and lets not forget your restaurant managers are the same as your project managers, who generally know sod all about writing code, they promise your customers everything under the sun, generally work their brigade to death to deliver to unrealistic time frames and have a penchant for blaming the coders when it all goes tits up!

Worst of all the customer rarely gets what he or she actually ordered, because so many software company’s still persist on following dated project management and development methodologies trying to gather requirements up front, do exhaustive analysis and design then coding, and finally delivering a system 12 months later to a customer who’s requirements have now changed.

Did I sound bitter then? It’s probably because I realise that I’ve spent the better part of a decade working for the kinds of software companies where this kind of thing is considered the norm. Sadly for many company’s it still is the norm!

So lets apply the Ramsay formula to this. How do you turn teams that are building software in the manner above around?

Firstly, you have to empower your brigade. In the past I’ve worked in places where the job of most developers is simply to “fill-in-the-blanks”… in other words just implement method stubs that are generated by a design tool that the architects use. This creates hierarchical divisions within teams, and is generally extremely de-motivating. Like a cook who has given up using his imagination, has no passion, and has given up thinking creatively and has been reduced to doing little more than reheating pre-cooked frozen hash.

In order to empower the team, they need to OWN the code collectively. It doesn’t belong to one person, it belongs to everyone. As a team they’re passionate about it. You have to be break people out of the traditional thinking that I wrote this class so people have to check with me if they want to change it!

We need “fresh” ingredients. Yes we should re-use software but only if its appropriate. How many times have you worked on software projects where the architecture isn’t based on whether its the right technology or tool for the job, but is based on other factors like the company you work for has a cool licensing agreement with a vendor and wants to use their technology because they don’t have to fork out for something more appropriate? This, using a square-peg to fill a round hole approach, in variably leads to difficulties.

Have “simpler menus”! We need to get rid of up-front complicated over-architected designs, and over analysing solutions – which are a product of old-style waterfall approaches. Teams need to be moving towards iterative, agile development methodologies. These processes engage the customers who are able to provide feedback on the product at the end of each iteration so this approach to developing software encourages customers to change their requirements if and when they want to which means that when you finally deliver … your actually giving them what they want! And this squarely puts the customer first!

Keep management out of the kitchen. If your the kind of organisation willing to invest in bringing a smart bunch of technical people together, then trust them to do their job. They don’t need project managers standing over their shoulders asking them for an hourly update on their progress. This kind of Orwellian micro-management is culturally ingrained into some large software companies and in my opinion it is very, very damaging. It fosters resentment, encourages bullying and totally de-motivates developers because of the perpetual monkey on their backs!

I am so glad im not working for an organisation that suffers from these problems, and perhaps if you are then maybe you should consider a change?

Buildix – KNOPPIX Based Linux Distro

Heard about this on a Google tech talk, its a linux distro distributed as an rpm or VM ( which I have downloaded ), it provides a pre configured install of CruiseControl, Trac and Subversion …

Martin Fowler mentions it on his site, obviously hes recommending it because of his involvement with thoughtworks. Nevertheless its a great idea, based around shortening iteration zero.

There’s a good blog by one of the “authors” describing why they made this, which is well worth reading.

I’ve found it easy to use and configure and it gives a great head start to project teams trying to get continuous integration environments set up.