Scrum Didn’t Work For Us

Last year, our fearless team of interaction designers, creative designers and interface engineers (about 20 of us at the time) took the decision to embrace Scrum, the “agile” methodology for project management.

We were all given training courses to attend, and I myself volunteered (along with several others) to become a certified Scrum Master. As we began on sprints, attended sprint planning and reviews, and got together for sprint retrospectives afterwards, we debated the details of what we were trying to do. Our goal was to produce better things, possibly faster, but certainly more efficiently by controlled iterations and close contract between team members.

I believe we tried as hard as we could to make it work. However, after 6 months we could see it was not going the way we had hoped. Reluctantly, we began to prepare to transition the UX capability out of Scrum and back into something more like our old “waterfall” method. We enforced closure on this with a “failure party” a few weeks ago, to mark an episode from which we learnt a great deal.

The lessons we learnt, in summary, were that managing the UX design process within an overall Scrum framework can probably only work if the entire team is in one room (and not in three countries as we are), and that all members of that team are employed by the same company (and not 70% outsourced as we are). Even then, we had serious doubts about whether team members with almost completely separate skill sets (as is the case with UX teams) would ever be able to “swarm” on design problems in the way that they would had they all been members of a similar discipline, as the Java developers on the team are.

What went wrong for UX went wrong quickly. We saw “epics” dropping into backlogs for the UX team to deal with that we could not commit to in one (2 or 3 week) sprint. It was impossible to avoid a waterfall of activity between the three UX disciplines, so even the smallest stories began to take 3 sprints to do. For a creative designer to commit to a story before the UxD had worked on it was not going to happen. Similarly, if no markup had been produced for a story, back-end development starting on it was also impossible – or at least hugely impractical. We were simply doing waterfall with standups. That said, we ended up trying to do each other’s jobs at times, and the saw the quality of our output suffer because of it. We were certainly not going to achieve more speed through Scrum, and we could see no obvious way to raise quality either.

On top of procedural issues, we also saw cultural problems emerge. In constantly iterating on stories, the teams developed an attitude that user experience could somehow be developed piecemeal. Re-design a single menu, create a module for a page, but ignore the effect of that on the overall experience. The result was chaotic progress as we wandered through trees without regard to the wood we were in. This, coupled with an inability to really decide when a UX story is “done” was a recipe for a rather depressing mode of “good enough” in our deliverables to the dev team. It also put a huge amount of responsibility in the hands of the product owners, who were ill-equipped to judge UX design work. The UX leads (operating outside of the Scrum teams) in turn lost visibility of what their teams were working on, and whether any overall UX continuity was being achieved. The urgency of a sprint time-box also meant that we often didn’t give ourselves the ability to sweat the details. Capturing complex edge cases, error states, and other things fell away as sprints concentrated on the happy-day scenarios in the name of “shipping code”.

Perhaps some of this would have been cured through remedial activity such as more “story grooming”. But the need to break down epics into manageable chunks meant that such grooming was often like nailing jelly to a wall. A user experience cannot really be “broken down” in the way scrum demands. Attempts at grooming often took huge amounts of discussion time because of this. This was work that we would have done in the past in design workshops and kick-off meetings, which the sprint juggernaut appeared to flatten in its path. Scrum therefore brought all our deadlines hard up against us as we were unable to keep the UX sprints sufficiently far away from the development work for them, which in turn produced vicious circles of declining quality.

So we bailed, arranging an orderly “roll-off” of staff from each Scrum team so that we could put them into a single resource pool and get back to a more formal design process. This was far from easy as the UX leads had to step in to fast-track stories ahead of time to buy us the time we needed for waterfall to work for us while the incumbent team members finished what they had committed to in Scrum. So the parting shot of Scrum was to double our workload, albeit temporarily.

It’s important to note that the back-end (Java) development and testing teams have remained in Scrum, and the process appears to be working relatively well for them. They are, however, all present in the same physical place, employed by the same company, and for the most part have common skills between them.

I have no doubt that Scrum works. It’s just that it works for other people. I do have my doubts that even for them it works in the way I want it to work for UX (although I have read the Nielsen/Norman study on this that indicates it can do). I’m sure the debate will continue, but this, for what it’s worth, was our story at hotels.com.