Obeng, Waterfall, Agile and Hybrid: Start With the Project

Recently, a friend who had been on an AI training course mentioned that part of the discussion had focused on what delivery approach should be used for AI projects.

Should they be Agile? Should they be hybrid? Is a more traditional approach still appropriate?

It got me thinking that, before choosing the method, it is worth considering the nature of the project itself.

AI projects can vary enormously. Some are relatively straightforward implementations of established tools. Others involve a great deal of experimentation and learning. In some cases the outcome is clear but the route is not. In others, even the end goal is still emerging.

So rather than starting with the methodology, perhaps the more useful starting question is:

What sort of project are we actually dealing with?

That is where Eddie Obeng’s model can help.

Obeng looks at two fairly simple questions:

• Do we know what we are trying to achieve?

• Do we know how we are going to achieve it?

From those two questions come four different types of project.

And I think they provide a useful way of thinking about whether Waterfall, Agile or a hybrid approach is appropriate.

Painting by Numbers

This is probably the easiest one to recognise.

We know what the outcome should be and we broadly know how to achieve it.

It might be replacing a piece of equipment with something similar, rolling out an established process or delivering a type of project that the organisation has done several times before.

There will still be risks and problems. There always are.

But fundamentally, both the destination and the route are reasonably well understood.

This is where a more traditional Waterfall approach can work perfectly well.

We can define the outcome, build a plan, sequence the work, understand the dependencies and manage delivery against that plan.

There sometimes seems to be a reluctance to say that Waterfall is the right answer, as if Agile must automatically be more modern or more effective.

It isn’t.

If we know where we are going and we know how to get there, then being able to plan the work properly is an advantage.

Going on a Quest

This is different.

We know what we are trying to achieve, but we are less certain about how we are going to achieve it.

That is much closer to the environment where Agile approaches can work particularly well.

We may know that we need a better customer experience, a new digital service, a more efficient process or a particular business outcome.

What we do not yet know is exactly what the solution will look like.

In that situation, trying to define every step at the start can give us a very detailed plan based on assumptions that may turn out to be wrong.

It makes more sense to work iteratively.

Try something.

Learn from it.

Adjust.

Then move forward again.

The important point is that the learning is not getting in the way of the project.

The learning is part of the project.

This feels particularly relevant to some AI projects, where the business need may be reasonably clear but the best application, technology or operating approach is still being worked out.

Making a Movie

This one works the other way around.

We have a reasonable understanding of the process we are going to follow, but the final outcome is not completely clear at the start.

A film is the obvious example behind Obeng’s description.

There will be a budget, a schedule, locations, equipment, people and deadlines.

A lot of that can be planned.

But the final product develops as the work progresses.

This is where a hybrid approach starts to make a lot of sense.

Some parts of the project can be managed in a structured and predictable way.

Other parts need flexibility.

That is probably much closer to the reality of many projects than the idea that an entire project has to be either Agile or Waterfall.

We might be very clear about the procurement process, infrastructure, governance and key milestones while still allowing the actual solution to evolve.

Trying to force both types of work into exactly the same management approach is unlikely to help.

Lost in the Fog

Then there is the difficult one.

We are not sufficiently clear about what the outcome should be and we are not clear about how we are going to achieve it either.

At this point, I am not sure the main question should be whether the project is Agile or Waterfall.

We probably do not yet have a delivery-method problem.

We have a definition problem.

The first job is to understand what we are actually trying to do.

That might mean talking to users and stakeholders, exploring the problem, testing ideas, running pilots or prototypes and looking at different options.

In other words, reducing the uncertainty.

Only when we have learned enough does it really make sense to decide how the project should be delivered.

And this raises another useful feature of Obeng’s model.

Projects can move between the four types.

A project might start Lost in the Fog.

As we learn, the desired outcome becomes clearer and it turns into a Quest.

Later still, enough may have been learned about the solution for parts of it to become much more predictable.

The delivery approach can therefore change as our understanding of the project changes.

So where does Hybrid fit?

I sometimes think hybrid is described too casually as:

“A bit of Agile and a bit of Waterfall.”

That does not really explain why we would use it.

Most significant projects contain different types of work.

Imagine introducing a new digital service.

The infrastructure might be reasonably predictable.

The procurement process might follow a defined route.

The customer-facing solution might need iterative development.

The way people will use the service operationally may still be evolving.

Those are different types of uncertainty within the same project.

Why would we necessarily manage all of them in exactly the same way?

A sensible hybrid approach allows us to apply the appropriate level of structure to each part.

That is rather different from simply taking a Waterfall methodology and adding a few Agile meetings to it.

The Method Should Follow the Project

For me, that is the useful connection between Obeng and the Waterfall, Agile and hybrid discussion.

The danger comes when we start with the methodology.

“We are an Agile organisation.”

“This programme uses Waterfall.”

“All our AI projects will use a hybrid model.”

Maybe.

But first I would want to know what sort of project we are dealing with.

How clear is the outcome?

How well understood is the route?

Where is the uncertainty?

If we understand both the destination and the route, plan it.

If we understand the destination but need to discover the route, iterate and learn.

If different elements of the project have different levels of certainty, use a hybrid approach.

And if neither the outcome nor the route is clear, spend some time understanding the problem before pretending that we have a delivery plan.

The No Fluff Takeaway

There probably isn’t a single correct methodology for AI projects.

Just as there isn’t a single correct methodology for projects generally.

The better starting point is:

What do we know, what don’t we know and where is the uncertainty?

Obeng gives us a simple way of thinking about that.

Once we understand the type of project we are dealing with, the discussion about Waterfall, Agile or hybrid becomes much more useful.

Methodology should follow the project, not the other way round.