The Evolving Large Project Structure

Defining the structure of a large software development project can be a daunting challenge. The functions designed into a project's governing model are highly dependent on the goals of the project, the organization into which the resulting product will be deployed, and (to some extent) the project's technological selections and/or constraints. In my experience, most large project structures evolve in an organic manner and as a response to some condition or stimuli. For example, a dependency coordinator function is added to address deficiencies in upstream and downstream management.

An evolving project structure would appear to be a good thing, so long as the organization always focuses on what it needs and is able to adapt to the new reality within a reasonable time frame. Nevertheless, all too often no attention is paid to a project's structure until an event occurs that exposes a serious weakness. In traditional projects, this could mean that critical functions (like how change is managed) can be left undefined until the project is in trouble and unable to meet its dates. Agile projects do not appear to be immune to this phenomenon even though frequent process and scope change are expected.

Why do organizations undertake large multi million-dollar projects and not take the time to prepare themselves for the inevitable pitfalls that the lack of a flexible, well thought out governing structure bestows? A myriad of reasons come to mind: lack of perceived time to set up governance, lack of experience, a desire to not spend project funds on scaffolding, etc. Perhaps the more important question is what should we do about it?

Start Small, but Start Smart

Despite our desire to jump right into a project, some time needs to be spent up front identifying how a project will be organized and who will be assuming certain roles to make sure that essential functions are covered. In an agile project, this may take the form of defining the project's stakeholders and how they will be represented to the team(s) or how inter-team communication will occur (between multiple agile teams, the so-called "scrum of scrums" concept, or between agile teams and their non-agile counterparts). In a traditional project, the traditional functions of change management, dependency management, etc., should be dealt with up-front—even if one person owns them initially.

Look at Past Successes and Failures

It would seem to be a rare case that an organization would staff an entire project with inexperienced team members. Even if the majority of team members lack specific development experience, there are always those that have done it before. This knowledge needs to be mined. Expert help and best practices can also help avoid some of the pitfalls of a bad project organization structure.

Don't be Afraid to Change Course

If a structure is not working, change it! Sometimes we try to push a bad position just to maintain the status quo. It is inevitable that you will not identify all of the structural components that you need up-front. When you encounter one of these deficiencies, address it. Be equally mindful of outdated structural components and get rid of them.

Doing Things the Old-fashioned Way

My work gives me a lot of insight into how software projects can (and frequently do) go bad. Much of my focus of late has been on the more adaptive software development methodologies (the distributed agile software development experiments I did for my doctorate, for example). But, one project, or perhaps more correctly, program that has caught my attention as of late is one that is traditionally considered too large for using agile as a methodology -- with 250+ individuals involved. Officially, they used a plan-driven methodology.

Some agile purists might point out that the methodology they chose (actually, that they were forced to use) is the root of the problem. Perhaps it is, but as I look deeper into the project and its troubles I notice that they really did not pay much heed to any particular (official) methodology. As they move out of their dark days and into their next project, I wonder whether just following any rigorous methodology would help them be more successful (yes, I consider agile methodologies to be rigorous -- some of the most disciplined and rigorous, actually).

My First Entry

This is just a test entry to get things going. I currently have this blog marked so that everyone can read, but comments are moderated. I am going to test it out and see if I like it or not. I do not expect this will replace my normal journal, but I may be able to include some thoughts that are less personal, but are running through my head.