Lack of technical knowledge in leadership is a key reason why so many IT projects fail

Implementing information technology projects in the public sector is challenging. And we seem to experience these challenges with a regularity that is both perplexing and frustrating. Think #censusfail, the myki smartcard fiasco and the Queensland Health payroll debacle, to name just a few.

Indeed, the failure rates of large-scale IT projects are unreasonably high across both the public and private sectors, with costs of failure reported in the hundreds of millions of dollars.

Read more: As census failure blame points at IBM, why we shouldn't be surprised by its failings

Researchers have been studying the causes of this kind of failure for decades – yet this appears to have had no impact on the problems organisations face when they undertake large-scale IT projects.

Only a few of these studies have considered the role played by the technical knowledge of project leaders. With colleagues, my own research finds the technical knowledge of leadership plays a key role.

Understanding the advice being given

We examined previous research alongside a case study of a large-scale public sector IT project failure. Reports on the project and documents obtained through freedom of information requests created a rich pool of data that allowed us to examine the life of the project as it unfolded over many years.

One of our main findings was underpinned by the idea that leaders require more than a passing familiarity with the technical skills required to do the job if they are to identify competence in those carrying out the work. Without this, the projects have a poor chance of success.

Technological competence needs to be specific, not generalised. The most senior executive with day-to-day accountability for the project, and who has a direct and material impact on project outcomes, must have experience with, and knowledge of, the technology being developed.

An inexperienced project leader will be incapable of comprehending the advice being provided if they lack the specific experience in the technical domain being managed. That means that it’s not sufficient to surround an inexperienced manager with experts upon whom they would theoretically turn to for advice.

Read more: 'Digital by default' – efficient eGovernment or costly flop?

The research and the case study demonstrate that an inexperienced senior executive defers to inappropriate sources for advice and support, choosing not to trust the advice of their internal experts. Instead, they are apt to treat technical disagreements as personality conflicts, and characterise critiques of the vendors’ performance as interference.

Technical skills on Australian boards

Australian public companies are investing in very large information systems projects and many of these are at risk. When these projects fail it can have a direct material impact on a company and its share price. And when projects are delivered with less than the required functionality, or at an inflated price, they negatively impact upon the performance and effectiveness of those businesses.

Effective oversight requires competence and experience. This means that the boards need some measure of IT knowledge if they expect to provide effective governance, risk management and strategic oversight of IT projects.

We have analysed Australian Stock Exchange publicly available information on the qualifications and experience of Australian directors in order to gain some insight into the digital competence of Australian boards.

Read more: The road to failure is paved with good intentions – here's how to turn them into action

After examining 35,000 director positions, with 37,500 reported qualifications, we found that just 6% of directors had qualifications in a STEM-related field, or possessed a PhD.

Most directors have qualifications in finance (18%) or accounting (19%). Lawyers figure strongly on boards with 9% representation, while mining qualifications count for 8% of board membership. Those with arts, business or other qualifications accounted for 40%.

The current composition of Australian boards of directors is heavily weighted towards finance, accounting and legal. While this background is not particularly surprising, it means that the accumulated knowledge and experience of Australian boards is not adequate to provide effective governance and oversight when it comes to the significant challenge of IT projects.

Focus on the problem, not the symptoms

Lots of factors contribute to the success of information systems projects: support from senior management; clear and realistic goals; a strong, detailed plan that’s kept up to date; good communication and feedback; the involvement of both clients and users; suitably qualified and sufficient resources; and effective change management.

But the absence of some or all of these factors are not a cause of project failure. Rather they are consequences of a poorly run project due to the situational incompetence of project leaders who have direct oversight and accountability for the day-to-day workings of the project.