Endeavour Programme and BMD Urban: Lessons learnt from AI in practice on major projects
By Dr George Quezada, CSIRO Data61 on secondment to Arup Australasia
The world is amidst an artificial intelligence (AI) arms race with tech giants and national governments vying to create the best tech and skills ecosystem. According to CSIRO’s Data61 recent report, Australia’s AI Roadmap[1], AI could be worth over $22 trillion to the global economy by 2030. That’s new value created by humans working collaboratively with a new breed of software tools. Software that can harness and use enormous amounts of data to realise untapped value and achieve benefits and important goals, such as early detection of cancer, agricultural robots, vehicle crash avoidance, targeted infrastructure maintenance and more.
But what does it take to implement AI technology and create a successful business use case? I’ve explored this question with Australian-based AI start-up Endeavour Programme which is working with Brisbane-based civil construction company BMD Urban. BMD Urban is presently implementing Octant AI, an AI-enabled project management tool to help the business and project managers detect problematic projects, 30-60 percent ahead of time compared to current processes. The collaboration between Endeavour Programme and BMD Urban provides a useful case study to illustrate how adopting advanced AI technology enhances capabilities to tackle practical business problems and can provide valuable business insights and identify commercial opportunities and benefits.
With an annual turnover in excess of $450 million, BMD Urban faces the challenge of running many civil infrastructure projects within an increasingly complex urban development environment. In general, infrastructure projects can be notorious for running overtime and budget. Oxford-based expert on megaprojects Bent Flyvbjerg maintains a global database of the performance of thousands of infrastructure projects (see Figure 1). The data shows that overruns are typically large and all too frequent.[2],[3],[4]
Figure 1. Cost Overruns for various industrial Project Types. Adapted from Flyvbjerg & Budzier (2019).
What’s failing is human prediction and control of these projects. For project managers, there can be simply too many inter-related factors that could push a project over budget or cause massive delays.
Octant AI uses a technique called machine learning to make Forecast Final Cost Predictions and calculate predicted project and portfolio gains and losses. For BMD Urban, Octant AI has been trained on 680 civil projects spanning nearly seven years of historical data. Projects include bridges, intersections and roads, pipelines and others. Octant AI has demonstrated a range of performance improvements depending upon the level of detail accessed, and as such, Octant AI is scalable and can be implemented off the back of available data.
Over the course of a year, Endeavour Programme conducted four rounds of AI validation on BMD’s business data. This was done to demonstrate that Octant AI’s performance was stable and accurate, and to show how Octant AI creates business value by providing early warning of significant project cost blowouts. Results from the most recent testing has been encouraging: Octant AI demonstrated particular value in the first half of projects, providing reliable early warning and prediction of final cost overruns (at 70-80 percent precision) and improved portfolio forecasting accuracy (circa 20 percent; see Figure 2).
Figure 2. Sample comparison of Business forecast cost to complete and AI Prediction for extreme projects.
So, what has the implementation team learned about using AI tools? In brief, three key lessons have come from this case study to date:
- Invest in data quality. To gain the most out of AI tools, organisations need to know their data, warts and all, and invest in initiatives to improve quality and collection. Being data smart means making sure you collect the right data that links closely with your organisation’s ultimate objectives. For BMD Urban, its drivers of project performance, and their understanding of what drives performance might change as they do more analysis with tools like Octant AI.
- Developing a useful AI reporting interface can take time. There are hard and soft issues with providing AI predictions and implementation teams need to understand what the AI tool outputs mean to the business and how to convey that meaning to end-users. For BMD Urban, it’s about understanding what’s valuable to the enterprise and helpful to the individual project managers as end-users.
- AI doesn’t replace human judgement and decision making. Instead, the Octant AI system helps project managers and reviewers crunch through huge volumes of project data to give an ‘early warning’ of problems that are highly likely to result in project overruns. It’s important not to blindly follow AI systems but in this case, to view AI as another tool in the project management toolbox.
[1] Hajkowicz S, Karimi S, Wark T et al. 2019. Artificial Intelligence: Solving problems, growing the economy and improving our quality of life. Data61 CSIRO: Australia.
[2] Flyvbjerg B. Over Budget, Over Time, Over and Over Again: Managing Major Projects In: Morris Peter W. G. et al. The Oxford Handbook of Project Management. Oxford: Oxford University Press; 2011.
[3] Flyvbjerg B, Ansar A, Budzier A et al. 2018. Five things you should know about cost overrun. Transportation Research Part A: Policy and Practice, 118(174-190).
[4] Flyvbjerg, Bent, and Alexander Budzier. 2019. ‘Oxford Global Projects – What We Do: Benchmarking’. Oxford: Oxford Global Projects. https://static1.squarespace.com/static/5c312863266c07d084fcd39e/t/5c59e47124a694ac1159d451/1549395097028/OxfordGlobalProjects.pdf.
