In the first episode of Quay Conversations, We explore the issue of risk assessment in large programs and why getting it wrong can have some serious consequences.
Listen on
Perspectives from the Quay Bulletin
Transcript
Hosts:
Rod Adams | Co-founder of Quay Consulting
Rebecca Bennett | Co-hostIn this episode, we’re focusing on risk management and large programs and why scope is so critical to assessing risk, ensuring that the right skills are in play and the consequences of not having growth in focus. As Victoria’s recent COVID-19 church has shown, getting it wrong can have extraordinary consequences.
REBECCA BENNETT: We’re looking at the most recent outbreak of COVID-19 in Victoria, seeing that there are a lot of issues around containing the virus in the quarantine hotels and that there are issues that parallel some of the challenges that arise when a project in crisis.
ROD ADAMS: Yeah, most definitely. There are clear parallels.
REBECCA: Let’s look at what we know about the crisis in Victoria, that there are a number of players involved, that there are some key failings, but also there’s a lot of unknowns. So from your perspective, what do you think we are seeing as the starting point of this project or program?
ROD: Well, I think at the moment there’s a number of reviews going on and some of the accountable people ducking, waving, and you can kind of understand it. They’re waiting for the information to come out of the review. But there’s also opportunities in flight to provide information the way things have unfolded. I think you need to take it back to really first principles from a project perspective. And the first thing you need to look at is scope. What was it they were trying to achieve in terms of the hotel quarantine system?
REBECCA: From my understanding, the Victorians had already stood up a program for quarantine hotels around March, and initially it was to manage accommodation for quarantined health workers and for the homeless.
ROD: If you’ve already run a similar type of project or engagement, you can leverage those learnings. I think it’s worthwhile going all the way back to have a look at the scope of the project. And what you really want to look at is exactly what was it that the government was trying to achieve? What was the overall objective of this exercise? So how did we end up having people in quarantine to start with? That’s the first thing that needs to be answered.
REBECCA: So in your view, what was the scope of the project?
ROD: If you take it up to the highest level, the scope of the project was to keep COVID-19 or coronavirus out of Australia. What had been identified was that the sources of infection were coming from overseas. So at the very highest level, that was the scope. Now, how Victoria ended up trying to execute that scope is where the issues have been uncovered. So if you break it down into the next levels, so clearly what was happening was the infections were coming from overseas.
And the other thing that was known was that people weren’t infectious after more or less 14 days. So there’s a number of things within the scope that needed to be managed. The first thing was exiting people off planes. The second thing was transferring them somewhere that was safe. And the third element was once you had taken them to somewhere safe, making sure that facility or that process was run professionally and to a high degree of for want of a better term, quarantine standards.
REBECCA: The states approached it in similar and yet different way. So the overall goals were similar. The Ruby Princess incident in New South Wales had shown how quickly the virus could get out of control if it entered the community. And so moving towards the hotel quarantine for returning travellers was both a utilisation of capacity that with an impending lockdown but also a way of containing the spread of the virus. The ADF was involved in that early planning. … The Australian Defence Force… Was involved in the early planning and help was offered to the states. And from what we know, some of the larger states like New South Wales, Queensland and Western Australia adopted that assistance quickly and that the scope of that assistance was left to the states to determine. What had been set up in Victoria at that point and why do you think that they didn’t adopt the ADF support?
ROD: Well, look, it comes down to execution. I think the national cabinet, it so from a project perspective, more or less, was the steering committee made recommendations around quarantine and some of those recommendations were around the processes in and around the airports, the transporting of people from overseas and the quarantining of people from overseas. But these were just recommendations. They weren’t binding, which is an interesting question in itself, because actual border protection and you would argue by default quarantine and I think there’s a thing called the quarantine act from way back when is actually federal jurisdiction.
ROD: It’s not state jurisdiction. So I think there’s still some questions to be answered there around ultimate accountability. But setting that aside, the states agreed at that national cabinet to take accountability for running the quarantine programs. Now, at that national cabinet, there are specialists there and they provided specialist advice. And that advice was all around landing the people, disembarking them, transporting them to the quarantine facilities and then managing those quarantine facilities. Now, what you would do in a normal project situation is you would look at the scope of the project and at every point you should do a risk assessment.
And normally what you do is you run a risk workshop and you engage as many people as possible stakeholders, subject matter experts, whomever it may be, to help you identify the risks and the mitigation against those risks, so I don’t think it’s a criticism of the personnel that the Victorian government used. I think it’s the way they got there. In the first instance, it should have been identified that COVID-19 is extremely contagious. So to run a quarantine facility, you need personnel that are properly trained and properly equipped to manage those facilities.
If they had undertaken a proper risk assessment, this would have come out. In my view, they didn’t undertake that risk assessment.
REBECCA: One of the interesting things is actually come out of the early parts of the parliamentary inquiry that’s underway at the moment is that the Victorian Department of Health and Human Services was quite underfunded and had been using a program of workers as part of its workforce management. Would you see that that could be a parallel in the way that, say, a program had been … The skill had been brought into a project?
ROD: I think what you’re looking at, the first and foremost is the actual execution and you’re looking at the reason for the execution model that was adopted. Now, I’m keen to talk about that because there’s a number of theories floating around about why they went with the execution model they went with.
But I want to bring you back to the fundamental decision which should have been based around risk. Now, most projects have a number of key elements that constrain a project. Time’s one of the qualities. Another cost is another. Now, in this situation, cost was no object. What’s happened with COVID-19 is we’ve seen from a government perspective the rule book torn up in terms of what the government’s willing to spend, not just to keep our citizens safe, but ultimately the economy safe.
So normally when you do a risk assessment, you’ve got to look at it through the prism of certain constraints. So I think in this situation, money wasn’t a constraint. I don’t think time was particularly a constraint because it was deemed that people would remain in quarantine until they were no longer infectious. Quality absolutely was the key constraint. Quality from a risk perspective was the key thing that should have been focused on, because the higher quality of the outcome, the less chance of the infection escaping.
What’s the point of saving time and saving money if the quality is going to be poor and the situation, allows the virus to escape? You actually don’t achieve your overall objectives. Back to the risk focus. My contention is they didn’t actually do a risk assessment. They just went straight to execution mode and they made a series of assumptions around the quality of the execution that they didn’t actually run through a risk framework and actually ask the right people or workshop, ‘… Is this model going to deliver the quality outcome that is required or demanded of this challenge?’
REBECCA: What’s really interesting is to look at how decisions were made in the choices for the particular people that were used, the decisions that were made around using the ADF versus using, say, private contractors. What’s your perspective on how those decisions were made and the risk that that posed?
ROD: So there’s two aspects to it. There’s the how and there’s the why. Now, what are the big challenges about delivering project stakeholders’ is understanding stakeholders motivations. So, for example, there might be somebody pushing hard to engage a certain group of workers because that will benefit them. So their frame of reference is around getting a benefit for a particular group of workers. The frame of reference is not the overall scope or the overall outcome. So ultimately the buck has to stop with the sponsor in the typical project world.
ROD: So I contend that there wasn’t a risk assessment done because … You can do a risk assessment and you can decide to accept the risk. You can say, we are actually … We understand the risk of having these quarantined hotels managed by, at times low skilled, low paid, poorly equipped, poorly trained individuals. And we will accept that risk for the upside of and, no doubt there’s benefits probably around cost, probably around keeping, local people working. But I don’t think the decision was made with that information in hand because I don’t think a proper risk assessment was done.
REBECCA: So in the context of a project framework, if you look at a program as a whole, where is that risk assessment… Who is likely to initiate that type of risk assessment?
ROD: It is one of the responsibilities of the project manager to initiate that risk assessment and then provide the recommendations that come out of that risk assessment. But back to the how and why, I don’t think that process was followed. Then we get to the why. Why was the decision made to hire people that were, as I said, poorly equipped, poorly? And the how is I believe that there wasn’t a thorough risk assessment done? That doesn’t answer the why, but what it does answer is without having executed a proper detailed risk assessment, you don’t have the information at hand to make an informed decision to give you the best chance of making the right decision, particularly in the frame of reference that the quality outcome for this project, the quality aspect, was so high as opposed to the time and cost elements.
REBECCA: So, circling back to risk. If we look at the projects that we’ve been involved in -and we’ve had some high-risk projects…
ROD: Yes…
REBECCA: Not performing the risk assessments obviously exposed quite a serious fault line in the consequences of not adhering to the scope, which is to contain the virus and to contain it in a way that reduced the likelihood of community transmission. If we look at this from a project manager versus a sponsor perspective, if the project manager hasn’t done the risk assessment, where was the oversight from a stakeout or program perspective, looking at risk? If it hasn’t … If it hasn’t been raised by… At the layer of project delivery, where was the oversight in managing and addressing risk?
ROD: Well, you could probably contend that there wasn’t sufficient oversight and potentially the right governance hadn’t been set up to enable that oversight or even people with issues to be able to raise those issues. So there seems to be some email trails where different agencies at different times have raised red flags. But there doesn’t seem to be an overall governance structure to enable those to be surfaced because you can make the wrong decision and then you can pivot quickly. But that doesn’t seem to have happened.
A poor decision was made which accepted an exponential amount of risk into the project. Now, that’s not to say that they couldn’t have got lucky, because, as I understand it, Western Australia also employed a similar model and they don’t seem to have had the same problems.
REBECCA: Could that be down to the sheer scale of the arrival of international travellers?
ROD: Indeed, could be volume. It could be a volume thing. It could be that the way the hotels are set up, they’re a long way from each other. It could be that the security guards weren’t bouncing around between different hotels. They would work at one hotel. It could be a whole number of reasons. But the point I’m trying to make is it didn’t have to happen. But the way the Victorian government set up the program, they significantly increased the risk of a poor quality outcome and the virus escaping.
Now, they could have done a risk assessment and they may have rejected the use of the ADF and the state police and whatnot who are a better equipped and a more highly trained workforce. They still may have rejected that based on the risk assessment, but my contention is they didn’t do the risk assessment start with.
REBECCA: So the understanding I have from the initial offering of ADF personnel to support the states in its quarantine compliance is that it was offeredut, but at the time, which is back in March, the Victorian government felt that they had the capacity within their existing authorities to manage the outbreak. Possibly what’s happened in-between time is they haven’t revisited the risk.
ROD: So it’s the expertise thing because the key boundary for this project has always been quality. As I said, cost, time, infinite almost in terms of trying to achieve the objective, which has been quality. So, you know for example, we have some clients that are pharmaceutical clients. Their projects are all about quality because they need be. We have other clients who need to get things to market quick so they’re more focused on time when they look at what are the drivers and the levers for a project.This project was all about quality.
REBECCA: Could you argue in the context of time, quality and cost, that because Australia had achieved a level of flattening the curve, that a degree of complacency has crept in?
ROD: Yeah, look, I don’t know if it’s complacency. I don’t think enough people got in the room that had a broad enough understanding of the risk and everybody was just doing their piece. And that’s fine. But if you can’t piece together all of it, you can look at it from a … So it’s contiguous so, so you can see the end to end as the planes land, as the people get off the planes, as they get on the buses, as they get to the hotels.
ROD: So just some anecdotal evidence of walked around Sydney and I’ve walked past the quarantine hotel and it looks like a crime scene. There’s police tape all over the place. When the buses pull up, you’re being asked to move across the road. It’s clearly a… Almost a military-run exercise with occurring on the side of caution around risk to try and keep people separate. So I don’t think anybody had that contiguous view and it was just assumed that how hard can it be?
We’ll get security guards to do it. And without really working out, are they the right people? Do they have the skills? Do they have the equipment? And if you understand the security guard industry in Victoria, it went through an inquiry a couple of years ago. It’s a series of companies subcontracting to other companies. So the actual company you engage… They may not be using their employees. They might be the employees, might be three, four, five steps removed.
And that’s a known fact. And also the training is, you know, it’s not an industry that requires any real training to get into.
So already off the bat, you know, the people that you’re using are not highly trained. And the way the contracts were worded as I understand it, all the accountability around training and equipping these people were pushed on to the hiring companies, which to me is ludicrous because where’s the oversight from the health department or where’s the oversight from the government to ensure that there’s compliance?
REBECCA: So we’re really talking about, as you say, a failure of assessing the risk, inadequate governance as the program’s been rolled out and rolled on and a degree of hands-off in terms of delivery.
ROD: That would be a really good way to put it, a degree of outsourcing. So outsourcing doesn’t necessarily get you your outcome. You know, you can roll what you like in an RFP, in an outsourcing document and a contract. But if you’re not going to oversee them and have mechanisms to validate that what you’ve asked for is happening, how do you know it’s happening? Hope’s not a strategy. And to be able to say, well, you know, it was in the contract.
Well, that’s great. But you’ve actually outsourced something to an industry that’s actually not equipped to do that. And nobody has run quarantine hotels for 100 years. So it’s not like we’ve got companies out there that have a deep history in managing quarantine hotels now.
REBECCA: And the reality is that we’ve found ourselves with a significant surge in cases and unfortunately deaths as a result of either the hands-off approach or the general lack of oversight.
ROD: Look, I think it’s the lack of a risk assessment to understand where the risks were. If you’re going to outsource it from a quality perspective, you need to take a certain amount of accountability – that doesn’t seem to have occurred. But you’ve got a workforce that is actually … Has no safety net. JobKeepr certainly didn’t provide a safety net for the casualised workforce. So you’ve got people that are motivated to actually go to work, even if they’ve got the sniffles.
ROD: So they will potentially pick up the virus, not have any sort of safety net from a financial perspective. And they’ll keep working, go home, pass it on to the people they’re living with, who then go into other facilities. And that’s enabled to get into the aged care sector. There’s a whole proper risk assessment you would think would have gone down to that level of detail, even asking the question if there is a casualised workforce and they won’t get paid if they don’t work. Do you think they’ll keep turning up to work? And the answer is quite possibly.
REBECCA: And I suppose that this is what this is exposes the fault lines of moving to a more flexible way of working. And who bears the risk?
ROD: Yeah, well, look, it’s definitely something that needs to be investigated in terms of supercharging the virus once it got out of the quarantine hotels, because there’s a couple of categories of workers. They’re not well paid and incredibly casualised. That is a security guard industry, but it’s also the aged care sector. And a lot of those people live in the same areas and in the same housing situations. And again, they’re casualised. They work across multiple sites.
They had no access to JobKeeper because they were disqualified because for whatever reason, the federal government had rules and it was really focused on permanent employees. So you’re actually there’s no incentive for those people to take time off work when they’re ill. their mode of working is to just keep working because they need to.
REBECCA: We can see that when you have a highly contingent workforce with greater risk at their personal level, taking that risk, it actually introduces a heck of a lot more risk into a project.
ROD: Yeah, it does. And particularly if one of the things you’re trying to guard against is the moment somebody shows any signs of illness that they shouldn’t come into work and go and get tested.
REBECCA: Do you think that there’s a real challenge, though, that a lot of people who are asymptomatic just simply didn’t think that they could be carrying the virus into those situations?
ROD: No doubt. And the fact that this virus has people with who are asymptomatic, that’s an issue. But if you take it up a level, if people aren’t trained, if they don’t have the right equipment, you are inviting a significant amount of risk for things to go horribly wrong. And in this situation, things have gone horribly wrong.
REBECCA: I think that a lot of people would be arguing that this is an unprecedented crisis and that possibly there wasn’t the time to implement some of the risk assessments you’re talking about. What would your view on that be?
ROD: You know, what I’m seeing is a bit of a blame game at the moment with people saying, oh, well, you know, we never got offered the ADF. Well, to me, that’s neither here nor there, because if a proper assessment was done, you would have been requesting the ADF. So, it’s not a matter of the offer was never made. It should have been a request. So sometimes you only need to get one or two things right.
ROD: And in this situation, there would have been people accountable just to get a couple. A lot of decisions, right, and there is always enough time for the critical decisions to get them right, you burn them in order, you get the right people in the room and you go through the process and you get the right decision. It’s not acceptable to say it was rushed. Therefore, this is how we did it, because I keep coming back to the frame of reference about what was important about this project and the importance was the quality of the execution and not allowing the virus to transfer from travellers into the general population.
REBECCA: So if we look at a contrasting situation where the response of other states to the surge in Victoria of new cover cases and you look at, say, what New South Wales has done in terms of border control, in terms of how it is responding to links, cases surging in New South Wales. Would you say that there is a different thinking, different approach occurring in New South Wales against what’s possibly happening in Victoria?
ROD: You mean a different approach in terms of running quarantine hotels where a different approach and….
REBECCA: In managing the risk, in managing the risk. Do you think there’s a stronger …. Having learned possibly from, say, the Ruby Princess debacle, do you think New South Wales has possibly got some of its risk assessment stacks lined up?
ROD: Look, I think so. I think the offer was made. I think to me it’s pretty obvious the real risk is not getting them off the plane, what happens on the bus? The real risk is that for 10 days in a hotel. Keeping people in situ, keeping them away from each other, that’s if you look at where we are. Where’s that weak point there? To me, it clearly it’s… It’s trying to keep someone cooped up for 14 days and keeping them away from other people.
REBECCA: Once you realize that, then to me, you absolutely are erring ,on the side of caution and getting people that are trained, properly trained and equipped as best you can in dealing with the public and dealing with difficult situations. And there was even a thought around prison guards at one point. But that’s essentially where the key risk was, you know, I think is quite obvious. And it must have been obvious at the national cabinet level, which is why the offers were made.
ROD: So somebody at that level obviously knew that it was going to be a risk. And, you know, you’ve got to look again, you’ve got to come back to the quality assurance from the federal government. If the federal government is responsible for quarantine, what is their role to actually poke their head into what’s happening down in Victoria and say, you know what, we think you’re accepting too much risk by outsourcing, not just the management, but also the equipment and the training of the whole workforce to third party providers who wouldn’t have managed a pandemic in 100 years.
ROD: I’m yet to see the federal government again and outsourcing question once they’d outsource it to the state government, they seem to just walk away.
REBECCA: Which brings parallels to what’s happening in the aged care sector. And if you look at where the biggest impact has been in Australia, it has been in aged care in terms of the rate of infection and deaths occurring. It’s an oversight … It’s a sector overseen by the federal government, but executed by the states
ROD: Very tricky, where accountability and responsibility laws is… is very difficult to discern.
REBECCA: We often talk about difficult conversations and blame cultures. Do you see that at the moment, that the right people are stepping up and saying, hey, we didn’t do what we needed to do? Or do you see a blame culture developing where they’re going to shift the buck?
ROD: Look, it’s hard to say. So I’m saying to state premiers apologise and say they’re accountable, but I’m not saying any action.
REBECCA: Is it too soon?
ROD: No, it’s never too soon. For example, the Ruby Princess, Gladys Berejiklian, has stepped up and said that she apologises and there’s accountability. But nobody… There were failures, but nobody’s been moved on. Nobody seems to have paid the price for equally down in Victoria. I don’t think you need to wait for the inquiry to find people who’ve made mistakes. And if accountability means anything, then people have to be accountable and not just talk about it.
REBECCA: So we’ve talked about outsourcing. Can you outsource accountability?
ROD: Well no, you can’t outsource accountability? But in this situation, it looks like they have tried to outsource accountability. And the reason I say that is if you have a look at the contracts, the responsibility for training and equipping the security guards was with the hiring firms. Now, there’s nowhere in that contract or from what I can see in terms of how the project’s been structured, that any government agency remained accountable for ensuring that was happening.
So what they seem to have done is outsource the responsibility and the accountability, but somewhere in the chain of events or chain of command, there should have been … Accountability should have remained with the correct agency, and that accountability should have been taken seriously in terms of regular spot checks. However, I would argue that having tried to outsource responsibility was an extremely high-risk strategy that was not well understood and well worked through. That’s led to some pretty catastrophic outcomes.
REBECCA: So what do you think the levers would have been in a situation like this, knowing that the risk was so high and knowing that the potential outcomes could be so poor, what levers would have driven them to outsource the quarantine compliance?
ROD: Look, I can only speculate. People say that the emergency services in Victoria don’t get on with the ADF. There’s been other lines around the state government was looking to hire local people. It may have been that there were people within the ruling Labor Party that felt local workers should get first shot at every possible job opportunity. We won’t know people’s motivations and we may never know. But it comes back to everybody has a different frame of reference. And if someone’s not accountable for the quality outcomes, you can’t hold them accountable for making bad decisions.
ROD: I think it’s important that some decisions would have been made where people have their frame of reference is just around achieving a certain outcome; that they’re not accountable for delivering the overall quality outcome but the people that are accountable for that should have provided the processes and mechanisms for better decision making
REBECCA: As the inquiry makes its recommendations and as the inquiry explores a lot of the issues that we’ve been sort of talking around today, do you think that there are going to be some significant lessons that can be learned about risk in delivering projects for Victoria’s various programs?
ROD: Without a doubt. Without a doubt. What will come out of it is that there was no adequate risk assessment, that risk assessments always critically important. You have the right people in the room, you have the right information. And when you do make decisions and critical decisions like this, they’re made in such a way that if you are accepting risk, you acknowledge that and you have mitigation plans. If the risks turn into issues. There were no mitigation plans because my view is there was no risk assessment.
There was no acknowledgement of the risk. There was just an assumption, an assumption that this model will work and no quality controls around the model
REBECCA: Are the ways in a … In a project, we make assumptions about a range of elements that are involved in a project. What are the mechanisms that we could use to mitigate some of the unforeseen risks?
ROD: Well, it comes back to the original point. You can’t generate mitigation strategies unless you’ve identified the risk. So it comes back to that fundamental.
REBECCA: So one of the outcomes that we’ve seen as part of this process is that Melbourne is no longer taking in international travellers, which is distributing risk elsewhere. What does that tell you about Victoria’s management of this crisis?
ROD: Oh, it tells me that the whole program has been a complete failure to the point that the scope and at the highest level the scope would have been to be able to land, pick a number three, four or five hundred people a day and quarantine them safely and successfully before they went back into the general population has been an abject failure,
REBECCA: Given that the outcome has been that we have a substantial surge in Victoria and they have stopped accepting international travellers. What’s your perspective on this as a project, as a whole?
ROD: Well, there’s only one perspective, and that is the project has been a complete failure. Some organisations can’t kill projects in this situation. The government has, and that’s the right thing to do. But it… It spells it out loud and clear that the projects fail
REBECCA: And it’s distributed the risk elsewhere.
ROD: It’s distributed the risk elsewhere. There’s also probably some other conclusions you can draw and that is there’s been a loss of confidence in the ability for the various agencies to run a quarantine program in Victoria to continue with this program. And they’ve decided to only lend people from overseas into other states who are running successful quarantine programs.
REBECCA: That’s it for today’s Quay Conversation. If you found today’s episode useful, you can find the show notes on our website and you can subscribe to the podcast by searching for us on your favourite podcast. Thanks for listening.
Quay Conversations is produced by Quay Consulting. Our theme song is Trumpet Funk by ColorThemeMusic. You can subscribe to the podcast on Apple Podcasts, Google Podcasts, Spotify, and Stitcher.