Project Risk Management

I have been fascinated since I read an article in the 60′s in Physics Today about peoples’ disconnect between the reality and perception of risk. Politicians manipulate this-and I could write pages about this fascinating defect in human perception. This paper is, however, about managing risk in projects, particularly technical projects. How can we overcome this human weakness and make sensible risk management decisions? I have some suggestions.

The Risk Matrix:

Risk is the combination of the likelihood of occurrence and severity of consequences of an (undesirable) event. Risk management is the deployment of resources to lower the probability of occurrence and/or (negative) consequences of the event to acceptable levels.

You are probably familiar with the most basic project risk management tool, the
risk matrix. I show a notional rendition below.

I have shown arbitrary red and green zones separated by a 45 degree line to illustrate the concept. A project with less risk tolerance would draw the 45 degree line lower and a project with more risk tolerance would accept a higher line.

The Four Core Risk Categories:

We can mitigate the risk in erroneous evaluation by human perception by organizing four risks in separately managed categories. For example, the human mind stubbles badly in evaluating the relative risk of death from terror attacks, highway accidents and the flu.

The annual number of Americans dying from the first is negligibly small (even with 9/11), the next 42,000, and then 36,000 (and potentially much higher in massive flu pandemic). We could, however, make rational decisions about protecting ourselves from each taken separately. I can decide how to, for example, lower the probability of death by
flu with simple hygiene. I lower the likelihood of death from massive pandemics with emergency supplies to ‘hole up’ for many weeks while it passes. Similarly, I recommend thinking about project risk mitigation in categories.

Here are the four risk categories:

Risk of unmanaged Social Contexts: This risk is (until the work in How NASA Builds Teams) grossly underestimated. It is, for example, at the root cause of all the major space disasters.
Risk of sloppy Process Practices and lack of core knowledge: This can be tricky because there is strong connection between process rigor and project cost. More process rigor means more manpower costs, and people are expensive. Process processes must be carefully tailored if you are to be cost competitive. Core knowledge in this model is the actual underpinning of process. Team members have to understand, for example, systems engineering processes before they can practice them.
Technical Risk is more straightforward. Technical project teams are usually staffed by highly trained technical people. Moreover, because this is their comfort zone, team members tend to enjoy technical analysis and retreat into it under stress.
Programmatic Risk, usually manifested as schedule slips and overruns generally has process failures as root cause. Technology might not have been sufficiently mature. Study phases before commitment to development may not have been adequate to produce realistic (grass roots) cost estimates.

I believe that competent project management team members can prioritize priorities within each risk group.

The Norm in Human Communications is ‘Not’

Unfortunately, the various risks and causes are highly interrelated. Technical difficulties cause programmatic difficulties. Inadequate resources demoralize teams causing low-performance social contexts. Process shortfalls fail to provide sufficient advance notice
for effective mitigation actions.

The only resolution to this dilemma is open and effective communications throughout the team. This, in turn, requires a high-performance team social context. Thus, Team Social Context is the most important and overarching risk that must be measured and mitigated. How NASA Builds Teams describes our Team Social Context risk mitigation process.

Team Social Context Risk Mitigation:

The NASA organization that funds our work (APPEL) was formed after Challengers’ explosion to prevent such accidents in the future. How NASA Builds Teams lays out the argument in detail that space mission losses are overwhelmingly caused by social context
shortfalls. These claims are supported by extensive evidence from the failure review boards for Challenger, Hubble, Columbia and others. Further we correlate the behaviors in these and other less consequential failures with those of low-ranking teams in our Team Development.

The figure below shows the progress of approximately 40 (20 per cent) NASA teams who measured in the perilous bottom quintile in their first team assessment. The grey diamond is where these teams started with their first Team Development Assessment.

These teams, on average, began in the toxic, disaster-prone bottom quintile (driven by unaddressed management / structural shortfalls). The successive triangles moving to the right are their second, third and fourth reassessments indicate sustained improvement in team social context and project-wide risk mitigation. Furthermore, these teams
developed in relative isolation which is less effective than the integrated and comprehensive approach described in ‘C-Gram – Managing Multiple Sub-Team TDAs’
also on this web site.

We now display the results in the risk mitigation matrix. Because each assessment and reassessment returns a single value, we chart the condition along the diagonal of the display.

The point at which mitigation is sufficient is, of course, up to the discretion of the team leader and their management. Clearly these teams, on average, are making excellent progress. They can continue and sustain their improvement by recurrent self-managed (full) Team Development Assessments at very low cost. Read ‘C-Gram Estimating 4-D ROI’ to see that these ROIs are in thousands of percentages.

Project Practices Risk Mitigation:

Frankly, when we began to study project teams with assessment data, I thought that NASA teams would demonstrate process excellence. We built a Project Practices assessment tool by entering process measurement language into our existing assessment software system. At the time, I saw Social Context and Process Practices as complementary essential elements. I imagined them as opposing cylinder banks of a metaphorical engine driving teams forward. I thought it was clever to place one process practice in the Social Context assessment tool and vice-versa with the Process Practices to cross-link the tools with each other.

So we added Roles, Accountability and Authority (RAAs) to the Social Context assessment. The idea was if the team’s RAA score was low, all the processes should be assessed. I was astounded to see how frequently the RAA score was in the dumpster for
large-scale NASA flight projects. We help teams enhance this score working as consultants, whereas our coaches have the lead in developing the other seven behaviors.

The Project Processes assessments measures project team utilization of the following processes:

1. ‘Cost Management‘ – Forecasting cost to complete

2. ‘Integrated Scheduling‘ - Critical path analysis

3. ‘Requirements Management‘ - Controlling scope

4. ‘Risk Management‘ – Classifying & proactively acting

5. ‘Systems Engineering‘ - Meeting technical requirements

6. ‘Acquisition Management‘ - Right contracts & incentives

7. ‘Safety & Mission Assurance‘ – Second set of eyes

8. ‘Gate Products‘ – The essence of project management

This assessment tool has not been used much, and I am not sure why.

Technical Risk Mitigation:

I have watched technical teams use the matrix to analyze technical risk many times. It is straightforward and completely doable.

Programmatic Risk Mitigation:

The matrix, in my experience, is too infrequently used for programmatic mitigation because of communication difficulties across the budgeter-engineer boundary.

Appendix – Project Practices Assessment Detail

Welcome Page:

This is an opportunity for you to provide feedback about your team’s use of eight core project management processes. The processes are: Cost Management; Integrated Scheduling; Requirements Management; System Engineering Processes; Risk Management Processes; Safety / Mission Assurance; Acquisition Management Processes; and Gate Products

We describe each process including a standard for ‘excellent’ utilization. You assess KeyName team’s practices against this standard by clicking on the most appropriate ‘radio button.’ Then, you can add explanatory comments if you want.

The last screen summarizes all your responses in an ‘integrated’ display. At this point, you can conclude the assessment or revisit any of your responses. Your scores and comments are anonymous. The system uses your e-mail address for the sole purpose of tracking
participation.

1. ‘Cost Management’

Estimating costs with ‘top-down’ (parametric) and ‘bottom-up processes’ depending on project maturity.
Placing budget ‘Baselines’ under appropriate configuration control.
Tracking and forecasting costs with systems appropriate to the project size / complexity.
Mitigating budget threats early as part of the risk- management process.
Maintaining appropriate levels of reserves for each program phase.
Understanding the sponsor’s sensitivity about cost growth and making the appropriate responses.
Implementing ‘Earned Value’ systems as appropriate.

Our KeyName team’s practices regarding Cost Management are:

Excellent: We fully meet the standard stated above.
Good: We usually meet the standard stated above.
Unsatisfactory: We too often fail to meet the standard.
Broken: We consistently fail to meet the standard.
Doesn’t apply, or I don’t know.

(THESE REPEAT FOR EACH FACTOR)

2. ‘Integrated Scheduling’