Playing around with Risks

(1)

Playing around with

Risks

Jurgen Cleuren

April 19th 2012

Introduction

• Projects are done in a probabilistic environment – Incomplete information

– Parameters change over time

– What is true in the beginning of the project, can be false some time later

(2)

Which games ?

• Poker

• Monopoly

• Backgammon

Succes:

First learn the rules, then play better than everyone else

• Rules -> requirements

• Every game starts with learning and agreeing on the rules -> every project starts with defining and agreeing on the requirements

(3)

TEXAS HOLD’EM POKER

Rules of Texas Hold’em

• 2 Blind cards • Betting round

• 3 Open Community cards (Flop) • Betting round

• 1 Open Community card (Turn) • Betting Round

(4)

Hand Ranking

General test flow vs Poker round

• Create FTT/Risk Matrix • Software Delivery

• Get Blind cards – evaluate risk - Bet • Flop (3 cards)

Software Delivery • Full functional test • Bugfixing

• New Software Delivery • Retesting and regression • Bugfixing

Flop (3 cards)

• Re-evalute hand and Risk • Betting round

• Turn (1 card)

• Re-evaluate hand and Risk • Betting round

(5)

Similarities / Differences

3 Recurring phases

1 big phase (Complete Functional testing vs Flop) 2 small phases (retest + regression vs Turn + River)

Determine Risk before 1st_{test run (Risk Matrix or FTT vs 1}st _bet)

Adapt Risk Matrix or FTT after each test run

Should the FTT or Risk Matrix be adapted ?

• Every Test run gives new information • Likelihood of risks change

– Failed test cases get a higher likelihood

– Passed test cases in unchanged code have a lower likelihood • Closer to deadline => Risks can change

• Reporting is more clear

(6)

(7)

(8)

Who ?

• Test manager should take the lead

• Preferably Project Board

• Test manager can do it by himself, but the board should at least be aware that this activity is done

Justification

• A Poker hand needs justification at any point in the game

– Having the best hand in the beginning doesn’t imply that you are going to win

– You must be prepared to fold when you are not winning anymore – The money you’ve already bet doesn’t count

• It is in the pot so it is not your money anymore • Don’t put more money in a losing hand No justification anymore: get out of the hand – No justification anymore: get out of the hand – Cut your losses

(9)

Project justification

• A project needs justification at any point in time • During testing: justification after each test run

– New information is given – Risks are updated

• No Justification means project should be closed • Previous investments do not count

– Don’t put more money in a failing project • Cut your losses

Test process justification

• Get Blind cards – evaluate risk - Bet • Flop (3 cards)

• Create FTT/Risk Matrix • Software Delivery • Re-evalute hand and Risk

• Betting round • Turn (1 card)

• Re-evaluate hand and Risk • Betting round

• River (1 card)

• Full functional test • Justification • Bugfixing

• New Software Delivery • Retesting and regression • Justification

(10)

Result Oriented Thinking

• A decision can be right even if the outcome is not favourable • Focus more on the decision and the ‘why’ instead of the result

– Tester A completes a test set in 4 days by skipping tests

– Tester B completes the same test set in 6 days through thoroughly testing

– No defects were discovered in production – Who would you reward the most ?

Thought process

• In poker, to anticipate and understand your opponents actions, you need to think as your opponent and not as you in his situation

• What motivates you, does not necessarily motivate another person

• Successful people ask better questions

– WHY? is more important than HOW? or WHO?

(11)

Luck

“Luck is when preparation meets opportunity” - Seneca –

• Be prepared to get lucky

– In poker, sometimes you need to get lucky. When you get lucky, be sure to take a big pot out of it.

– Sometimes a best case scenario happens but we need to takeSometimes a best case scenario happens, but we need to take advantage of it.

– Being ahead of planning is more valuable if you can actually do something with this situation

(12)

Aspects of the game

• Investing in houses

• The higher the investment, the bigger the payoff • Some spaces have a higher probability • Random element: dice

(13)

Analogy to Risk Based Testing

• Different probability of landing on spaces = Likelihood • Higher Payoff = Impact

• What can monopoly teach us ?

– What is more important: Impact or Likelihood ?

(14)

Winning Monopoly strategies

• Complete Orange Colour group and invest as soon as possible

– Why Orange ? It has the biggest probability of other players landing on it

– Be prepared to even trade down to get this colour group

• Complete Red Colour group as a second priority

– Why Red ? It has the second biggest probability of other players landing on it

landing on it

What lessons are to be learned

• Likelihood >>>>>> impact

– The weight of Likelihood should be > 50% – The weight of Impact should be < 50%

(15)

BACKGAMMON

(16)

Important rules of Backgammon

• Goal: get all your tiles from one end to the other

• Only tiles that are standing alone on a pillar can be captured

• A captured tile has to be brought back in play at the beginning of the board

• Random element: Dice

Translation to risks

• Your own checkers: Risks • Opponents checkers: Causes

• Whenever one of your checkers is captured it’s in fact a cause hitting a risk

• A risk is mitigated when the checker cannot be captured (2 or more on one pillar)

(17)

Translation of risk priority

• Low risk: checker in the first quadrant

• Medium risk: checker in the second or third quadrant • High risk: checker in the fourth quadrant

What can Backgammon teach us?

• Which risks should be mitigated and which are low priority ?

• There might be a possibility to remove a cause, but in the same way creating a new risk. Should we do it ?

(18)

Backgammon Strategies

• Checkers in the 1st _{quadrant don’t have to be protected. Moving forward}

is the better play.

⇒Risks with low priority don’t have to be mitigated. The correcting cost is usually way less than the mitigation costs

• Checkers in the 4th _{quadrant need to be protected at all costs.}

⇒Risk with high priority need to be mitigated at all costs

• For checkers in the 2nd_{and 3}rd_{following rule applies: Always take a}

chance to capture p

⇒If you can remove a cause and therefore create a medium or low risk, do so

(19)

Conclusions

• What Poker taught us:

– Adapt FTT/Risk tree after each test run – Priorities of test items can change

– Justification has to be done after each test run – Don’t focus on results, focus on decisions – Be prepared to get lucky!

Conclusions

• Create FTT/Risk Matrix

• Create FTT/Risk Matrix • Software Delivery • Full functional test • Adapt FTT/Risk Matrix C ea e / s a

• Software Delivery • Full functional test • Bugfixing

• New Software Delivery • Retesting and regression • Bugfixing

• Final Software Version

dap / s a

• Justification • Bugfixing

• New Software Delivery • Retesting and regression • Adapt FTT/Risk Matrix • Justification

• Bugfixing

(20)

Conclusions

• Monopoly

– Likelihood >>>>> Impact

• Backgammon

– Don’t mitigate low priority risk – Always mitigate high priority risks

– Removing a cause and creating a medium or low risk is the way to play

Jurgen Cleuren ([email protected])