Problem Solving Workshop

(1)

Problem Solving Workshop

rar

d

Paul Gerrard

[email protected] gerrardconsulting.com

Helping clients transform their testing through

INNOVATION, COACHING and LEADERSHIP Our CLIENTS

– Want to be agile rather than follow Agile dogma

– Have a pragmatic approach and are focused on delivery

(2)

• Introduction

• A New Model for Testing

• Critical Thinking

• Systems Thinking (a tiny intro)

• Design of Experiments

• Debugging Rules (courtesy of David J Agans)

• Some Practicalities and Other Sundry Topics

• Close

Slide 3

Creating a Problem –Solving

Workshop

• More than one client asked for a Problem

Solving Workshop

• There is very little on the market that focus

on testing and debugging software and systems

• This class is a compilation of things I have

found useful and my own experience

• There’s a lot of material – I’ll skip slides

• It’s a work in progress – let’s keep it informal.

(3)

Before we start

• Introductions

• Today should be conversational, rather than a

lecture

• I will tell stories from my experience, but it

would help a lot if you share some of yours

• It will help others to relate what I say to what

you do.

Slide 5

What do you want to

get from today’s

(4)

Testing

• What’s Happening?

• A New Model for Testing

(5)

A New Model for

Testing

The old ways won't

work in the future

We need a New Model of Testing (free from logistics)

(6)

Forget Logistics

(for the time being) Document or not? Automated or manual?

Agile v waterfall?

This business or that business? This technology v that technology?

ALL Testing is

Exploratory

We explore sources of knowledge ...

... to build test models ...

(7)

Models Pub Quiz

How many models can YOU name?

rar

d

Paul Gerrard

[email protected]

gerrardconsulting.com

(8)

Testing (the system)

Our model(s) are adequate

Our model(s) are not adequate

Exploring (sources) Judgement Creates test models Uses test models

BTW – Do Developers explore the same way? I think so.

Slide 27

Exploration process

Exploration Definitions specs/stories People (& you) Sources Require-ments Test Models Enquiring Challenging Sources: People, documents, experience, system under test

Modelling Test Models: Can be documented or mental models Predicting System under test Slide 28

(9)

Testing process

Testing _{Under Test}System

Refining Informing Applying Interpreting Test Models Revise the System

More exploring Reporting

Slide 29

New Model Testing

90 Minute talk: https://www.youtube.com/watch?v=G7xAhPACyso(go to 23m 05s) 29 page paper: http://dev.sp.qa/download/newModel

(10)

One Consequence

There are others

Capabilities

Enquiring, Modelling, Predicting, Challenging Informing, Applying, Interpreting, Refining

(11)

• Analysis, enquiry and elicitation • Modelling

• Creation of custom models, using heuristics, guesses, brainstorming, ideation, creative thinking • Custom test design techniques

• Comparison of models, value, advantages, disadvantages, compromises

• Identification, validation and use of oracles

• Predicate logic and proof • Hypothesis and inference • Socratic method

• Rapid Review and Inspection techniques • Test case design

• Test models and the meaning of coverage • Testing as controlled experiment • Observation, Note taking, recording

A very different skillset

• Basic data analysis and statistics • Decision-making with incomplete data • Computer forensics

• Fault tree analysis • Failure diagnosis

• Bug advocacy, triage processes and negotiation

• Meaningful software and test metrics • Visual presentation of data

• Reporting and presentation skills • Understanding stakeholders • Test analytics

• Risk management, risk-based testing and decision-making

• Critical Thinking • Interpersonal skills

• Dealing with uncertainty/fallibility

Slide 33

Testing Career Development

(speculative)

Foundations Technical Management Strategic

Test Strategy Project Intelligence

Test Assurance

Exploration Forensics Interpretation Scripting/

Programming

Test Automation Technical (Excel, SQL, OS utils etc) Stakeholder management Analytics & visualisation Managing uncertainty Critical Thinking ISTQB, TMMi etc...

Supplier Management

Test Process Management

(12)

Critical Thinking

• How are we deceived?

• What is critical thinking?

• Working in Teams (I’m critical)

• Evidence, reasons and conclusions

• Credibility of sources

• Distinguishing causes and effects

• Systems thinking

(13)

test

We can be deceived by most of

what we deal with

People • Users • Stakeholders • Project Managers • Developers • Ourselves Artefacts • Project Plans • Requirements • Designs Infrastructure • Environment • Test Tools • Test Data Software • Under Test • Not Under Test

Slide 37

How we can be deceived - 1

• By People

– They may be uninformed and/or may not be able to

express themselves well

– We cannot understand them easily

– They have agendas, prejudices and vested interests

– They push for unauthorised requirements/changes

– Our own prejudices and shortcomings get in the way

• By artefacts (requirements, designs, project plans)

– Gaps in content

– Ambiguities, contradictions, defects

– Unsupported assertions

(14)

– Environments may not be correct, stable, dedicated

– Tools may not work or not work the way we expect and

mislead us

– Test data may not be appropriate, complete, controlled

• Software

– Software under test may reveal its secrets slowly

– Software not under test may hide or influence behaviour

– Software does not obey the laws of physics (or any reliable

law for that matter).

Slide 39

What is Critical Thinking?

• Critical thinking is the ability to think clearly

and rationally, understanding the logical connection between ideas

• Critical thinking has been the subject of much

debate and thought since the time of early Greek philosophers such as Plato and

Socrates and has continued to be a subject of discussion into the modern age.

Slide 40 https://youtu.be/l9SqQNgDrggPhilospher’s song

(15)

Critical thinkers

• Rigorously question ideas and assumptions

rather than accepting them at face value

• Challenge ideas, arguments and findings and

whether they represent the entire picture

• Are open to finding that they do not

• Identify, analyse and solve problems

systematically rather than by intuition or instinct.

Slide 41

We need to be critical

• Testers are responsible for collecting, analysing and disseminating intelligence on software and systems

• We are reliant on people, artefacts, infrastructure and software and our ‘genes’ tell us to be

distrustful

• We need to be critical of the fallibility of:

– Our sources of information and knowledge – Our processes, approaches and tools – Our project colleagues and stakeholders – Our own prejudices, biases, blind spots …

(16)

Working in Teams

I am a sceptic when it comes to Team-Working and Problem-Solving

training courses

Working in teams?

• Surviving in an Open Office Space

• What Do I Do When I Hate My Colleague?

• Are You Resisting Change?

• Good People Do Bad Things in the Afternoon

• Ten Back Pocket Phrases to Disarm Conflict

• The Value of Saying I'm Sorry

• Are You Being Defensive?

• How to Say Something Nice

• Is Your Vulnerability Making Others Uncomfortable?

• Don't Let Your Voice Be Silenced by Your Boss

• Quiz: Are You Making These Promotion Killing

Mistakes?

• Is Gossip Your Guilty Pleasure?

• More Guilt, Less Shame

• Insulting Your Colleagues Without Saying a Word?

• Are You the Victim on Your Team?

• 3 Resolutions if You Want a Promotion This Year

• Will You Be Turned On Over Christmas?

• 8 Tips to Survive Your Office Holiday Party

• To Give or Not to Give? Unwrapping Issues of Gifts

at Work

• A Magic Trick to Increase Your Credibility

• Letter to Those Who Don't Put Their Mug in the

Dishwasher

• Happy No-vember: 10 Things to Say "No" to This

Month

• Can I Be Happy at Work if I Don't Like My

Teammates?

• Only You Can Turn a Mountain into a Mole Hill

• 4 Tips to Overcome Your Conflict Avoidance Issue

• Spread Too Thin: 4 Secrets of Reducing Your

Workload

• Are You Drowning Out Minority Voices?

• Overworked and Undercontributing?

• Time to Unpack Your Emotional Baggage

• 8 Bullies Who are Really Weaklings

Slide 45

https://www.psychologytoday.com/blog/making-your-team-work

• You don’t choose your colleagues • You can’t select their skills • No choice over their personality

• I’m a great fan of great teams. But I’m sceptical of team-loving, team-building, trainers (who probably never worked in a team)

(17)

Slide 46

Consulting Assignment

Read the initial findings of a site visit to a company having problems in

testing

(18)

Evidence, reasons and

conclusions

Evidence + Reasons = Conclusions

Start with Conclusions

• Are there reasons that support them?

• What evidence or examples are offered to

support the reasons and assertions?

(19)

Reasons

• A reason is a statement made that aims to persuade the reader to accept a conclusion

– Is the reason an assertion of fact or opinion? – Is the reason relevant?

– Does the reason support (or make a difference to) the conclusion?

– Would other evidence (not available now) make a difference to the conclusion?

– Are there undocumented assumptions being made in the reasoning?

– Are these assumptions reasonable?

Slide 50

Evidence

• We assemble evidence and examples from our

sources and question them:

– Is our source reliable, expert, trustworthy?

– How was the evidence gathered?

– Who gathered the evidence?

– Who paid for the research?

– Is the evidence meaningful?

– Is the evidence relevant to our problem?

– Is it trustworthy?

– It is out of date?

– Is the sample size representative, large enough?

(20)

Slide 52

Pay attention to

• Small words: some, all, only, always, never

• Comparisons:

–More, less, better, worse, faster, slower, cheaper… –Ask “compared to what?”

• Timing: “when?”

• Who: “is this person motivated to lie or tell

the truth?”

(21)

(22)

• The person making the argument is human, has a different perspective, biased, uninformed

• It could be you or your team

• People don’t need to be politicians to be

unreliable witnesses, claimants, salesman

• We have to look beyond the superficial

evidence and look at perspectives and motivations too.

Slide 56

Analyse this 1

• I usually buy my computers on eBay

• There’s an old, spare computer in my study that is a file server; it backs up our data

• It keeps crashing without warning

• When I power it up, it restarts OK but then after a time it fails again

• We haven’t lost any data yet, but I’m afraid we will do eventually

• My wife says we need to buy a new server.

(23)

Analyse this 2

• Mark each statement on the previous slide as

evidence, reasons and conclusions

• Do the reasons support my conclusion(s)?

• Any assumptions?

• What other questions might you ask?

• Would you buy another computer?

• If not, what would you do next?

Slide 58

Distinguishing causes and effects

• It is claimed by some people that severe illness

is caused by depression and anger

• After all, people who are severely ill are very

often depressed and angry

• Thus, it follows that the cause of severe illness

actually is the depression and anger

• So, a good and cheerful attitude is key to

(24)

Systems Thinking

1 minute intro

• Systems Thinking (ST) is an approach to

understanding complex systems and focuses on problem solving

• Model building is fundamental – to simplify the

system and inform problem solving

• Scoping models and causal loop models are of

some relevance to us here.

(25)

Soft Systems Methodology

Slide 62

A system of systems

Web site

Mobile App ERP systems

Accounting CRM

(26)

Share common functionality A single user journey Systems of Record

Causal loop diagrams

(27)

Design of Experiments

skip to slide 72 Slide 66

Design of Experiments

• Purpose of experiments • Errors in experiments

• What you can and can’t control

• Single factor experiments

(28)

• An experiment asks a question with a goal

–To demonstrate something happens when it should happen

–To demonstrate something doesn’t happen when it should not

• Sounds like testing doesn’t it?

• I use the term demonstrate rather than prove.

Slide 68

Correlation and causation

• Correlation (or non-correlation)

–“Does the response time increase with the size of the file submitted?”

–“With the same file size, do we get varying response times?”

• Causation (or non-causation)

–“Does the failure always occur when I connect a new device?”

–“When I add a new device, does it sometimes work?”

(29)

Errors in Experiments

Slide 70

• Type 1 Error: Saying something happened

when it really didn’t (a false alarm)

• Type 2 Error: Not discovering that something

really happened (a failure to alarm)

• Type 3 Error? Failing to ask the right question.

Decision-Making Outcomes What you say based on experiment

Yes No

The Truth Yes Correct T2 Error

No T1 Error Correct

X Factors

System

Controllable (X) factors

Uncontrollable Variables (Z)

Response Measures (Y) Distinguish between what you can control

in production and what you can control

(30)

Slide 72

System Under Test Program State including

relevant data System state Intended inputs Configuration and system resources From other cooperating processes, clients or servers

Program state including uninspected outputs

System State Monitored outputs Impacts on connected devices and system resources To other cooperating

processes, clients or servers Doug Hoffman

Is what I can’t control the cause of

the problem?

• Load on a client device

• Number of concurrent users

• New or existing user account

• Failure to complete a transaction

• Size of file, length of session, age of customer, experience of user

• What are you not in control of that causes problems?

(31)

Single factor experiments

• Focus on one thing at a time, keeping all other

factors constant

• Each experiment changes one thing

• If a problem occurs you know where to look

• Run a series of experiments to compare

–Tuning parameters for performance

–Configuration setting that causes/fixes a problem

• Easy to go ‘back one step’.

Slide 74

2-Factor experiments

• To simplify, here we have two factors Lo/Hi or on/off setting

• Of course factors might have many settings

– E.g. OS/Browser/firmware version combinations – A continuous range e.g. file size, volume, maximum,

typical or minimum allowable values • Choose extreme values, but inside the

‘operational envelope’

• Randomize the run order or re-set to eliminate time-related issues (another factor?)

1 2 3 4 Factor A Fa ct o r B Lo Hi Hi

(32)

• For each of the pairs of factor-values cover it in at least one test

• Easy to do in principle, but hard to minimise

the number of tests without a tool

• So, use an All-Pairs program to generate test

combinations

–http://www.pairwise.org/tools.asp

–I’ve used Pict (free, Microsoft) with some success.

Slide 76

Debugging Rules

Draws on Debugging David J Agans

(33)

Debugging Rules

• A short book that summarises a set of rules

that will ‘help you to find bugs faster’

• The author, Agans says

–“When it took us a long time to find a bug, it was because we had neglected some essential,

fundamental rule; once we applied the rule, we quickly found the problem”

Slide 78

Aren’t these rules obvious?

• Most of them will either be obvious or what

you do instinctively

• But applying rules isn’t always easy

• Seeing instinctive rules laid out might help you

to remember them and how they fit into a debugging strategy

(34)

• Software and systems people need to find the cause of a problem, where a design did not work as expected

• Car mechanics need to find the cause of a

problem where a part fails or wears out (and the design is assumed to work OK)

• Software people debug, car mechanics

troubleshoot (although most debugging rules apply to both).

Slide 80

9 Debugging Rules

• Understand the system

• Make it fail

• Quit thinking and look

• Divide and conquer

• Change one thing at a time

• Keep an audit trail

• Check the plug

• Get a fresh view

• If you didn’t fix it, it isn’t fixed

(35)

Understand the System

Understand the system

• This could be as simply put as ‘Read the

******* Manual!’ or RTFM

–There’s even a website of that name

• What you never want to happen is to learn

(from your boss or colleague) that you wasted days finding a solution that was on page 2 of the manual

• More broadly, the manual is a ‘source of

(36)

• Sources of knowledge are sometimes plentiful, but fallible and time consuming to gather

• We often to want to ‘make progress’ before

we have gathered the knowledge available

–We don’t consult domain experts –We don’t ask the boss or colleagues –We trust our limited knowledge

–We are fooled by our prejudices and biases.

Slide 84

Summary

• RTFM – the answer is in there 80% of the time

• Consult domain experts, colleagues etc.

• Know what’s reasonable

• Know the roadmap, how things interact

• Know your tools

• Look it up, don’t rely on memory.

(37)

“I found it by accident. It was

in the manual”

Slide 86

Make it fail

“If you can’t reproduce it, you can’t fix it”

(38)

• So you can look at it

– Bugs have a nasty habit of occurring when it is inconvenient to see what’s going on

• So you can focus on the cause

– The failure is the start of your investigation (not an endpoint)

• So you can eliminate possible causes one by one

• So you can tell if you’ve fixed it

– You need a trusted reproduction of the failure before you can be confident any fix has ‘worked’.

Slide 88

Intermittent failures

• Intermittent failures are harder to reproduce

• Problem is, you don’t know how to reproduce

the failure so it’s hard to start your analysis

• Need to seek out a possible cause that has

the same intermittent pattern as the failure.

(39)

Intermittent failures 2

• Intermittent bugs tend to have the most

obscure causes

• You often have to guess the causes and

experiment to find the one responsible

• When you reproduce the bug, you aren’t

aware of all of the conditions

• There may be many causes you can’t control

–Initial conditions, inputs, timing, external processes, noise, temperature, vibration, network traffic…

Slide 90

My car keeps overheating

• In December1988 I bought a new Lancia Delta LX

• My one and only ‘new car buy’

• All worked well until May/June

• On some trips the water gauge suddenly flipped from normal To HOT!

• Usually had to pull over, to cool the engine.

(40)

Slide 92

• What could cause the temperature problem?

• How would you test for each cause?

Remember

• Do it again

• Start at the beginning

• Stimulate the failure; don’t ‘simulate’ it

• Find the uncontrolled condition that makes it intermittent

• Log everything, find the ‘signature’ of the bug

• Logs might tell you what happens when, but might not indicate the source

• Know that “that” can’t happen

• Maintain your debugging tools – they might help you.

(41)

Quit thinking and look

“You can think of thousands of reasons for a failure.

You can see only the actual cause.”

Slide 94

Thinking is good, but no substitute

for looking

• When you can reproduce a failure, it’s

tempting to ‘think’ your way to a solution

• Of all the many ways something could go

wrong – do you really think you found the true cause?

• Don’t jump to conclusions, the ‘fix’ you apply

(42)

• See the failure, not what you think is a failure, or a simulation

• Follow through – see the details not just the

superficial evidence

• Use instrumentation and any tools to capture

or log the evidence you need

• Use guessing to focus the search; don’t go

fixing guesses before you know the bug.

Slide 96

My balcony roof

• In our last house we have a small balcony –

cosmetic rather than actually useful

• The flat roof seemed to leak and the water

stained the wallpaper in our living room

• I spent a day laying a new asphalt roof and

drain hole

• Next time it rained, our living room wall was

damp again.

(43)

Divide and Conquer

“It’s hard for a bug to hide when its hiding place keeps getting cut in half.”

Slide 98

Narrow the search

• When you reproduce the problem, there could be many causes

• This is the only rule that actually helps you find the problem

• All other rules help you to follow this one

• I think of a number between 1 and 100

• You guess the number; I say too high or too low

• What is the maximum number of guesses required?

(44)

• We make choices of experiments that eliminate half the possibilities

• If you try and eliminate 99% and fail – you still

have 99% to eliminate

• Having run your test, which side are you on?

• How do you halve the possibilities now?

• What is a half anyway?

Slide 100

What is ‘half’ anyway?

• Start with the bad

• Don’t trace from the beginning to

demonstrate good data gets treated properly

• Start where the problem emerges and work

from there.

• How does it work in your environment?

(45)

What does this app do?

• Here is a very simple app

• Your task is to find out what it does

• Can’t be that difficult can it?

• http://dev.sp.qa/workshop - example 1

• Think of questions that

• “halve” the possibilities

• You have 20 guesses.

Slide 102

Summary

• Narrow the search with successive

approximation

• Get the range:1 to 100 or…?

• Determine which side of the bug you are on

• Use easy-to-spot test patterns

• Start with the bad

• Fix the bugs you know about

(46)

Change One Thing at a

Time

Slide 104

Use a rifle not a shotgun

• Suppose you change three things at the same

time and the problem disappears

–Which of the three changes improves matters? –Is it two in combination? (Which two)

–Or is it all three and you got lucky?

• If your change doesn’t have the desired effect

– back it out right away

• Changing one thing at a time leaves you in

control.

(47)

Grab the brass bar with both

hands

• Agans tells the story of controllers in US

nuclear power plants

• When something goes wrong, they are told to

grab a brass bar in front of all the dials until they’ve checked every dial and indicator

• Be careful of “trying this”, “trying that” – it

might not help, confuse the system and you.

Slide 106

Compare with one that worked

• You should have a trusted test that exposes

the failure

• When you get a test that ‘works’, you have a

control that you can compare against

–Are there differences in the logs?

–Are there differences in the inputs or starting states?

(48)

• Often, the difference between a working system and system that fails is a single change

–The fix is easy – roll back the change

• When testing, you might find the test that

failed now passes or another failure emerges

• It could be your test or anything related to

the environment that changed

–With or without your knowledge perhaps?

Slide 108

What does this app do?

• Here is a very simple app

• Your task is to find out what it does

• Can’t be that difficult can it?

• http://dev.sp.qa/workshop - example 2

• Think of tests that change

one thing at a time.

(49)

Summary

• Isolate the key factor

• Grab the brass bar with both hands

• Change one test at a time

• Compare with one that worked

• Determine what you changed since the last

time it worked.

Slide 110

Testing Air Traffic Controller

Rostering

• I was asked to suggest a test strategy for a system to manage Air Traffic Controller rostering

• The problem was creating expected results – it took a full day to produce one roster

• I suggested create one trusted test case manually • Change one input, trace the change through the

algorithm and predict a new expected result.

“A system produces shift rosters for air traffic controllers the day before the shift. For every shift, radar positions must be filled by staff who performed the same role no more than ninety days previously. On the roster-input screen, when a controller is entered for a radar position, the system must check the ‘role last done’ is within the acceptable range.

(50)

Keep an Audit Trail

“The shortest pencil is longer than the longest memory”

Slide 112

Note taking is a critical skill

• Your notes are an audit trail of what happens

• But it’s not just that – it’s a trace of how you

are thinking too

• Sometimes it’s the most insignificant thing that

is the key

• But you can’t record everything.

(51)

What you need from your log

• How you set up the system under test

• What you did and when

• What happened, what you saw and when

• But logs aren’t all manually kept

–Source control systems

–System logs (on different machines)

–Have you thought about recording sessions (e.g. camcorder, Snipping tool, Snagit or Camtasia)? –Mobile phones for pictures or videos.

Slide 114

Summary

• Write down what you did, in what order, and

what happened

• Understand that any detail could be the

important one

• Timestamp and correlate events

• Instrumentation and logging captures a vast

amount of value

(52)

Check the Plug

“Assume makes an ass of you and me”

Slide 116

Question your assumptions

• What is obvious sometimes doesn’t even

register in our minds

• We are so focused on the difficult problem,

that the obvious stuff gets forgotten quickly

• Subconscious assumptions are a killer

• You might have to think very carefully to

remember what your assumptions actually are.

(53)

Are where you think you are?

• Your test fails

–Are you absolutely sure you performed the test set up accurately?

–(Testing without the indexes applied to the database)

• Your test fails

–Is it the system, your test, the test tool or your monitoring that is failing?

Slide 118

Hitch-hiking in France

• A friend and I spent a few days in Paris but

missed our coach to the coast

• We had little money and had to hitch-hike

• Our plan was to get the metro to the start of

the autoroute and stand at the toll booth

• “Allez-vous au Calais?”

(54)

• Question your assumptions

• Start at the beginning

• Test the tool.

Slide 120

Get a Fresh View

(55)

Ever had this experience?

• You’re stuck on some really tough problem

and decide, after all else has failed, to ask the advice of a colleague

• As you explain your problem, suddenly you

realise what you’ve been doing wrong

• “Thanks – you’ve been really helpful!”

• “I didn’t do anything”.

Slide 122

Get a fresh view

• Ask for fresh insights

• Tap expertise; listen to the voice of

experience

• Know that help is all around

• Don’t be proud

• Report symptoms, not theories

(56)

If You Didn’t Fix it, it

isn’t Fixed

Slide 124

Summary

• Check that it’s really fixed

• Check that it’s your fix that fixed it

• Know that it never just goes away by itself

• Fix the cause

• Fix the process.

(57)

Some practicalities and

Dealing with bug

reports from the

support team

What is your experience?

(59)

First line support

• Support desks are remote from users and

have all the usual communication issues

• The user is either:

–Naïve: and finds it hard to follow instructions –“Expert”: has tried their own botched solutions

and lost all evidence of the original problem

• Can only troubleshoot, not debug – may only

have a proxy operating the live system

• Some of the debugging rules still apply though.

Slide 130

Dealing with support requests

• Likely to be under-evidenced

– Single example of failure and no deeper analysis

– Little or no knowledge of user’s context, environment, technology, bad habits

– You might get some half-baked hunches though • You might not be able to reproduce initially

• Do you expect a support request to give you a great start?

• Encourage your support team to be as smart at debugging as you.

(60)

When you’re in a

hole…

…stop digging

A typical scenario

• You have a production failure to look at

• Looked straightforward – you just need to

– Trim down the data to speed up analysis, to home in on what caused the program crash

• Several days of data editing later, the failure disappears and you can’t reproduce it

• But:

– You don’t have a backup of the data – Your environment is all over the place – Your notes are sketchy and unhelpful – You just ran out of ideas.

(61)

When you’re in a hole, stop digging

• How do you know you are in a hole?

• Make a list of symptoms that would make you

think you have reached a dead-end

• Could these be used as a set of criteria for

spotting problems early?

Slide 134

(62)

• It is good to be confident in your ability but…

• Complacency: makes you take short cuts – that

don’t’ work

• Insecurity: makes you fearful of telling the boss what

you have (or haven’t) achieved

• Stubbornness: makes you unwilling to ‘start again’

• Inattentional blindness: blind to things you aren’t

looking for

• Cognitive dissonance: blind to things you know

“can’t happen” because you convinced yourself they can (or vice versa)

Slide 136

Some short videos

• https://youtube.com/watch?v=z-Dg-06nrnc • https://www.youtube.com/watch?v=ubNF9QN EQLA • https://www.youtube.com/watch?v=SGjuJwc2g s4 • https://youtu.be/9Y17YaZRRvY Slide 137

(63)

Cognitive dissonance

Slide 138

All I have to do is work hard, the result will come

I am making no progress

I can’t stop now, I am nearly there Belief Reality Says publicly

Idea Generation

• Reframing questions • Attribute listing • SCAMPER • Check lists • Analogies

• Most useless idea • Mental excursion • Greenfield site

• Assumption Reversal

• There are lots of sources for ‘idea generation’

• These come from the BCS book, “The Human Touch”

• You might know Brainstorming already, but it’s been shown to be not as effective as popularly believed.

(64)

• Foundations of Software Testing, Kaner, Fiedler

• http://www.softwarequalitymethods.com/both.php, Doug Hoffman’s writings on Oracles

• Human Touch, Personal Skills for Professional Success, BCS, 2012

• DOE Simplified, Anderson and Whitcomb

• https://www.moresteam.com/toolbox/design-of-experiments.cfm

• How to Break Software, James Whittaker

• The Laws of Software Process, Philip G Armour

• Systematic Fault Diagnosis, Engineering Equipment Users Association, 1980

• Why we Make Mistakes, Joseph T. Hallinan

Slide 140

Critical Thinking sources

• OCR AS level Critical Thinking

• Straight and Crooked Thinking, Thouless, Penguin

• Critical Reasoning, Anne Thompson, Routledge

• Critical Thinking, Bowell and Kemp, Routledge

• Critical Thinking Skills, Cottrell, Palgrave

• A Rulebook for Arguments, Weston, Hackett

• Lines of Thought, Young, Oxford

•

https://www.skillsyouneed.com/learn/critical-thinking.html

(65)