Risk management practices and
tools: A pilot study of Australian
software development projects
.
Bee Bee Chua
University of Technology, Sydney Australia
June M. Verner
Empirical Software Engineering National ICT Australia
Sydney
Topic Outlines
{ Literature review of requirements problems and
software practices (and) tools used in software development projects.
{ An understanding of the research investigation.
{ An introduction of the research methodology.
{ An overview of the data results.
Reasons for software development
project failures
Literature Review :
{ Standish Group [1]
One third of projects were never completed and one half were challenged, i.e. were delivered with only partial functionality, had major cost overruns and significant delays.
Problems were due to :
Lack of user involvement (13%) Incomplete requirements (12%). Changing requirements (12%). Unrealistic expectations (6%) Unclear objectives (5%)
Reasons for software development
failures
{ European Software Institute [2] conducted a similar survey
to investigate software practitioners’ perceptions of software problems:
areas of requirements specification (more than 50%)
requirements management (50%)
{ Other problems include incomplete requirements [3],
ambiguous requirements, unclear requirements.
{ An empirical study [4] on requirements volatility (RV) is a
cause to project failure that significantly impacted on project cost and time during software development.
Other type of problems on project
failures
{ Other type of problems leading to project failures include
organizational complexity technical complexity
people relationship complexity requirements complexity.
{ Large numbers of software metrics and risk tools have been
suggested in literature reviews however these metrics were specifically introduced to help software development team to identify and measure goals associated with risks with
software processes and software products
{ However, these metrics do not sufficiently address risks
The Objectives
{ To examine what risk management practices and
tools are used in Australia within the context of requirements risk management for software development projects.
{ To discover what tools and methods are used for
software developed for both internal and external customers.
{ To investigate how useful practitioners consider
Research method
{ Questionnaire was developed for our pilot study
based on a broad study of the literature.
{ Our questionnaire is based on the literature that
addresses risks related to requirements changes and their impact on software development
projects.
{ A survey was chosen because of its simplicity and
because we wanted to find the frequencies of tool and technique usage as well as identify
Quantitative research
{ Two types of method use:
{ Postal Survey
z 9 out of 25 respondents answered the survey
that was posted to the local software companies.
{ Email Survey
z 7 out of 17 respondents replied email survey
{ 99% of our respondents had been involved in the development of more than one software development project :
Electronic Commerce Mobile Commerce Banking Financial Healthcare Shipping Telecommunication Information Technology Information Systems Engineering Education
Import and Export Transportation
Property and Construction Airlines
Our goal is to investigate the types of risk tools and risk management practices used by the
respondents in Australia for the following type of customers:
{ 1. Internal Customers.
{ 2. External Customers.
Three sections in the survey were chosen for our analysis.
{ Section 1: Respondent Software Development Experience.
{ Section 2: Organization Details.
Section 1- Respondent software
development experience
{ Has your organization developed software development
projects?
{ Were you involved in the development?
{ Indicate the types of software development project areas
Section 2- Organization Details
z What type of industry does your organization
belong to?
z Indicate your position in the organization.
{ Does your organization use a risk assessment model? 0 6.25 12.5 18.75 25 0 6.25 6.25 0 6.25 6.25 12.5 0 5 10 15 20 25 30
Yes No Not Sure Did not answer the
question
Int ernal Cust omers Ext ernal Cust omers Int ernal and Ext ernal cust omers
{Organizational size may be a factor and that risk assessment
may not be used commonly in small and medium
Section 3- Risk Management
{ What risk management strategies were used to help
control the number of requirement changes?
4 12 8 8 4 0 16 12 12 8 20 0 8 4 4 0 0 4 0 5 10 15 20 25
Risk Analysis Risk Identification
Risk Assessment
Risk Evaluation Did not answer the question Others Internal Customers External Customers Internal and External customers
{Organizations adopted risk analysis, risk identification and riisk
assessment commonly used in external projects rather than internal projects.
{The result indicates that these strategies give external
customers a structured mechanism to provide visibility into threats to project success. for example: ambiguous wording use in government policies and guidelines
{ Did your customers accept requirement
changes based on risk analysis?
12.5 0 6.25 6.25 18.75 0 12.5 25 6.25 6.25 6.25 0 0 5 10 15 20 25 30
Yes No Not sure Did not answer the question
Internal Customers
External Customers
Internal and External customers
6. 25 0 12. 5 6. 25 18. 75 6. 25 12. 5 18. 75 0 6. 25 12. 5 0 0 2 4 6 8 10 12 14 16 18 20
S a t isf ie d wit h t o o ls No t sa t isf ie d wit h t o o ls No t su r e Did n o t a n swe r t h e q u e st io n Internal Cus tomers External Cus tomers Internal and External cus tomers
Are you generally satisfied with the risk assessment tools that you use?
{ Does your role in the organization involve you in
conducting requirements changes risk assessment? 6.25 12.5 6.25 12.5 25 18.75 12.5 6.25 0 0 5 10 15 20 25 30
Yes No Did not answert the question
Internal Customers External Customers
Internal and external customers
{ What types of tools were used to control the risk
impact of requirement changes?
0 6.25 6.25 12.5 6.25 6.25 25 6.25 6.25 0 6.25 6.25 0 5 10 15 20 25 30
S imula t ion Tools P r e dic t ive t ools Not sur e of a ny t ools Did not a nswe r t he que st ion
Internal Customers
External Customers
Internal and External customers
{Simulation and predictive tools are not very useful for risk
Preliminary Results
• Organizational perspective
• The people in the organization who govern software
projects are at a senior level of management.
• Organization size may make a difference in the use of
risk management practices.
• IT practitioners’ perspective
• Risk Management practices are used for all projects no
matter what type of customer.
• Practitioners tended to be more satisfied with the tools
used if they have external customers and simulation are tools are more likely to be used for these external customers.
• Type of customer does not affect the use of predictive
Conclusion and Future Research
{ The pilot study is a first step in our empirical research.
{ In subsequent studies, we intend to investigate
{ what other tools are used for risk management,
{ are the tools used for anything other than an assessment of
the degree of risk,
{ are they really useful for risk management,
{ do they provide good value,
{ how useful are these tools and where are they used, and
{ what are their shortfalls ?
{ What type of risk management practices and tools used in US,
References
{ 1. Standish.1995. The Scope of Software Development Project
Failures; The Standish Group: Dennis MA; [verified 20th Dec 2004] http://www.standishgroup.com/choas.html
{ 2. European Software Institute. 1996. “European User Survey
Analysis.” Report USV_EUR 2.1, ESPITI Project.
{ 3. Krasner, H. 1989. “Requirements Dynamics in Large Software
Projects.” Proceedings of the 11th World Computer Congress (IFIP89), Amsterdam, the Netherlands.
{ 4. D. Zowghi and N. Nurmuliani, “A study of the Impact of
Requirement Volatility on Software Project Performance,” in Proceedings of the Ninth Asia-Pacific Software Engineering Conference (APSEC’ 02), 2002.
{ Questions and Answers
