Diagnosing

The purpose of the diagnosing phase was to understand the current problems and practices within the organization that may need changing. This section describes the data collected between November 2004 and May 2005 to assess TelSoft‟s software practices from the viewpoint of

relevant stakeholders (see Table 2: Diagnosing Key Dates). At TelSoft, this effort involved 22 semi-structured interviews, two 3-hour workshops, a standardized assessment, and nearly a dozen meetings of the problem solving and research teams.

Table 2: Diagnosing Key Dates

Date Activity

November 29, 2004 First diagnosing interview of software development manager

January 19, 2005 Workshop: Software Development Problem Diagnosis [6, 7]

January 19, 2005 New Division President announced

March 16, 2005 SC meeting: Interim Status and first contact with new Division President

March 16, 2005 Workshop: Internal Customers Problem Diagnosis [8, 9]

March 30, 2005 REGPG Assessment completed [11]

May 25, 2005 Last diagnosing interview with external customer May 30, 2005 First draft of diagnostic report [11]

June 1, 2005 Intervention Cycle 1 begins: First PST meeting to plan improvement strategy

In thinking about the diagnosing plan, the PST valued the context-specific judgments of the

TelSoft‟s employees and customers as well as the general insights that could be provided by

standardized assessment methods. To accommodate the desire for both perception-based and process-based assessment, we developed an assessment framework that integrates the two approaches. Our combined approach to RE assessment consists of three steps: initiating the assessment, executing multiple inquiry cycles, and making recommendations based upon the findings (Napier, Mathiassen et al. 2006).

Figure 1: Combined RE Assessment Approach

The assessment was organized as one process-based and three perception-based inquiries. During this time, the PST met as needed (roughly once a month). At these meetings, the research team would present initial findings and describe any issues that arose during data collection. The

TelSoft members of the PST identified representatives to be interviewed in each of the stakeholder groups and facilitated creation of the group workshops.

For the process-based portion of the assessment, the research team selected the assessment from the book Requirements Engineering: A Good Practice Guide (REGPG) (Sommerville and Sawyer 1997). REGPG has been successfully used in both academia and industry. In addition, the research team had access to a REGPG assessment tool (Sommerville and Ransom 2005) that simplified data collection, provided process guidance, ensured accurate calculation of requirements maturity, and automated report generation. The REGPG assessment was conducted during a two hour meeting with members of the PST on March 30, 2005. Participants were provided a written report containing a description of each of the 66 practices and expected benefits to including the practice. Each relevant practice was read aloud and categorized as being standardized, normalized, discretionary, or never followed. For questions the group did not feel prepared to answer, they solicited response from appropriate people after the meeting. The REGPG assessment identified TelSoft‟s strengths as being in the areas of documenting, eliciting,

and describing requirements [10]. Areas for improvement were in analyzing, validating, and managing requirements. The company‟s overall RE maturity level was assessed at the lowest level: initial.

The perception-based portion of the assessment was designed based upon my prior knowledge of

TelSoft. We identified three stakeholder groups involved in RE: software development, internal customers, and external customers. The research team created interview guides [5] which asked objective and subjective data on requirements-related documentation and activities that were tailored for each stakeholder group. To ensure participant confidentiality, the research team took

responsibility for data collection and analysis, reporting results at an aggregate level. I was the primary interviewer joined by either Dr. Johnson or Dr. Mathiassen where possible.

The first perception-based inquiry cycle focused on the software development group at TelSoft; this group is responsible for interacting with clients to generate a software requirements specification, creating the GIS software based upon these software requirements, evaluating the impact of requirements changes, and ensuring the quality of the resulting software product. We interviewed nine representatives from the software development group (see Table 3: Summary of Diagnosing Interview Sources). The research team analyzed interviewees‟ responses for similar themes. This analysis produced two key documents: a summary of TelSoft‟s actual requirements

process and a list of seventeen potential problem areas. On January 19, 2005, all members of the software development group participated in a three-hour workshop to evaluate this list. For each problem area, workshop participants individually provided an assessment of criticality, feasibility, and priority. These individual responses were then debated and again prioritized in break-out sessions during the workshop. A plenary session was then held in which representatives from each of the break-out groups described their top issues. The primary outcome from this cycle was a prioritized list of problem areas as perceived by the software development group [7, 11].

Table 3: Summary of Diagnosing Interview Sources

Stakeholder Group Count Role

Software Development Group 9 2 Development Managers 2 Project Managers 2 Software Engineers 2 Systems Analysts

1 Quality Assurance Analyst Internal Customers 6 1 Liaison to Software Group

3 Project Managers 2 Sales Representatives External Customers 7 6 Managers

1 Engineer Total 22

The second inquiry cycle focused on the internal groups that interacted with the software development group in generating and managing software requirements. The software development group receives requirements from both the marketing organization and an internal production group that uses its GIS software. Once the interviews were completed, the research team again analyzed the interview data for common themes that suggested potential problem areas. On March 16, 2005, the PST sponsored a workshop for validating and prioritizing the 14 identified problem areas. Workshop participants included those interviewed as well as other users within the internal production group. The primary outcome from this cycle was a prioritized list of problem areas as perceived by the internal customers [9, 11].

In the final perception-based inquiry cycle, we interviewed external customers who interacted with TelSoft to generate software requirements, request requirements changes, and perform user acceptance testing. The PST selected seven client representatives from three of TelSoft‟s long-

time customers. A new interview guide was created that asked about requirements documentation, requirements management, and process innovation. In this cycle, there was no workshop used as a discussion forum. The customers praised the TelSoft personnel for understanding their business, responding promptly to customer requests, and adapting internal practices to client‟s needs; however, they also identified areas for improvement (e.g. customer relationship management, software release packaging procedures and documentation). The primary outcome from this cycle was a list of strengths and areas for improvement [11].

The research team met to synthesize information from the four inquiry cycles. Although the initial focus was on requirements management practices, the inquiry revealed broader issues that prevented TelSoft from effectively satisfying its customers. In total, the research team identified seven improvement areas: software vision management, project portfolio management, software configuration management, customer relations management, requirements management, software quality assurance, and end-user interaction (see Table 4: Identified Improvement Areas for description). In light of these findings, we expanded our research interests to focus more broadly on improving software practices.

Based upon the diagnosing data, we diagnosed TelSoft as lacking enterprise agility, the ability to sense opportunities and respond as an intrinsic part of organizational practices (Overby et al., 2006). Enterprise agility is related to existing literature streams on agility (Abrahamsson et al., 2002; Borjesson and Mathiassen, 2005; Dove, 2001; Gunneson, 1997), alertness (Zaheer and Zaheer, 1997), and adaptive enterprises (Haeckel, 1995; Haeckel, 1999). Sensing capability refers to the organization‟s ability to recognize new business opportunities and technologies as they appear and interpret the impact they might have for the organization (Overby et al., 2006).

TelSoft was unable to sense new opportunities; instead, the organization was dominated by old ways of thinking. Responding capability refers to the organization‟s ability to act based upon the information gathered (Overby et al., 2006). Even in those instances when TelSoft sensed the need for change, they were not able to respond appropriately; they lacked the capability to effectively adapt and innovate. Seen from the standpoint of sensing capability and responding capability,

TelSoft needed to combine the ability to sense customer needs and technological and market opportunities while dynamically responding once aware of suitable opportunities. Based upon this assessment, we recommended that TelSoft abandon strict command-and-control approaches and use governing principles and defined roles to become a more adaptive enterprise (Haeckel, 1995). Principles from Haeckel‟s (1995; 1999) sense-and-respond model were chosen to address this issue.

The research team documented these findings in a comprehensive Phase 1 Diagnostic Report which was revised and approved by the PST [11]. The improvement strategy would be addressed through a number of focused and dedicated project teams with clear success criteria and specified deliverables. These project teams would be established, monitored, and coordinated through the PST. The SC would be responsible for approving the overall plans for the improvement.

The SC was kept informed of the PST‟s activities through periodic status meetings. It is important to note that there were several personnel changes in the SC during this cycle. By the end of Intervention Cycle 1, a new Division President was named. To introduce the new Division President to the initiative, an interim presentation and report was provided on March 16, 2005. The next SC meeting was held on June 9, 2005 to describe the findings and overall recommendations moving forward. Within two weeks of this meeting, SC committed to the improvement strategy and to further collaboration with the research team through December 2006.

Table 4: Identified Improvement Areas

Area Issues

1. Software vision management

TelSoft strategy for software development and customer service should be explicated, maintained, and communicated. This provides a value-based foundation for requirements coordination and management that is consistent with TelSoft‟ business strategy.

2. Project portfolio management

TelSoft software project portfolio should be managed explicitly and coordinated across internal and external stakeholders. This creates the necessary dynamic capability to respond effectively to different and emerging customer and innovation requests.

3. Software configuration

management

TelSoft software configuration management should be improved to ensure consistent and transparent modification and packaging to individual customers. This ensures effective coordination with customers and minimizes adverse effects across projects.

4. Customer relations management

TelSoft should improve its management of customer relations to ensure more symmetric information sharing and proactive expectation and change management. This leads to increased customer satisfaction.

5. Requirements management

TelSoft must improve the transparency and consistency of requirements change management as well as the approach to specify requirements. This lead to improved efficiency, transparency throughout the process, fewer errors, and increased customer satisfaction.

6. Software Quality assurance

TelSoft must build a consistent and systematic software quality assurance process and commit people on all levels to adopt it. This will lead to early detection of errors, improved efficiency, and increased customer satisfaction.

7. End-user interaction TelSoft must establish closer interaction between software development and end-users. This will lead to improved understanding of requirements and to enhance change management in collaboration with internal and external customers.