Effect. of the Document Quality. of the Project Start Architecture. on Project Success. An exploration. Master thesis

(1)

Effect

of the

Document

Quality

of the

Project Start Architecture

on

Project Success

An exploration

Master thesis

University: Radboud University, Nijmegen, The Netherlands Curriculum: Architecture in the Digital World (2006)

Author: Ir Ernst de Boer Student number: 3600337

Supervisor: Prof. Dr H.A. (Erik) Proper

Second reader: Dr S.J.B.A. (Stijn) Hoppenbrouwers Date: 17 March 2009

(2)

Copyright Ernst de Boer.

Parts of this thesis may be reproduced for scientific or educational reasons without prior approval. For all other reasons, prior written approval is needed before reproduction.

In all reproductions including scientific reproductions, references to the author and this thesis must be included. Informing the author when references or reproductions are made is much appreciated.

(3)

Abstract

Enterprise architecture is a maturing field that helps organisations align their internal organisational structure, processes and information technology to their strategy. One of the instruments used in this alignment is the project start architecture document (PSA). Changing the organisation and its information technology is mainly realised by projects which may or may not be successful. This thesis explores the relationship between PSAs and project success. The exploration builds upon existing knowledge about project success and applies this knowledge to the PSA.

The concept that links project success with the PSA emerges from literature to be tangibility. Using a small number of case studies, we validated this expected relationship between the PSA and tangibility. Next, we were able to determine in which cases a PSA is useful and when it is not, the important factors being the type of project and the project management style.

Finally, we identified a number of relevant document quality characteristics that make a PSA a good one.

(4)

(5)

Samenvatting

Enterprise architectuur is een zich ontwikkelend vakgebied dat organisaties helpt hun interne organisatiestructuur, processen en informatietechnologie op de bedrijfsstrategie af te stemmen. Een van de instrumenten daarbij is de projectstartarchitectuur (PSA). Het werkelijk veranderen van de organisatie en de informatietechnologie wordt

voornamelijk door projecten gerealiseerd. Dit onderzoek verkent de relatie tussen PSA's en project succes. We gebruiken daarvoor bestaande kennis over projectsucces en passen deze kennis toe op het gebruik van de PSA.

Het concept dat projectsucces verbindt met de PSA blijkt in de literatuur tastbaarheid te zijn. Door middel van een klein aantal case studies kon deze relatie tussen de PSA en tastbaarheid worden gevalideerd. Vervolgens bleken we in staat om aan te geven in welke gevallen een PSA wel en niet zinvol is. De belangrijkste factoren daarbij blijken het type project en de projectmanagementstijl te zijn.

Tevens waren we in staat een aantal relevante kwaliteitsaspecten te herkennen die een PSA een goede PSA maken.

(6)

(7)

1 Introduction

Many projects fail. This is also the case for IT projects, even though there is extensive literature on project management and its effect on project success. The usual ingredients of the recipes to improve project success are project management control instruments, management style and people.

To further improve the success rate of IT projects, other expertise fields are contributing as well. One of those areas is Enterprise Architecture, which is about aligning an

organisation’s structure and behaviour with its strategy and goals. One of the instruments that emanate from that field is the so-called Project Start Architecture or Project Solution Architecture (PSA). Several organisations now require such a document before starting the realisation phase of an IT project. PSAs supposedly make projects more successful. However, this has never been verified or falsified. This research explores that claim. Another question is which aspects of a PSA influence the chances on project success. One of the consequences of not having this knowledge is that neither effectiveness nor efficiency of the PSA is guaranteed. This research explores which aspects are relevant and which are not for effective PSAs, hence creating the opportunity to reduce the efforts needed for creating PSAs. Some of the conclusions can also help project managers and architects to use the PSA better, ultimately leading to more successful IT projects. Finally, the results can be used as a starting point for follow-up research.

The steps leading to the knowledge sought are: first, from literature, the conceptual relation between the PSA and project success is deduced. Going into more detail, a relationship between PSA contents and its contents’ quality, and the factor(s) influencing project success is derived. A third step consists of exploring the relationships between the different aspects of PSA contents’ quality and the relevant factors for project success, by means of interviews and scoring tables. For this purpose, a number of potential PSA users are interviewed about actual projects at ASR Netherlands1. Based on these interviews, the relationships are explored between different content aspects and the relevant factors influencing project success.

The thesis ends with concrete conclusions and suggestions for future research leading to an even more effective and efficient use of the PSA instrument.

1

Research started at one of the subsidiaries of Fortis: Fortis Insurance Netherlands. During research execution, Fortis sold parts of the company. ASR Netherlands is the continuation of Fortis Insurance Netherlands. This did not influence the research.

(10)

(11)

2 Theory

2.1 Projects, project success and architecture

Most organisations use information technology (IT) to support their business processes, which are necessary to realise products or services. In an ever changing world, these business processes, or the products or services they realise, change continuously. This in turn makes it necessary for IT to change as well, thus creating the changed support that is needed. Some of these changes can not be carried out by the line organisation and need to be managed by a project. In this thesis, a project is defined as “a

management environment that is created for the purpose of delivering one or more business products according to a specified business case” (CCTA, 1999, p. 7). When such a project includes changes in IT, it is called an IT project. In this research, only IT projects are considered.

Many of these IT projects are not successful (Ernst & Young, 2007), which poses a problem for the organisation: the goal is not realised and the project is thus a waste of resources.

Several definitions and understandings of project success exist (Shenhar, Dvir, Levy, & Maltz, 2001). Traditionally, project success used to mean that the project had realised its results on time, within budget and to specification. Currently, another way of looking at project success prevails which is oriented more towards creating business value: does the project satisfy relevant stakeholders? (van Aken, 2002; Pinto & Slevin, 1988b). This research uses the latter view in which the relevant stakeholders are limited to the end-users and business executive. The definition of a project is in line with this view. The reasons why this strong reduction in relevant stakeholders is possible in this research, are given in Appendix 1.

Today, the first view is sometimes called “project management success” (de Wit, 1988) as opposed to “project success”, in this thesis defined as “the level to which the project results satisfy the end-users and business executive”.

A lot of research has been done on increasing project success rates by project management techniques, a clear example provided by a recent special issue of the International Journal of Project Management (Khazanchi & Reich, 2008). Applying the knowledge from that research area has done a great deal of good, but project success can not be realised by project management techniques alone. For instance: the level of confidence that customers and users have in the project manager is important for project success, but this can not be managed easily (Xia & Lee, 2005). Other scientific areas therefore try to contribute to more successful projects as well. Enterprise architecture is one of these areas. The theory and practice of Enterprise Architecture (van den Berg, Luijpers, van Steenbergen, & Wagter, 2005), (Lankhorst, 2005) introduces the so called Project Start Architecture or PSA. It is a document that is created during project initiation for projects that contain IT changes, and is supposed to achieve higher project success rates. It is used in two ways: as a summary of all relevant principles, guidelines,

reference architecture etc. that are relevant for the project and which the project then needs to use when crafting a solution; or, taking into account the principles, guidelines etc. and the project objective, as a description of the solution itself. This research explores the link between the PSA and project success. In this thesis, only the second type of PSA is therefore relevant.

Many definitions of Enterprise Architecture exist, for an overview see Op 't Land, Proper, Waage, Cloo, & Steghuis (2008). Because the research is about a specific use

(specification of results) of a specific artefact (PSA) in a specific context (project), it is not important to know in which theoretical frameworks this usage occurs and in which

(12)

Figure 1. Place of PSA in Prince2

To illustrate in which phase the PSA is positioned, the relationship between the PSA and the project phases is shown. Since one of the most commonly used project management methods for IT projects is Prince2 (CCTA, 1999), this method is used for the illustration. The use of a PSA is not confined to any specific project management technique though. In Prince2 terminology, the PSA is a basis for writing the project plan (project initiation document or PID) and creating a product breakdown structure: the PSA explains what will change; the PID uses this to plan how.

As mentioned above, van den Berg & Steenbergen (2004) claim that PSAs always contribute to project success. The authors explain this by reasoning that the PSA gives more clarity about the deliverables, thereby giving the project manager better possibilities to manage the project. This has no scientific background though. They propose a

standard PSA template for all projects. An example of such a template is included as Appendix 2. Other authors (Felix & Harrison, 1984) also found the need for solid (start) architectures in complex, distributed systems development projects some time ago, yet they don't go into detail on the demands on such an architecture.

In the day-to-day practice of the organisation studied, PSAs are not regarded to always contribute to project success – neither project managers nor architects see the direct relation between the use of PSAs and project success that is claimed by van den Berg & Steenbergen. The question arises: when does a PSA contribute to success, and when does it not; the next question being: what factors contribute to success and which factors do not?

To summarise, no scientific research has been done on the effect that the PSA has on project success, yet a one-size-fits-all template is provided to be used in all cases. To improve project success rates, knowledge is needed on both the PSA creation process and the PSA contents. This knowledge has not been made explicit yet. This research creates some of that explicit knowledge and shares the knowledge with the architectural and project management communities.

(13)

In order to keep this research feasible, the scope is limited. One of the restrictions is that the PSA creation process is left out and focus is on the resulting document only2. A second limitation is introduced by focusing only on the use of the PSA in the project initiation phase, although the PSA can be used in other phases as well.

2.2 Project success model

Before going deeper into the question of what a PSA should contain to achieve optimal positive effect on project success, a model must be in place to align project success (in the “value” sense) with relevant aspects of “hard core” project management techniques. Such a model is created by van Aken (2002). His work is summarised in this section. He uses the concept of project success as defined in section 2.1 and adds an overview of all factors influencing that success, based on a thorough literature review. Van Aken has been able to conclude that three relevant dimensions exist that can be inhibited by projects and that influence project success. These dimensions of projects and their respective influences on project success are shown in the conceptual model (Figure 2). The dimensions are:

- controllability (van Aken, 2002, pp. 71-77); a combination of the complexity and the tangibility of the project;

- working style (van Aken, 2002, pp. 124-125); the management style of the project manager;

- use of instruments (van Aken, 2002, pp. 79-86).

Figure 2. Conceptual model of project success (van Aken, 2002, p. 129) The associated conclusions on project success are (2002, p. 149):

1) to achieve project success, structuring is desired in projects that are barely controllable, but not in controllable projects;

2) a goal oriented style of working affects project success favourably to a high degree; 3) when work is goal-oriented, the project manager can work successfully with only a

few instruments;

4) the more instruments used, the less chance of project success.

The numbers are changed into positive (+) or negative (-) effects in the following figure. Connections not mentioned do not have a significant effect.

2

Another reason is that some good practices on the PSA creation process are already shared among architects, although no formal publications are known.

(14)

This thesis builds upon van Aken’s work. The limits from his research therefore also apply to this research: the results are valid only for projects in organisations over 50 people, and that are organised with a separate project manager, business executive, and project team (no small projects).

2.3 Project success and PSA

In order to determine the effect of a PSA on project success, all possible influences in the model must be investigated: what is the possible effect of a PSA on the respective influence? This is shown in the following figure which extends the previous model:

Project success Instrument use Controllability: - tangibility Structured working style Goal oriented working style + / + / -few instruments needed + PSA ? + + if contrasted, - if the same

Figure 3. Conceptual model of PSA contents effects on project success

The following subsections examine each of these influences. The two types of working style are discussed together.

2.3.1 Effect of PSA on instrument use

Van Aken gives a good overview on possible types of instruments (2002, pp. 83-85). The PSA can be considered to be an instrument and therefore affects instrument use

positively. Since van Aken’s end conclusion is that instrument use is not a major

contributing factor but it will reduce project success if too many instruments are used too rigidly, the conclusion can be drawn that, through the factor instrument use, the PSA is not a major contributing factor to project success: however, too much emphasis on the PSA will reduce project success chances.

Further exploration of the effect of PSA on instrument use is not going to help improve project success and therefore is kept out of scope in the remainder of this research. 2.3.2 Effect of PSA on working styles

Working style (both goal oriented and structured) is important for project success, especially in combination with the controllability of a project (van Aken, 2002, p. 149). Working style is solely based on the project manager’s style of working. Although the working style can result in the use of a PSA, the reverse is not true: PSAs have no direct effect on the working style. The project manager may choose to change his working style though because a PSA is available. This is not considered to be an effect of the PSA itself on project success and the relationship will not be explored further. The effects are shown as dotted lines in Figure 3.

2.3.3 Effect of PSA on controllability

Uncontrollable projects can be managed to realise success by applying a structured working style. For controllable projects, it is wise to apply a less structured style. Changing uncontrollable projects into controllable projects therefore influences project success – it will improve project success as long as it is complemented by the right

(15)

project management style. Since a PSA can influence controllability directly, a PSA can affect project success.

In controllable projects, the project members can commit themselves to results, while in the less controllable projects only a commitment to do ones best can be made. Assuming that project managers and executives prefer results above efforts, a controllable project is preferred for project success.

2.3.4 Summary of the PSA effects

The influence of the PSA on controllability of the project is the only relevant factor for project success. At the same time, the PSA must be used as little as possible as a project management instrument because that use will only reduce chances on project success. Since controllability is that important, the following subsection describes the term more thoroughly.

2.3.5 Controllability

Projects can be distinguished by type. Van Aken uses the Dutch word “grijpbaarheid” as the discerning variable. He translated this word as “governability”. However, in today’s world the word governability is commonly used in a more political sense. This research chooses to change the translation of the Dutch word “grijpbaarheid” to “controllability”. The original definition of controllability can then be translated into “the degree to which the project manager can influence both contents and course of a project”.

The controllability of a project is a weighed sum of (van Aken, 2002, p. 76); see Appendix 3 for more detail:

- 1 x number of disciplines involved;

- 2 x tangibility: Concrete, physical end products are highly tangible and can be specified in much detail, yet low tangible end products are hard to specify, and - 3 x numbers of parties involved;

In this thesis, the solution is assumed not to change from the bare fact that a PSA is created – the PSA just describes the solution and makes the design decisions more explicit. As a consequence, the number of disciplines and the number of parties involved are fixed. This results in only one variable that can be changed by the PSA: tangibility. Van Aken chose to define tangibility indirectly by providing a scoring tool to measure tangibility which is part of Appendix 4. A dictionary defines tangibility as “capable of being understood and evaluated, and therefore regarded as real” or “capable of being given a physical existence” (Webster's, 1990).

2.4 PSA contents and the contents’ quality

The question to be answered is now: what characteristics of the PSA influence tangibility? Literature (e.g. van den Berg et al., 2005) tells that completeness and accuracy of a PSA are important. For these, templates are available that tell what must be described, e.g. Zammuto (n.d.). The how question is never answered, e.g. if the PSA is easily understood, apart from a thesis by Veltman – van Reekum in 2006. She created an instrument for measuring the quality of Enterprise Architecture products. However, she built this instrument on software, information and data quality instruments which seems to be the wrong starting point: when considering the PSA, it is the quality of the description itself that is relevant, not the quality of what is described3. Another difference is the purpose of the measurement tool: she developed a method to determine quality as such of all kinds of enterprise architecture products, whereas the goal of this research is

3

Nevertheless, the solution that is described by the PSA must be correct – a good description of a bad solution will not make a project successful.

(16)

to determine the effect of the different quality characteristics of one of the architecture products on one of the project’s characteristics.

This thesis assumes that the how question is very important though: the tangibility of the project as felt by the project members is based upon reading the PSA, and reading involves more than contents. This research explores which quality aspects are indeed important (both what and how related) and therefore should be given attention when creating a PSA, and which aspects can safely be ignored when writing a PSA. The different quality aspects of (technical4) documents in general, and therefore of a PSA, can be captured by nine characteristics (Hargis et al., 2004). A scoring list is available in their book and reproduced in Appendix 5.

The nine quality characteristics are, grouped by three main characteristics: - Easy to use • task orientation • accuracy • completeness - Easy to understand • clarity • concreteness • style - Easy to find • organization • retrievability • visual effectiveness.

All nine characteristics are scored by four to nine questions each.

There is no reason to expect or assume that one or more of the characteristics are irrelevant in influencing tangibility. On the other hand, If a particular quality characteristic shows to be very relevant, it is possible to explore that particular characteristic more in depth. For example, if completeness shows to be very important, the elements that are most important for completeness can be checked using the list in Appendix 2.

2.5 Summary

This ends the theoretical background to this research. In this chapter, it was shown that PSAs can have an influence on project success. Focusing on the project initiation phase, the factors and prerequisites that play a role were discussed. Two of those factors are further explored in the practical research part: tangibility and the quality characteristics influencing it. These two will give some insight into the questions: when can a PSA have a positive effect on project success, and what characteristics make this happen.

4

The term “technical” applies to process, product and organizational models too, which are also part of PSAs. Since these models are still “hard”, the whole PSA is considered to be of a technical nature.

(17)

3 Research execution

The execution of the practical part of the research gives the data to answer the question: What are the relationships between the different quality characteristics and tangibility? How this question is answered is the subject of this chapter.

3.1 Research strategy

For this explorative study, a small number of the same experiments are chosen. The project initiation phase is imitated in this experiment in a controlled way: a person, a typical user of PSAs, is given the standard set of project documents that are normally available at project initiation. The person is given some time to read this information, in order to simulate a real project. To determine this period, the person is asked how much time he5 usually takes to read the initial project documentation. The time the user usually takes to read a PSA is also asked. Having read the project documentation for this predetermined period, the person is interviewed about the tangibility of the project, using a questionnaire. Next, the person is given the actual PSA of the project. The reader is asked to use the aforementioned amount of time to read this PSA. After reading, the tangibility is determined again and the reader is interviewed about the perceived document quality, again using a questionnaire. Finally, some open questions are asked for cross checking the validity of the answers and possible new clues. The extended interview script is included as Appendix 6.

To be able to analyse both variations in readers and in PSAs, the same persons are asked to read different PSAs, and different persons are asked to read the same PSAs. To keep the research feasible within a limited time frame, three people are interviewed and three PSAs are analysed. This results in 27 contact moments: three projects, three readers and three contact moments per reader / project combination.

An important consideration is to make sure the reader does not have knowledge about the project studied or the functional domain the project is about. If this happens to be the case anyway, another project is selected. E.g., if a project is about life insurance and one of the readers would be or would have been practicing in this field, another project in another field is selected. The next section gives the project and reader selection criteria in more detail.

Alternative approaches have been considered but cast away as not feasible: better separation of the different effects (project / reader / PSA) could be realised by creating several PSAs for the same project and comparing these PSAs on their effect on tangibility by having the same person read the different PSAs; to correct for the organisational bias, more readers in more organisations could be interviewed. In other types of research or follow-ups, these approaches can be valuable.

3.2 Sources for research

How to determine tangibility with and without the PSA?

There is a scale available in van Aken’s book which is used with minor adaptations. The scale is reproduced in Appendix 4. The score is a number between 1 (very low) to 5 (very high).

How to determine the quality characteristics levels?

This research is about only relative effects: which quality characteristics are the most important? An absolute and objective scale is therefore not a must. The subjective opinions of the readers suffice as long as they score consistently through all

measurements. Interviews and the questionnaire from Appendix 5 (Hargis et al., 2004)

5

(18)

are used for this purpose. The scale goes from 1 (“among the worst”) to 5 (“can be used as a model”). The assumption made is that the interviewees will apply the same

(personal, subjective) scale during research. A cross check is introduced to find a shifting norm, if present. This is done during the interview for the third project by asking again some of the same questions for the first project, as remembered by the interviewee. If the reader now happens to assign different values to the first project than he did earlier, the interviews for the first and project must be repeated.

Which projects to select?

For comparison reasons, it is important to use projects that only have tangibility as varying factor, so the projects must be comparable in the number of disciplines involved and the number of parties involved (simple versus complex). Measuring these values objectively is realised by listing possible parties and disciplines, asking all readers and the researcher to state which parties and disciplines were/are involved, and counting them. If the numbers differ considerably, another project is chosen. The list used to determine number of parties and number of disciplines is part of Appendix 3.

The projects are all started recently (2007 and later). Projects don’t need to be finished yet since the projects’ successes are not relevant for this study. This approach does allow for follow-up research which can look into the actual project success and project working style. All projects are selected from the same organisation, where access to all PSAs is assured.

Which people to interview?

Since this research is not about the effect of using a PSA as such, interviews are held with people that are familiar with PSAs in general and usually are involved in project execution. For practical reasons, the readers come from the same organisation as the projects. Care is taken they are not familiar with the projects that are the objects of the PSAs. Otherwise, tangibility changes before and after reading the PSAs can not be measured truthfully. Using people from the same organisation also guards against the bias of differences in generic organisational knowledge, architecture process knowledge and PSA template experience.

(19)

4 Results

This chapter gives a short characterisation of the three projects and the main measurement results. Conclusions follow in Chapter 1.

4.1 Results per project

As one of the prerequisites for project selection and comparison, all projects must have a comparable number of disciplines and number of parties involved. This is indeed the case (6 or 7, and 9-11 respectively). The exact scores are irrelevant and therefore omitted in the following subsections. They are included in Appendix 8 - Appendix 9. 4.1.1 Project 1: SAP

The first project is “Real Estate on SAP”. The project aims to replace the current (legacy) software set for one of the company’s subsidiaries with SAP. The subsidiary manages and develops assets in real estate (housing, shops, offices, land). Integration with other applications in the company’s application environment (data warehouse, archiving, document management, geographical information system, to name some) is an important aspect.

At the start of the PSA writing, many documents on SAP implementation details were already available. The readers in this research were allowed to browse through all documents and take with them what they wanted to read. All consciously and realistically chose to read only the project initiation document (PID). After reading this PID, the scores for tangibility were 3.3, 2.7 and 3.3 respectively.

After reading the PSA, the scores changed to 4.3, 3.3 and 3.3.

All readers found this PSA a very good addition to the PID, especially since the PID only focused on project management and hardly on how things were to be solved on a technical level. The PSA gave clear answers and good direction. Although one of the readers did not like one of the fundamental technical choices, it was clear to him that this choice was final and all consequences were taken into account.

The PSA scored also well on the clarification of the quality norms and the amount of knowledge needed for project members.

Out of the three PSAs, this one scored highest on the quality aspects “easy to use” and “easy to understand”.

4.1.2 Project 2: web site

The second project is about an addition to a web site. The insurance company studied traditionally works with brokers (in Dutch: assurantietussenpersonen), and does not sell to or communicate with consumers directly. To lower the administrational overhead for these brokers, other projects have already realised small web elements that these brokers can use on their web sites to allow their current and new customers to calculate premiums and to buy so-called simple products, like car insurance.

The goal of this project is to realise a similar solution on the public web site of the

insurance company itself. These new customers and their applications for new insurance policies must be routed to one of the brokers. The project has a very challenging

deadline.

After reading the project documentation, the readers scored mixed on tangibility: twice 2.7 and once 4.0.

After reading the PSA, the values changed to 4.3, 3.3 and 2.0. The last value was explained by the reader with the words “I was confused after reading the PSA”.

(20)

All readers found the PSA useful in giving clear directions to the project members and were convinced the PSA had helped the project, even though it was clear to them that the PSA had only been given formal approval when the project was almost finished. They were sure that the unfinished and informal PSA had been used during the project as if it was formally approved. This was even true for the reader who had said he was confused. The PSA scored high on the clarification of the quality norms.

The PSA scored low on accuracy and completeness, which led to a low score on understanding. Its quality scored average on both content and format. The PSA was directed at project members who were already familiar with the subject; the readers were not.

4.1.3 Project 3: accounts payable

This project is aimed at enhancing the process regarding accounts payable. At the time of the project start, the current process was a manual process in which a paper copy of the invoice and an accompanying slip were distributed by the financial department to the department that can state whether the invoice should be paid or not. The signed slip is then sent back to the financial department, processed manually and the invoice is paid. The new process replaces this paper flow by an electronic flow, and automates the processing step. No changes take place in the process of reading the invoices and entering them into the accounts payable system: these steps continue to be manual. The workflow process of approving invoices is part of the accounts payable system, in which a link to the scanned image of the invoice is made available.

The project had seen three unsuccessful attempts before, due to discussions on technical issues during project execution. A PSA had not been created for those attempts.

A PSA has been created for this (4th) attempt6.

After reading the project documentation but before reading the PSA, the readers scored this project the highest on tangibility (average 3.9).

After reading the PSA, one of the readers surprisingly lowered his score somewhat, while the other scores remained unchanged and stayed to be the highest (average 3.8).

Asking whether they thought the PSA had improved the chances on project success, readers one and three thought the PSA would indeed have benefited the project because the technical choices were clearly made (although none of the readers liked the choices), while reader two found the PSA useless. This conclusion was in line with his scoring: reading the PSA had not changed any of his tangibility scores. Interestingly, the other readers had not, or not significantly, changed their scores either but still deemed the PSA useful for the project.

The PSA quality scored lower than the other PSAs on understanding and retrieving. Although the relative scores are consistent for all three readers, the variance within the answers is very high.

4.2 Results per reader

Reader one raised his tangibility scores for the first and second PSAs to 4.3. The third PSA received a final lower score of 3.0.

In the tangibility questionnaire, the reader especially liked the clear results and the good explanation of the quality norms in the first PSA. The second PSA was valued about the same but with the critical verbal note in the open questions that the PSA was not very

6

(21)

good regarding the quality of the contents. The values measured for the quality

characteristics were consistent with this remark, although the correlation is not strong. The reader was not impressed by the third PSA: almost all tangibility measurement scores were lowered, only the knowledge needed to execute the project was a bit less (positive). This was clearly visible as well in the quality characteristics measurement, which showed particularly low scores on content. The cross-check questions corroborate the results.

The second reader also found PSAs #1 and #2 of the same quality and a substantial improvement to the original set of project documentation. Most scores were raised with the emphasis on the quality requirements7. Interestingly, the third PSA does not improve the score on tangibility, yet the project’s tangibility receives the highest score. As

mentioned above, the reader thought this PSA was useless. When checked with the quality attributes scores, it is observed that the reader found this PSA not directed towards him (task orientation scores).

The third reader did not score any project’s tangibility higher after reading the respective PSAs. As a matter of fact, most of his scores did not change after reading the PSAs. Only the quality requirements and the knowledge needed improved for PSAs #1 and #2, while the direct tangibility question was awarded higher for the third PSA. The latter score was explained by the reader: it came from one single word in the PSA which had made him understand the only thing unclear to him before reading the PSA. For the second project, the reader even lowered the tangibility score. He explained this by telling: “I’m confused. I miss the context and the relations, the alignment between the different parts. The same was true when I had read the project documentation, but I had expected to find it in the PSA.” On this basis, it can be reasoned that the score should be corrected to 3

(unchanged). Expectations play a role, apparently.

4.3 Cross checks and open questions

In search for the quality characteristics that are most important for a PSA that enhances project success chances, the interview ended with open questions on this subject. The same questions were also used to verify the consistency between the other measurements and conclusions. The questions are listed in Appendix 7.

In this section, first the answers to the question “What contributed to the clarification? (The PSA as a document, not the contents as such)” are given. They give some insight into the “softer” aspects of PSAs and the readers’ appreciation of the PSAs. The answers are presented per reader since this appreciation is expected to be a personal preference. Reader 1:

“The other source documents clarified the most.” (web site)

“Especially the infrastructure pictures and the Archimate figure.” (SAP) Reader 2:

“The sentence ‘from this email message, the (…) application is initiated’ answered the only question I had after reading the initial project documentation.” (accounts payable) “I only started to understand when I saw the pictures.” (web site)

“The contents are not good enough, so the form is irrelevant.” (web site) “How wonderful the PSA template was used.” (serious remark on SAP)

7

Quality requirements are a well-known term in architecture. They are mostly non-functional in nature. They answer questions on e.g. portability, availability and time-to-market of changes. In software, they are also known as “software quality attributes” and defined in ISO standard 9126.

(22)

“This is a good PSA: it shows the solution and explains why it was chosen. The issues are mentioned, respected and a choice is made explicit and explained.” (SAP)

Reader 3:

“The structure of the PSA and the tables.” (SAP and web site)

“Process scheme; the relationship email-application mentioned in 4.2 (of the PSA). I missed the link application – process and the consistency.” (accounts payable)

Next are all answers to the question: “Do you think this PSA increases the quality of the solution and the chances on project success? Why? Why not?”. They are presented per PSA.

SAP:

“No alternatives are shown, only the preferred solution. So I think it directed the project well, but I do not know if the solution is the best one. “ (reader 1)

“The solution is shown well, but I don’t agree with it!” (reader 3) Web site:

“These two chapters (infrastructure and application) were clear about the solution, but no rationale was given. It probably helped the project in delivering, but I don’t know about the quality of the solution.” (reader 1)

“It lists the issues and gives answers, so it does increase project success chances. But I don’t think it enhances the quality of the solution.” (reader 2)

“No alternatives are given and no reason why this solution is chosen. So it enhances project success, but I don’t know about the quality of the solution” (reader 3)

Accounts payable:

“Although the PSA gives answers, it does not give the rationale so discussions may arise during execution. If the document is the result of discussions with these people, then it is all right..” (reader 1)

“Not really.” (reader 2)

“The solution is shown well; it will certainly enhance project success.” (reader 3) The third question is: “Which (content) elements have the biggest effect on the score?” Keep in mind that this question does not play a role in answering the research questions of this thesis but is only asked for exploration of possibilities for follow-up research. This question is answered less specific by the readers. They all said it was “the PSA as a whole”. One interviewee pinpointed a specific table (functionality vs. software modules, i.e. relation between business process and application), but this was not supported by the other readers for the same PSA. Another reader said that one specific sentence made the biggest difference in understanding. This sentence explained a specific relationship between application and process, as did the previous answer. The answers to the previous question may also be used (with caution) to provide some insight in the relevancy of the different content elements.

4.4 Summary

With one disputable exception, the tangibility of all projects after reading the PSAs, were at least 3.

Readers did not always agree but trends were clearly visible. The scores on quality requirements benefited most from the PSAs.

The overall document quality characteristics scores are in line with the tangibility scores and the open questions.

(23)

5 Conclusions

5.1 Tangibility scale

The scale determined by van Aken (2002) is adapted to this research, as explained in detail in Appendix 4. Summarized, the planning and team questions are omitted. Question B, how is the project to be executed, is a special one. This aspect was expected to be important, since PSAs not only explain the final situation (solution) but also show the roadmap, e.g. intermediate stages. All three projects studied were however about new processes, applications and infrastructure, so no intermediate stages were present. The question was therefore interpreted by all readers as a planning question. All PIDs and other source documents somehow contained this planning information, while, logically, the PSAs didn’t. Therefore, the readers scored this question higher or equal before reading the PSA than after, or answered that the PSA hadn’t changed anything in their understanding because “the PSA is not about planning”. Considering this, question B was not used for analysis and conclusions.

Question E is not about tangibility, yet is important for the quality of the solution itself. During the first test interview, questions A (“clear result”) and K (“I do not know what the result will be”) appeared to be the same question, but asked using a positive and negative sentence structure respectively. The second question was removed and the relative weight of the score on question A was doubled.

Staying as close to van Aken as possible, the formula becomes: Tangibility = 2 x “clear results” + 1 x “dependency on external factors”.

As a cross check, it was also tested whether tangibility measurement would be different when excluding question I. The differences in the answers on question C are only minimal (2 answers +1, one answer -1) and the net effect on the tangibility is hardly present (3.37/3.55 becomes 3.44/3.67). In a qualitative sense, nothing changes. The formula did therefore not need to be changed and question C was kept included.

5.2 Tangibility

As expected, the readers experienced and scored tangibility differently, probably because of differences in knowledge, role and experience. This confirms that the scale is

subjective. However, the overall picture from the measurements was consistent among readers, which confirms that tangibility measurement can be used as an instrument. The measurement itself only takes 5-10 minutes, which makes the instrument usable in practice as well. A summary of the tangibility scores is pictured in Figure 4 below. Two reasons were uncovered for the reader-project combinations in which the tangibility did not rise:

1. the tangibility is already high;

2. the PSA is expected to give more clarity than it actually does.

If tangibility is indeed as important as expected, the PSA needs to be more tailor-made. It needs to take into account the already present tangibility and give more emphasis to the areas that are still unclear. Other researchers have already answered what areas should be covered in the PSA. This research shows that, depending on the knowledge and information already available, one should elaborate only on those subjects where

knowledge is to be gained. In the PSAs studied, the quality requirements are an example of these needs, as are the relationship between business process and applications. The current PSA template in this organisation and in literature is suitable for this tailoring, but from this small sample it is clear that authors are not aware of this need or do not have the necessary competences to act upon this knowledge.

(24)

Tangibility before and after PSA reading 0,0 0,5 1,0 1,5 2,0 2,5 3,0 3,5 4,0 4,5 5,0 r1 S A P r2 S A P r3 S A P r1 w e b s it e r2 w e b s it e r3 w e b s it e r1 a c c o u n ts p a y a b le r2 a c c o u n ts p a y a b le r3 a c c o u n ts p a y a b le

reader & project

ta n g ib il it y s c o re Before After

Figure 4. Tangibility changes for all reader / project combinations

5.3 Quality characteristics

The first analysis is the separation of content (what) and format (how). For each project, the scores of the three readers are averaged. This is allowed since the readers have different norms, but these norms are constant. This was checked a couple of times by asking the readers during the interview for the third project: “what is your score for this question for PSAs #1 and #2?”. The answers were consistent. The overall scores for the PSAs were 3.6, 3.4 and 3.3.

For the following paragraphs, one must bear in mind that the number of content questions was only 13 against 49 format questions.

When only the answers to the content related characteristics are taken into account, the scores are 3.9, 3.4; 3.4. For the format related questions, the scores were 3.5, 3.4 and 3.3 respectively. Figure 5 summarizes these scores, Appendix 10 contains all scores. The coloured bars show the average scores; the vertical lines denote the minimum and maximum scores.

The results for format are pretty close to each other. The question arises whether the readers are relatively indifferent to the format elements of the PSAs and thus score all questions about the same, or that the PSAs are comparable in quality. Might it be that the template has an important influence on the format score?

For that purpose, all questions are separated into template related and not template related. Keep in mind that this is another viewpoint than content or format: some questions can be content as well as format related, see Appendix 5. When these template questions (content as well as format related) are not taken into account, the PSAs score almost equal on quality (3.50, 3.37 and 3.37 respectively, n=37). When only the template related questions are taken into account, the scores become 3.79; 3.57; 3.21 (n=16). The scores for these template related questions stay almost the same when differentiated between the content related template questions and the format related template questions and therefore neglected further. Figure 6 shows the integral scores.

(25)

1,0 1,5 2,0 2,5 3,0 3,5 4,0 4,5 5,0 SAP w eb site accounts payable SAP 3,58 3,50 3,89 w eb site 3,43 3,42 3,44 accounts payable 3,32 3,28 3,44

overall score format only content only

Figure 5. Quality characteristics: content and format separated

1,0 1,5 2,0 2,5 3,0 3,5 4,0 4,5 5,0 SAP w eb site accounts payable SAP 3,58 3,79 3,50 w eb site 3,43 3,57 3,37 accounts payable 3,32 3,21 3,37

overall score template only non-template

Figure 6. Quality characteristics: template and non-template separated

From these scores the conclusion is drawn that the non-template related questions are even more evenly answered. The PSAs only clearly scored differently on the aspects of how the information was presented (from the viewpoint of the reader or not), and if

(26)

graphics were used appropriately and clearly. All other questions were scored evenly and around average.

This leads to the conclusion that the PSA template is very important for the overall PSA quality, both for format and for contents. Interestingly, another conclusion is that a PSA is considered better if the template is used.

Since the first paragraph of this section already showed that contents were important we can conclude that the quality characteristics that are only format oriented and not

template related, are unimportant for this group of readers.

The set of both template and content related characteristics were not relevant in the context of the template related characteristics as discussed above; on their own, they are worth looking into. The scores are 3.93; 3.53; 3.20. Since only five questions are in this set, it is difficult to draw hard conclusions. Surely, the readers’ preferences for a template which is nothing more than a structure can not explain content related scores, so it can only mean that the PSA template makes sure that all relevant content elements are covered. This correlates well with the strongest difference in answers to the specific question “All topics that support users' tasks are covered, and only those topics.”. From this, it is concluded that the PSA template is a good one. Improvements are possible though, as can be deduced from the open interview questions; see section 5.4. Content related questions were less present in the questionnaire, as intended. The biggest discriminators are “all topics (...) are covered”, “the subjects in general are appropriate for the audience, and “the subjects are focused, realistic, accurate, and up to date”. Formulated differently: the information needs to be relevant to the reader and correct. This does not come as a surprise, yet it confirms the hypothesis that contents are a prerequisite for a good PSA.

A reminder: the results from this part can only be interpreted qualitatively. The exact looking calculations above were only meant to find extremes in values, variations and to find discriminating factors.

5.4 Open questions

The open questions were primarily used as cross check for the conclusions drawn from the other measurements. One earlier conclusion about the relationship between the different content elements of the PSA (e.g. link between organisation and process, or between application and infrastructure) can now be made more specific: for these

readers, it is mainly the relationship between process and applications that is of particular importance.

Another conclusion is that PSAs that raise tangibility are considered useful by the readers: they expect that the PSAs for those projects do indeed guide the project in a positive way. Put differently: high tangibility leads to successful projects, according to the readers. This is in line with the theoretical basis of this research.

From the open questions, it appeared that the index, style and mood questions were considered not to be as important as the emphasis and legibility questions. From the format related questions, the ones that are the biggest discriminators are “the information is presented from the user’s view”, “the elements are short”, “scenarios illustrate tasks and provide product overviews (use cases)”, “style is active”, “mood is appropriate”, “templates (..) are implemented appropriately”, “main points are emphasised”, “the index is complete”, and three questions about legibility and putting emphasis on the right issues. The combined answers from the open questions and from the quality

characteristics measurement lead to the conclusion that a good PSA is mainly about understanding the overall solution within its context, and, to a lesser extent,

(27)

5.5 Summary

The assumptions made were all validated which allows us to use the measurements. Nevertheless, all conclusions must be met with some reservation, since many restrictions were introduced in this research: a small number of readers and projects, one type of reader, one organisation and one type of project controllability are the most important limitations. The following conclusions can therefore only serve as a guideline in which direction to research further, or as non-falsification of current knowledge.

From theory, it is deduced that PSAs can help project success as long as a goal oriented project management style is used. Next, projects that are already tangible do not benefit from a PSA directly; other, indirect influences may have an effect though. A third

conclusion is that too much emphasis on the use of project management instruments has a negative influence, which also applies to the use of architectural instruments as the PSA.

From the practical results, the following conclusions were drawn about PSAs, tangibility and project success chances:

- The PSA can influence tangibility as long as tangibility is not high already;

- The influence between PSA and tangibility is valid both for contents and for format.; - The hypothesis that the format aspects are a hygienic factor is falsified;

- The hypothesis that the contents are a hygienic factor is plausible;

- Good PSAs are considered by the readers to have a positive impact on (probable) project success.

Since a good PSA correlates with a tangibility rise,

- The deduced relationship between tangibility and project success chance is confirmed;

- PSAs that raise tangibility are considered most valuable by the readers. This correlation is even stronger for the effect on quality requirements score. The following conclusion was drawn about the quality characteristics and PSAs:

- The PSA template used at the company enhances both the content scores and the other quality characteristics scores.

“The knowledge needed to be able to execute the project” was an element in van Aken’s initial scale (2002, p. 105) but it was not part of his final scale. This element indicates the knowledge transfer by the PSA. As the answers to that question showed, the PSA can be used well for that purpose.

In the chapters on literature and research description, it was concluded that for some types of projects the PSA does not contribute to project success, and for other types it does. It is now possible to add that the PSA can have another beneficial effect: - The PSA is a good document to assemble and transfer knowledge to project

members.

The net effect between knowledge transfer (positive) and instrument usage (negative) can not be deduced from this research.

Two questions were added to the original tangibility measurement: Additions to the questionnaire regarding "requirements" R It is clear to me what the objectives of the business executive are. S The requirements already give a solution

They were not necessary to answer the research question, but added for possible follow-up research. A short note on these questions though.

The answers to the questions are completely coherent with the overall picture of the PSAs: both the SAP and the web site projects were said to have the solution already lined out in the source documents, and the PSAs were good summaries of the source documents. They elaborated on some aspects (mainly non functional requirements also

(28)

know as the quality requirements) and how to fulfil these. This elaboration led to a better understanding of the goal of the project and how to achieve that goal; the quality of the solution itself got less attention. The third PSA made the solution a tiny bit more tangible, but the goal of the project and how that goal would be achieved were not clarified at all. Apparently, understanding the goal of the project is very important. This supports the assumption made in the theory chapter that a PSA can improve the goal orientated style of the project manager.

5.6 Impact on PSA usage and writing in practice

In the context of project initiation, writing a PSA is useful when: - the project’s tangibility is not high already AND

- the project manager applies the correct style (goal oriented).

We can use a simple measurement of tangibility by asking just two questions. The PSA is also useful when

- there are many project members who read it (knowledge transfer). When writing PSAs, focus on:

- relationships between process and applications;

- what is not known already by the readers (quality requirements being one of the probable elements);

- fast knowledge transfer: keep the quality characteristics in mind. Since this was not on top of mind for the readers who usually also act as writers, some courses may be needed to make the authors more conscious of this aspect.

The use of PSAs should be related to the type of project and project management style. PSAs also play an important role in architecture governance. This is a completely other use of the PSA. Whether the same PSA template is the best way to make sure the project complies with the architecture guidelines, or that another, simpler template is sufficient, was not part of this research.

5.7 Limitations

Summarizing the most important limitations for this research:

- Only one type of projects is considered, with the same amount of parties and disciplines;

- Cases are from one company only; - A small number of cases is used;

- Only experienced readers are interviewed;

- Readers came only from the application development department; - All PSAs adhered to one template;

- The quality requirements did not appeal to readers directly. Therefore, they were unable to state the relative importance of the different quality characteristics. Since the number of interviews were much too low compared to the number of questions, factor analysis or other statistical tests were also impossible. All questions were thus considered to be equally important. Most probably, more conclusions can be drawn if the relative importance is taken into account.

(29)

6 Reflection

6.1 Reflection on the research

Combining project management knowledge with enterprise architecture knowledge was a good idea. It brings together two completely different fields of expertise and the people working in those fields; people who (usually) are driven by other values, have different characters and yet need to work together to create enterprise-wide value.

A lot of this terrain is still barren.

Enterprise architecture is a young profession but one that originates from fields that sometimes have been around for a very long time. Enterprise architects sometimes forget that latter part. Combining knowledge from software development, change management, people management, business process management, IT (operational) management, financial management, project management, strategy development, to name just some of the related fields, can be very powerful.

The time it takes to interview people has been brought back to almost zero by the use of checklists and interview scripts. It is very easy to do future research: all ingredients are in place. The tangibility questionnaire might need a little improvement. If future researchers want to be able to compare quantitatively, improvements are also needed for the other questionnaires.

6.2 Suggestions for future research

It was already known what the PSA should contain from an architecture governance perspective, as this was the origin of the PSA. This research has explored the effect of the PSA on project success. It also explored which quality characteristics of the PSA are important. We now know what the main points of attention are in a PSA from a project perspective.

The assumption is that the PSA only serves an architecture governance objective and a project goal. The first suggestion for future research is: are there any more goals or objectives the PSA supports?

If we want the instrument to be used even more effectively and efficiently, the PSA creation process is next on the list of research topics. The process should be quick, but how do we know who are to be included in this process and how? Do these people need to be informed, committed, consulted? When? Which ways of communication and collaboration fit this process best?

Following up on the conclusion that a PSA can indeed help the goal oriented project management style, it would be interesting to know if clarity about the project’s goal is a condition sine qua non or that a PSA could still be useful if only the solution is outlined but not the goal or the rationales. Especially for the projects that are less controllable due to a high number of parties and disciplines, this effect can be important. The field of knowledge management can give a headstart for such research.

Of course, the research should be extended to other organisations and other groups. It may be nice to know what architects think of the effect on the project, but the project manager may have a different opinion. Extending to the actual project results and evaluating the complete project, including the PSA document and process, will shed more light on other relevant effects.

This research compared PSAs that were all based on the same PSA template. It would be interesting to compare PSAs based on a PSA template and PSAs that are not based on a PSA template. This may also make the influence of the template clearer.

(30)

Currently, the PSA is used to show that the project’s solution complies with the

architecture guidelines as well as to describe the project solution to such an extent the project can manage the realisation of the project’s goals. This is fine if the project benefits from such a document, but can also hinder the project as was shown earlier. In those situations, having twos document and template may work better. Research could focus on the requirements for such a document and process from the architecture governance viewpoint, keeping in mind that the project does not benefit directly from the document.

(31)

7 References

van Aken, T. (2002). De weg naar projectsucces; eerder via werkstijl dan via instrumenten (3rd ed., p. 178). 's-Gravenhage: Reed Business Information. van den Berg, M., Luijpers, J., van Steenbergen, M., & Wagter, R. (2005). Dynamic

Enterprise Architecture: How to Make It Work (p. 256). Wiley. van den Berg, M., & Steenbergen, M. V. (2004). DYA®: Stap voor stap naar

professionele enterprise-architectuur. Ten Hagen & Stam Uitgevers.

van Brandwijk, L., & van den Daal, F. (2004). Werken onder Architectuur, het maken van architectuurbeschrijvingen. Utrecht: Fortis Verzekeringen Nederland.

CCTA. (1999). Managing Successful Projects with PRINCE2 (Fourth impression (with amendments).). Norwich, United Kingdom: The Stationary Office.

Ernst & Young. (2007, June 20). ICT Barometer. Ernst & Young. Retrieved December 15, 2007, from

http://www.ey.nl/download/publicatie/Rapportage_ICT_Barometer_over_ICT_Proj ecten_20_juni_2007.pdf.

Felix, R. G., & Harrison, W. L. (1984). Project Management Considerations for Distributed Processing Applications. MIS Quarterly, 8(3), 161-170.

Hargis, G., Carey, M., Hernandez, A. K., Hughes, P., Longo, D., Rouiller, S., et al. (2004). Developing Quality Technical Information: A Handbook for Writers and Editors (2nd ed., p. 445). IBM Press. Retrieved from

http://safari.oreilly.com/0131477498/pref01.

Khazanchi, D., & Reich, B. H. (Eds.). (2008). Special Issue: Achieving IT Project Success. International Journal of Project Management, 26(7), 699-770. doi: 10.1016/j.ijproman.2008.06.005.

Lankhorst, M. (2005). Enterprise architecture at work : modelling, communication, and analysis (p. 334). Berlin; Heidelberg; New York: Springer.

Op 't Land, M., Proper, E., Waage, M., Cloo, J., & Steghuis, C. (2008). Enterprise Architecture: Creating Value by Informed Governance (1st ed.). Springer. Pinto, J. K., & Slevin, D. P. (1988a). Project Success: Definitions and Measurement

Techniques. Project Management Journal, 19(1), 67-72.

Pinto, J. K., & Slevin, D. P. (1988b). Critical Success Factors Across the Project Life Cycle. Project Management Journal, 19(3), 67.

Shenhar, A. J., Dvir, D., Levy, O., & Maltz, A. C. (2001). Project Success: A

Multidimensional Strategic Concept. Long Range Planning, 34(6), 699-725. doi: 10.1016/S0024-6301(01)00097-8.

Sogeti. (n.d.). Template Project Start Architecture. Retrieved October 19, 2007, from http://eng.dya.info/Images/Template%20Project%20Start%20Architecture_tcm14-23439.pdf.

Veltman - van Reekum, E. (2006, January). Determining the Quality of Enterprise Architecture Products. Utrecht University. Retrieved December 1, 2007, from http://www.dya.info/Images/Thesis%20E_van_Reekum_Determining_Quality_Ent erprise_Architecture_Services%20v2_tcm13-24174.pdf.

Webster's. (1990). Webster's New Dictionary and Thesaurus Consise Edition. New York, N.Y.: Russell, Geddes & Grosset.

de Wit, A. (1988). Measurement of project success. International Journal of Project Management, 6(3), 164-170. doi: 10.1016/0263-7863(88)90043-9.

(32)

Xia, W. D., & Lee, G. H. (2005). Complexity of information systems development projects: Conceptualization and measurement development. JOURNAL OF

MANAGEMENT INFORMATION SYSTEMS, 22(1), 45-83.

Yu, A. G., Flett, P. D., & Bowers, J. A. (2005). Developing a value-centred proposal for assessing project success. International Journal of Project Management, 23(6), 428-436.

Zammuto, M. (n.d.). Soluton Architecture Template. Retrieved November 18, 2007, from http://www.michaelzammuto.com/writing/downloads/index.jsp.

(33)

Appendix 1 Project success and stakeholders

Project success is defined as “the level to which the project results satisfy relevant stakeholders” (van Aken, 2002), (Pinto & Slevin, 1988a). Whether stakeholders are relevant for determining project success solely depends on their role: are they

stakeholders in the project result or in the realisation process? Since the definition clearly states that only the results count, project team members and others involved in the realisation of the project are no relevant stakeholders. In this matter, this research takes a different standpoint than van Aken (2002, pp. 91-98) who chooses to include the project team members. This different standpoint is shared by many authors, e.g. Yu et al. (2005). Van Aken doesn’t use this definition in ways that impact his conclusions, so this

difference can safely be ignored. The only groups not covered at this point are indirect stakeholders and stakeholders in society. For this research, they are not relevant either since they are more concerned with the results of the change in a broader sense than the results of the solution as such (as described in the PSA). The relevant stakeholders can therefore safely be reduced to the users and the business executive, and project success to the level to which these two groups are satisfied with the project results. Implicitly contained in the definition of project success is the notion that project success can only be determined when the project is finished.

(34)

(35)

Appendix 2 Generic PSA contents template

A standard Project Start Architecture document (Sogeti, n.d.; Zammuto, n.d.) has the following contents. The PSAs in this research should adhere to the ASR template as defined by van Brandwijk & van den Daal (2004), consistent with the aforementioned contents.

- Author, document versioning, distribution history; - Introduction; • Objective of project • Stakeholders • Budget • Time • Related projects

• Related prescriptive / reference architectures - Overview.

- Business architecture;

• Relevant business processes and changes

• Relevant business organisation entities and changes • Relevant business products and changes

• How the architecture answers the objective (also called: fit with requirements) - Information architecture;

• Relevant information objects and usage changes • Quality requirements for information

- Information Systems architecture; • Relevant applications and changes

• Main functionality and mapping to information systems components • Structure

• Interfacing • Data • Services

• Quality requirements (mostly non-functional) and how the solution fulfils the requirements

- Infrastructure (or technical) architecture; • Relevant infrastructure and changes • Technical architecture mapping • Growth scenarios

- Security as a separate view, explaining how the solution fulfils the requirement; - Administration as a specific view, and how concerns are dealt with;

- Implementation and migration aspects; - Rationale for deviation of standards; - Design decisions (if already made);

- Issues to be dealt with in later project phases that don’t have to be answered at this point in time.

A complete PSA is one that has all elements described or explained why that particular element is omitted.

(36)

Effect. of the Document Quality. of the Project Start Architecture. on Project Success. An exploration. Master thesis

Effect

of the

Document

Quality

of the

Project Start Architecture

on

Project Success

An exploration

Master thesis

Abstract

Samenvatting

Table of Contents

1 Introduction

2 Theory

3 Research execution

4 Results

5 Conclusions

6 Reflection

7 References

Appendix 1

Project success and stakeholders

Appendix 2

Generic PSA contents template