The results of the study may be prone to threats of external validity. External validity concerns the correctness of generalising towards particular target users, settings and times. Let us look at the possible threats to external validity of the generalisation of the research results.
Target users
One of the possible threats to external validity is a choice of certain types of users, who may be not representative of the target population of potential end users. It has been attempted to collect the most representative sample available. Some limitations are noted in this respect.
The participants for the diary study (Chapter 4) were selected mainly from the em-ployees of the university campus. The reason for this was a very low reaction to the ad-vertisement placed in the local newspaper. Participation of university administrative employees was a solution. The diary study involved 10 people who cannot possibly be considered representative of a population as large as the market for online EPSs. The participants related to the university may be a rather homogenous group in many re-spects, but their involvement in the university is irrelevant to their relation to EPSs.
They were of course geographically very similar, but this seems to be a difficult effect to avoid. However, the diary study served as a data collection technique, and at-tempted to provide explanations of users’ opinions and experiences, rather than gen-eralise to a target population. The focus of the study was on the detail and depth of ex-planations, rather on the breadth of coverage. The final number of the participants was in accordance with the goals of the study.
To address this type of threat in the experimental study (Chapter 5) the users were carefully selected by the call centre of Postbank, based on the requirements of the study. The users of the nation-wide consumer survey (Chapter 3) with the sample size of more than 1300 respondents can be treated as representative of the population of potential users of payment systems in the Netherlands.
It would be possible to generalise the results to the heterogeneous population repre-sented by all participants, and not possible to single out specific subpopulations. It cannot be said that the payment system under test would successfully appeal to e.g.
just young people, pensioners, or students. It would be an error of external validity to generalise across these subgroups of the whole sample. Consequently, the DRs can be assumed to hold for the average user and not to be applied to any subgroup.
Context of use and the scope
The studies of this research attempted to be as realistic as possible. However, we should be cautious in claiming that generalisation of the results could transcend the setting and the context of the studies and be generalised to a wider range of settings, i.e. other EPSs, applications and context of use.
While admitting this, it has to be noted that the scope of the research was clearly de-fined from the very start and followed through the whole course of the research activi-ties. Moreover, this research has identified a number of cases where the context of use is highly important for certain systems’ requirements, users’ attitudes towards EPSs, and consequently user acceptance. Therefore, the implications for design can be treated as valid only for the given scope and the context of use, described in Chapter 1.
The design recommendations emerged out of the qualitative research that considered electronic payments in real life situations, with the diary study recording real pay-ments. The design recommendations were applied to a commercial payment system by the company-developer and therefore their application is tested in a realistic context.
It can be said with confidence that the validation experiment and the diary study had a quite high degree of realism. In both studies the setting was consistent throughout the process of the studies and data collection. It can be concluded that the studies in this thesis are done with the high degree of ecological validity.
6.2.3 Conclusions
Having examined a comprehensive list of potential threats to the validity and general-iseability of the research presented in this thesis, it is argued that the design knowl-edge provided is useful and valid. The research activities of this thesis have a high de-gree of realism. The research included the nation-wide consumer survey, eliciting user attitudes towards EPSs of a large sample of Netherlands-based respondents. The qualitative diary study was able to investigate the actual user experience with online EPSs, and has provided grounded data, used for the hypothesising of the design rec-ommendations.
For the validation of the design recommendations it was possible to form an alliance with the actual developers of an EPS and validate the DRs with a commercial payment system. The outcome of the experiment makes it possible to draw conclusions about the validity of certain DRs and the possibility to use them for the design of e-commerce EPSs. The high ecological validity and realism of the studies allow us to conclude about the success of this research.
6.3 Future work
This research attempted to embrace a wide spectrum of possible issues with user ac-ceptance of e-commerce EPSs. Future research may focus on the further development and validation of the concept of user acceptance of EPSs. For instance a model of user acceptance of e-commerce EPSs may be developed and validated to become a reliable tool for gauging user acceptance of electronic payment systems and similar related technology. Future work can be concentrated on the validation of specific factors that can influence user acceptance. It can concern itself solely with just one of the issues, e.g. privacy, trust or security, usability.
Of course, the most natural continuation of this research would be to take the design recommendations even further. They can be further validated, enhanced and substan-tiated in the context of actual use or in larger scale experiments. It would be an inter-esting long term study to observe the effect of the design recommendations in a real life system on the market, to observe their relevance in a longer span of time, and to track down their development.
Further work on the design recommendations should try to resolve potential threats to their validity. On the one hand, the design recommendations can be used differently than in the presented study. Another way of the application and implementation of the design recommendations can improve a chance to avoid mono-operation bias, i.e. ap-plying the design recommendations only in one way. This can also help to refine the details of the design recommendations. On the other hand, the design recommenda-tions should be applied to other types of payment systems in order to avoid mono-method bias that could emerge if applying the DRs only to one type of EPSs. While the system used for the experimental study suits the scope of this research well, it would be interesting to test the design recommendations with a different type of payment systems. This will allow generalising the validity of the design recommendations to different EPSs and contexts of use.
A promising direction of future research is developing a system for evaluation of EPSs.
This direction presumes creating evaluation models, methods, tools and techniques, etc. For instance, heuristic evaluations or checklists can be created for revealing prob-lems with EPSs at the design stage, paving the way for improvements and changes in the current and future systems. These evaluation methods and tools can be then vali-dated empirically.
In conclusion, future research has a great number of exciting opportunities. It can transcend the field of online EPSs and delve into other areas of e-commerce and future information technology.