Data Sources

“All is data” as long as it is relevant to the substantive area (Glaser, 1998). Throughout this research, the researcher used five main sources of information.

As the initial source of information, the researcher gathered more than 350 project documents. They covered all phases of the CRM initiatives, from the early stages of project initiation until the end of the research projects. The researcher signed a non-disclosure agreement with each of the three companies in order to benefit from unlimited access to their documents. Analyzed documents included CRM project presentations and progress reports made by the project manager, Microsoft Visio process maps, specifications and results of the 'proof of concept' phase, business requirements documents such as the project charter and the Request for Proposal (RFP), minutes of interviews held by the project team with the main stakeholders, presentations of the legacy systems to be replaced, workshops minutes, organisational charts, Microsoft Project plans, technical documentation (technical architecture, data models, data governance, integration with the company's other information systems, customization of the CRM system, data migration plans, security/access management issues, bugs and change requests reports), new functionalities required by users after the launch of the CRM system, emails exchanged during the course of the project and stored by the project team, user training documentation, and finally User Acceptance Testing (UAT) reports. The

researcher stored all documents and corresponding memos in QSR NVIVO, and used this abundant documentation in two ways. Firstly, he read the available documentation before the first round of interviews in order to familiarize himself with the project, the actors, the deliverables and the issues faced during and immediately following deployment. When necessary, he asked project team members and users to provide additional information on document contents and on his initial comments and analysis. Secondly, information contained in documents helped the researcher triangulate with other sources of information such as users’ interviews and log data. The researcher did not perform a quantitative content analysis on those documents. However, he extracted open codes from them in the same way as he did for transcripts and memos. All documents gathered in this research were grouped in three main directories (Ozkan, 2004). The ‘internals’ directory included primary source materials such as audio interviews and their transcripts, field notes taken during interviews and meetings, and key papers relevant to this thesis. The ‘externals’ directory was composed of secondary documents such as general company information (e.g. annual reports and newspaper articles), training documentations, presentations made at the company’s local and global headquarters, interview templates, and results of personal research conducted on GT and information system usage. The ‘memos’ directory contained the records of the researcher’s thoughts and observations and was organised around a project journal composed of a day-to-day research diary, as well as conceptual, explanatory and theoretical memos.

Face-to-face interviews served as the second source of information. The researcher chose the interviewee profiles (departments, job functions and seniority) after reading the project documentation and defining who the main user groups were. The interviewees’ population of each research project included a wide cross-section of users in terms of age (from mid-20s to mid- 50s), seniority in the company (from two months to 12 years), and position (from junior staff to executive committee members).The researcher made sure to have a balanced representation of each of the departments involved in the CRM initiative. Whenever it was possible, he selected at least two participants per department in order to be able to cross-check their comments. The researcher held 135 interviews at the three research sites, with an average interview

duration of about 50 minutes. He based the initial round of interviews on the semi-structured questionnaire initially designed for the first case study.

The third source of information was the researcher’s observation of participants working with the CRM system. Observation was mainly conducted as a follow- up to the interviews when users showed the researcher CRM system functionalities they used and the business processes automated by their new tool. The researcher did not conduct systematic observation of daily usage. Out of the 135 face-to-face interviews, 25 were followed by a demonstration of the CRM tool. Users were usually keen to show the CRM system in order to highlight issues such as user interface or data quality problems, but also benefits such as improved process efficiency or new online reporting and analysis capabilities.

The fourth source of information was CRM systems logs. The researcher tracked individual usage (e.g. who uses the system, which functionalities are used, how often each functionality is used). Monthly data were collected by the project manager, sent to the researcher, and stored in Microsoft Excel for further analysis. The researcher used CRM system usage statistics to track usage evolution of individual users, cross-check users’ statements about their use, and trigger discussions during interviews. Whenever possible, the researcher decided not to gather self-reported usage data directly from CRM system users to avoid response bias and to allow him to gather a significant amount of data over a fairly long period without having to re-interview staff. The researcher gathered tool functionality usage and productivity gains through the CRM system logs. Table 1 presents a summary of the collected statistics for the first case study. The researcher collected measures related not only to the frequency of use or the number of functionalities used but also to individual productivity and business benefits. This fourth source of information brought multiple new insights into individual system usage. For example, Table 1 clearly shows that even though the TMS is used more frequently (items 1-3), better supports users’ tasks and increases their productivity (items 4, 5 and 6), it does not provide significant business benefits at company level (item 7) after 19 months of operation. The researcher tried to systematically verify the statistics during interviews. The following excerpt from the interview with a trade marketing manager (TMM) illustrates the difficulty in moving beyond the

individual productivity phase and generating business benefits such as improved sales or market share, and therefore seems to corroborate the collected statistics.

“The benefits of the Siebel introduction were not really visible to me in 2008; they only came with the availability of better reports in 2009. Reports and dashboards bring more visibility on my business. I can better track the activities of my staff … Does it help me achieve my objectives? Well … not really but I am not now wasting time collecting data from different sources, they are almost all there in Siebel. For my staff, the functionalities of the analytics module such as trade segmentation and improved access to information have made them more professional, especially in front of customers. Unfortunately, data synchronization (updates) can only be done at the office during the night, therefore when my TMAs do not come back to office for a while they have outdated data… I believe we are not yet at the level where we should be with such a tool: I do not see any real benefits except increased productivity.”

The above statement is partially confirmed by one of his employees, who states that:

“I do not see much benefit from using Siebel for top management. Most benefits are for TMAs like me in terms of improved access to data and a better planning of activities. The other main benefit should be reporting but the module is so slow and difficult to use that this discourages people from using those analytics functionalities. I am more efficient, that’s it.”

Statistics Focus is on May- July ‘08 Aug- Oct ‘ 08 Nov’ 08- Jan ‘09 Feb- Apr‘ 09 May- July ‘09 Aug- Oct ‘09 Nov ‘09 1 Nbr of Analytics users logged during period Functionalities used 7 18 24 27 28 29 31 2 Nbr of dashboards available to users Functionalities used 24 20 27 29 40 68 82 3 Nbr of activity types created by users (to report customer visits) Functionalities used 12 12 14 15 14 14 13 4 % of planned visits completed by field staff Individual productivity n/a n/a 79% 75% 79% 93% 95% 5 Nbr of visited outlets by field staff Individual productivity 7’514 8’027 8’960 9’520 10’038 10’302 10’021 6 Task accomplishment (planning, reporting, transactions) – qualitative Individual productivity +: improving -: deteriorating 0: stable - + + + 0 0 0

assessment. 7 Local market

share (%)

Benefits 27.8 26.5 25.7 25.2 25.1 24.5 n/a Table 1. Quarterly TMS Usage Statistics (Case 1)

The fifth and last source of information was the researcher’s participation in the company’s internal meetings such as training sessions, project steering committee meetings, staff presentations about the CRM initiative, and department meetings. The researcher had access to all materials presented by the project team during these meetings.