Chapter 4: Methodology
4.4 Data: reliability and validity
To grasp the full picture of the labor market intervention, data must be reliable and valid. Reliability refers to consistency. In this research, it includes consistency both across items and across different researchers. The effect of the LCL includes both direct and indirect impacts. It demonstrates that the result from the three datasets in this thesis match each other perfectly over different time periods. In addition, the results of the qualitative study support the findings of the quantitative studies. This consistency in the overall data shows the reliability of the sample in this research.
There are two major sources of reliability in the interview data: intended and unintended. To avoid unintended errors, such as clerical mistakes and ambiguous questions, it is important to be careful in designing and recording the data. However, both the interviewer and interviewee can be dishonest and create a biased result. Ethics training and other tests or recordings can be adopted to restrain the interviewer from intentional manipulation of the data. On the other hand, as LCL is a sensitive subject with possible illegal activities, interviewees may worry that the answer they provide could damage their safety. According to Salkind (2009), one of the basic assumptions in a reliable survey is that the questionnaire does not put unreasonable demands on the respondent. Alkin, Daillak, and White (2006) suggest that trust in the researcher is of equal importance to the adequacy of the procedures themselves. Therefore, I tried to build personal relationships with the interviewees to improve trust, in order to increase the reliability of the data. Otherwise, refusal of the interview would increase. Other solutions involved clarifying the purpose and safety procedures of the research with the interviewees. A snow-ball sampling technique was attempted in the fieldwork. It was aimed to not only expand the sample size, but also to improve the trust level between
the interviewer and interviewees. However, in practice, the amount of information provided by an interviewee is highly reliant on the personal relationship. In general, the younger generation are more vocal than their parents on this sensitive matter and retirees are more willing to provide information than those still in the workplace. Discourse analysis has been used to determine the true meaning of their responses.
Validity is a measure of whether research actually studies what it purports to study. As reviewed in the literature review section, the current literature only measures part of the direct costs of LCL, which indicates that it suffers from validity problems. Hence, to study the full-impact of LCL, both direct and indirect costs of LCL must be included.
The economic agent that decides the use of resources, including labor, is the owner of the firm. Workers may be experts in a specific production process, but they don’t always have information on the firm’s organizational decisions. Thus, to understand the economic impact of LCL on firms, such as investment decisions and contractual arrangements, the key stakeholder is the business owner. Intensive interviews have been used to collect data about the indirect impacts of LCL. In this case, the population is heterogeneous and contains different groups with inequality of information. Certain groups, such as lawyers, local officers and owners possess more knowledge of LCL than others. Thus, stratified sampling has been adopted to collect valid information. As firms with various backgrounds and characteristics behave differently with the impacts of LCL, the sample has been collected accordingly to provide valid results.
Appendix 1:
The benefits of this research may not accrue to each individual. Most participants in this research, such as businessmen and workers, could financially benefit from the de- regulation of the law. Thus, these stakeholders have strong incentives to participate in the research to make their voices heard. However, the data collected from the interviews can be sensitive, which may lead to personal loss by the interviewee, if the information was exposed to the public. These risks include financial risk, legal risk and social risk. To minimize these risks, confidentiality is key.
Only the nominated researchers have access to the material provided by the interviewees and confidentiality has been protected as far as the law allows. In order to protect the interviewees, the data collected has been modified, coded and saved confidentially, without personal information that can be traced back to the interviewees. For instance, instead of recording the specific industry, such as footwear manufacturing, I have recorded manufacturing as the industry. If a firm is located in Qingdao city, I record the location as a large city in North China. The large population of China also reduces the possibility of others identifying the interviewees. Moreover, businesses in
different locations in China have similar methods to get around the law; the homogeneous nature of the information increases the difficulties in locating interviewees personally. In publications, any personal information, such as full names, that could be potentially traceable to interviewees has been deleted to avoid personal damage to the interviewees. Finally, the information collected from each individual interviewee has been broken down and re-organized into different topics in my papers, which will make it difficult to re-identify the interviewee. For instance, the 20 to 30 firms in my sample have been used to study the increased use of informal workers or machinery after the law, without any identifiable information for each firm.
Following the rules of the ANU Code of Research Conduct, the data has been stored at the ANU. Storage on any computer is password-protected. All information has been securely backed up on University computers. Normally data must be stored for a period of at least five years from the date of any publication arising from the research. I have followed the standard practice. At the end of the storage period the data will be used for future research. As this use does not expose participants to new or additional risk, after the data is no longer needed for the current research, it may be archived or retained in a de-identified format by me, if the participants agree with this usage.
The interview data has been mainly used in Chapter 7, which studies capital deepening with a case study. The information related to changes in the production function is not very sensitive as the Chinese government openly supports the use of more machines in production. On the other hand, the interview data related to changes in contractual arrangements to avoid the impact of the LCL has not been directly quoted in this thesis, as the information includes illegal activities.