An Approach Towards Designing A Prescriptive
Analytical Logic Model for Software Application
Root Cause Analysis
Hoo Meng Wong
Sagaya Sabestinal Amalathas Tatana Zitkova
Abstract: Most of the business operations utilize software application heavily in today’s business world. Software application cannot afford to stop its functioning in the production environment during business hours. However, it is hard to restore the software application ser vice without identifying the root cause. Software application error can occur either within the software application layer or any other factor outside the software application layer. With an enterprise level of software application, the complexity is high as it involves multiple tiers and many middleware are running in the environment. The root cause analysis of software application will be easily prolonged due to more than one log file required. The duration on restoring the software application service back to the users can be hard to determine whenever error is occurred. Therefore, the objective is to mitigate prolonging on conducting root cause analysis activity. Indeed to identify the root cause of software application error in a more accurate manner, and shorten the duration of root cause analysis activity conducting on software application error. A Prescriptive Analytical Logic Model incorporates with Analytic Hierarch y Process (AHP) is proposed. This logic model contributes a new knowledge in the area of log file analysis to shorten the total time spent on root cause analysis activity. At the same time it contributes the knowledge in AHP to close the knowledge gap.
Keywords: Application log analysis, Log file analysis, Analytic Hierarchy Process.
1. INTRODUCTION
The computing era evolved since data-processing until today, and software application is still required to host on a server box (either on a physical server box or a virtual server box). Server is required to connect with network communication, it is whether a Local Area Network, LAN, or a Wide Area Network, WAN. At the same time server is required to store its data at the location either in a file or in a database. The database is another software application that focuses on handling and processing data into collection of information under an organized manner for accessing. This database can be hosted on another server box. Due to the complexity of having hardware (physical and virtual hardware), Operating System, Networking for software application communication, and database to allow the software application to store its data. This would make the support team facing much more challenge and difficult to identify the valid software application error and to keep up the software application execution up-time to support the business operation. As per the figure 1.1, Operating System is the middle layer that communicates between software application and server resources. Software application has to interact with Operating System to obtain allocated server resources such as CPU, Memory and Hard Disk Space to handle software application processing, which means software application has highly dependency on server resources. Without the server resources, software application will not execute by itself. By depending software application log file alone to conduct root cause analysis would definitely be insufficient whenever error occurred beyond the layer of software application. Hence a proposed research for a new model is required.
Figure 1.0 - Software Application is required sufficient server resources to execute all its functionality.
2. Background of the Study
problem statement, there are comments obtained from the Internet search regarding to the time consuming on application trouble-shooting, which are REDDIT (2015) and StackOverflow (2015). Note that the total time taken to resolve the software application error consists of the duration for conducting the root cause analysis activity and the duration for applying the resolution. This approach may or may not involve the Software Development Life Cycle (SDLC) activity for developing the fix by depending on the requirements of the fix. Hence, the proposed resolution is focusing on shorten the time consumption duration on conducting the root cause analysis activity.
3. Scope of the Study
To the best of my knowledge, whenever a software application is executing, it logs events including error event into its log file. However anything that happens beyond the software application boundary, the software application may not be possible to log that kind of events into its log file. The supporting reason can be in a situation that server is lack of server resources such as CPU, memory, hard disk space, or break down in network communication and etc. The important point to stress out here is that if the root cause analysis activity is solely focusing on software application log file alone as the input information, it is insufficient. Under such circumstance, multiple logs or input information from different software applications are required. The input information can be:-
Performance and Capacity monitoring system,
Server and Networking monitoring system, and
Configuration Management system.
By obtaining all the required log files from various software applications, this activity must not create any disturbance to the involved software applications during activity execution. Hence the exact amount of logs from various software applications or databases must be exacted and integrated to form a dataset for the input information of the root cause analysis. Indeed, the objective of this proposed research is to shorten the duration of conducting root cause analysis, and the sub aim is to identify the related log files as input data of the root cause analysis accurately. Finally the output is the Prescriptive Analytical Logic Model. The objective is sat based on a problem, which is the longer duration for conducting root cause analysis, and how it can be quantified to determine that it is a real problem. Therefore, a simple mathematics approach can be used to evaluate the statement of problem.
4. Significant of the Study
The proposed design of the logic model basically is looking for the contribution and the benefit that can be gained from the research. As for the contribution to the business, by lower down the risk of the software application error, and improve the reliability of utilizing the software application would bring business advantages to compete in today’s business industry. This is because with today's rapid business competitive world, time consuming on analysis and trouble-shooting activities is unacceptable. This is a continuous battle for the support team to face day-to-day software application error challenge in order to provide reliable up-time for the software application utilized in the
business organization. On the other hand, business companies can still continue to utilize their existing software applications (without incur any additional operation budget) and at the same time to allow the companies to save the investment budget on spending the capital amount to replace all or partial of the software applications and re-training their users on using the new software applications. This propose model not only can bring the above benefits to business industries but other industries which are using software application for their daily operation, they need to have the software application error fixed without further re-occurrence. Furthermore, it is also good to look at the perspective of the technical beneficial. Over the years there were various researches had been done at this area such as consolidate the logs or integrate the logs for analysis but there had been very attempts to propose a model to deliver a complete package for analyzing and fixing software application error which consists of the activities such as log integration, error analysis, decision making of preferred resolution, and automated on applying the error fix. There is a great potential in this research which brings contribution to business intelligent studies.
5. Statement of the Problem
To conduct the root cause analysis activity on software application error, it is time consuming to identify the valid error.
6. Research Objectives
The primary objective is to mitigate prolonging on conducting root cause analysis activity. There are also different perspectives along with the research objective. There are shown as follows:-
Business Perspective
i. To reduce the duration of root cause analysis activity and lead to shorten the total time taken duration of software application downtime.
ii. To mitigate the same software application error for re-occurrence.
iii. To shorten the total software application downtime percentage in a financial year.
Technical Perspective
i. To shorten the unnecessary time consuming on conducting root cause analysis.
ii. To improve accuracy to identify the software application error's root cause.
iii. To identify the valid software application error accurately by applying Analytic Hierarchy Process (AHP) for decision making process.
iv. To decide the best resolution to the software application error by applying Analytic Hierarchy Process (AHP) for decision making process. v. To improve root cause analysis experience with
the input of past analyzed result and resolution activities stored in the knowledge-based database.
Knowledge Contribution Perspective
scripting language. Which means each module will be carried out specific set of functions, and all these modules will be utilized to form a complete root cause analysis activity towards to achieve the outcome which is to shorter the analysis time spent duration.
7. Literature Review
Stewart, D (2012) is focusing on debugging real-time software application error using logic analyzer debug macros. Eick , S, Nelson , M, and Schmidt, J (1994) they are focusing on presenting the error logs in a readable manner. Wendy, P and Dolores, R (1993) suggested to focus on error detection in software application at the time of software development and maintenance. Salfner, F and Tschirpke, S (2015) are focusing on analyzing error logs by applying the proposed algorithms in order to predict future failure. Murínová, J (2015) had attempted to integrate multiple log files from various software monitoring tool and network devices for better root cause analysis on Web application error, however there is no proposed model stated in his research. Only Valdman (2001) had showed more detailed feature whereas other studies indicated intentions or suggestions. By reviewing the mentioned approaches and suggestions on conducting root cause analysis activity on software application error, mainly it is either focus on software application error log file alone for root cause analysis activity, or suggested the software application error analysis activity should be built in during the software development process. Indeed, these approaches and suggestions are narrow if the software application root cause is beyond the software application layer. On the other hand, based on the comparison of market log file monitoring product such as Solarwinds Log Analyzer, Logstash and Cisco AppDynamics that they
claim to have predictive analytic ability, and create a graphical visualization for trending report. However, any error beyond the common errors, the decision making to identify the valid error is still on human decision. On the AHP side, Vaidya and Kumar (2004) had reviewed Analytic Hierarchy Process (AHP) with 150 application papers in total, and there is no knowledge in applying AHP on identifying valid software application error. Since there is no published article showing that AHP had been applied to software application error analysis, the proposed research is to fill up the knowledge gap.AHP was developed by Thomas L. Saaty stated in Wikipedia (2015), with the proposed algorithm of the prescriptive analytical logic, the proposed algorithm can utilize and apply the theory of AHP to shortlists the best resolution, and to be proposed or to be applied against the software application incident automatically or manually. By referring to R. W. Saaty (1987), the proposed algorithm can apply the three principles in identifying valid software application error, and on the other hand in identifying preferred resolution. The three principles which are the decomposition principle, the comparative judgments, and the synthesizing priorities. In addition, Vaidya and Kumar (2006) had provided the discussion on how to apply AHP under the analytic hierarchy process, and this would help us to understand how the proposed algorithm would apply it in the scenario after identifying the valid software application error and identifying preferred resolution during the analysis process under the proposed model behind the prescriptive analytical logic.There are published articles and journals available for the analysis and studies at the area of AHP, which are good to investigate how the past suggestions and techniques of AHP can be adopted and utilized similarly in this proposed research activity.
No. Title Author Analysis and Discussion
1
Analytic Hierarchy Process: An overview of applications
Omkarprasad S. Vaidya and Sushil Kumar, 2004.
This is a review paper for the applications of Analytic Hierarchy Process (AHP) with 150 application papers in total are referenced and reviewed. Carefully went through the detail of the review paper and cross checked the references of this paper, there is no published paper discussing about Log File Analysis using AHP approach for decision making process.
2
A Spatial Analytic Hierarchy Process for Identification of Water Pollution with GIS Software in an Eco-Economy Environment
Razvan Serbu, Bogdan Marza and Sorin Borza. 2016.
This paper used AHP multi-criteria method to evaluate each sampling station’s physicochemical parameters, and based on the obtained result to place respective river in the water quality classes. The research was using GIS software which was allowed an integrated automatic data collection system and displays interactive results.
3
Applying the Analytic Hierarchy Process (AHP) to build a strategic framework for technology
roadmapping
Nathasit Gerdsri and Dundar F. Kocaoglu. 2007.
This paper introduces the Technology Development Envelope (TDE) with the AHP as part of the proposed framework that structured by obtaining strategic information on the development of technologies. Then, this strategic information can be utilized to evaluate the value of each technology based on the impacts of its characteristics on an organization’s objectives.
4
Applications of the extent analysis method on fuzzy AHP
Da-Yong Chang. 1996. This paper introduces the use of triangular fuzzy numbers for pairwise comparison scale of fuzzy AHP as the new approach.
5
Decision making with the analytic hierarchy process
Thomas L. Saaty. 2008. This paper introduces and explains AHP as a theory of measurement through pairwise comparisons and relies on the judgements of experts to derive priority scales.
6
The Analytic Hierarchy Process - What It Is And How It Is Used
R. W. Saaty. 1987. This paper introduces the AHP as a measurement on ratio scales, and discuss some of the ideas relating to AHP process and ramifications.
7 The Analytic Hierarchy Process:
Dr. Benito L. Teehankee and De La Salle
Capturing Quantitative and Qualitative Criteria for Balanced Decision-Making
University. 2009. a meaningful and rational way in a decision situation.
8
Using the Analytic Hierarchy Process for Decision Making in Engineering
Applications: Some Challenges
Evangelos Triantaphyllou and Stuart H. Mann. 1995.
This paper is from the engineering application perspective to examine some of the practical and computational issues are encountered when the original AHP method is applied. In addition, the paper also discussed that using Revised-AHP (was introduced by Belton and Gear (1983)) to overcome the deficiency of the original AHP when the original AHP was used in the pairwise comparisons. Further more, on the Multi-criteria decision-making (MCDM) problem whenever alternatives appear to be very close with each other, this paper suggested to use Revised-AHP as it would be the most efficient method to examine in such situation.
Table 1.0 – Analysis and discussion of published articles related to AHP approach.
After investigating the area of knowledge in AHP based on the publications, so far there is no knowledge in applying AHP on identifying valid software application error. This has clearly shown a strong support reason to conduct the proposed research in this area in order to fill up the knowledge gap, which is applying AHP on software application error analysis to decide the valid software application error. Hence, with the Prescriptive Analytical Logic Model, it carries high potential on knowledge contribution as the new logic model has the algorithm to be able to identify the root cause of the software application error more accurately under the AHP processing approach, and deliver the outcome to shorten the analysis duration of conducting root cause analysis activity during the software application downtime
8. Proposed Design
The research is to propose a Prescriptive Analytical Logic Model design that consists of a set of proposed algorithm. It is used to analyze the root cause of the software application from the log events retrieved from various software
application log files. With this proposed logic model design, it is aiming to resolve the statement of problem.
Figure 2.0 - The proposed software application analysis algorithm will analyse across multiple databases.
The proposed algorithm under the Prescriptive Analytical Logic Model includes the following crucial activities:-
Activity Algorithm
1 By given the permission, retrieve the log events from the involved software application log files.
2 Identify whether the newly reported software application error is first time occurrence or re-occurrence by cross-checking the database which is associated to the prescriptive analytical logic.
3 Identify possible log data and select the necessary log data for analysis under the defined software application error classification. 4 Allocate weight to each possible software application error based on Analytic Hierarchy Process (AHP).
5 Shortlist the software application error under the highest weight.
6 Analyze the selected log data for shortlisted software application errors and define possible resolution option. 7 Allocate weight to each possible resolution option based on AHP.
8 Shortlist the preferred resolution option under the highest weight.
9 Deploy the preferred resolution option to fix the software application error under the predefined condition.
10 Store the analysis result and resolution action into a database which is associated to the prescriptive analytical logic for future reference and knowledge base activities.
Table 2.0 - Proposed process activity under proposed algorithm.
The software application error can be categorized under the predefined categories of possible errors depending on the error aspect. The objective is to focus within known category of error and narrow down the root cause. There are changes that error is hard to determine and hard to match into any known error category. Under this circumstance it is
analyzed result for the same issue retrieved from the knowledge base database. The analysis process will identify the need of additional required information retrieved from various software application databases. These software application databases are Configuration Database, Production Support Ticketing System Database, and Application and System Monitoring Database, and etc). With all the obtained information, the analysis process must identify the possible involved areas with this error. Whether it is solely being fixed at the software application layer, or other layers such as:-
Hardware capacity layer (i.e. CPU, memory, hard disk),
Operating System layer,
Network layer,
Database layer, or
Any combination which more than one layer.
Again, the analysis process will identify the need of additional required information retrieved from specific or various log files from different layers. There is also circumstance that when a major software application error occurs, it would also rigger a list of related and even non-related errors or warning messages in the software application log, with such the analysis process will need to retrieve software application log information under a specific time duration from the software application log file as for the input information of the analysis process. The AHP process is applied to allocate weight to the software application error as well as errors found from different layers under the same “timing”. Therefore, AHP process is playing an important role to allocate weight to each possible error based on the error aspect. Whenever the weight is allocated on each error, the AHP process will shortlist the error under the highest weight. Once the shortlisted error is determined, the AHP process will proceed to evaluate all the possible resolutions based on the weight allocation to each possible resolution. Finally, the AHP will shortlist the resolution based on the highest weight. If the possible resolution is more than one, then the PAL will generate a report by providing the possible resolution option. The PAL is designed to have two configuration files, one is designed to record down the common errors and room to cater for new found error. Then, another one is designed to record down the resolution steps for the common errors as well as room to cater for human to provide new resolution steps based on the new found error. These preferred resolution actions under the resolution are manual, semi-automatic, and fully automatic. The PAL must obey the configurations in order to carry out the resolution under the preferred actions. Especially for the “Complex Analysis”, it is required to store the analysis result and the resolution action into a knowledge based database which is associated to the PAL for future reference. Regardless the analysis activity is under simple analysis or complex analysis, the knowledge based is catered for upcoming analysis activity and fine tuning the logic. It is mainly for improving the decision making on selecting the final resolution to be applied to the new found error in the near future. Lastly, from a training purpose perspective, since the knowledge base of the proposed PAL will play an important role for the entire analysis process during the error analysis activity. It can be utilized as training material to train any new joining IT support engineers to pick up the support experience rapidly. At the same time it benefits for those who are experience support engineer to improve the
knowledge of software application support. To ease of retrieving the information from knowledge base, the knowledge base database can be either stored locally at the database server or in the cloud. This is because to access the knowledge based in the cloud, the information is accessible at any time.
9. Prerequisites of the Proposed Model
Multiple Tiers Environment
In this proposed research, it is required to run on a multiple-tier environment which involves Web tier, application tier, and database tier. The proposed algorithm will be implemented into a software plug-in component sittplug-ing at the logic tier to plug-integrate all the required log information from various software applications and store in a separate location for information retrieval later. This will help on retrieving log event from log file as input data from various software applications when the root cause analysis activity is triggered at the logic tier.
NTP Server
Whenever a software application involves in a multiple-tier environment, it is good to standardize the server time across all the involved servers. The supporting reason is that the proposed Prescriptive Analytical Logic Model is required to collect log data from different software applications based on the specific given time as the primary key, and therefore the server time on all the involved servers are required to have the network time synchronization. As per Masterclock (2016), “Accurate time stamping is key to root-cause analysis, determining when problems occurred and finding correlations. If network devices are out of sync by a few milliseconds or, in extreme cases a few seconds, it can be very difficult for network administrators to determine the sequence of events.”.
Technologies of the Proposed Model
The proposed prototype of the software plug-in component will be coded using Java Programming as this programming language is platform independent. At the platform, it required to install Java Runtime Environment (JRE) and configure the Java home path and memory allocation for the JRE.
10. Conclusion
duration of conducting root cause analysis, and the sub aim is to identify the related log files as input data of the root cause analysis accurately. Finally the output is the Prescriptive Analytical Logic Model (PAL). The objective is sat based on a problem, which is the longer duration for conducting root cause analysis, and how it can be quantified to determine that it is a real problem. Therefore, by using a simple mathematics approach, it helps to evaluate the statement of problem.
References
[1] Hoo Meng W., Amalathas S.S. (2019). A New Approach Towards Developing a Prescriptive Analytical Logic Model for Software Application Error Analysis. In: Silhavy R., Silhavy P., Prokopova Z. (eds) Intelligent Systems Applications in Software Engineering. CoMeSySo 2019 2019. Advances in Intelligent Systems and Computing, vol 1046. Springer, Cham. DOI: https://doi.org/10.1007/978-3-030-30329-7_24. https://link.springer.com/chapter/10.1007/978-3-030-30329-7_24
[2] Hoo Meng Wong and Sagaya Sabestinal Amalathas. (2019). An Approach towards developing an Algorithm for Software Application Error Analysis. David Publishing Company. DOI:10.17265/2328-2185/2019.04.006.
http://www.davidpublisher.com/index.php/Home/Articl e/index?id=39805.html
[3] Wikimedia Foundation Inc. 2017. Logfile. https://en.wikipedia.org/wiki/Logfile (Accessed on 19th February 2017).
[4] Loggly Inc. 2017.
https://www.loggly.com/blog/measuring-the-impact-of-logging-on-your-application/ (Accessed on 19th February 2017).
[5] Archanaa Panda. 2011. High Performance And Smarter Logging. DZone. https://dzone.com/articles/high-performance-and-smarter (Accessed on 20th February 2017).
[6] Linda V. Ornelas. 2003. SANS Institute. Important of
Event Logging.
https://www.giac.org/paper/gsec/3297/importance-event-logging/105437 (Accessed on 23th February 2017).
[7] Edward Mercer, 2015. How Technology Affects Business Operations. OpposingViews.com. http://science.opposingviews.com/technology-affects-business-operations-1659.html (Accessed on 28th November 2015)
[8] SuccessFactors, 2015. Using Technology to Increase
Your Business Productivity.
https://www.successfactors.com/en_us/lp/articles/using
-technology-to-increase-your-business-productivity.html (Accessed on 28th November 2015) [9] Alex Bloom, 2014. The Cost of Application Downtime
is Priceless. StatusCast.
http://www.statuscast.com/cost-application-downtime-pricess/ (Accessed on 28th November 2015)
[10] Hasan, Mohamad K., 2010. A Framework for Intelligent Decision Support System for Traffic Congestion Management System. Kuwait city, Kuwait. http://www.umsl.edu/divisions/business/pdfs/Mohamad %20Hasan%20IDSS.pdf (Accessed on 13th November 2015)
[11] Kristof Horvath, 2015. Using Root Cause Analysis to Drive Process Improvement. Intland Software. http://intland.com/blog/safety-engineering/using-root-cause-analysis-to-drive-process-improvement/ (Accessed on 28th November 2015)
[12] Management Logic, 2012. Root-Cause Analysis. http://www.management-logic.com/toolbox/sales/Root-Cause%20Analysis/Index.html (Accessed on 28th November 2015)
[13] Margulius, David. 2006. Tech jobs take stress to whole
new levels. InfoWorld, Inc.
http://www.infoworld.com/article/2655363/techology-
business/tech-jobs-take-stress-to-whole-new-levels.html (Accessed on 13th November 2015) [14] REDDIT. 2015. Experienced Dev's: How much of your
time do you spend troubleshooting? https://www.reddit.com/r/webdev/comments/3sldcc/exp erienced_devs_how_much_of_your_time_do_you/ (Accessed on 14th November 2015)
[15] StackOverflow. 2015. How much time do you spend in
production troubleshooting?
http://stackoverflow.com/questions/1425069/how-
much-time-do-you-spend-in-production-troubleshooting (Accessed on 14th November 2015) [16] Wikipedia, 2015. Analytic hierarchy process.
Wikimedia Foundation, Inc.
https://en.wikipedia.org/wiki/Analytic_hierarchy_proce ss (Accessed on 13th November 2015)
[17] www.evolven.com, Labels: Data Center, Downtime. 2014. Downtime, Outages and Failures - Understanding Their True Costs. Evolven Software. http://www.evolven.com/blog/downtime-outages-and-failures-understanding-their-true-costs.html (Accessed on 13th November 2015)
[18] SANS Institute. 2001. Importance of Understanding Logs from an Information Security Standpoint.
https://www.sans.org/reading- room/whitepapers/logging/importance-understanding-logs-information-security-standpoint-200 (Accessed on 6th March 2017)
[19] Andreas Grabner. 2012. Dynatrace LLC. Top Performance Mistakes when moving from Test to
Production: Excessive Logging.
https://www.dynatrace.com/blog/top-performance- mistakes-when-moving-from-test-to-production-excessive-logging/ (Accessed on 6th March 2017) [20] Robert Rinnan. 2005. Benefits of Centralized Log File
Correlation. Gjøvik University College. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.121.8787&rep=rep1&type=pdf (Accessed on 16th March 2017)
[21] Wikimedia, 2016. Electronic data processing.
Wikimedia Foundation, Inc.
https://en.wikipedia.org/wiki/Electronic_data_processin g (Accessed on 4th March 2018)
[22] Wikimedia, 2017. Service-level agreement. Wikimedia Foundation, Inc. https://en.wikipedia.org/wiki/Service-level_agreement (Accessed on 21st August 2017) [23] ResearchGate. 2019. How would you define a "model"
within a theoretical research? https://www.researchgate.net/post/How_would_you_de fine_a_model_within_a_theoretical_research
[24] WhatIs.com. 2019. framework. TechTarget. https://whatis.techtarget.com/definition/framework (Accessed on 17th February 2019)
[25] Collins English Dictionary. 2019. Definition of 'model'. https://www.collinsdictionary.com/dictionary/english/m odel (Accessed on 17th February 2019)
[26] David B. Stewart, PhD. 2012. Troubleshooting real-time software issues using a logic analyzer. InHand
Electronics, Inc.
http://www.embedded.com/design/debug-and- optimization/4236800/Troubleshooting-real-time-software-issues-using-a-logic-analyzer (Accessed on 12th December 2015)
[27] Felix Salfner and Steffen Tschirpke. 2015. Error Log Processing for Accurate Failure Prediction. www.usenix.org.
https://www.usenix.org/legacy/event/wasl08/tech/full_p apers/salfner/salfner_html/ (Accessed on 12th December 2015)
[28] Gerald Stermsek, Mark Strembeck, Gustaf Neumann. A User Profile Derivation Approach based on Log-File Analysis. Institute of Information Systems, New Media Lab Vienna University of Economics and BA, Austria. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.141.6449&rep=rep1&type=pdf (Accessed on 20th February 2017)
[29] James H. Andrews. 1998. Theory and practice of log file analysis. Dept. of Computer Science, University of Western Ontario. London, Ontario, Canada. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.44.292&rep=rep1&type=pdf (Accessed on 20th February 2017)
[30] James H. Andrews. A Framework for Log File Analysis. Dept. of Computer Science, University of Western Ontario. London, Ontario, Canada. https://pdfs.semanticscholar.org/9d1a/97988d8b41354c d4bf85ace96648d1684555.pdf (Accessed on 20th February 2017)
[31] Jan Valdman. 2001. Log File Analysis. https://www.kiv.zcu.cz/site/documents/verejne/vyzkum /publikace/technicke-zpravy/2001/tr-2001-04.pdf (Accessed on 18th January 2017)
[32] Júlia Murínová. 2015. Application Log Analysis. http://is.muni.cz/th/374567/fi_m/thesis_murinova.pdf (Accessed on 18th January 2017)
[33] Reyha Verma and Jasbir Singh. 2015. A Comparative Analysis of Deterministic Sorting Algorithms based on Runtime and Count of Various Operations. Department of Computer Science, National Institute of Technology, Srinagar, India. Department of Computer Science & IT, University of Jammu, Jammu, India. http://web.a.ebscohost.com.proxy.unitar.my:8080/ehost /detail/detail?vid=13&sid=9b1c93a9-f593-45b7-ba91-47543dbb05d4%40sessionmgr4001&hid=4207&bdata =JnNpdGU9ZWhvc3QtbGl2ZSZzY29wZT1zaXRl#A N=111599948&db=iih (Accessed on 18th January 2017)
[34] Stephen G. Eick, Michael C. Nelson, and Jeffery D. Schmidt. 1994. Graphical Analysis of Computer Log Files. Communications of the ACM. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.43.4832&rep=rep1&type=pdf (Accessed on 12th December 2015)
[35] Samreen Laghari and Muaz A. Niazi. 2014. Modeling the Internet of Things, Self-Organizing and Other Complex Adaptive Communication Networks: A Cognitive Agent-Based Computing Approach. Computer Science Department, Islamabad Model College for Girls, Street #25, F-6/2, Islamabad, Pakistan. Computer Science Department, COMSATS Institute of IT, Park Road, Islamabad, Pakistan. http://web.a.ebscohost.com.proxy.unitar.my:8080/ehost /pdfviewer/pdfviewer?sid=44857e1c-62af-4109-8c98-da19baece96a%40sessionmgr4005&vid=9&hid=4104 (Accessed on 14th November 2016)
[36] Shigeru Oyanagi, Kazuto Kubota and Akihiko Nakase. Application of Matrix Clustering to Web Log Analysis and Access Prediction. Shigeru Oyanagi, Kazuto Kubota and Akihiko Nakase. Parallel Application TOSHIBA Laboratory, Real World Computing Partnership. R&D Center, Toshiba Corp., Komukai Toshiba-cho, Saiwai-ku, Kawasaki, 212-8582, Japan. http://robotics.stanford.edu/users/ronnyk/WEBKDD20 01/shigeru.pdf (Accessed on 14th November 2016) [37] Software Testing Help. 2017. 15 Best Bug Tracking
Software: Top Defect/Issue Tracking Tools of 2017.
Software Testing Help.
http://www.softwaretestinghelp.com/popular-bug-tracking-software/ (Accessed on 25th October 2017) [38] Wendy W. Peng and Dolores R. 1993 Wallace. Software
Error Analysis. NIST Special Publication. http://www.geocities.ws/itopsmat/SoftwareErrorAnalys is.pdf (Accessed on 12th December 2015)
[39] Wikipedia. 2015. Analytic Hierarchy Process. https://en.wikipedia.org/wiki/Analytic_hierarchy_proce ss (Accessed on 12th December 2015)
[40] Alberto Gutierrez. 2009. Debugging: 7 steps to fix an error. Software Development Theory. http://www.makinggoodsoftware.com/2009/06/14/7-steps-to-fix-an-error/ (Accessed on 12th December 2015)
[41] Colin Eberhardt. 2014. The Art of Logging. CodeProject.
https://www.codeproject.com/Articles/42354/The-Art-of-Logging (Accessed on 12th December 2015) [42] Brian Kirsch. 2018. How CPU and memory affect
application performance. TechTarget. http://searchservervirtualization.techtarget.com/tip/Ho w-CPU-and-memory-affect-application-performance (Accessed on 12th December 2015)
[43] John B. Bowen. 1980. Standard error classification to support software reliability assessment. Hughes-Fullerton, Hughes-Fullerton, California. National Computer Conference.
https://www.computer.org/csdl/proceedings/afips/1980/ 5088/00/50880697.pdf (Accessed on 12th Decenber 2015)
[44] SoftwareTestingHelp.com. 2018. 7 Types of Software Errors That Every Tester Should Know. Basics of
Software Testing.
http://www.softwaretestinghelp.com/types-of-software-errors/ (Accessed on 8th March 2018)
[46] MariaDB. 2018. Operating System Error Codes. https://mariadb.com/kb/en/library/operating-system-error-codes/ (Accessed on 8th March 2018)
[47] Amy Echeverri and Sadequl Hussain. 2018. Troubleshooting with Linux Logs. The Ultimate Guide to Logging. https://www.loggly.com/ultimate-guide/troubleshooting-with-linux-logs/ (Accessed on 8th March 2018)
[48] SparkTrust. 2018. How to Fix High CPU Usage. http://www.sparktrust.com/fix-high-cpu-usage/?gclid (Accessed on 9th March 2018)
[49] Wikipedia. 2018. Database application. Wikimedia
Foundation, Inc.
https://en.wikipedia.org/wiki/Database_application (Accessed on 9th March 2018)
[50] Rafay Ansari 2017. How to Fix Error Establishing a Database Connection Error. Design Bombs. https://www.designbombs.com/how-to-fix-error-establishing-a-db-connection-error/ (Accessed on 9th March 2018)
[51] Wikipedia. 2018. Multitier architecture. Wikimedia
Foundation, Inc.
https://en.wikipedia.org/wiki/Multitier_architecture (Accessed on 22nd March 2018)
[52] Microsoft TechNet. 2018. Troubleshooting Server and Database Connection Problems. Microsoft Inc.
https://technet.microsoft.com/en-us/library/ms156468(v=sql.105).aspx (Accessed on 22nd March 2018)
[53] EventLog Analyzer 2018. Syslog & Windows Event
Log Analysis. ManageEngine.
https://www.manageengine.com/products/eventlog/eve
ntlog-analysis.html?cam=279894757&loc=&kwd=log%20fil e%20analysis&gclid=EAIaIQobChMI7euVn4OC2gIV DX69Ch06UQS5EAAYASAAEgKM3PD_BwE (Accessed on 22nd March 2018)
[54] IBM Knowledge Center. 2018. Introduction to message
queuing. IBM.
https://www.ibm.com/support/knowledgecenter/en/SSF KSJ_9.0.0/com.ibm.mq.pro.doc/q002620_.htm (Accessed on 27th March 2018)
[55] Pinal Dave. 2016. 2 Common Errors and Reasons for SQL Server Login Failures. LogicalRead. https://logicalread.com/errors-sql-server-login-failures-pd01/#.Wrr8Ba2k1zU (Accessed on 28th March 2018) [56] Microsoft 2018. View a List of Databases on an
Instance of SQL Server. Microsoft Crop.
https://docs.microsoft.com/en-us/sql/relational- databases/databases/view-a-list-of-databases-on-an-instance-of-sql-server (Accessed on 28th April 2017) [57] Oracle Documentation. 2018. Defining Database
Services. Oracle.
https://docs.oracle.com/cd/B28359_01/server.111/b283 10/create007.htm#ADMIN11125 (Accessed on 28th March 2018)
[58] Modeling the Internet of Things, Self-Organizing and Other Complex Adaptive Communication Networks: A Cognitive Agent-Based Computing Approach. http://web.a.ebscohost.com.proxy.unitar.my:8080/ehost /pdfviewer/pdfviewer?sid=44857e1c-62af-4109-8c98-da19baece96a%40sessionmgr4005&vid=9&hid=4104 (Accessed on 25th April 2017)
[59] Social Cognitive Theory and Individual Reactions to
Computing Technology.
http://web.a.ebscohost.com.proxy.unitar.my:8080/ehost /pdfviewer/pdfviewer?sid=44857e1c-62af-4109-8c98-da19baece96a%40sessionmgr4005&vid=15&hid=4104 (Accessed on 22th March 2017)
[60] A Comparative Analysis of Deterministic Sorting Algorithms based on Runtime and Count of Various Operations.
http://web.a.ebscohost.com.proxy.unitar.my:8080/ehost /detail/detail?vid=13&sid=9b1c93a9-f593-45b7-ba91-47543dbb05d4%40sessionmgr4001&hid=4207&bdata =JnNpdGU9ZWhvc3QtbGl2ZSZzY29wZT1zaXRl#A N=111599948&db=iih (Accessed on 12th December 2015)
[61] A Novel Algorithm for Independent Component Analysis with Reference and Methods for Its Applications.
http://web.a.ebscohost.com.proxy.unitar.my:8080/ehost /detail/detail?vid=18&sid=9b1c93a9-f593-45b7-ba91-47543dbb05d4%40sessionmgr4001&hid=4207&bdata =JnNpdGU9ZWhvc3QtbGl2ZSZzY29wZT1zaXRl#A N=96281979&db=fsr (Accessed on 12th December 2015)
[62] John P. Rouillard. Real-time Log File Analysis Using the Simple Event Correlator (SEC). https://www.usenix.org/legacy/publications/library/proc eedings/lisa04/tech/full_papers/rouillard/rouillard_html / (Accessed on 12th December 2015)
[63] Gerald Stermsek, Mark Strembeck, Gustaf Neumann. A User Profile Derivation Approach based on Log-File Analysis.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.141.6449&rep=rep1&type=pdf (Accessed on 22th March 2017)
[64] K. R. Suneetha and Dr. R. Krishnamoorthi. Identifying User Behavior by Analyzing Web Server Access Log File.
https://www.researchgate.net/profile/Krishnamoorthi_R /publication/255583124_Identifying_User_Behavior_b y_Analyzing_Web_Server_Access_Log_File/links/56a 5ab8208ae1b65113382aa.pdf (Accessed on 22th March 2017)
[65] James H. Andrews. 1998. Theory and practice of log
file analysis.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.44.292&rep=rep1&type=pdf (Accessed on 16th June 2017)
[66] Dileepa Jayathilake. A mind map based framework for automated software log file analysis. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.456.4146&rep=rep1&type=pdf (Accessed on 16th June 2017)
[67] Method and apparatus for graphically analzying a log-file. https://www.google.com/patents/US5847972 (Accessed on 7th July 2017)
[69] System and method for analyzing web-server log files. https://www.google.com/patents/US6317787 (Accessed on 5th September 2017)
[70] G. Castellano, A. M. Fanelli, M. A. Torsello. LODAP: A Log Data Preprocessor for mining Web browsing
patterns.
http://www.wseas.us/e-library/conferences/2007corfu/papers/540-274.pdf (Accessed on 5th September 2017)
[71] Log file analysis and evaluation tool. https://www.google.com/patents/US20090144699 (Accessed on 5th September 2017)
[72] Mark Guzdial. Deriving Software Usage Patterns from
Log Files.
https://smartech.gatech.edu/bitstream/handle/1853/365 0/93-41.pdf?sequence=1&isAllowed=y (Accessed on 5th September 2017)
[73] Table-based error log analysis. https://www.google.com/patents/US6598179 (Accessed on 19th August 2017)
[74] James E. Prewett. Analyzing cluster log files using Logsurfer.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.5.8610&rep=rep1&type=pdf (Accessed on 19th August 2017)
[75] Method, system, and program for merging log entries from multiple recovery log files. https://www.google.com/patents/US7076508 (Accessed on 15th October 2017)
[76] System using footprints in system log files for monitoring transaction instances in real-time network.
US 7908365 B2.
https://www.google.com/patents/US7908365 (Accessed on 15th October 2017)
[77] Method using footprints in system log files for monitoring transaction instances in real-time network.
US 7912946 B2.
https://www.google.com/patents/US7912946 (Accessed on 15th October 2017)
[78] An overview of preprocessing of web log files for web
usage mining.
http://etms-eg.org/cms/upload/1354650330.pdf (Accessed on 15th October 2017)
[79] Event Logs for the Analysis of Software Failures: A
Rule-Based Approach.
http://web.b.ebscohost.com.proxy.unitar.my:8080/ehost /pdfviewer/pdfviewer?sid=6bf25055-2122-4a13-bf98-5169a4c9ff0b%40sessionmgr103&vid=8&hid=102 (Accessed on 15th October 2017)
[80] Distributed Log Analysis on The Cloud using MapReduce.
http://web.a.ebscohost.com.proxy.unitar.my:8080/ehost /pdfviewer/pdfviewer?sid=7f05d220-a895-41e3-bd1e-7482e8c6b86e%40sessionmgr4008&vid=10&hid=420 4 (Accessed on 29th October 2017)
[81] Implementing an error log analysis model to facilitate faster problem isolation and repair. US 7872982 B2. https://www.google.com/patents/US7872982 (Accessed on 29th October 2017)
[82] Jiaqi Tan, Xinghao Pan, Soila Kavulya, Rajeev Gandhi, Priya Narasimhan. Mochi: Visual Log-Analysis Based
Tools for Debugging Hadoop.
http://static.usenix.org/events/hotcloud09/tech/full_pap ers/tan.pdf (Accessed on 29th November 2017)
[83] Object oriented processing log analysis tool framework
mechanism. US 6144967 A.
https://www.google.com/patents/US6144967 (Accessed on 29th November 2017)
[84] A Framework for Log file Analysis. https://pdfs.semanticscholar.org/9d1a/97988d8b41354c d4bf85ace96648d1684555.pdf (Accessed on 29th November 2017)
[85] Tao Li, Sheng Ma, Feng Liang and Wei Peng. An Integrated Framework on Mining Logs Files for
Computing System Management.
http://users.cis.fiu.edu/~taoli/tenure/p776-li.pdf (Accessed on 29th November 2017)
[86] Mislav Šimunić, Željko Hutinski and Mirko Čubrilo. Log File Analysis and Creation of More Intelligent
Web Sites.
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&s ource=web&cd=13&ved=0ahUKEwjZztKT8cPSAhV MrY8KHSkQC4A4ChAWCCAwAg&url=http%3A%2 F%2Fhrcak.srce.hr%2Ffile%2F116374&usg=AFQjCN
Eo2GBP9vRjBxZ_HthVBv-gt08L6Q&bvm=bv.148747831,d.c2I (Accessed on 29th November 2017)
[87] Robert Rinnan. 2005. Benefits of Centralized Log file Correlation. Gjøvik University College. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.121.8787&rep=rep1&type=pdf (Accessed on 29th November 2017)
[88] StorageCraft. 2018. Which Hardware Fails the Most and Why. StorageCraft Technology Corporation. https://blog.storagecraft.com/hardware-failure/
(Accessed on 8th April 2018)
[89] Wikipedia. 2018. Hard disk drive failure. https://en.wikipedia.org/wiki/Hard_disk_drive_failure. Wikimedia Foundation, Inc. (Accessed on 8th April 2018)
[90] Eassos. 2018. How to fix "Windows detected a hard disk problem"? Eassos Ltd. http://www.eassos.com/how-to/windows-detected-a-hard-disk-problem.php (Accessed on 9th April 2018) [91] TechRepublic. 2018. Using smartctl to get SMART
status information on your hard drives. CBS Interactive. https://www.techrepublic.com/blog/linux- and-open-source/using-smartctl-to-get-smart-status-information-on-your-hard-drives/ (Accessed on 9th April 2018)
[92] ArchLinux. 2018. S.M.A.R.T.
https://wiki.archlinux.org/index.php/S.M.A.R.T. (Accessed on 9th April 2018)
[93] Backblaze. 2018. What SMART Stats Tell Us About Hard Drives. https://www.backblaze.com/blog/what-smart-stats-indicate-hard-drive-failures/ (Accessed on 9th April 2018)
[94] WikiBooks. 2015. Minimizing Hard Disk Drive Failure and Data Loss/Detecting an Impending Drive Failure. https://en.wikibooks.org/wiki/Minimizing_Hard_Disk_ Drive_Failure_and_Data_Loss/Detecting_an_Impendin g_Drive_Failure (Accessed on 9th April 2018)
[96] ITPro Today. 2018. How can we relocate the event log files of our Windows Server 2003 and Windows Server 2008 file servers to a different drive? http://www.itprotoday.com/management-mobility/q- how-can-we-relocate-event-log-files-our-windows-server-2003-and-windows-server (Accessed on 9th April 2018)
[97] VPSie. 2018. Where to find Windows 2012/2012 R2 Log Files? https://vpsie.com/knowledge-base/where-to-find-windows-20122012-r2-log-files/ (Accessed on 9th April 2018)
[98] RootUsers. 2018. Where Are Windows Server 2016 Log Files Stored? https://www.rootusers.com/where-are-windows-server-2016-log-files-stored/ (Accessed on 9th April 2018)
[99] Rakesh Kanojia. 2018. Are you spending too much time on Root Cause Analysis and IT Issue Triage? Service Berry Technologies. https://www.service- berry.com/are-you-spending-too-much-time-on-root- cause-analysis-and-it-issue-triagecheckout-micro- focus-operations-bridge-for-a-seamless-issue-triage-experience/ (Accessed on 2nd July 2018)
[100]Kishore Ramamurthy. 2016. The Importance of Root
Cause Analysis.
Optanix.
https://www.optanix.com/blog/the-importance-of-root-cause-analysis (Accessed on 2nd July 2018)
[101]Solarwinds. 2018. Automate log analysis, improve IT operations, mitigate threats, and help ensure compliance. SolarWinds Worldwide, LLC. https://www.solarwinds.com/topics/event-log-analyzer-tool#step_benefit_1 (Accessed on 2nd July 2018) [102]Logstash. 2018. Centralize, Transform & Stash Your
Data. Elasticsearch.
https://www.elastic.co/products/logstash (Accessed on 2nd July 2018)
[103]AppDynamics. 2018. Transform the way modern business is built and run. AppDynamics, Cisco. https://www.appdynamics.com/ (Accessed on 2nd July 2018)
[104]Evangelos Triantaphyllou and Stuart H. Mann. 1995. Using the Analytic Hierarchy Process for Decision Making in Engineering Applications: Some Challenges. Inter’l Journal of Industrial Engineering: Applications
and Practice.
http://bit.csc.lsu.edu/trianta/Journal_PAPERS1/AHPapl s1.pdf (Accessed on 18th February 2019)
[105]Belton, V. and Gear, T. (1983). On a Short-coming of Saaty's Method of Analytic Hierarchies. Omega, 228-230. https://www.academia.edu/19364689/On_a_short-coming_of_Saatys_method_of_analytic_hierarchies (Accessed on 18th February 2019)
[106]Stephan Weißleder and Hartmut Lackner. 2013. Top-Down and Bottom-Up Approach for Model-Based
Testing of Product Lines.
https://arxiv.org/pdf/1303.1011.pdf (Accessed on 18th February 2019)
[107]Wikipedia. 2019. Top-down and bottom-up design. https://en.wikipedia.org/wiki/Top-down_and_bottom-up_design (Accessed on 18th February 2019)
[108]Atos. 2019. Applications As a Service. Atos SE.
https://apprenda.com/library/software-on-demand/applications-as-a-service/ (Accessed on 20th February 2019)
[109]Unomaly. 2019. Root cause analysis: Using the right tools for the job. https://unomaly.com/product/root-cause-analysis/ (Accessed on 21st February 2019) [110]John Flaig. 2013. The Risks in Root Cause Analysis.
Quality Digest.
https://www.qualitydigest.com/inside/quality-insider-article/risks-root-cause-analysis.html (Accessed on 21st February 2019)
[111]Jyoti Soni, Ujma Ansari, Dipesh Sharma and Sunita Soni. 2011. Predictive Data Mining for Medical Diagnosis: An Overview of Heart Disease Prediction. International Journal of Computer Applications (0975 – 8887).
https://pdfs.semanticscholar.org/fbd6/5a18f6653b56138 cd5196d20e2f39de189e3.pdf (Accessed on 21st February 2019)
[112]Max Landauer, Markus Wurzenberger, Florian Skopik, Giuseppe Settanni and PeterFilzmoser. 2018. Dynamic log file analysis: An unsupervised cluster evolution approach for anomaly detection. https://www.sciencedirect.com/science/article/pii/S016 7404818306333/pdfft?md5=e114106f24719a22492bdc 5df17f2742&pid=1-s2.0-S0167404818306333-main.pdf (Accessed on 21st February 2019)
[113]Sallam Osman Fageeri and Rohiza Ahmad. 2013. An Efficient Log File Analysis Algorithm Using
Binary-based Data Structure.
https://www.sciencedirect.com/science/article/pii/S187 7042814028912/pdf?md5=5b0ef9817478b4ad8339553 bd652fa84&pid=1-s2.0-S1877042814028912-main.pdf (Accessed on 21st February 2019)
[114]Omkarprasad S. Vaidya and Sushil Kumar, 2004. Analytic hierarchy process: An overview of applications. Department of Mechanical Engineering, Army Institute of Technology, Pune 411 015, India. National Institute of Industrial Engineering (NITIE), Vihar Lake, Mumbai 400 087, India. http://ac.els-
cdn.com/S0377221704003054/1-s2.0- S0377221704003054-main.pdf?_tid=39850adc-5d78-
11e7-b5bf-00000aab0f26&acdnat=1498815840_e0c9a10c99c46ad 30db8da4ef17e817b (Accessed on 12th September 2018)
[115]Nathasit Gerdsri and Dundar F. Kocaoglu. 2007. Applying the Analytic Hierarchy Process (AHP) to build a strategic framework for technology roadmapping. ScienceDirect. ELSEVIER. http://ac.els-
cdn.com/S0895717707001069/1-s2.0- S0895717707001069-main.pdf?_tid=e0165298-5d78-
11e7-9000-00000aab0f6b&acdnat=1498816119_34b7b92d988eea 0c318b8d23810e293a (Accessed on 21st February 2019)
[116]Razvan Serbu, Bogdan Marza and Sorin Borza. 2016. A Spatial Analytic Hierarchy Process for Identification of Water Pollution with GIS Software in an
Eco-Economy Environment. MDPI.
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&s
ource=web&cd=8&ved=0ahUKEwjTk9bz-cLbAhUFupQKHcXaCmAQFghkMAc&url=http%3A
[117]Da-Yong Chang. 1996. Applications of the extent analysis method on fuzzy AHP. ELSEVIER. http://ac.els-cdn.com/0377221795003002/1-s2.0- 0377221795003002-main.pdf?_tid=e251d09a-7732-
11e7-b173-00000aacb35d&acdnat=1501644789_e6ae23503d2e2f 883d2413693f63275a (Accessed on 12th September 2018)
[118]Thomas L. Saaty. 2008. Decision making with the analytic hierarchy process. Int. J. Services Sciences,
Vol. 1, No. 1.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10. 1.1.409.3124&rep=rep1&type=pdf (Accessed on 12th September 2018)
[119]R. W. Saaty. 1987. The Analytic Hierarchy Process - What It Is And How It Is Used. Pergamon Journals Ltd.
Great Britain.
http://ac.els-
cdn.com/0270025587904738/1-s2.0- 0270025587904738-main.pdf?_tid=55956250-6b90-
11e7-b75e-00000aab0f02&acdnat=1500365511_5a54a3ecf035ad3 e45ab72b909b7632e (Accessed on 12th September 2018)
[120]Dr. Benito L. Teehankee and De La Salle University. 2009. The Analytic Hierarchy Process: Capturing Quantitative and Qualitative Criteria for Balanced
Decision-Making. ResearchGate.
https://www.researchgate.net/publication/256009323_T he_Analytic_Hierarchy_Process_Capturing_Quantitati ve_and_Qualitative_Criteria_for_Balanced_Decision-Making (Accessed on 12th September 2018)
[121]Evangelos Triantaphyllou and Stuart H. Mann. 1995. Using the Analytic Hierarchy Process for Decision Making in Engineering Applications: Some Challenges. http://s3.amazonaws.com/academia.edu.documents/317 54775/AHPapls1.pdf?AWSAccessKeyId=AKIAIWO WYYGZ2Y53UL3A&Expires=1498819440&Signatur e=iIMAp3tvmRbBTX8k6rUvRIorb7o%3D&response-
content-disposition=inline%3B%20filename%3DUSING_THE _ANALYTIC_HIERARCHY_PROCESS_FOR.pdf (Accessed on 12th September 2018)
[122]Explorable.com. 2015. Experimental Research. https://explorable.com/experimental-research
(Accessed on 31st December 2015)
[123]Thinking Dimensions. 2018. Reliability - Root Cause Analysis. Thinking Dimensions Global. http://www.thinkingdimensions.com/blog/industrial-problem-solving/reliability-root-cause-analysis (Accessed on 1st July 2018)
[124]CodeBetter.Com. 2018. Benefits of Root Cause
Analysis. WordPress.
http://codebetter.com/raymondlewallen/2008/12/20/ben efits-of-root-cause-analysis/ (Accessed on 1st July 2018)
[125]Kristof Horvath. 2015. Using Root Cause Analysis to Drive Process Improvement. Intland Software.https://content.intland.com/blog/safety- engineering/using-root-cause-analysis-to-drive-process-improvement (Accessed on 2nd July 2018) [126]Darrell Stiffler. 2009. Performing Root Cause Analysis
on Personnel? Global Knowledge Training LLC.https://www.globalknowledge.com/blog/2009/05/ 11/performing-root-cause-analysis-on-personnel/
(Accessed on 2nd July 2018) Apparatus for log data
collection and analysis.
https://www.google.com/patents/US5819094 (Accessed on 5th September 2017)
