1. Introduction
1.7. Previous work and automated algorithms for identification of structural alerts
Knowledge-based structural alert methods, such as the commercially expert systems Lhasa’s
Derek Nexus42 and Genetox Expert Alerts from Leadscope,70 have been used for toxicity
predictions. However, constructing these types systems is very time consuming, requires input from experts, and may suffer from human bias. Automated approaches using statistics to identify structural alerts do not suffer from these drawbacks. There have been numerous different approaches to automated generation of structural alerts, each with their own strengths and weaknesses.
Computer Automated Structure Evaluation (CASE) is a fragment-based approach.71 Chemicals are
broken down into linear subunits containing between three and twelve interconnected non- hydrogen atoms. All possible linear fragments between these sizes are derived from each chemical and the occurrence of each fragment in active and inactive chemicals is calculated. These numbers are analysed statistically. If the distribution of active and inactive chemicals is significantly skewed towards active chemicals, the fragment is identified as an activating “biophore”. If the distribution is significantly skewed towards inactive chemicals, the fragment is identified as non-activating. Significant skew was initially defined as a distribution that would have had at most a 5% chance of being observed if the occurrence was random, assuming a binomial distribution. Multiple Computer Automated Structure Evaluation (MultiCASE) is similar
to CASE, using hierarchical statistical analysis.72 Where CASE uses all statistically significant
fragments, MultiCASE uses the most statistically significant fragment at each iteration, any chemicals containing that fragment are removed from the training set, and the process repeated. For each fragment identified, other correlated fragments and physicochemical properties are used to create a QSAR specific to that biophore. The CASE and MultiCASE approaches are limited by the use of only linear subunits, meaning branching substructures are not accounted for.
Bioalerts is an open source Python library for automatically constructing structural alerts.73
Substructures are defined by Morgan fingerprints of increasing size. As with CASE, the occurrence of each substructure in the active and inactive chemicals is counted and the probability of that distribution occurring randomly, assuming a binomial distribution, is calculated. A substructure is identified as a structural alert if this probability is below a threshold (for example 5%). The use of Morgan fingerprints means substructures are limited to circular environments, which may not give optimal results.
SARpy uses string mining to automatically construct structural alerts.74 Molecules are input as
by likelihood ratio (as used in diagnostic testing), defined as the proportion of active chemicals containing the substructure divided by the proportion of inactive chemicals containing the substructure. A potential limit of likelihood ratio is that it returns a value of infinity for any fragment contained by no inactive chemicals and at least one active chemical. As a result, specific substructures, contained by no inactive chemicals and few active chemicals, will have large likelihood ratios. These overly specific alerts may not generalise well, giving poor predictions outside of the training set. The use of string mining limits substructures to atoms which occur next to each in the SMILES string, making branching difficult to account for.
Each of these algorithms for automatic generation of structural alerts differ in two key ways: 1. How substructures are derived.
2. The statistical approach used to accept or reject a substructure as a structural alert. The approaches to derivation of substructures are not capable of dealing with branching substructures (fragment-based or string mining approaches) or non-circular environments (fingerprint-based approach). Whilst these approaches may be effective at identifying small substructures, such as those that are electrophilic and capable of causing DNA mutagenicity or skin sensitisation, they would struggle to deal with larger substructures, such as rings with branching features. Hence, these algorithms may not find the optimal substructures and may not be suitable when the optimal substructure is large or branching.
SAR models have different requirements when used for different purposes. For example, in risk assessment, a false negative prediction is the most dangerous type of error and as such, a SAR model for risk assessment should have as high sensitivity as possible, often at the expense of specificity. However, in drug discovery, confidence in active predictions is most important, so a model should have as high specificity as possible, often at the expense of sensitivity. The statistical approaches used in the previously discussed methods for automatically constructing structural alerts do not allow for this type of flexibility.
In this work, maximal common substructure searcher has been used to find the largest substructures common to chemicals in the training set. This does not limit the size or shape of the substructures. A statistical approach to accepting substructures as structural alerts has been used that is flexible, allowing the user to adjust a parameter to change the relative importance of number of actives and inactives in selection of substructures.
Prior work from within the Goodman group
Prior to this work, Allen et al have constructed structural alerts for the Bowes targets, published
in 2016.43 For each biological target, data was extracted from ChEMBL75. Substructures common
to active chemicals were found using a maximal common substructure algorithm. Human analysis, aided by literature searches, was used to select which would be used as structural alerts. A small number of structural alerts were developed for each target (an average of 2.93 per target), but each covered many active chemicals. The ChEMBL database generally contains relatively few inactives for each target. For validation of the structural alerts, an assumption is made to provide additional inactive chemicals: for each target, chemicals present in data sets of other targets in the study are assumed to be inactive at the target of interest if they are not already present in that target’s data set.
Concurrent to this work, Allen et al published updated structural alert-based models for the same
targets in 2018.44 The same database and methods for extracting data were used as the previous
study, including the same method for collecting assumed-negatives for validation. The largest substructure common to 2% of the training set active chemicals was found, coded as a structural alert and chemicals containing the chemical removed from the training set. This is iteratively repeated until only one chemical is contained by the largest common substructures. Different filters are applied to the list of generated structural alerts to create two models for different purposes. A model designed with the highest possible sensitivity (at the cost of specificity) for use in screening chemicals is created by using all alerts that are contained by at least two chemicals in the training set. A second model, designed to have a higher specificity and overall performance, for use in risk assessment is created by using alerts that are contained by at least five chemicals in the training set and which are contained by more active chemicals than (assumed) inactive chemicals in the test set. The overall process for creating these models is summarised in Figure 1.11.
Both the screening and risk assessment models have significantly better performance metrics than the previous work in terms of sensitivity (proportion of experimentally active chemicals correctly predicted), specificity (proportion of experimentally inactive chemicals correctly predicted), accuracy (proportion of all chemicals correctly predicted) and Matthews Correlation Coefficient (MCC). Compared to the previous work, the individual structural alerts used in the new approach are generally larger in size and cover far fewer active chemicals. However, a greater number of structural alerts are used in each target, and the combination of these alerts leads to a model with better overall performance. In this thesis, comparisons will be made to Allen’s updated models only, as these are the latest and better performing of the published models. New models will be compared to the “Screening” and “Risk Assessment” models.
Figur e 1. 11 : O ver view o f t he p ro cedu re us ed by Al len et al to ge ner ate str uctur al a ler t- ba sed m odels. 44 Im age ada pted fr om A llen et a l (20 18 ).