- Notifications
You must be signed in to change notification settings - Fork2
AutoML system for building trustworthy peptide bioactivity predictors
License
IBM/AutoPeptideML
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
- Documentation:https://ibm.github.io/AutoPeptideML
- Source Code:https://github.com/IBM/AutoPeptideML
- Webserver:http://peptide.ucd.ie/AutoPeptideML
- Google Collaboratory Notebook:AutoPeptideML_Collab.ipynb
- Paper:https://doi.org/10.1093/bioinformatics/btae555
- Blog post:Portal - AutoPeptideML Tutorial
- Models and Paper SI:Zenodo Repository
AutoPeptideML allows researchers without prior knowledge of machine learning to build models that are:
- Trustworthy: Robust evaluation following community guidelines for ML evaluation reporting in life sciencesDOME.
- Interpretable: Output contains a PDF summary of the model evaluation explaining how to interpret the results to understand how reliable the model is.
- Reproducible: Output contains all necessary information for other researchers to reproduce the training and verify the results.
- State-of-the-art: Models generated with this system are competitive with state-of-the-art handcrafted approaches.
We recommend the use of the Google Collaboratory Notebook for users that want greater control of the model training process and the webserver for users that prefer ease of use.
Installing in a conda environment is recommended. For creating the environment, please run:
conda create -n autopeptideml pythonconda activate autopeptideml
pip install autopeptideml
pip install git+https://github.com/IBM/AutoPeptideML
# static build with AVX2 (fastest) (check using: cat /proc/cpuinfo | grep avx2)wget https://mmseqs.com/latest/mmseqs-linux-avx2.tar.gz; tar xvfz mmseqs-linux-avx2.tar.gz;export PATH=$(pwd)/mmseqs/bin/:$PATH# static build with SSE4.1 (check using: cat /proc/cpuinfo | grep sse4)wget https://mmseqs.com/latest/mmseqs-linux-sse41.tar.gz; tar xvfz mmseqs-linux-sse41.tar.gz;export PATH=$(pwd)/mmseqs/bin/:$PATH# static build with SSE2 (slowest, for very old systems) (check using: cat /proc/cpuinfo | grep sse2)wget https://mmseqs.com/latest/mmseqs-linux-sse2.tar.gz; tar xvfz mmseqs-linux-sse2.tar.gz;export PATH=$(pwd)/mmseqs/bin/:$PATH# MacOSbrew install mmseqs2
To use Needleman-Wunch, either:
conda install -c bioconda emboss
or
sudo apt install emboss
Data used to benchmark our approach has been selected from the benchmarks collected byDu et al, 2023. A new set of benchmarks was constructed from the original set following the new data acquisition and dataset partitioning methods within AutoPeptideML. To download the datasets:
- Original UniDL4BioPep Benchmarks: Please check the projectGithub Repository.
- ✅ New AutoPeptideML Benchmarks (Amended version): Can be downloaded from thislink. Please note that these are not exactly the same benchmarks as used in the paper, those are kept in the next line for reproducibility (seeIssue #24 for more details).
⚠️ New AutoPeptideML Benchmarks (Paper version): Can be downloaded from thislink. This benchmarks have a small number of overlapping sequences between training and testing due to a bug described inIssue #24. They are kept here for reproducibility.
1. Model builder options
Dataset construction
dataset
: File with positive peptides inFASTA
orCSV
file. It can also contain negative peptides in which case the files should contain the labels (0: negative or 1: positive) either in the header (FASTA
) or in columnY
(CSV
).--balance
: IfTrue
, it balances the datasets by oversampling the underrepresented label.--autosearch
: IfTrue
, it searches for negative peptides.--autosearch_tags
: Comma separated list of tags that may overlap with positive activity that are going to be excluded from the negative peptides.--autosearch_proportion
: Negative:positive ration when automatically drawing negative controls from the bioactive peptides database (Default: 1.0).
Output
--outputdirdir
: Output directory (Default:./apml_result/apml_result
).
Protein Language Model
--plm
: Protein Language Model for computing peptide representations. Available options:esm2-8m
,esm2-35m
,esm2-150m
,esm2-650m
,esm2-3b
,esm2-15b
,esm1b
,prot-t5-xxl
,prot-t5-xl
,protbert
,prost-t5
. (Default:esm2-8m
). Please note: Larger Models might not fit into GPU RAM, if it is necessary for your purposes, please create a new issue.--plm_batch_size
: Number of peptides for which to batch the PLM computation.(Default: 12).
Dataset Partitioning
--test_partition
: Whether to divide the dataset in train/test splits. (Default:True
).--test_threshold
: Maximum sequence identity allowed between train and test. (Default: 0.3).--test_size
: Proportion of data to be assigned to evaluation set. (Default: 0.2).--test_alignment
: Alignment algorithm used for computing sequence identities. Available options:peptides
,needle
. (Default:peptides
).--splits
: Path to directory with train and test splits. Expected contents:train.csv
andtest.csv
.--val_partition
: Whether to divide dataset in train/validation folds.--val_method
: Method to use for creating train/validation folds. Options available:random
,graph-part
. (Default:random
)--val_threshold
: Maximum sequence identity allowed between train and validation. (Default: 0.5).--val_alignment
: Alignment algorithm used for computing sequence identities. Available options:peptides
needle
. (Default:peptides
).--val_n_folds
: Number of folds (Default: 10).--folds
: Path to directory with train/validation folds. Expected contents:train_{fold}.csv
andvalid_{fold}.csv
.
Model Selection and Hyperparameter Optimisation
--config
: Name of one of the pre-defined configuration files (seeautopeptideml/data/configs
) or path to a custom configuration file (see next section).
Other
--verbose
: Whether to display information about runtime (Default: True).--threads
: Number of threads to use for parallelization. (Default: Number of cores in the machine).--seed
: Seed for pseudorandom number generators. Controls stochastic processes. (Default: 42)
2. Predict
dataset
: File with problem peptides inFASTA
orCSV
file.--ensemble
: Path to the a file containing a previous AutoPeptideML result.--outputdir
: Output directory (Default:./apml_predictions
).--verbose
: Whether to display information about runtime (Default: True).--threads
: Number of threads to use for parallelization. (Default: Number of cores in the machine).--plm
: Protein Language Model for computing peptide representations. Must be the same as used to train the model. Available options:esm2-8m
,esm2-35m
,esm2-150m
,esm2-650m
,esm2-3b
,esm2-15b
,esm1b
,prot-t5-xxl
,prot-t5-xl
,protbert
,prost-t5
. (Default:esm2-8m
). Please note: Larger Models might not fit into GPU RAM, if it is necessary for your purposes, please create a new issue.--plm_batch_size
: Number of peptides for which to batch the PLM computation.(Default: 12).
3. Hyperparameter Optimisation and model selection
The experiment configuration is a file inJSON
format describing the hyperparameter optimisation search space and the composition of the final ensemble. The first level of the file is a dictionary with a single key (ensemble
ormodel_selection
ormodel_selection
) and a list of search spaces for the hyperparameter optimisation. For each model within theensemble
list,n
different models will be trained one per cross-validation fold; in the case ofmodel_selection
, only one of the algorithms will comprise the final ensemble; in the case ofmodel_selection
, only one of the algorithms will comprise the final ensemble.
Each experiment requires the following fields:
model
: Defines the ML algorithm. Options:KNearestNeighbours
,SVM
,RFC
,XGBoost
,LGBM
,MLP
, andUniDL4BioPep
. More options will be added in subsequent releases and they can be implemented upon request.trials
: Defines the number of iterations for the hyperparameter optimisation search.optimization_metric
: Defines the metric that should be used for directing the optimisation search. Always, the metric will be calculated as the average across then
cross-validation folds. For the metrics available all of the binary classification within the list in thescikit-learn documentation are supported (Default: Matthew's correlation coefficient, MCC).hyperparameter-space
: List of dictionaries that defines the hyperparameter search space proper. Each of the dictionaries within correspond to a different hyperparameter and may have the following fields:name
: It has to correspond with the corresponding hyperparameter in the model implementation. Most of the simpler ML models use thescikit-learn
implementation,LGBM
uses the Microsoft implementation (More information onLGBM Repository) andUniDL4BioPep
uses the PyTorch implementation (More information onUniDL4BioPep PyTorch Repository), though for this model hyperparameter optimisation is not recommended.type
: Defines the type of hyperparameter. Options:int
,float
, orcategorical
.min
andmax
: Defines the lower and upper bounds of the search space for typesint
andfloat
.log
: Boolean value that defines whether the search should be done in logarithmic space or not. Accelerates searches through vast spaces for example for learning rates (1e-7 to 1). It is not optional.value
: Defines the list of options available for a hyperparameter of typecategorical
for example types of kernel (linear
,rbf
,sigmoid
) for a Support Vector Machine.
There is an example available in thedefault configuration file.
autopeptideml dataset.csv
autopeptideml dataset.csv --val_method graph-part --val_threshold 0.3 --val_alignment needleautopeptideml dataset.csv
autopeptideml dataset.csv --val_method graph-part --val_threshold 0.3 --val_alignment needle
For running predictions
autopeptideml-predict peptides.csv --ensemble AB_1/ensemble
AutoPeptideML is an open-source software licensed under the MIT Clause License. Check the details in theLICENSE file.
Special thanks toSilvia González López for designing the AutoPeptideML logo and toMarcos Martínez Galindo for his aid in setting up the AutoPeptideML webserver.
About
AutoML system for building trustworthy peptide bioactivity predictors