Protocol for predicting single- and multiple-dose-dependent gene expression using deep generative learning

Summary

Variational autoencoders (VAEs) can be used to model the gene expression space of single-cell RNA sequencing (scRNA-seq) data. Here, we present a protocol for using single-cell variational inference of dose response (scVIDR), a VAE designed to model single-cell gene expression for dose-dependent chemical perturbations. We describe steps to access the scVIDR code and data using a containerization application called Docker. We then detail procedures for training the sVIDR model and predicting gene expression.

For complete details on the use and execution of this protocol, please refer to Kana et al.¹

Graphical abstract

Highlights

• •Prediction of dose-dependent single-cell gene expression after chemical perturbations
• •Biological interpretation of model predictions
• •Complete Docker-based workflow with step-by-step instructions
• •Support for both ARM64 and AMD64 architectures, with or without the original data

Publisher’s note: Undertaking any experimental protocol requires adherence to local institutional guidelines for laboratory safety and ethics.

Variational autoencoders (VAEs) can be used to model the gene expression space of single-cell RNA sequencing (scRNA-seq) data. Here, we present a protocol for using single-cell variational inference of dose response (scVIDR), a VAE designed to model single-cell gene expression for dose-dependent chemical perturbations. We describe steps to access the scVIDR code and data using a containerization application called Docker. We then detail procedures for training the sVIDR model and predicting gene expression.

Before you begin

The single-cell variational inference of dose-response (scVIDR) framework is a computational tool based on variational autoencoders (VAEs) designed to predict dose-dependent changes in single-cell gene expression, enabling cellular response predictions across multiple doses and cell types. It employs a leave-one-cell-type-out strategy—training on all cell types and doses except the target cell type at non-control doses, then testing by comparing predicted expression for the target cell type treated above control against the ground-truth data. Here, portal hepatocytes are the target, using single-cell RNA-seq data from male C57BL/6 mouse livers orally gavaged with TCDD at 0.01, 0.03, 0.1, 1.0, 3.0, 10.0, or 30.0 μg/kg or vehicle every 4 days for 28 days (GSE184506); immune cells were removed to avoid dose-response biases,² leaving six cell types—cholangiocytes, endothelial cells, stellate cells, central hepatocytes, portal hepatocytes, and portal fibroblasts.

Institutional permissions

The datasets used in this protocol are the same as those from our previous study.¹ All animal procedures were approved by the Michigan State University (MSU) Institutional Animal Care and Use Committee (IACUC) and reporting of in vivo experiments follow the Animal Research: Reporting of In Vivo Experiments (ARRIVE)³ and Minimum Information about Animal Toxicology Experiments (MIATE) guidelines (https://fairsharing.org/FAIRsharing.wYScsE).

Preparation: Software installation and container access

Timing: <30 min

To improve user experience and avoid package dependency difficulties, we have containerized scVIDR with all the data and code required to use the software via Docker. Container options are available for both ARM64 and AMD64 architectures, with versions that either include or exclude data files based on your specific requirements or preferences. If you are unfamiliar with Docker, more information can be found here: https://docs.docker.com/get-started/. In brief, Docker enables software to be packaged into isolated “containers” that have been pre-configured to include the proper versions of various dependencies and any other requirements for running the software, including data files. These containers are built from Docker “images” which are hosted on Docker Hub for users to pull and run locally. The images for scVIDR can be found here: https://hub.docker.com/r/bhattacharyalab/scvidr/tags.

Singularity and high-performance computing cluster users

For users that routinely work on high-performance computing cluster (HPCC) environments, instructions on how to run the scVIDR containers via Singularity and bind their own data to them can be found on the documentation website: https://bhattacharyalab.github.io/scVIDR/installation/getting-started/#running-scvidr-in-an-hpc-environment.

To pull the scVIDR image and run the container on your local machine, the following steps should be performed.

• 1.Download and install Docker. You can install the desktop Docker application from Docker’s website. https://www.docker.com/products/docker-desktop/.
• 2.Once installed, open the Docker desktop application. This will start the Docker “daemon” which is required to run Docker command line arguments.
• 3.Open your local terminal.
• 4.To download the scVIDR Docker image that includes all the data, run this command in your terminal:

docker pull bhattacharyalab/scvidr:with-data

Note: This will take about 5 min. The image is about 11 GB in size as it contains the data used in the original paper. After pulling the image, it should appear in your Docker Desktop application within the Images tab.

• 5.Use the image to run the container with this command:

After this command, the container should appear in your Docker Desktop application within the Containers tab.

docker run -dit --name scvidr_container \

-p 8888:8888 bhattacharyalab/scvidr:with-data

Note: See troubleshooting 1.

Note: If you would like to utilize GPUs within this container you can add the flag --gpus all flag to the command. However, this requires NVIDIA GPUs and CUDA compatible hardware, so Mac users with Apple Silicon chips will be unable to use this flag. Instructions on using GPUs for HPCC environments can be found on the documentation website included in the previous section.

• 6.Access the Docker container with this command:

docker container exec -it scvidr_container /bin/bash

• 7.Start the Jupyter Notebook instance within the container with this command:

jupyter lab --port=8888 --ip=0.0.0.0 --allow-root --no-browser

• 8.Copy and paste the IP address output from the terminal into your web browser of choice. This will open a Jupyter Lab notebook with the data, code, and dependencies. The IP address should begin with http://127.0.0.1.

Preparation: Understanding the JupyterLab structure

On the lefthand side are several folders (bin, data, figures, metadata, notebooks, vidr) and several files (Dockerfile, LICENSE, README.md, requirements.txt) (Figure 1).

Figure 1.

Screenshot of the Docker container’s JupyterLab interface

Bin

This folder contains the Python files used to train the scVIDR model, predict gene expression, calculate gene scores, and evaluate its performance compared to scGen.⁴ These files are run directly from the command line (within the Jupyter Lab Docker container instance).

Data

This folder contains all the data files necessary to run the scripts and notebooks within this container. Users can also upload their own data into this folder to run scVIDR on their data. scVIDR requires input data to be AnnData files in h5ad format. More detailed instructions on using scVIDR and the expected format of data is included in the README.md file.

Figures

This folder contains the figures and supplementary figures from the main paper.

Metadata

This folder contains a file detailing the liver cell types analyzed in this study.

Notebooks

This folder contains the Jupyter notebooks with all the necessary code to create the figures step by step.

Vidr

This folder contains the source code for scVIDR. Including this allows users to make direct changes to the code for their specific use cases. Instructions on how to do this are included below.

Dockerfile

This file was used to build the Docker image. Users can modify this for their own projects. Detailed instructions on how to do this can be found at https://docs.docker.com/reference/dockerfile/.

License

This specifies the license agreement for the scVIDR software (MIT License).

README.md

This markdown file provides instructions on how to use scVIDR for its original use case. Input data for the model is expected to be an AnnData file in h5ad format. More detailed instructions on using scVIDR and the expected format of data are included in the README.md file under the “VAE model training (single- and multi-dose models)” section.

Requirements.txt

This file lists the dependencies necessary to run scVIDR.

∗Warning: creating a virtual environment (e.g. a Conda environment) on your personal machine with these dependencies to run scVIDR can be tricky. This is why we opted to use a Docker container.∗

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Deposited data
Single-nucleus RNA-seq dataset (mouse liver, TCDD exposure)	Nault et al.2	GEO: GSE184506
Single-cell RNA-seq dataset (PBMC, IFN-γ treatment)	Kang et al.5	GEO: GSE96583
Single-cell RNA-seq dataset (cross-experiment PBMC)	Zheng et al.6	SRA: SRP073767
Single-cell RNA-seq dataset (LPS6 species perturbations)	Hagai et al.7	E-MTAB-5919
Sci-Plex multiplexed chemical transcriptomics dataset (188 treatments)	Srivatsan et al.8	https://doi.org/10.1126/science.aax6234
Software and algorithms
Python (v.3.8.5)	Python Software Foundation	https://www.python.org/downloads/release/python-385/
Docker Desktop	Docker, Inc.	https://www.docker.com/products/docker-desktop/
Singularity (v.4.1.2)	Sylabs	https://github.com/sylabs/singularity/releases/tag/v4.1.2
JupyterLab	Project Jupyter
scVIDR (v.1.0)	Bhattacharya Lab GitHub	https://github.com/BhattacharyaLab/scVIDR
scanpy (v.1.9.1)	Wolf et al.9	https://scanpy.readthedocs.io/
scvi-tools (v.0.13.0)	Lopez et al.10	https://docs.scvi-tools.org/
scGen (v.2.0.0)	Lotfollahi et al.4	https://github.com/theislab/scgen; https://doi.org/10.1038/s41592-019-0494-8
scPreGAN	Wei et al.11	https://doi.org/10.1093/bioinformatics/btac357
CellOT	Bunne et al.12	https://doi.org/10.1101/2021.12.15.472775
NumPy (v.1.23.5)	Harris et al.13	https://numpy.org/
SciPy (v.1.10.0)	Virtanen et al.14	https://www.scipy.org/
scikit-learn (v.1.2.1)	Pedregosa et al.15	https://scikit-learn.org/
GeomLoss (v.0.2.5)	Feydy et al.16	https://www.kernel-operations.io/geomloss/
GSEApy (v.1.0.3)	Fang et al.17	https://gseapy.readthedocs.io/
Other
Zenodo archive of scVIDR repository	Bhattacharya Lab/Zenodo	https://doi.org/10.5281/zenodo.8025235

Step-by-step method details

After successfully activating the Docker container, follow the steps below to train the model and predict gene expression.

Example usage—Single-dose model training, prediction, and gene scores

Timing: 15–20 min (without GPU)

Timing: 3–5 min on a 32 GB NVIDIA V100 GPU (with GPU)

This section walks users through training and using the single-dose scVIDR model to predict gene expression and gene scores after a chemical perturbation.

• 1.Start the scVIDR Docker container as described above in the Preparation section.

Within the JupyterLab environment of the scVIDR Docker container, navigate to the terminal (Figure 2).

Note: See troubleshooting 2.

• 2.Change the directory to ‘bin’:

cd bin

• 3.Enter the following command to begin training the single dose prediction model for portal hepatocytes (predicting the gene expression of TCDD treated portal hepatocytes):

python scvidr_train.py single_dose \

  --celltypes_keep ../metadata/liver_celltypes \

  --test_celltype "Hepatocytes - portal" \

  ../data/nault2021_singleDose.h5ad \

  ../trained_models_tutorial/single_dose/portal_hepatocytes.pt

Note: This string of commands specifies the following: (1) train the model on a single dose, (2) select the appropriate metadata file, (3) use portal hepatocytes as the test cell type (in this case, but any cell type in the metadata file can be used), (4) select the appropriate dataset to load into the model, and lastly, (5) where to save the trained model by specifying the output location.

Note: Entering a new output location (../trained_models_tutorial/single_dose/) is important because the container already includes all of the trained models within the data directory. If we do not specify a unique output directory, the training will fail, as the code will not overwrite the existing model by default. The pre-trained models are in the data folder.

Note: With early stopping enabled (default), this training will take about 15–20 min without GPU use (depending on the computer’s specific resources). With GPU, it will take about 3–5 min. Early stopping causes fewer training epochs to be completed if the validation metric stops improving. This helps prevent overfitting and reduces computation time. If you would like to disable early stopping, add the following parameter to the previous command:

--early_stopping=False

• 4.Make a new directory to store the results of the prediction:

mkdir -p ../predictions_tutorial/single_dose

• 5.Run the prediction:

python scvidr_predict.py single_dose \

  --test_celltype "Hepatocytes - portal" \

  --celltype_column celltype \

  --dose_column Dose \

  --control_dose 0 \

  --treated_dose 30 \

  --celltypes_keep ../metadata/liver_celltypes \

  ../data/nault2021_singleDose.h5ad \

  ../trained_models_tutorial/single_dose/portal_hepatocytes.pt \

  ../predictions_tutorial/single_dose/portal_hepatocytes

Figure 2.

Screenshot of the Docker container’s JupyterLab interface

The red rectangle surrounds the terminal button that users can select to open the container’s terminal.

Warning: Be mindful of introducing extra spaces when copy and pasting this command. Any additional spaces may cause errors.

Note: The output of this command will be an h5ad file located in the predictions_tutorial/single_dose directory. This file contains the predicted gene expression profile of the cell type used for prediction (treated portal hepatocytes, in this case).

• 6.Calculate the gene scores of the predicted cells’ gene expression by entering the following command:

python scvidr_genescores.py \

  ../data/nault2021_singleDose.h5ad \

  ../trained_models_tutorial/single_dose/portal_hepatocytes.pt \

  ../predictions_tutorial/single_dose/portal_hepatocytes \

  --dose_column Dose \

  --celltype_column celltype \

  --test_celltype "Hepatocytes - portal" \

  --control_dose 0 \

  --treated_dose 30

The output of this command will be a CSV file that provides information on which genes are likely to change the most as the treatment dose is increased. The above command takes the original data as input, but the gene scores are calculated on the predicted gene expression. This is built into the scvidr_genescores.py functionality.

Example usage—Multi-dose model training, prediction, and gene scores

Timing: 1–2 h with early stopping enabled (without GPU)

Timing: 5–7 min with early stopping disabled on a 32 GB NVIDIA V100 GPU (with GPU)

This section walks users through training and using the multi-dose scVIDR model to predict gene expression and gene scores after a chemical perturbation at various doses.

• 7.Change the directory to ‘bin’:

cd bin

• 8.Enter the following command to begin training the multi dose prediction model for portal hepatocytes (predicting the gene expression of TCDD treated portal hepatocytes):

python scvidr_train.py multi_dose \

  --control_dose 0.0 \

  --celltypes_keep ../metadata/liver_celltypes \

  --test_celltype "Hepatocytes - portal" \

  ../data/nault2021_multiDose.h5ad \

  ../trained_models_tutorial/multi_dose/portal_hepatocytes.pt

Note: See troubleshooting 3.

• 9.Make a new directory to store the results of the prediction:

mkdir -p ../predictions_tutorial/multi_dose

• 10.Run the prediction:

python scvidr_predict.py multi_dose \

  ../data/nault2021_multiDose.h5ad \

  ../trained_models_tutorial/multi_dose/portal_hepatocytes.pt \

  ../predictions_tutorial/multi_dose \

  --model scVIDR \

  --dose_column Dose \

  --celltype_column celltype \

  --test_celltype "Hepatocytes - portal" \

  --control_dose 0.0 \

  --treated_dose 30.0 \

  --celltypes_keep ../metadata/liver_celltypes

• 11.Calculate the gene scores of the predicted cells’ gene expression by entering the following command:

python scvidr_genescores.py \

  ../data/nault2021_multiDose.h5ad \

  ../trained_models_tutorial/multi_dose/portal_hepatocytes.pt \

  ../predictions_tutorial/multi_dose/portal_hepatocytes \

  --dose_column Dose \

  --celltype_column celltype \

  --test_celltype "Hepatocytes - portal" \

  --control_dose 0.0 \

  --treated_dose 30.0 \

  --celltypes_keep ../metadata/liver_celltypes

• 12.
  Recreate the figures in the paper (Optional steps):

  • a.
    Within the JupyterLab instance, recreate the figures from the paper by navigating to the “notebooks” directory.

Within the “notebooks” directory, there will be several “.ipynb” files that are ready to run.

Adding files and making changes to the code

Adding files to the container’s Jupyter Lab environment can be achieved by clicking the “Upload Files” (up-arrow) at the top of the web page (Figure 3).

Note: Any changes you make to the Docker container will be retained unless the container is deleted. To start fresh, the container can be removed by entering the following commands in your local terminal:

docker stop scvidr_container

docker rm scvidr_container

Note: Instead of removing the container, if you would like to save any changes you made to a new container, you can follow the next commands:

• 13.Open a new terminal session and enter the following command, inserting the appropriate arguments for the placeholders:

docker commit <container_name_or_id> <new_image_name>

Note: This command creates a new image based on the current instance of the container. Here, <container_name_or_id> is scvidr_container. The <new_image_name> argument can be changed to anything you would like. For this example, we will name it ‘updated_scvidr_container.’

Note: After running this command, a new Docker image should appear in the Docker Desktop application within the Images tab.

• 14.Run the new image to start the container with the saved changes by running this command:

docker run -dit --name updated_scvidr_container -p 8889:8889 updated_scvidr_container

Note: The port has changed from 8888 to 8889, so you can see both the original container and your updated container at the same time.

Figure 3.

Screenshot of the Docker container’s JupyterLab interface

The red rectangle surrounds the upload arrow that users can select to upload files to the container.

After this command, a new container should appear in your Docker Desktop application within the Containers tab with the name “updated_scvidr_container.”

• 15.Enter the container with this command:

docker container exec -it updated_scvidr_container /bin/bash

• 16.Start the Jupyter Notebook instance within the container with this command:

jupyter lab --port=8889 --ip=0.0.0.0 --allow-root --no-browser

• 17.As before, copy and paste the IP address that begins with http://127.0.0.1 into your browser of choice.

In this new Jupyter Lab environment, you should see that any changes made to the original files are maintained.

Expected outcomes

Our protocol provides a robust and user-friendly computational workflow for modeling the effects of chemical perturbations on single-cell gene expression dynamics using scVIDR. This approach leverages advanced generative modeling to systematically characterize cellular responses to chemical perturbations using single-cell RNA-sequencing data. By following this protocol, users can quickly recreate scVIDR’s findings, modify the code as they see fit, or completely rework it for their specific use cases and data.

Limitations

Like other computational tools, the quality of scVIDR’s predictions is constrained by quality of the input data. High levels of noise, gene dropout events, or insufficient sample replicates, among other factors, will compromise the reliability of the results. In addition, scVIDR has limited ability to infer causal relationships from its predicted gene expression profiles, and experimental validation should be considered to confirm its predictions.

Troubleshooting

Problem 1

The browser is unable to connect to Jupyter Lab after successfully pulling and running the Docker image.

Potential solution

Make sure the local port is set to 8888 (or whatever was used when running the Docker container from the image). This was set using the –p 8888:8888 argument.

Incorrect: https://127.0.0.1/lab.

Correct: http://127.0.0.1:8888/lab.

Problem 2

Getting lost in the container’s JupyterLab terminal.

Potential solution

All of the directories and files in the container are within the scVIDR directory. To navigate back to the appropriate directory enter the following command.

cd /scVIDR

Problem 3

The kernel dies while running the code.

Potential solution

To remedy this, allocate more memory to the container through the Docker Desktop application within the “Resources” tab in the Docker Desktop settings.

Resource availability

Lead contact

Further information and requests for resources and reagents should be directed to and will be fulfilled by the lead contact, Dr. Sudin Bhattacharya (sbhattac@msu.edu).

Technical contact

Technical questions on executing this protocol should be directed to and will be answered by the technical contact, Vishal Panda (pandavis@msu.edu).

Materials availability

No materials were generated in this protocol.

Data and code availability

• •For additional information on using scVIDR, including HPCC environment-specific information, please visit the documentation website: https://bhattacharyalab.github.io/scVIDR/.
• •The Docker Hub for scVIDR can be found at https://hub.docker.com/r/bhattacharyalab/scvidr/tags.
• •In addition to the information provided here, more information about the package, along with a step-by-step tutorial, can be found on the documentation website: https://bhattacharyalab.github.io/scVIDR/. Information on how to use scVIDR within HPCC environments can also be found on the documentation website.

Acknowledgments

This work was supported by Michigan AgBioResearch, the National Institute of Environmental Health Sciences (R01 ES031937, P42 ES004911, and T32 ES007255), and the National Human Genome Research Institute (R21 HG010789) of the National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. This work was supported in part through computational resources and services provided by the Institute for Cyber-Enabled Research at Michigan State University. The graphical abstract demonstrates the workflow for using scVIDR. Images for steps 2 and 3 of the figure are adapted with permission from Kana et al., 2023.

Author contributions

D.E.B.: writing (original draft, review, and editing), visualization, software, and validation; V.P.: software and validation; D.M.: conceptualization, formal analysis, and methodology; O.K.: conceptualization, formal analysis, investigation, methodology, and software; S.B.: conceptualization, formal analysis, funding acquisition, project administration, supervision, validation, and writing (review and editing).

Declaration of interests

The authors declare no competing interests.

Declaration of generative AI and AI-assisted technologies in the writing process

During the preparation of this work, the author(s) used ChatGPT (basic version) for light editing and assistance with formatting terminal commands. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the publication.