Figure 1.

Study design and neural network architecture. (a) Deep Profiler and iGray are obtained using a 5-fold cross-validation with an 80:20 ratio for training and validation within each fold, respectively. iGray is calculated by combining image and clinical data and using the cumulative incidence function of the regression model. (b) Neural network structure. The neural network consists of three main parts: an encoder for extracting imaging features and building a task-specific fingerprint, a decoder for estimating handcrafted radiomic features and a task-specific network for generating an image signature for therapy outcome prediction. A 3-D convolutional neural network (CNN) was used as an encoder for extracting imaging features. A fully-connected layer was adopted to link the latent fingerprint space with classical radiomics. Another neural network was used to predict the outcome based on latent fingerprint variables. Since therapeutic outcomes contain time-to-event information, we used a proportional hazards model to relate the time that passes before local failure occurs to classify failures.