ML comprises various data analysis algorithms that help extract valuable features from complex data and predict the outcome of interest. It can be divided into Unsupervised ML (UML) or Supervised ML (SML) (Figure 2). UML is usually used to extract features (clinical, imaging, genetic, and many other features) without focusing on the outcome. The goal is to discover features which can divide the populations into unique clusters (unbiased by our clinical judgement) and could demonstrate a difference in clinical outcomes. Once we have these features, we use them to define a predictive and prognostic model for our outcome of interest by using SML methods(1).

NLP

ML algorithms can use imaging and genetic data with relative ease compared to clinical data. Although clinical data is human-readable, it is tough for machines to comprehend it until we pre-process it manually. If we have a massive pile of clinical data, manual pre-processing becomes arduous, and it is often impossible to decipher helpful information. In this situation, NLP significantly extracts valuable information from the narrative text to assist clinical decision-making. For example, it can read Chest X-ray reports and alert us about antibiotic use (2) or automatically alert us if there are some issues with laboratory results (3).

Deep Learning: a more advanced ML technique

A neural network with a single layer of nodes (each node is a mathematical linear regression expression) to capture non-linear patterns in data is classically used as one of the SML methodologies. Deep learning is the broader and deeper extension of this classic neural network. In most cases, deep learning is used in conjunction with complex medical images with a vast amount of numerical data in each image. One commonly used algorithm in this context is Convolutional Neural Network (CNN).

Medical images are taken as input, and relevant, helpful information is extracted from those images. It is correlated with the clinical outcome of interest, and finally, we have a fully trained model which can predict the outcome based on medical images. This whole feature extraction process from medical images and model building is fully automated without human intervention. This is also one of the drawbacks, as this entire process works as a ‘Blackbox’, where we do not have any access to the logic of decision-making.

However, recent advances have somewhat weakened this criticism by plotting the weights of different layers of CNN into the image and thus highlighting the area most helpful in decision-making. This allows us to understand and confirm the biological basis of decision-making. For example, in lung cancer, a pre-treatment CT scan can be used to predict the survival outcome (4, 5) and post-SBRT risk of relapse, as well as radiation pneumonitis (6, 7).