When investigating the relationships between multivariate data, principal component analysis (PCA) is used as a method to reduce the dimensions to about two or three dimensions and examine the relationships between the data.) is often used.
However, in recent years, methods such as UMAP and t-SNE have been gaining attention as dimensionality reduction methods. These are nonlinear dimensionality reduction methods, and are said to be effective even when the data structure cannot be expressed well using principal component analysis, which is a linear dimensionality reduction method.
These techniques are also dimensionality reduction methods, allowing you to reduce multidimensional data to two or three dimensions and visually examine patterns in the data.
In the field of machine learning, UMAP and t-SNE are widely used for dimensionality reduction, which is useful for a variety of applications, including image recognition, text classification, and genetic analysis.
JMP Pro has a platform called "Multivariate Embedding" that includes the UMAP and t-SNE methods.
Example of sample grouping using t-SNE
In this blog, we use spectral data to reduce the dimensionality using principal component analysis, UMAP, and t-SNE, and visualize the relationships in the data, while examining whether the labeled data can be well separated.
Spectral data used
The data we will use this time is mid-infrared spectrum data of olive oil.*
Spectra of 60 kinds of olive oils were collected by sampling. Each sample was tagged with country of origin. We will investigate whether the country of origin (Greece, Italy, Portugal, Spain) can be classified from this multivariate spectral data.
In machine learning, country of origin information is equivalent to a "label" and is supervised data. In subsequent analysis, the focus is on whether the classification can be done according to the label.
Using Graph Builder, we graphed the spectra for each country of origin. The intensity for each wavenumber is shown, allowing us to visually confirm the characteristics of each country's spectrum and its differences from other countries, such as peaks at specific wavenumbers and spectra with large or small fluctuations in intensity.
*Data source:
Data for: Spectra Data Classification with Kernel Extreme Learning - Mendeley Data
Dimensionality reduction by principal component analysis
First, we perform a principal component analysis (PCA) on this spectral data and visualize the first two components in a score plot.
Since we are measuring the intensity at 570 wavenumbers, we are reducing the 570 dimensions to 2 dimensions.
Looking at the score plot, we can see that there is a lot of overlap between different plot points, and the olive oil countries of origin are not well separated. Since the dimensionality reduction performed by principal component analysis is based on a linear transformation, it may not be able to capture the structure of the data well if the data is distributed nonlinearly.
Dimension reduction with UMAP and t-SNE
Next, we run UMAP and t-SNE on the olive oil data.
In JMP Pro, choose Analyze > Multivariate > Multivariate Embedding .
- In the launch dialog, specify the continuous column (in this example, column 570 showing intensity at wavenumber) as [Y, Column].
- Options include the method, the dimension after dimensionality reduction, etc.
- You can choose [UMAP] or [t-SNE] in "Method". The default is [UMAP].
- You can select the "dimension after dimensionality reduction". The default is "2" (2 dimensions).
The plots shown below show the results when both UMAP and t-SNE were run using these operations (bottom left: UMAP, bottom right: t-SNE).
Comparing this to the score plot of the principal component analysis, we can see that the olive oil countries of origin are separated to some extent. In particular, the orange color Spain can be clearly distinguished from the other countries of origin.
Next, let's set the "Dimension after dimensionality reduction" to 3 and visualize it in 3 dimensions. The "Multivariate Embedding" report shows a 3D plot. You can move the cursor around in this plot by dragging it, and the classification is well done, including the 3rd dimension.You can consider what is being done.
In the 2D plot of t-SNE, classification was not done well for anything other than the orange color Spain, but in the 3D plot, it was able to classify the green colors (Italy), red colors (Greece), and blue colors (Portugal) to some extent.
Both UMAP and t-SNE are nonlinear dimensionality reduction methods that reduce the dimensionality in a way that preserves the relationships between data points that are close in a high-dimensional space, even in lower dimensions. As such, they are excellent ways to visually group data.
However, while principal component analysis allows you to interpret the axes from reports such as loading plots, UMAP and t-SNE methods focus on finding patterns, such as relationships between data (samples) and dividing them into groups, so it can be difficult to interpret the meaning of the axes.
For example, in the UMAP plot above, a lot of the orange Spain data is plotted in the bottom right, but it is difficult to determine what being located in the bottom right on the two-dimensional coordinate system indicates.
Comparison of UMAP and t-SNE
UMAP is said to be faster than t-SNE in terms of calculation speed. It is also said that UMAP is easier to adjust parameters.
In fact, when we ran t-SNE with the default parameter settings for the spectral data example, we received a warning about convergence calculations. As described in the message, we resolved this by relaxing the convergence criteria (1e-8 to 1e-4) and increasing the maximum number of iterations (1000 to 10000).
Comparing UMAP and t-SNE, UMAP is generally said to be superior, but in practice it is better to try both UMAP and t-SNE and compare the results. However, when dealing with large-scale data, t-SNE may take a long time to calculate, and parameter adjustments may be required in some cases.
by Naohiro Masukawa (JMP Japan)
Naohiro Masukawa - JMP User Community
