Food Analysis Lab · W14

PLS Quantification II: VIP, PLS-DA and Outliers

Variable Importance in Projection, PLS-DA, and Outlier Diagnostics

Food Analysis Lab · 4-hour session
This week is mostly data analysis in the computer lab: VIP variable importance · classification and the confusion matrix · Hotelling T²/Q outliers

VIPVIP>1 filterPLS-DALogistic regressionConfusion matrixHotelling T²Q residualApply DomainOrange OR14Colab NB12

A question to start with

The model says this wavelength matters most —
do you believe it?

A true story (anonymized): while preparing the materials for this course, a script that computes VIP had one extra .sum() inside its loop, which collapsed “each wavelength’s own contribution” into a single number too early — so all 228 wavelengths received exactly the same VIP value, and the “top 20 VIP wavelengths” the script printed were meaningless.

Three simple checks would have caught this bug:

🟰

All VIP values identical

228 wavelengths should differ — all being the same is a warning sign in itself

➗

mean(VIP²) ≠ 1

VIP is defined so that “the mean of VIP² over all wavelengths = 1”; if you don’t get 1, the formula or the coefficients are wrong

🧪

High VIP does not match the chemistry

If the “most important” bands don’t match any known functional-group absorption, be suspicious instead of copying them down

In this class we will use methods that check themselves and walk through VIP, classification, and outlier judgment again.

W14 / VIP + PLS-DA + MODEL DOMAIN

Verify VIP, then interpret classification and model domain

Full narration and captions; three simulated NB12 datasets. Player and downloads · Interactive lab

W14 / PREDICT · OBSERVE · EXPLAIN

Six acts: VIP, classification, and T-squared/Q

Open the full interactive lab

Today’s learning goals

After W14 you should be able to…

Beginner No coding: Orange OR14 (PCA + logistic regression classification, Apply Domain to find out-of-model samples) Advanced Colab NB12 (VIP, PLS-DA, Q residual)

📈 VIP variable importance

Advanced NB12

Say what VIP computes: each wavelength’s importance across all LVs, weighted by SSY (the y variance explained by each LV); VIP>1 = above average
Tell VIP (importance, always positive) from the regression coefficient b (direction, positive or negative)
Verify VIP with three checks: all identical? mean(VIP²) = 1? matches the chemistry?
Compare before vs. after variable selection using RMSEP on a test set not used in the selection, to avoid selection bias (advanced, NB12)

🎯 Classification and the confusion matrix

Beginner Orange Advanced NB12

Explain how PLS-DA turns classification into regression with one-hot + argmax
Classify with Orange PCA + logistic regression and read the confusion matrix (accuracy, sensitivity, specificity)

🚨 Outlier diagnostics

Beginner Orange Advanced NB12

Tell T² (how far from the center within the plane) from Q (how far from the plane), and work a simple example
Read the 95% limits on a T² vs. Q influence plot; use Orange Apply Domain to find out-of-model samples
Check outliers first, record the reason, never delete them silently; know that changing instruments often gives high Q

4-hour lab schedule

This week is mostly data analysis in the computer lab

Time	Contents
0:00–0:25	Lecture: W13 recap (w, t, b), VIP principle and one hand calculation
0:25–1:15	Reading VIP: VIP vs. coefficient, VIP>1 filtering, selection bias (NB12 demo)
1:15–1:40	Lecture: classification principles (one-hot/argmax, PLS-DA, logistic regression, confusion matrix)
1:40–2:30	Orange OR14, part 1: PCA + logistic regression on six oils, read the confusion matrix
2:30–2:55	Lecture: Hotelling T² and Q residual, influence plot
2:55–3:30	Orange OR14, part 2: Apply Domain to find out-of-model samples + Corn outlier module
3:30–4:00	Group sharing: do the high-VIP bands match the chemistry? Which two oils are confused most easily?

First, review W13 PLS ↗

The three letters in a PLS model: w, t, b

LV (latent variable): a “composite indicator” PLS builds from the spectra; each LV is related to both the spectra and the concentration
w (weights): one set per LV, one number per wavelength = “how much this wavelength counts in this LV”
t (scores): the spectrum (after subtracting the mean spectrum) multiplied by w and summed. Example: w=(0.6, 0.8), x=(0.5, 0.25) → t=0.3+0.2=0.5
b (regression coefficients): obtained by linking w and q; one b per wavelength, and the sign tells you the direction

Principle 1

VIP: variable importance in projection, each symbol in plain words

VIP_j = √( J · Σ_a SSY_a·(w_ja/‖w_a‖)² / Σ_a SSY_a )

j = the j-th wavelength; J = total number of wavelengths (written p in the NB12 code; J=81 here); a = the a-th LV
SSY_a = q_a²·Σt_a² = the y variance explained by the a-th LV (lowercase q is the slope of y on t, unrelated to the Q residual later)
(w_ja/‖w_a‖)² = the share of wavelength j in the weights of the a-th LV; the shares within each LV sum to 1

In plain words: VIP² = J × “the share across LVs, weighted-averaged by SSY” — a wavelength with a big share in the LV that explains the most y gets a high VIP
Shares sum to 1 → the mean of VIP² over all wavelengths is exactly 1 → VIP>1 = above average (a rule of thumb, not a test)
Share of y variance explained by the 4 LVs in NB12: 84.5% / 13.6% / 1.6% / 0.3% — the first LV pretty much decides everything

Calculating VIP once

3 wavelengths, 2 LVs: work it out by hand

Assume SSY = [8, 2] (LV1 explains 8, LV2 explains 2, total 10); J = 3

Wavelength	LV1 share	LV2 share	Σ SSY × share	VIP² = J × … ÷ ΣSSY	VIP
Wavelength 1	0.64	0.00	8×0.64 + 2×0.00 = 5.12	3×5.12÷10 = 1.536	1.24
Wavelength 2	0.36	0.20	8×0.36 + 2×0.20 = 3.28	3×3.28÷10 = 0.984	0.99
Wavelength 3	0.00	0.80	8×0.00 + 2×0.80 = 1.60	3×1.60÷10 = 0.480	0.69

Check: mean(VIP²) = (1.536 + 0.984 + 0.480) ÷ 3 = 1.00 ✓ This holds even with only 3 wavelengths.

Wavelength 1 counts only in the big LV (SSY=8) → highest VIP; wavelength 3 has a large share, but in the small LV (SSY=2) → VIP<1.

VIP ≠ direction

VIP has no sign; the coefficient b does

NB12	VIP	Coefficient b
1200 nm (acetic acid C–H, illustrative)	2.57	+4.96
1450 nm (water O–H first overtone, illustrative)	1.82	-4.01

Both bands have VIP >1: both are important
b is one positive and one negative: holding the other inputs fixed, a higher input at 1200 nm raises the predicted concentration, while at 1450 nm (water) it does the opposite; how the actual spectra change with concentration must be checked in the data separately — it can’t be read directly from b
To talk about “direction”, always look at b; VIP only answers “how important”

A 4-LV PLS model built from the NB12 demo data (simulated spectra, not this class’s measurements); blue line = VIP (always ≥0), orange line = coefficient b.

Data · VIP

VIP vs. wavelength: rebuild the model by filtering with VIP > 1

NB12 model	No. of wavelengths	LV	RMSECV	RMSEP
Full range	81	4	0.1756	0.1658
VIP>1	18	3	0.2256 ↑	0.1329 ↓

Wavelengths kept: 1150–1250 nm (11) and 1420–1480 nm (7) — exactly the absorption bands of acetic acid and water ✓
RMSECV got worse, RMSEP got better: the two metrics tell opposite stories
The test set has only 20 samples, so an RMSEP difference of 0.03 may be just luck — neither metric can be trusted on its own

NB12 demo data (simulated): training set 60, test set 20; red line = VIP=1. Unit: acetic acid %.

Selection bias

Using the same data to select wavelengths and score them = setting your own exam and grading it

❌

Select and evaluate on the same data

Compute VIP and pick wavelengths from all the data, then score on the same data — the model has peeked at the answers, so the score is optimistic

✅

Select on the calibration set only

Compute VIP using only the calibration (training) set; after filtering, compute RMSEP once on a test set that has never been touched

🔁

Or re-select within each fold

In cross-validation, recompute VIP and re-filter in every fold so that RMSECV is fair

What NB12 did: VIP was computed on the 60 training samples, and RMSECV (after filtering) was also computed on the same training samples → this is “select and evaluate on the same data”, so beware of optimistic bias; RMSEP uses 20 test samples never touched → fair, but there are too few samples. The VIP>1 threshold was set in advance, not tuned by looking at RMSEP.

Mini-game ①VIP and regression coefficient: live quiz0 / 5

Classification ①

One-hot encoding and argmax: turning classes into numbers

	CAN	COC	EVOO	PEA	ROO	SOY	Result
One-hot of EVOO (the answer)	0	0	1	0	0	0	Column 3 = 1
NB12 test sample ŷ (actual COC)	-0.13	0.91	-0.17	0.22	0.37	-0.21	argmax → COC ✓
Misclassified sample ŷ (actual CAN)	0.25	-0.07	0.25	0.10	-0.13	0.60	argmax → SOY ✗

One-hot: 6 oils → 6 columns of 0/1, and the column of the class it belongs to is 1 (column order follows NB12: alphabetical by English code)
argmax: the model gives a score ŷ for each of the 6 columns, and the column with the largest score is the answer
ŷ is not a probability: it can be negative or exceed 1, and is only used for comparing sizes

Codes: CAN = canola oil, COC = coconut oil, EVOO = extra virgin olive oil, PEA = peanut oil, ROO = refined olive oil, SOY = soybean oil.

Classification ②

PLS-DA: turning a classification problem into regression

y changes from “one concentration” to “6 one-hot columns”; PLS regresses all 6 columns at once (the same PLS as in W13)
The number of LVs is again chosen by cross-validation: NB12 picked 6 LVs
Each new sample gets 6 ŷ values → argmax decides the class
Finally, the test-set results are summarized in a confusion matrix (next two pages)

The six-oil data in NB12: 6 oils × 15 samples = 90; 30% were drawn as the test set, stratified by class (27 samples), and the remaining 63 were used to build the model.

Orange has no PLS-DA widget; the beginner route does the same thing with “PCA scores + logistic regression” (next page).

Classification ③ Orange route

PCA scores + logistic regression: scores → probability

Compress first: PCA condenses 70 wavelengths into 6 scores (W12 PCA ↗), and the classifier sees only these 6 numbers
Logistic regression: add up the weighted scores to get z; the S-curve p = 1/(1+e^−z) in the figure is the demo for two-class problems (a single sigmoid is used only when there are two classes)
The six oils need multinomial logistic regression: one z_k per class, normalized together by softmax into probabilities p_k = exp(z_k)/Σexp(z_j), the six probabilities sum to 1, and the largest class is chosen — the same idea as argmax
C=100: the strength of regularization (keeping the weights from getting too large); the larger C is, the weaker the restriction and the more closely the model fits the training data

Orange OR14 part 1

Six-oil classification: File → PCA → Logistic Regression → Confusion Matrix

PCA: set the number of components to 6 and leave Normalize unchecked (spectra share the same unit, so no standardization is needed); connect Transformed Data to Test and Score
Test and Score uses 5-fold cross-validation; but PCA was fitted on all 90 samples before being connected — so the validation folds have effectively seen that PCA (data leakage) and CA will be optimistic; use the independent NB12 test set for the formal evaluation (PLS-DA 24/27 = 88.9%)
Double-click Confusion Matrix: rows = actual, columns = predicted; try changing the number of PCs or C and see how accuracy changes

How to read a confusion matrix

NB12 PLS-DA: test set 24 / 27 = 88.9%

Actual \ Predicted	CAN	COC	EVOO	PEA	ROO	SOY
CAN	4	0	0	0	0	1
COC	0	4	0	0	0	0
EVOO	0	0	4	0	0	0
PEA	0	0	0	5	0	0
ROO	0	0	1	0	2	1
SOY	0	0	0	0	0	5

Rows = actual class, columns = model prediction; the green diagonal = correct
Accuracy = sum of the diagonal ÷ total = 24 ÷ 27 = 88.9% (6 LVs)
Red = misclassified: CAN→SOY (1), ROO→EVOO (1), ROO→SOY (1)
ROO (refined olive oil) has the most errors: being called EVOO is because the two share the olive-family baseline; being called SOY is a cross-family misclassification and may just be chance from too few test samples

NB12 demo data (simulated spectra, not this class’s measurements).

One class at a time

Sensitivity and specificity: ROO as an example

	Predicted ROO	Predicted not ROO
Actually ROO	TP = 2 caught	FN = 2 missed
Actually not ROO	FP = 0 false alarm	TN = 23 correctly excluded

Collapse the 6×6 matrix into a 2×2 of “ROO / not ROO”.

Sensitivity = TP÷(TP+FN) = 2÷4 = 0.50: only half of the true ROO were caught
Specificity = TN÷(TN+FP) = 23÷23 = 1.00: no other oil was ever misclassified as ROO
Compare SOY: sensitivity 1.00, but 2 other oils were misclassified as SOY → specificity = 0.909
Low sensitivity = missed cases; low specificity = false alarms — look at both together

Mini-game ②Is this a classification or a quantification problem?0 / 6

Sort 6 food-analysis scenarios into “classification problem (PLS-DA)” or “quantification problem (PLS regression)”.

Outliers ①

Hotelling T²: how far from the center within the model

T² = Σ t_a² / λ_a: t_a = the sample’s score on the a-th PCA principal component; λ_a = the variance of the calibration set on that component (eigenvalue, W12 PCA ↗)
In plain words: use each component’s own width as the ruler — one step in a narrow direction is more “abnormal” than one step in a wide direction (B is farther than A)
95% limit: an ellipse computed from the F distribution, with about 95% of normal samples falling inside; NB12 (40 calibration samples, 2 principal components) uses the approximate limit k(n−1)/(n−k)·F = 6.66
Strictly speaking, this limit is for the calibration set itself; for new samples multiply it by (n+1)/n, about 6.83 — the difference is small and the outlier conclusions don’t change

Outliers ②

Q residual (SPE): how far from the model plane

Q = Σ(x − x̂)²: x̂ = the spectrum “reconstructed” by the model from the principal components; the difference is the part the model cannot explain
Example: residuals at three wavelengths [0.01, −0.02, 0.02] → sum of the three squared residuals Q = 0.0001 + 0.0004 + 0.0004 = 0.0009; this is only a calculation exercise — whether it exceeds the limit must be judged against a limit built from the same model, same wavelengths and same preprocessing, never a limit computed from another model
95% limit (Jackson–Mudholkar approximation): from how much variance the calibration set still has in the discarded principal components, it estimates the Q value that 95% of normal samples will not exceed; NB12 = 0.00118
Note: Q (uppercase) is the residual; the lowercase q in the VIP formula is a slope — the two are unrelated

Outliers ③

T² and Q together: four cases

✅

Low T², low Q: normal

Within the calibration range, and the model recognizes this kind of spectrum

📐

High T², low Q: extreme but recognized

The model understands the chemical composition, it is just more extreme than anything in the calibration set. NB12 leverage point: T² = 28.4 (limit 6.66), Q = 0.00091 (limit 0.00118)

🌀

Low T², high Q: the model has never seen this

New variation the model cannot explain (a new impurity, a different instrument). NB12 instrument-change samples: T² = 0.63, 3.17, with Q about 33 times the limit

🚨

High T², high Q: check first

Both extreme and carrying new variation — go back to the spectrum and the sample first, then decide whether it can be used

Mini-game ③High T² vs. high Q0 / 6

Using the definitions you just learned, decide: is this scenario more like “high T² (leverage)” or “high Q (new variation the model can’t explain)”?

Data · T² vs. Q influence plot

NB12 demo: calibrated on one instrument, applied to an extreme sample and to another instrument

Orange leverage point: T² = 28.4 is far above the limit of 6.66, but Q = 0.00091 is within the limit → extreme but recognized by the model
Red instrument-change points: T² is normal, Q is about 33 times the limit → an instrument difference the model has never seen
The vertical axis is a log scale: each step up = 10 times larger
Looking at T² alone misses the red points; looking at Q alone misses the orange point — look at both

NB12 demo data (simulated PCA, 40 calibration samples, 2 principal components; not real Corn values). Dashed lines = 95% limits.

Orange OR14 part 2

Apply Domain: project new samples onto the calibration-set PCA

Applicability domain: a model is reliable only for samples “similar to the calibration set”; Apply Domain transforms all samples using the calibration-set PCA
A point far from the center of the calibration cluster on the scatter plot = intuitively a high T² (e.g., leverage_outlier)
Orange has no Q residual: an instrument-change sample may look perfectly normal, and you need Colab NB12 to compute Q
OR14 uses 3 principal components and NB12 uses 2, so positions in the plot will not reproduce numbers like 6.66 or 28.4 — just look at “who is outside the cluster”

Found an outlier — what now?

Check first, never delete silently

🔍

First go back to the spectrum/sample itself

Bubbles, impurities, a sample placed the wrong way round, a dilution error? Rule out operational problems first, then talk about statistical judgment

🚫

Never delete data silently

Record the reason before deleting; deleting when you can’t find a reason is manufacturing false accuracy (the same principle as W08 Grubbs outliers ↗)

🔁

Re-validate when transferring a model

A model built on one instrument often gives high Q on another instrument — re-check when changing instruments or batches, and do calibration transfer if needed

📖 Outlier detection teaching module (Corn dataset, T²/Q influence plot) → — practice with the T²/Q influence plot on the Corn dataset (80 samples × 700 wavelengths, three instruments m5/mp5/mp6).

Mini-game ④A model result looks odd — how do you investigate?— / 6

Use ▲▼ to put the checks you would do when “you doubt a model result” into a sensible order.

Mini-game ⑤Decision challenge: VIP / classification / outlier judgment0 / 5

Mini-game ⑥Calculation challenge: mean(VIP²) of these 5 VIP valuesUnanswered

From the VIP table printed by NB12, 5 wavelengths (900, 1100, 1300, 1500, 1700 nm) were picked out, with VIP values 0.753, 0.564, 0.342, 0.707, 0.292. Find mean(VIP²) of these 5 values (hint: these are only a part of all 81 wavelengths, not all of them).

Homework

Beginner Orange OR14 + Advanced Colab NB12

🎯

Beginner: confusion matrix

OR14 part 1: attach a screenshot of the six-oil confusion matrix, give the accuracy and the two most-confused oils, and calculate the sensitivity/specificity of one class

🔎

Beginner: who is outside the model?

OR14 part 2: attach the Apply Domain scatter plot, point out the samples outside the calibration cluster, and explain why an instrument-change sample may not be visible on the plot

📈

Advanced: VIP plot + chemical interpretation

NB12: plot VIP and the coefficient b, mark the bands with VIP>1 and relate them to the chemistry; compare RMSECV/RMSEP before and after filtering and discuss selection bias

🚨

Advanced: T²/Q case

NB12: pick one sample that exceeds a limit on the influence plot, explain whether its T² is high, its Q is high, or both, and how you would handle it

Open NB12 in Colab ↗ · Beginner (no coding): Orange workflow OR14 ↗. Please also include the conclusions from each group’s sharing in your report: whether the high-VIP regions match the chemistry, and which two oils are confused most easily.

Self-check

By the end of today you should be able to…

✓Say what VIP computes: each wavelength’s importance across all LVs, weighted by SSY (the y variance explained by each LV); VIP>1 = above average
✓Tell VIP (importance, always positive) from the regression coefficient b (direction, positive or negative)
✓Verify VIP with three checks: all identical? mean(VIP²) = 1? matches the chemistry?
✓Compare before vs. after variable selection using RMSEP on a test set not used in the selection, to avoid selection bias (advanced, NB12)
✓Explain how PLS-DA turns classification into regression with one-hot + argmax
✓Classify with Orange PCA + logistic regression and read the confusion matrix (accuracy, sensitivity, specificity)
✓Tell T² (how far from the center within the plane) from Q (how far from the plane), and work a simple example
✓Read the 95% limits on a T² vs. Q influence plot; use Orange Apply Domain to find out-of-model samples
✓Check outliers first, record the reason, never delete them silently; know that changing instruments often gives high Q

Tap to tick — make sure you really can do it.