進階課：Time-series e-nose Feature Engineering

電子鼻原始資料通常是一段反應曲線，不是一個數字

當樣本放進腔體後，感測器反應會從 baseline 開始上升，接著進入較穩定的區域；清洗或抽氣時，又會逐漸回復。這整段曲線包含很多資訊：反應強度、反應速度、穩態值與恢復能力。

建議資料：葡萄酒酸敗或釀造醋電子鼻時間序列

進階課需要原始時間序列資料。最適合的公開資料是葡萄酒酸敗 e-nose dataset，因為每個檔案就是多個 sensor 隨時間的反應；也可以使用釀造醋或水果新鮮度資料。

time-seriesbaseline correctionfeature extractionOrange-ready CSV

一條曲線可以萃取哪些特徵？

特徵工程不是亂取數字，而是把感測器反應的物理意義轉成模型能使用的欄位。你可以把每個 sensor 的曲線拆成四類資訊：背景、強度、速度與形狀。

從一堆 txt / xlsx 檔，整理成一張特徵表

原始資料夾
  ├── sample_001.txt      每個檔案 = 一個樣本的時間序列
  ├── sample_002.txt
  └── ...
        ↓ Python 逐檔讀取
        ↓ baseline correction
        ↓ 對每個 sensor 萃取 max / mean / AUC / slope / time_to_max
        ↓ 從檔名或 metadata 取得 class label
        ↓ 合併成 sample × features table
features.csv
  sample_id | class | MQ3_max | MQ3_auc | MQ3_slope | MQ4_max | ...

這張 features.csv 就是接下來做 PCA、PLS-DA、SVM、Random Forest 或 Orange workflow 的共同起點。

把 time-series 轉成 feature table

下面程式碼用通用寫法示範。不同資料集的欄名與檔案格式可能不同，你只需要修改 sensor_cols、標籤解析方式與取樣時間欄位。

1. 讀取一個 time-series 檔案

from pathlib import Path
import pandas as pd
import numpy as np

# 例：wine spoilage dataset 解壓後，每個 txt/csv 是一個樣本
file_path = Path("Dataset/HQ_Wines/HQ_Wine01-B01_R01.txt")

# 依實際分隔符調整 sep：常見是 comma、semicolon、tab 或 whitespace
df = pd.read_csv(file_path, sep=None, engine="python")
print(df.head())
print(df.columns.tolist())

2. 對單一 sensor 萃取特徵

def extract_sensor_features(t, x, baseline_seconds=10, steady_seconds=30):
    """Return feature dict for one sensor time series."""
    t = np.asarray(t, dtype=float)
    x = np.asarray(x, dtype=float)

    # baseline: first N seconds
    base_mask = t <= (t.min() + baseline_seconds)
    r0 = np.nanmean(x[base_mask])

    # baseline-corrected and normalized response
    dx = x - r0
    if abs(r0) > 1e-12:
        ndx = dx / r0
    else:
        ndx = dx

    # steady-state: last N seconds
    steady_mask = t >= (t.max() - steady_seconds)

    # initial slope: first 20 seconds after baseline window
    slope_mask = (t > t.min() + baseline_seconds) & (t <= t.min() + baseline_seconds + 20)
    if slope_mask.sum() >= 2:
        slope = np.polyfit(t[slope_mask], ndx[slope_mask], 1)[0]
    else:
        slope = np.nan

    idx_max = int(np.nanargmax(np.abs(ndx)))

    return {
        "baseline": r0,
        "max_abs": float(np.nanmax(np.abs(ndx))),
        "max_signed": float(ndx[idx_max]),
        "time_to_max": float(t[idx_max] - t.min()),
        "steady_mean": float(np.nanmean(ndx[steady_mask])),
        "steady_std": float(np.nanstd(ndx[steady_mask], ddof=1)),
        "auc_abs": float(np.trapz(np.abs(ndx), t)),
        "initial_slope": float(slope),
    }

3. 對一個樣本的所有 sensor 萃取特徵

def extract_sample_features(df, sensor_cols, time_col=None, sample_id="sample"):
    if time_col is None:
        # 如果沒有 time 欄位，就用 row index 當時間；若知道取樣率，請換成秒
        t = np.arange(len(df), dtype=float)
    else:
        t = df[time_col].to_numpy(float)

    row = {"sample_id": sample_id}
    for col in sensor_cols:
        feats = extract_sensor_features(t, df[col].to_numpy(float))
        for k, v in feats.items():
            row[f"{col}_{k}"] = v
    return row

sensor_cols = [c for c in df.columns if "MQ" in c or "sensor" in c.lower()]
row = extract_sample_features(df, sensor_cols, time_col=None, sample_id=file_path.stem)
row

4. 批次處理整個資料夾

def label_from_path(path):
    # wine dataset 範例：父資料夾 AQ_Wines / HQ_Wines / LQ_Wines 可當 class
    parent = path.parent.name
    if parent.startswith("AQ"):
        return "average_quality"
    if parent.startswith("HQ"):
        return "high_quality"
    if parent.startswith("LQ"):
        return "low_quality"
    if parent.lower().startswith("ethanol"):
        return "ethanol"
    return parent

root = Path("Dataset")
files = sorted(list(root.rglob("*.txt")) + list(root.rglob("*.csv")) + list(root.rglob("*.xlsx")))
rows = []

for path in files:
    if path.suffix.lower() == ".xlsx":
        raw = pd.read_excel(path)
    else:
        raw = pd.read_csv(path, sep=None, engine="python")

    sensor_cols = [c for c in raw.columns if "MQ" in c or "sensor" in c.lower()]
    if not sensor_cols:
        # 如果沒有標準欄名，可手動指定，例如 raw.columns[2:8]
        sensor_cols = list(raw.columns[2:8])

    feat = extract_sample_features(raw, sensor_cols, time_col=None, sample_id=path.stem)
    feat["class"] = label_from_path(path)
    feat["source_file"] = str(path)
    rows.append(feat)

features = pd.DataFrame(rows)
features.to_csv("enose_time_series_features.csv", index=False)
features.head()

5. 產生 Orange 可讀的版本

# Orange 對乾淨 CSV 最友善：一列一樣本、class 欄當 target、其他數值欄當 features
orange_df = features.drop(columns=["source_file"], errors="ignore")
orange_df.to_csv("orange_enose_time_series_features.csv", index=False)
print(orange_df.shape)

特徵表完成後，再回到熟悉的 PCA 與機器學習

一旦把 time-series 轉成 sample × features，接下來就和前兩章一樣：先 PCA 看結構，再用分類或回歸模型驗證特徵是否真的有預測能力。

from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.svm import SVC
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report

features = pd.read_csv("enose_time_series_features.csv")
y = features["class"]
X = features.drop(columns=["class", "sample_id", "source_file"], errors="ignore")
X = X.apply(pd.to_numeric, errors="coerce").fillna(X.median(numeric_only=True))

# PCA visualization
Xp = StandardScaler().fit_transform(X)
T = PCA(n_components=2).fit_transform(Xp)

# Classification
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.3, stratify=y, random_state=42
)
model = Pipeline([("scaler", StandardScaler()), ("svc", SVC(kernel="rbf", C=10))])
model.fit(X_train, y_train)
pred = model.predict(X_test)
print(classification_report(y_test, pred))

Orange 從 feature table 開始，不直接吃原始曲線

Orange 很適合做 PCA、分類與模型比較，但它不適合直接批次處理大量 time-series 檔案。因此建議用 Python 先輸出 orange_enose_time_series_features.csv，再進 Orange。

Python feature extraction
  → orange_enose_time_series_features.csv
  → Orange File
  → Select Columns
      Features: MQ*_max_abs, MQ*_auc_abs, MQ*_initial_slope, ...
      Target: class
      Meta: sample_id
  → Preprocess
      Impute / Standardize
  → PCA
  → Scatter Plot
  → Test and Score
       ← SVM
       ← Random Forest
       ← Logistic Regression
  → Confusion Matrix

完成本章後，你應該能回答這些問題