{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "intro",
   "metadata": {},
   "source": [
    "# TimesFM × UCI Online Retail：Jupyter 實作教材\n",
    "\n",
    "這份 Notebook 搭配同目錄的完整程式包使用，目標是讓你逐格看懂並重跑：\n",
    "\n",
    "1. 讀取已處理的每日銷售資料。\n",
    "2. 使用 chronological holdout 保留最後 28 天。\n",
    "3. 重跑 Seasonal Naive(7) 與 ETS。\n",
    "4. 計算 MAE、RMSE、sMAPE、WAPE。\n",
    "5. 讀取本次已驗證的 TimesFM 3.0 輸出，公平比較三個模型。\n",
    "6. 選擇性下載資料與 TimesFM 權重，另存完整重跑結果。\n",
    "\n",
    "> 本案例的 target 是每日正向商品銷售毛額（GBP），不是淨收入、需求量或行銷增量。TimesFM 3.0 權重採 Non-Commercial License；預設不下載、不執行大型模型。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "setup",
   "metadata": {},
   "source": [
    "## 0. 執行環境\n",
    "\n",
    "請先下載並解壓完整程式包，從程式包根目錄啟動 Jupyter。Baseline 所需套件：\n",
    "\n",
    "```bash\n",
    "python3 -m venv .venv\n",
    ".venv/bin/python -m pip install -r requirements-baseline.txt\n",
    ".venv/bin/python -m pip install jupyterlab  # 只有本機尚未安裝 Jupyter 時才需要\n",
    ".venv/bin/jupyter lab\n",
    "```\n",
    "\n",
    "Notebook 不會讀取帳號、Cookie、Token、GA4 或 CRM 資料。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "imports",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import json\n",
    "import subprocess\n",
    "import sys\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from statsmodels.tsa.holtwinters import ExponentialSmoothing\n",
    "\n",
    "\n",
    "def find_project_root(start: Path) -> Path:\n",
    "    \"\"\"從目前目錄往上找程式包根目錄，找不到就明確失敗。\"\"\"\n",
    "    for candidate in (start.resolve(), *start.resolve().parents):\n",
    "        if (candidate / \"data/processed/daily_gross_sales.csv\").is_file():\n",
    "            return candidate\n",
    "    raise FileNotFoundError(\n",
    "        \"找不到 data/processed/daily_gross_sales.csv；\"\n",
    "        \"請先解壓完整程式包，並從程式包根目錄啟動 Jupyter。\"\n",
    "    )\n",
    "\n",
    "\n",
    "ROOT = find_project_root(Path.cwd())\n",
    "DATA_PATH = ROOT / \"data/processed/daily_gross_sales.csv\"\n",
    "OUTPUT_PATH = ROOT / \"outputs/forecast_holdout.csv\"\n",
    "METRICS_PATH = ROOT / \"outputs/metrics.json\"\n",
    "print(f\"專案根目錄：{ROOT}\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "load-md",
   "metadata": {},
   "source": [
    "## 1. 讀取與檢查每日資料\n",
    "\n",
    "輸入是程式包內已處理的公開 UCI Online Retail 日資料。這裡只保留聚合後欄位，不使用客戶層級資料。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "load",
   "metadata": {},
   "outputs": [],
   "source": [
    "daily = pd.read_csv(DATA_PATH, parse_dates=[\"date\"])\n",
    "daily = daily.sort_values(\"date\").reset_index(drop=True)\n",
    "\n",
    "required = {\"date\", \"gross_sales_gbp\"}\n",
    "missing = required.difference(daily.columns)\n",
    "assert not missing, f\"缺少欄位：{sorted(missing)}\"\n",
    "assert daily[\"date\"].is_monotonic_increasing\n",
    "assert daily[\"date\"].is_unique\n",
    "expected_dates = pd.date_range(daily[\"date\"].min(), daily[\"date\"].max(), freq=\"D\")\n",
    "assert daily[\"date\"].tolist() == expected_dates.tolist(), \"日期序列不連續\"\n",
    "assert np.isfinite(daily[\"gross_sales_gbp\"]).all()\n",
    "assert (daily[\"gross_sales_gbp\"] >= 0).all()\n",
    "\n",
    "summary = pd.Series({\n",
    "    \"開始日期\": daily[\"date\"].min().date(),\n",
    "    \"結束日期\": daily[\"date\"].max().date(),\n",
    "    \"日數\": len(daily),\n",
    "    \"零銷售日\": int((daily[\"gross_sales_gbp\"] == 0).sum()),\n",
    "    \"平均每日毛額（GBP）\": daily[\"gross_sales_gbp\"].mean(),\n",
    "})\n",
    "summary\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "split-md",
   "metadata": {},
   "source": [
    "## 2. Chronological holdout\n",
    "\n",
    "最後 28 天只用於評估；所有模型只看 holdout 之前的資料。不能 random split，否則模型可能先看到未來模式。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "split",
   "metadata": {},
   "outputs": [],
   "source": [
    "HORIZON = 28\n",
    "SEASONAL_PERIOD = 7\n",
    "assert len(daily) > HORIZON + SEASONAL_PERIOD\n",
    "\n",
    "train = daily.iloc[:-HORIZON].copy()\n",
    "test = daily.iloc[-HORIZON:].copy()\n",
    "y_train = train[\"gross_sales_gbp\"].to_numpy(dtype=float)\n",
    "y_test = test[\"gross_sales_gbp\"].to_numpy(dtype=float)\n",
    "\n",
    "split_info = pd.Series({\n",
    "    \"train_start\": train[\"date\"].min().date(),\n",
    "    \"train_end\": train[\"date\"].max().date(),\n",
    "    \"holdout_start\": test[\"date\"].min().date(),\n",
    "    \"holdout_end\": test[\"date\"].max().date(),\n",
    "    \"horizon\": HORIZON,\n",
    "})\n",
    "split_info\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "models-md",
   "metadata": {},
   "source": [
    "## 3. 定義兩個可重跑模型與四項指標\n",
    "\n",
    "- Seasonal Naive(7)：把訓練資料最後 7 天重複四次。\n",
    "- ETS：加法趨勢、阻尼趨勢、加法週期，週期長度 7。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "models",
   "metadata": {},
   "outputs": [],
   "source": [
    "def seasonal_naive(train_values: np.ndarray, horizon: int, period: int = 7) -> np.ndarray:\n",
    "    if len(train_values) < period:\n",
    "        raise ValueError(\"訓練資料短於季節週期\")\n",
    "    return np.resize(train_values[-period:], horizon).astype(float)\n",
    "\n",
    "\n",
    "def ets_forecast(train_values: np.ndarray, horizon: int, period: int = 7) -> np.ndarray:\n",
    "    model = ExponentialSmoothing(\n",
    "        train_values,\n",
    "        trend=\"add\",\n",
    "        damped_trend=True,\n",
    "        seasonal=\"add\",\n",
    "        seasonal_periods=period,\n",
    "        initialization_method=\"estimated\",\n",
    "    )\n",
    "    fitted = model.fit(optimized=True, use_brute=False)\n",
    "    return np.asarray(fitted.forecast(horizon), dtype=float)\n",
    "\n",
    "\n",
    "def metric_values(actual: np.ndarray, predicted: np.ndarray) -> dict[str, float]:\n",
    "    actual = np.asarray(actual, dtype=float)\n",
    "    predicted = np.asarray(predicted, dtype=float)\n",
    "    error = predicted - actual\n",
    "    denominator = np.abs(actual) + np.abs(predicted)\n",
    "    smape_terms = np.divide(\n",
    "        2.0 * np.abs(error),\n",
    "        denominator,\n",
    "        out=np.zeros_like(error),\n",
    "        where=denominator != 0,\n",
    "    )\n",
    "    return {\n",
    "        \"MAE\": float(np.mean(np.abs(error))),\n",
    "        \"RMSE\": float(np.sqrt(np.mean(error ** 2))),\n",
    "        \"sMAPE (%)\": float(100.0 * np.mean(smape_terms)),\n",
    "        \"WAPE (%)\": float(100.0 * np.abs(error).sum() / np.abs(actual).sum()),\n",
    "    }\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "run-baselines",
   "metadata": {},
   "outputs": [],
   "source": [
    "naive_pred = seasonal_naive(y_train, HORIZON, SEASONAL_PERIOD)\n",
    "ets_pred = ets_forecast(y_train, HORIZON, SEASONAL_PERIOD)\n",
    "\n",
    "baseline_metrics = pd.DataFrame({\n",
    "    \"Seasonal Naive(7)\": metric_values(y_test, naive_pred),\n",
    "    \"ETS\": metric_values(y_test, ets_pred),\n",
    "}).T\n",
    "\n",
    "# 這些 assert 是教材的最小可執行檢查：避免資料或公式悄悄改變。\n",
    "assert np.isclose(baseline_metrics.loc[\"Seasonal Naive(7)\", \"MAE\"], 15184.707259285717)\n",
    "assert np.isclose(baseline_metrics.loc[\"ETS\", \"MAE\"], 17457.93741020681)\n",
    "baseline_metrics.round(2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "plot-md",
   "metadata": {},
   "source": [
    "## 4. 看預測曲線，而不只看一個分數\n",
    "\n",
    "三個模型都會在最後一天的極端尖峰失手。圖形用來看週期與尖峰；冠軍仍須用相同 holdout 上的多項指標判定。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "plot",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(12, 5))\n",
    "ax.plot(test[\"date\"], y_test, label=\"實際值\", color=\"#111827\", linewidth=2.4, marker=\"o\", markersize=3)\n",
    "ax.plot(test[\"date\"], naive_pred, label=\"Seasonal Naive(7)\", color=\"#d97706\", linewidth=2)\n",
    "ax.plot(test[\"date\"], ets_pred, label=\"ETS\", color=\"#0f766e\", linewidth=2, linestyle=\"--\")\n",
    "ax.set_title(\"28 天 holdout：實際每日銷售額與 baseline 預測\")\n",
    "ax.set_ylabel(\"GBP／日\")\n",
    "ax.grid(alpha=0.25)\n",
    "ax.legend()\n",
    "fig.autofmt_xdate()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sensitivity-md",
   "metadata": {},
   "source": [
    "## 5. 尖峰敏感度檢查\n",
    "\n",
    "移除最後一天只為理解誤差是否完全由單一尖峰主導；這不是重新挑選 holdout，也不能取代 rolling-origin validation。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "sensitivity",
   "metadata": {},
   "outputs": [],
   "source": [
    "sensitivity = pd.DataFrame({\n",
    "    \"Seasonal Naive(7)\": metric_values(y_test[:-1], naive_pred[:-1]),\n",
    "    \"ETS\": metric_values(y_test[:-1], ets_pred[:-1]),\n",
    "}).T\n",
    "sensitivity.round(2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "timesfm-md",
   "metadata": {},
   "source": [
    "## 6. 讀取已驗證的 TimesFM 3.0 輸出\n",
    "\n",
    "下一格讀取程式包內的既有 TimesFM 輸出；它不會重新下載或執行模型。這個證據界線很重要：看到輸出不等於你剛完成一次新的 TimesFM 推論。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "timesfm-output",
   "metadata": {},
   "outputs": [],
   "source": [
    "artifact = pd.read_csv(OUTPUT_PATH, parse_dates=[\"date\"])\n",
    "saved = json.loads(METRICS_PATH.read_text(encoding=\"utf-8\"))\n",
    "assert len(artifact) == HORIZON\n",
    "assert artifact[\"date\"].tolist() == test[\"date\"].tolist()\n",
    "assert np.allclose(artifact[\"actual_gross_sales_gbp\"], y_test)\n",
    "assert artifact[\"timesfm_3_point\"].notna().all()\n",
    "\n",
    "timesfm_pred = artifact[\"timesfm_3_point\"].to_numpy(dtype=float)\n",
    "all_metrics = pd.DataFrame({\n",
    "    \"Seasonal Naive(7)\": metric_values(y_test, naive_pred),\n",
    "    \"ETS\": metric_values(y_test, ets_pred),\n",
    "    \"TimesFM 3.0（既有輸出）\": metric_values(y_test, timesfm_pred),\n",
    "}).T\n",
    "\n",
    "assert np.isclose(all_metrics.loc[\"TimesFM 3.0（既有輸出）\", \"MAE\"], saved[\"models\"][\"timesfm_3\"][\"mae\"])\n",
    "all_metrics.round(2).sort_values(\"MAE\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "interpretation",
   "metadata": {},
   "source": [
    "### 正確判讀\n",
    "\n",
    "- Seasonal Naive(7) 在 MAE、RMSE、sMAPE、WAPE 四項都最佳。\n",
    "- TimesFM 在 MAE、WAPE 排第二；ETS 在 RMSE、sMAPE 略優於 TimesFM。\n",
    "- 只有 28 個 holdout 點，不能外推成「哪個模型永遠比較好」。\n",
    "- 下一步應做 rolling-origin validation，再加入 ARIMA／SARIMAX、梯度提升與未來已知 covariates。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "optional-md",
   "metadata": {},
   "source": [
    "## 7. 選用：完整重跑 TimesFM\n",
    "\n",
    "下格預設為 `False`，避免無意下載大型 checkpoint。若要執行，先閱讀 Hugging Face 上的 TimesFM 3.0 Non-Commercial License，安裝 `requirements-timesfm.txt`，再將旗標改為 `True`。輸出另存到 `outputs/notebook_timesfm/`，不覆蓋教材附帶的已驗證輸出。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "optional-run",
   "metadata": {},
   "outputs": [],
   "source": [
    "RUN_FULL_TIMESFM = False\n",
    "\n",
    "if RUN_FULL_TIMESFM:\n",
    "    subprocess.run(\n",
    "        [sys.executable, str(ROOT / \"scripts/download_data.py\")],\n",
    "        cwd=ROOT,\n",
    "        check=True,\n",
    "    )\n",
    "    subprocess.run(\n",
    "        [\n",
    "            sys.executable,\n",
    "            str(ROOT / \"scripts/run_forecast.py\"),\n",
    "            \"--timesfm\", \"required\",\n",
    "            \"--output-dir\", \"outputs/notebook_timesfm\",\n",
    "        ],\n",
    "        cwd=ROOT,\n",
    "        check=True,\n",
    "    )\n",
    "    print(\"完成：outputs/notebook_timesfm/\")\n",
    "else:\n",
    "    print(\"未執行 TimesFM：將 RUN_FULL_TIMESFM 改為 True 才會下載資料與模型。\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "next-steps",
   "metadata": {},
   "source": [
    "## 8. 舉一反三\n",
    "\n",
    "你可以沿用相同骨架替換模型，但請維持相同時間切分：\n",
    "\n",
    "1. ARIMA／SARIMA：先看 ACF、PACF 與殘差，再用 rolling origin 比較。\n",
    "2. SARIMAX：只加入預測當下已知的未來排程、假日或預算。\n",
    "3. LightGBM／XGBoost：lag 與 rolling 特徵必須先 `shift(1)`，避免當日答案進入當日特徵。\n",
    "4. GA4／CRM：先定義 target、資料延遲與可用時間，再做 point-in-time correct join。\n",
    "5. 因果 uplift：需要 treatment 與反事實設計，不能從這份 forecast 直接推論。\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.10+"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}