🔬 Wang's Five Laws — LLM Spectral Analyzer
王氏五定律 — 大模型谱分析工具
Mathematical Foundations of Large Language Models (MF-LLM)
Reads HF weights via HTTP Range Request — no full model download required. Auto-detects model structure (GQA / MHA / K=V shared / heterogeneous head_dim), computes all Five Laws metrics per attention head, persists results to SQLite.
通过 HTTP Range Request 直接读取 HF 权重，无需下载整个模型。 自动识别模型结构，逐头计算王氏五定律全部指标，结果持久化到 SQLite。
| Law | Metric | Ideal |
|---|---|---|
| Law 1 | Pearson r (Q–K spectral alignment) | → 1 |
| Law 2 | SSR (spectral shape residual) | → 0 |
| Law 3 | Condition number κ | smaller = more stable |
| Law 4 | cosU(Uq, Uv) super-orthogonal | < 1/√d_head |
| Law 5 | cosV input subspace random orthogonal | ≈ 1/√d_model |
| 定律 | 指标 | 理论极值 |
|---|---|---|
| 第一定律 | Pearson r（Q-K 谱线性对齐） | → 1 |
| 第二定律 | SSR（谱形状残差） | → 0 |
| 第三定律 | 条件数 κ | 越小越稳定 |
| 第四定律 | cosU(Uq, Uv)（超正交） | < 1/√d_head |
| 第五定律 | cosV（输入子空间随机正交） | ≈ 1/√d_model |
Step 1: Inspect model structure — auto-detect components, head_dim, and K=V shared layers. Results are used by the Analyze tab.
No weights are downloaded — structure is inferred from safetensors headers only.
✅ Recommended Models
google/gemma-4-e2b
google/gemma-4-e4b-it
google/gemma-4-31b-it
Qwen/Qwen2.5-14B-Instruct
deepseek-ai/DeepSeek-R1-Distill-Qwen-14B
meta-llama/Meta-Llama-3-8B （Need access right）
Layer Index
- Layer index = N in
layers.{N}of safetensors keys - Raw index, not re-numbered per component
- Multi-modal models (e.g. Gemma-4):
layers.0~11may contain audio / vision / text layers- All components output separately, distinguished by prefix
Example: Gemma-4-E2B
| Component | Layer Range |
|---|---|
| audio_tower | 0 ~ 11 |
| language_model | 0 ~ 34 |
| vision_tower | 0 ~ 15 |
Example: Gemma-4-31B
| Component | Layer Range |
|---|---|
| language (local) | 0 ~ 59 |
| language (global) | 5, 11, 17 … 59 |
| vision_tower | 0 ~ 26 |
Layer Structure Overview