Routes and drift
The machine underneath does not hold still.
A route that was fastest last week may not be fastest now. MMPF learns route preference with a bandit, and runs a state machine whose only job is to notice when what it learned has stopped being true.
The routes
Five routes appear in the v1.0 benchmark. Each is a different combination of trial division bound and the heavy method that follows it.
| Route | Shape |
|---|---|
no_tree_ecm_light | no remainder tree, light ECM |
tree_1000_rho | tree to 1,000, then Pollard rho |
tree_1000_ecm_medium | tree to 1,000, then medium ECM |
tree_10000_rho | tree to 10,000, then rho |
tree_100000_rho | tree to 100,000, then rho |
No route dominates. That is the premise of the whole runtime — if one did, there would be nothing to select and nothing to certify.
Learning the preference
v0.6 and v0.7 compared LinUCB, Thompson sampling and discounted Thompson sampling across 20 seeds, on equal footing. The 600 resulting decisions are recorded as an MMLF ledger, which is why they can be replayed by a runtime that was not involved in producing them.
mmpf_bandit_decisions — 600 transactions, PASS under official MMLC Runtime 1.0.0.
The drift state machine
Discounting old evidence is not enough on its own, because it cannot tell you when the world changed. The state machine makes that explicit.
- STABLE
- SUSPECTED
- CONFIRMED
- COOLDOWN
- RECOVERING
The value of naming the states is that SUSPECTED and CONFIRMED are different, and so are COOLDOWN and RECOVERING. A system with one boolean for "is drifting" cannot distinguish a suspicion from a finding, or a system that has stopped drifting from one that has recovered.
Machine profiles
A predicted cost is only meaningful relative to a machine. mmpf calibrate measures the host and writes a profile, and a certificate records which profile it was issued against.
mmpf calibrate --output machine/my_machine.json- A machine profileportable — a profile from one host predicts that host
- Format and sensitivity testsan independent physical-machine dataset
The second of those is the second of the four reasons this release is a candidate. The synthetic profiles exercise the format and the sensitivity of the selector; they are not evidence that a profile transfers between real machines.