PREDICT tells you how a deck is likely to do in ranked play: its chance to win, how it fares against every color combination, which cards move the number, and head to head against any other deck, going first or second. This page is the technical version: where the numbers come from, which model was chosen and why, how it was tested, how honest its percentages are, and where it falls short.
The short answer
- Two numbers, from two kinds of data. The headline number comes from real ranked games between people. The second number comes from a machine learning model that learned from over a million games our simulator played. They are kept separate on purpose, and you see both.
- Both were tested on games and decks they had never seen, and both beat every simple way of guessing.
- The model is not overfit. Five-fold cross-validation shows it does almost exactly as well on new decks as on the decks it learned from.
- The percentages are honest, with one measured exception. When the simulator model says 60%, about 60% of those games are won. On brand new kinds of decks it is a little too sure of itself at the extremes, and we show exactly how much below.
- It keeps itself current. The ranked data refreshes every week, and every week last week's numbers are scored on the games played since.
Two numbers, two sources
How decks on your three Legends actually do in ranked play between people, adjusted for the cards in your list. A statistical model fitted on ranked results, refreshed every week.
Shown in blue on the panel.
A machine learning model trained on 1,019,892 games our simulator played between 2,203 decks. It reads the cards in two decks and predicts the winner, going first or second.
Shown as the yellow tick on the panel.
Why both? Real games know things a simulator cannot, like how well people actually pilot each deck. The simulator knows things ranked data cannot, like how two specific decklists do against each other when nobody has played that exact pairing yet. Each covers the other's blind spot, and the panel never blends them into one number you cannot check.
The players' number
It starts from the ranked win rate of decks on the same three Legends, then adjusts for your cards. Three steps:
- The Legends. Take the ranked win rate of every deck on your three Legends. A trio with few games gets pulled toward the win rate of its color combination, and that toward 50%, as if it had 40 extra games at the average. This is called shrinkage: it stops a lucky streak on a rare trio from looking like a great deck.
- Your cards. For each card in your list, compare how lists on those Legends do with it against the trio's rate, again shrunk by how many games back it up. Add those up, capped at 12 points either way, so no handful of cards can swamp the Legends.
- Keep it sensible. The result is held between 15% and 85%. Single games are too random for anything outside that to be believable.
Does it work on games it never saw?
What this shows. The numbers fixed on 2 Nov were scored on the ranked games played after that, which they had never seen. Each bar is a different way of guessing the winner. The score is log loss: lower means better guesses, and saying 50% every time scores 0.693.
Data table
| Way of guessing | Log loss |
|---|---|
| Coin flip, 50% | 0.6932 |
| Guess from the colors | 0.6842 |
| Raw win rate of the Legends | 0.6819 |
| PREDICT's players' number | 0.6765 |
Is it calibrated?
What this shows. Calibration asks whether the percentages mean what they say: of all the decks given 60%, do about 60% of their games end in a win? This chart is a reliability diagram, also called a calibration curve. Results are grouped by the predicted chance (across) and show the share actually won (up). A perfectly calibrated number sits on the dashed line.
Data table
| Predicted | Actually won |
|---|---|
| 30.7% | 25.9% |
| 42.7% | 37.2% |
| 47.9% | 63.2% |
| 53.1% | 49.4% |
| 58.3% | 58.3% |
| 61.8% | 64.5% |
The simulator and its data
CHOOMDECK's own rules engine, the same code you play against on the site, runs outside the browser at a few thousandths of a second per game. Its built-in player pilots both decks. Every game is seeded, so any one can be replayed exactly.
- The decks. 441 real lists: our own themed and meta decks, 321 public lists, and 57 consensus lists built from what ranked players on each Legend trio actually run. Each one was checked legal by the deck builder's own rules, then given four legal variants with one to four cards swapped: 2,203 decks in 385 families. A deck and its near copies share a family.
- The features. Each deck is described by what is printed on it, 176 numbers in all: how many of each card, the three Legends, the cost curve, card types, cards that can be sold, keywords and RAM. That is what lets the model score a deck it has never seen.
- An honest split. Whole families, not single decks, were split into training (70%), tuning (15%) and testing (15%), so a near copy of a training deck can never sneak into the test.
Data table
| Set | What it is for | Games |
|---|---|---|
| Training | the model learns from these | 799,932 |
| Validation | choosing settings, never for learning | 59,995 |
| Test | graded once, at the end | 59,990 |
| Dense pairings | 200 test pairings played 500 times each | 99,975 |
| Total | 1,019,892 |
Choosing the model, as a risk decision
Picking a model is not a contest won by the best score. A model becomes part of the site: it has to be right, honest about its own uncertainty, stable, explainable to the people who use it, cheap to run and easy to check. Seven candidates were compared, from simple guesses to gradient boosting.
What this shows. Every candidate, graded on test decks it never saw, in the hardest case: both decks new to it. Gray bars are the simple guesses every model must beat. The dashed line is the best any model could do, given how random single games are.
Data table
| Model | Log loss | Brier | AUC | Accuracy |
|---|---|---|---|---|
| Coin flip | 0.6931 | 0.2500 | 0.500 | 50.4% |
| Seat only | 0.6931 | 0.2500 | 0.500 | 50.4% |
| Table by colors | 0.6705 | 0.2387 | 0.627 | 58.9% |
| Bradley-Terry | 0.6430 | 0.2258 | 0.681 | 63.2% |
| Logistic, both decks | 0.6427 | 0.2257 | 0.681 | 63.2% |
| LightGBM | 0.6406 | 0.2248 | 0.683 | 63.3% |
| Bradley-Terry + cycle term | 0.6470 | 0.2272 | 0.679 | 62.9% |
Are the differences real?
What this shows. Each bar is how much better (left of zero) or worse (right of zero) one model is than another, per game, with the range the true difference most likely sits in. Blue: the whole range is on one side of zero, so the difference is real. Pink: the range crosses zero, so it could be luck.
Data table
| Comparison | Difference | 95% low | 95% high |
|---|---|---|---|
| Bradley-Terry vs Table by colors | -0.02748 | -0.03827 | -0.01890 |
| Logistic, both decks vs Table by colors | -0.02775 | -0.03887 | -0.01994 |
| Logistic, both decks vs Bradley-Terry | -0.00027 | -0.00101 | +0.00044 |
| LightGBM vs Table by colors | -0.02990 | -0.03915 | -0.02326 |
| LightGBM vs Bradley-Terry | -0.00242 | -0.00781 | +0.00220 |
| Bradley-Terry + cycle term vs Table by colors | -0.02344 | -0.03346 | -0.01553 |
| Bradley-Terry + cycle term vs Bradley-Terry | +0.00403 | +0.00202 | +0.00699 |
The scorecard
What this shows. The same three finalists, judged on everything a model has to get right once it is part of a site, not only on accuracy.
| What matters | Bradley-Terry (chosen) | LightGBM | Cycle term |
|---|---|---|---|
| Accuracy on new decks | 0.643 log loss | 0.641, better by an amount inside the noise | 0.647, worse |
| Calibration (average gap) | 1.9 points | 0.7 points, the best | 2.6 points |
| Stability | the same result in all 5 cross-validation folds, 0.610 to 0.643 | more settings to tune, more ways to drift | adds a second set of weights to keep stable |
| Explainable | one weight per card and Legend: the "cards that move the number" list comes straight from it | hundreds of trees; why it said 62% is hard to show | the cycle part has no plain meaning per card |
| Cost to run | 176 numbers in a small file, pure Python, a fraction of a millisecond per deck on the web server | a model library on the server, slower per reading | about twice Bradley-Terry |
| Going first or second | gives both directly | gives both | gives both |
| Known weakness | a little too sure on brand new kinds of decks, measured and correctable (see calibration) | harder to see where it fails | no advantage found |
Is it overfit or underfit?
What this shows. The deck families were split into 5 groups. Five times over, the model learned from 4 groups and was graded on the fifth, at several levels of caution (regularization). For each level: how well it did on the decks it learned from, on decks it never saw, and the gap between them.
Data table
| Caution | Decks it learned from | Decks it never saw | Gap |
|---|---|---|---|
| 3e-02 | 0.6131 | 0.6243 | +0.0111 |
| 1e-02 (chosen) | 0.6127 | 0.6243 | +0.0116 |
| 3e-03 | 0.6125 | 0.6251 | +0.0125 |
| 1e-03 | 0.6125 | 0.6256 | +0.0131 |
| 3e-04 | 0.6125 | 0.6259 | +0.0134 |
| 1e-04 | 0.6125 | 0.6260 | +0.0135 |
| 3e-05 | 0.6125 | 0.6260 | +0.0135 |
| 3e-06 | 0.6125 | 0.6260 | +0.0135 |
Simple guesses on the same unseen decks: by seat 0.6931, by colors 0.6724.
What this shows. Cross-validation picked a more cautious setting than the first model used. Both were then scored once on the untouched test decks. Negative means the new model is better; the interval says whether that is real.
Data table
| Untouched decks | Difference | 95% interval | Games |
|---|---|---|---|
| Test, both decks new | -0.00187 | [-0.00292, -0.00119] | 29,992 |
| Test, one deck new | -0.00088 | [-0.00141, -0.00045] | 29,998 |
| Validation, both new | -0.00052 | [-0.00105, -0.00006] | 29,996 |
| Dense pairings | -0.00153 | [-0.00308, -0.00063] | 99,975 |
Is the trained model calibrated?
What this shows. The reliability diagram again, for the model that shipped, on the untouched test decks. Pink: both decks new to the model. Yellow: one deck new. A perfectly calibrated model sits on the dashed line; points below it on the right and above it on the left mean the model is a little too sure of itself.
Data table
| Test set | Predicted | Actually won | Games |
|---|---|---|---|
| both new | 9.0% | 5.2% | 58 |
| both new | 16.4% | 22.2% | 1,234 |
| both new | 25.5% | 27.8% | 3,466 |
| both new | 35.1% | 37.8% | 4,403 |
| both new | 45.3% | 45.0% | 5,427 |
| both new | 54.7% | 54.9% | 5,765 |
| both new | 64.9% | 62.5% | 4,480 |
| both new | 74.6% | 70.5% | 3,714 |
| both new | 83.7% | 80.7% | 1,378 |
| both new | 91.2% | 94.0% | 67 |
| one new | 8.0% | 6.2% | 113 |
| one new | 16.1% | 14.0% | 1,409 |
| one new | 25.5% | 26.5% | 3,384 |
| one new | 35.2% | 37.5% | 4,750 |
| one new | 45.1% | 45.8% | 5,084 |
| one new | 55.0% | 54.6% | 5,049 |
| one new | 64.9% | 62.2% | 4,913 |
| one new | 74.6% | 73.1% | 3,587 |
| one new | 83.9% | 84.1% | 1,570 |
| one new | 91.9% | 92.1% | 139 |
How close is it on one specific matchup?
What this shows. Two hundred pairs of never-seen decks were each played 500 times, so their true result is well known. This is the hardest test: the exact number for one specific pairing.
More decks or more games?
What this shows. The model was trained on a quarter, a half, three quarters and all of the training families, and graded on never-seen decks each time. A line still falling at the right means more would help.
Data table
| Families | Log loss |
|---|---|
| 54 | 0.6529 |
| 108 | 0.6279 |
| 162 | 0.6263 |
| 216 | 0.6243 |
Data table
| Training games | Both new | One new |
|---|---|---|
| 25,000 | 0.6419 | 0.6270 |
| 50,000 | 0.6415 | 0.6261 |
| 100,000 | 0.6409 | 0.6252 |
| 200,000 | 0.6414 | 0.6257 |
| 400,000 | 0.6427 | 0.6258 |
| 799,932 | 0.6430 | 0.6259 |
Where it falls short
- The simulator does not play like people. Its built-in player pilots some strategies, blue control above all, worse than people do. By color matchup, the simulator's results line up only loosely with ranked play (a correlation of 0.25, where 1.00 would be a perfect match), and going first wins 50% of simulator games against about 60% in ranked play. That is why the headline number comes from real games, and the simulator's number sits beside it as a tendency.
- Every number is best of one. Ranked play here is single games. Tournaments play best of three, which favors the stronger deck more than a single game does.
- The card adjustment is a correlation. Lists that run a card may win for reasons beyond that card, such as who tends to play them. The 12 point cap is there because of that.
- One rules revision at a time. The simulator plays the rules as CHOOMDECK runs them today. A rules update or a new set means new games and a retrain.
How it stays current
- Every week the server pulls fresh ranked results and rebuilds the players' number and the deck grade's results half, for both CHOOMDEX and CHOOMDECK.
- Safety checks first. A pull with too few games, an odd first seat rate, win rates outside 5% to 95%, or a sudden collapse in games is refused, and last week's numbers stay up.
- Scored every week. The numbers that were live are graded on the games played since, beside the simple guesses, and the calibration check above grows with them.
- The trained model is retrained when the decks, the rules or the simulator's player change, and only ships if it beats the current one on untouched decks with an interval that stays below zero.
Words used on this page
- Win rate, 62%
- Out of 100 games, the deck wins about 62.
- Log loss
- A score for how good predicted chances were, game by game. Lower is better. Saying 50% every time scores 0.693. A difference of 0.01 is a real improvement.
- Calibration and reliability diagram
- Whether predicted chances match what happens. The diagram groups predictions and plots what was predicted against what was won; honest predictions sit on the diagonal.
- Calibration slope
- 1.00 is perfect. Below 1 means the predictions are too extreme, too sure of themselves; above 1 means too timid.
- Training data and unseen data
- A model learns from some decks and is graded on decks it never saw. Only the unseen grade counts.
- Deck family
- A deck and its near copies. Families are kept together so a near copy of a training deck never reaches the test.
- Cross-validation
- Split the decks into 5 groups, train on 4, grade on the fifth, and repeat so every group gets a turn.
- Overfit and underfit
- Overfit: memorized the training decks and does much worse on new ones. Underfit: too simple to learn much. A good model does nearly as well on new decks and far better than simple guesses.
- Regularization, or caution
- How strongly the model is held back from chasing noise.
- Shrinkage
- Pulling a rate built on few games toward a broader average, so luck does not look like skill.
- Bradley-Terry
- The trained model: logistic regression on the difference between two decks' cards and Legends. One weight per feature, which is what makes it explainable.
- 95% interval
- The range a true difference most likely sits in. If it does not cross zero, the difference is real, not luck.
PREDICT is part of CHOOM+ on CHOOMDEX and CHOOMDECK. This page was written on 3 Nov 2026 and is updated when the model changes. See it in action: How PREDICT reads a deck, step by step.
