Taming Balatro: Pushing White Stake Clear Rate to 99% with Reinforcement Learning

Core Conclusion (Top of the Pyramid)

To stably achieve a 99% Ante 8 clear rate on Balatro White Stake, end-to-end deep reinforcement learning (such as PPO) is not viable. The feasible route is “hierarchical architecture + hybrid search”:

1. Tactical layer (within a single round): model “selecting and playing cards” as a combinatorial optimization problem, and use MCTS or exact enumeration to find an approximately optimal solution—but note that this layer cannot be solved independently of the strategic layer (see §3.3: accumulating jokers give tactical actions long-term consequences)
2. Strategic layer (across rounds + shop): this is where RL should really focus—building (joker selection), resource management (gold/draws/discards), risk control, and providing long-term value estimates $V(s’)$ to the tactical layer
3. Reward shaping: the sparse “clear” signal must be densified, but we must also avoid the agent learning to “farm score” instead of “clear”

We break it down layer by layer below. First, clarify why this goal is both hard-core and feasible.

I. Calibrating the Goal First: What Does a 99% White Stake Clear Rate Mean?

1.1 Difficulty Baseline (Human Level)

White Stake is Balatro’s lowest difficulty (no stake modifiers). But even so, clearing Ante 8 is by no means easy. According to community statistics ¹:

📌 Key Judgment: The 99% target is significantly above human top performance. It means the agent cannot rely on “lucky builds” and must systematically avoid death risks and maximize the expected clear probability of every decision. This is precisely RL’s natural advantage over humans—RL can optimize long-term expectation, while humans are limited by the emotion and visible horizon of a single run.

1.2 The Score Wall: The Hard Threshold of Ante 8

The score required by Ante 8 Boss Blind on White Stake (Red Deck baseline) ²:

Some bosses (e.g., The Wall) can push the Ante 8 score wall to 300,000 ³. This means: entering Ante 7-8, the agent must have exponential score growth ability (xMult joker + hand upgrades). The build path chosen in the early-to-mid game already determines whether the late game can break through this wall.

II. Modeling the Game as an MDP: State, Action, Reward

This is the foundation of all RL work. The difficulty of Balatro lies in its decisions being hybrid, sparse, and highly build-dependent.

2.1 State Space $S$

A complete state needs to encode:

状态 = (
    手牌(花色/点数/增强),       # 当前可打出的牌
    牌堆/弃牌堆(残差信息),      # 推断剩余牌分布
    当前 blind 目标分数 + 已得分,
    剩余出牌次数 / 弃牌次数,     # 资源
    金币,                       # 商店购买力
    当前 joker 列表 + 每张效果,   # ★ 核心构筑,150+ 种
    # ★★ 每张 joker 的运行时累积状态(如 Green Joker 当前 mult、Ride the Bus 计数、Supernova 永久加成)
    #    这一项绝不能省——见 §3.3,它是跨回合耦合的根源
    牌组(deck)构成,             # 已增删/增强的牌
    消耗品(塔罗/星球/光谱),
    当前 ante / blind 类型 / Boss 效果,
    商店内容(若在商店阶段)
)

Encoding Challenge: Joker effects are extremely heterogeneous (some add chips, some multiply mult, some trigger conditionally, some rewrite scoring rules such as Burglar / Eternal types). They cannot be simply one-hot encoded and require structured/embedded encoding—this is the focus of subsequent network design. More critically, every joker must also carry its runtime accumulated state (current mult/chips bonuses, trigger counts, etc.); otherwise the model cannot understand long-range investments such as “discard to stack buffs” (see §3.3 for details).

2.2 Action Space $A$ (There Is a Big Pitfall Here)

Balatro’s actions are phased and heterogeneous:

⚠️ Key Insight: The “combinatorial explosion” of playing cards is a false proposition. Selecting 1–5 cards from 8 gives $\sum_{k=1}^{5}\binom{8}{k} = 218$ combinations, and after excluding invalid hand types the actually valid plays are often only a few dozen. This subproblem can be solved by enumeration + exact scoring; no RL needed!

2.3 Reward $R$: The Design That Needs the Most Care

The direct reward is “clear = +1, fail = 0”. But this signal is too sparse—a run lasts 15–40 minutes with hundreds of decisions, yet there is only one terminal signal, making credit assignment nearly impossible.

Drawing on OpenAI Five’s experience in Dota ⁴, we must do reward shaping, but we must watch the direction:

Core Principle (borrowing the zero-sum idea from OpenAI Five ⁴): The net effect of shaping rewards must not deviate from the “clear” objective. A practical approach is potential-based reward shaping $F = \gamma\Phi(s’) - \Phi(s)$, where $\Phi$ is a handcrafted or learned “clearance potential function” that theoretically does not change the optimal policy and only accelerates learning ⁵.

III. Algorithm Selection: Why Pure PPO Is Not Enough and Why an MCTS+RL Hybrid Architecture Is Needed

3.1 Why Pure End-to-End PPO Fails

Intuitively, Dota 2 succeeded with PPO, so Balatro should too? No. There are three reasons:

1. Sample efficiency: Dota was built on 256 GPUs + 45,000 years of game experience ⁶. Although Balatro community simulators (such as Jackdaw) are fast, a single run is long and decisions are dense; stacking up to Dota’s scale is unrealistic.
2. Precision of combinatorial actions: Playing cards is a discrete combinatorial problem; PPO samples from a stochastic policy and struggles to stably converge to “the optimal subset of the current hand”—which is exactly solvable exactly.
3. Long-range dependency of builds: Which joker to buy in Ante 3 directly determines whether Ante 8 can be cleared. PPO’s credit assignment span is too large.

3.2 Recommended Architecture: Exact Tactical Layer + Learning Strategic Layer

Borrowing the core idea of AlphaZero—use search as a “policy improvement operator” and use the network to guide search ^{7 8}—but hierarchical:

┌─────────────────────────────────────────────┐
│  战略层(跨回合,RL 的主战场)                │
│  - 商店决策:买/卖/跳过/重置                 │
│  - 资源管理:金币、抽牌、弃牌的分配         │
│  - 风险控制:何时安全过 blind、何时贪分     │
│  算法:PPO 或 AlphaZero-style 策略价值网络   │
└──────────────────┬──────────────────────────┘
                   │ (提供当前状态 + 战略意图)
                   ▼
┌─────────────────────────────────────────────┐
│  战术层(单回合内,组合优化)                │
│  - 出牌选牌:从手牌中选最优子集             │
│  - 精确计分:含 joker 链触发顺序            │
│  算法:精确枚举 / MCTS / 分支限界           │
└─────────────────────────────────────────────┘

Why does this division (preliminary idea) seem effective?

• The tactical layer seems “exactly solvable”—there are only 8 cards in hand, and the number of valid play combinations is only a few dozen; we can directly enumerate each combination’s actual score (considering all joker triggers) and pick the best.
• The strategic layer is where RL belongs—shop building and resource scheduling are the real long-term planning problems; RL learns “future-oriented” policies here.
• This seems to greatly compress RL’s action space and decision frequency.

⚠️ But this “division” has a fatal flaw; see the next section §3.3. The preliminary idea of “independently exactly solving the tactical layer” does not hold under Balatro’s real mechanics.

3.3 Key Correction: Cross-Round State Coupling Means the Tactical Layer Cannot Be Solved Independently

The above judgment that “the tactical layer is exactly solvable” only holds when pure scoring jokers (e.g., +chips, +mult) are present. But Balatro has a large class of jokers whose internal state changes with tactical actions and persists across rounds—this directly breaks the assumption of “single-round closure”.

Typical Example: Accumulating Jokers

A concrete deduction (using Green Joker as an example): there is a sure-win pair in hand; playing it directly clears the blind. But if we discard twice first and then play the pair, this round permanently gains +2 mult, which persists to all subsequent rounds. The real decision is:

方案 A:直接出对子 → 过关,mult 不变
方案 B:先弃牌 2 次 → 攒永久 mult → 再出对子 → 过关,mult +2(全周期有效)
方案 C:弃牌 5 次 → 攒更多 mult → 但可能把好牌弃了,过不了关

“Whether to discard to stack buffs” is a long-term investment decision, not a single-round optimum. The assumption of “exactly solving the tactical layer” directly collapses when such jokers exist.

Conclusion: On the surface Balatro is a card game, but underneath it is actually a resource-management + long-term-investment game disguised as a card game. Many of its mechanics (accumulating mult, deck evolution, consumable investment) are essentially trade-offs of “sacrifice now for future returns”. RL is better than MCTS at this kind of problem—because MCTS excels at discrete adversarial search (Go), while RL excels at long-horizon investment planning and credit assignment.

3.4 Corrected Architecture: Tactical Layer Depends on the Strategic Layer’s Long-Term Value

In the original architecture the tactical layer was an “independent black-box solver”; after the correction it must be guided by the strategic-layer network, and the two layers are jointly trained end-to-end:

┌─────────────────────────────────────────────────┐
│  战略层(RL:学习长期价值 V(s))                 │
│  - 输入:完整状态(★ 含 joker 运行时累积状态)  │
│  - 输出:V(s) = 期望通关率                      │
│  - 用途:① 商店决策  ② 给战术层提供长期价值    │
└──────────────────┬──────────────────────────────┘
                   │ 提供 V(s') —— 操作后状态的价值
                   ▼
┌─────────────────────────────────────────────────┐
│  战术层(组合优化 + 长期价值引导)               │
│  - 枚举出牌/弃牌组合 c                         │
│  - 对每个组合 c,精确计算综合价值:             │
│      单回合得分 + γ · V(操作后状态 s')        │
│  - 选综合价值最高的组合                         │
└─────────────────────────────────────────────────┘

Key Change: The tactical layer no longer pursues “highest single-round score”, but the combined optimum of “single-round score + discounted long-term value”. The long-term value of Green Joker’s “discard to accumulate mult” operation will be recognized and rewarded by the strategic-layer network $V(s’)$.

Impact on State Encoding: The input to the Set Transformer is no longer just “joker type embeddings”, but a joint embedding of “joker type + runtime accumulated state (current mult/chips bonuses, trigger counts, etc.)”.

Impact on Training: The two layers must be jointly optimized with gradient flow, approaching an AlphaZero-style “network guides search, search improves network” architecture—rather than “tactical layer independent, strategic layer independent”.

3.5 Concrete Network Design for the Strategic Layer

State encoding (the key innovation) uses Set Transformer / GNN to process the joker set, because joker order and combinations both matter:

joker 编码 = Transformer/SetEncoder(joker_embeddings)
全状态向量 = concat(标量状态, 手牌编码, joker编码, deck编码, 盲注信息)
        ↓
策略网络 π(a|s)  →  商店动作分布
价值网络 V(s)    →  "从当前状态出发的期望通关率"

The value network $V(s)$ outputs the expected clear probability, which is exactly the quantity we ultimately want to maximize—letting the agent directly learn to evaluate “given my current build and situation, how confident am I of clearing Ante 8”.

IV. Training Pipeline: Curriculum Learning + Self-Play

4.1 Curriculum Learning

Don’t train on Ante 8 right away. Referring to general RL engineering experience ⁹:

1. Phase 1: First train on Ante 1-3, letting the agent learn basic scoring and shop rhythm
2. Phase 2: Ante 4-6, introducing build synergy and resource management
3. Phase 3: Ante 7-8, focusing on breaking the score wall and Boss handling
4. Phase 4: Full-ante random seeds, complete-run training

⚠️ Note: OpenAI Five did not use a hand-designed curriculum in Dota ⁴. But Balatro is different—its ante is a naturally increasing difficulty ladder, so exploiting this structure for curriculum learning is reasonable and efficient.

2.2 Self-Play + Opponent Pool

Because Balatro is a single-player versus stochastic environment (non-adversarial), no opponent pool is needed. But we can maintain a seed pool / difficulty pool: weightedly sample random seeds on which the agent tends to lose (rare boss combinations, bad shops) for prioritized experience—this is equivalent to transferring AlphaStar’s “exploiter” idea ¹⁰ to the single-agent setting.

V. Key Technical Risks and Mitigations

VI. Is 99% Really Achievable? My Assessment

Achievable, but conditional:

1. The randomness of White Stake is limited: shop items are random, but it is not adversarial—there are no “unwinnable seeds”. Theoretically almost every random seed has a solution; only the difficulty differs.
2. Human top performance reaches ~80% ¹, showing that the remaining ~19% of failures mostly come from suboptimal decisions rather than “dead seeds”. RL systematically optimizes long-term expectation, leaving room to eliminate that 19%.
3. The bottleneck lies in long-term planning at the build layer, and this is exactly the strength of an AlphaZero-style “search + learning” architecture—MCTS can look ahead several rounds during shop decisions.

🎯 Realistic Expectation: With Jackdaw + exact tactical layer + AlphaZero-style strategic-layer hybrid architecture, after training for several weeks on moderate compute (single-machine multi-GPU), raising the White Stake Ante 8 clear rate from near 0% random baseline to 90%+ is supported by community precedent; pushing to 99% through adversarial sampling and architecture refinement is an aggressive but not fantastical goal.

There are already reports of RL projects achieving their first win in the real game ¹¹, proving that this path works.

VII. Recommended Production-Ready Tech Stack

VIII. Summary: Three Sentences to Remember This Design

1. The tactical layer cannot be decoupled from the strategic layer: accumulating jokers make single-round actions have long-term consequences (§3.3), so the tactical layer must be guided by the strategic layer’s long-term value $V(s’)$, and the two layers are jointly trained end-to-end.
2. Be careful with reward shaping: densifying progress signals is fine, but the terminal signal must lock onto “clear” to avoid reward hacking.
3. 99% on White Stake is achievable: because randomness is limited and human ~80% shows the remainder is caused by suboptimal decisions, RL’s optimization of long-term expectation has room to close the gap.

The next post will implement a minimal runnable Balatro RL demo (based on Jackdaw), write out the exact tactical-layer solver, and get the first baseline running. Happy hacking! 🃏

References

• Jackdaw — Balatro Simulator for RL
• BalatroBot — JSON-RPC API
• Balatro Wiki — Stakes