Hex and Counter Wargames are adversarial two-player simulations of realmilitary conflicts requiring complex strategic decision-making. Unlikeclassical board games, these games feature intricate terrain/unit interactions,unit stacking, large maps of varying sizes, and simultaneous move and combatdecisions involving hundreds of units. This paper introduces a novel systemdesigned to address the strategic complexity of Hex and Counter Wargames byintegrating cutting-edge advancements in Recurrent Neural Networks withAlphaZero, a reliable modern Reinforcement Learning algorithm. The systemutilizes a new Neural Network architecture developed from existing research,incorporating innovative state and action representations tailored to thesespecific game environments. With minimal training, our solution has shownpromising results in typical scenarios, demonstrating the ability to generalizeacross different terrain and tactical situations. Additionally, we explore thesystem's potential to scale to larger map sizes. The developed system is openlyaccessible, facilitating continued research and exploration within thischallenging domain. <<<

This paper proposes a new game-search algorithm, PN-MCTS, which combinesMonte-Carlo Tree Search (MCTS) and Proof-Number Search (PNS). These twoalgorithms have been successfully applied for decision making in a range ofdomains. We define three areas where the additional knowledge provided by theproof and disproof numbers gathered in MCTS trees might be used: final moveselection, solving subtrees, and the UCB1 selection mechanism. We test allpossible combinations on different time settings, playing against vanilla UCTon several games: Lines of Action ($7$$\times$$7$ and $8$$\times$$8$ boardsizes), MiniShogi, Knightthrough, and Awari. Furthermore, we extend this newalgorithm to properly address games with draws, like Awari, by adding anadditional layer of PNS on top of the MCTS tree. The experiments show thatPN-MCTS is able to outperform MCTS in all tested game domains, achieving winrates up to 96.2% for Lines of Action. <<<

Transformer models have demonstrated impressive capabilities when trained atscale, excelling at difficult cognitive tasks requiring complex reasoning andrational decision-making. In this paper, we explore the application oftransformer models to chess, focusing on the critical role of the positionencoding within the attention mechanism. We show that in chess, transformersendowed with a sufficiently versatile position encoding can match existingchess-playing models at a fraction of the computational cost. Our architecturesignificantly outperforms AlphaZero at 8x fewer FLOPS and matches priorgrandmaster-level transformer-based agents at 30x fewer FLOPS. <<<

The AlphaZero algorithm has been successfully applied in a range of discretedomains, most notably board games. It utilizes a neural network, that learns avalue and policy function to guide the exploration in a Monte-Carlo TreeSearch. Although many search improvements have been proposed for Monte-CarloTree Search in the past, most of them refer to an older variant of the UpperConfidence bounds for Trees algorithm that does not use a policy for planning.We introduce a new, improved search algorithm for AlphaZero which generalizesthe search tree to a directed acyclic graph. This enables information flowacross different subtrees and greatly reduces memory consumption. Along withMonte-Carlo Graph Search, we propose a number of further extensions, such asthe inclusion of Epsilon-greedy exploration, a revised terminal solver and theintegration of domain knowledge as constraints. In our evaluations, we use theCrazyAra engine on chess and crazyhouse as examples to show that these changesbring significant improvements to AlphaZero. <<<

This paper introduces ZeusAI, an artificial intelligence system developed toplay the board game 7 Wonders Duel. Inspired by the AlphaZero reinforcementlearning algorithm, ZeusAI relies on a combination of Monte Carlo Tree Searchand a Transformer Neural Network to learn the game without human supervision.ZeusAI competes at the level of top human players, develops both known andnovel strategies, and allows us to test rule variants to improve the game'sbalance. This work demonstrates how AI can help in understanding and enhancingboard games. <<<