July Feature: Archetype Determinant, A New Tool for TIA
One of the problems that I’ve faced a lot when building and working with the TIA system is the problem of how to properly classify decklists. How, that is, to determine when two different decklists are similar enough that they ought to be considered variations on the same idea, rather than as separate ideas. I’ve gone through a number of different approaches to solving this problem over the years, starting with a loose set of heuristics based on colour, Mane Character, and any tribal synergies that existed within the deck. I tried to refine these heuristics for a while, hoping to find some well-defined rules that I could use, but the problem proved much more complex than such a simple system could handle. Thus, for the last little while I’ve been doing classifications purely ad hoc, and it’s been working well enough. But the problem of trying to have a system to do it is one that I never fully left off thinking about.
In this post, I’m going to be going through the beginning of the most recent attempt that I’m making at this, a system that right now I’m calling Archetype Determinant (AD), which hopefully will eventually be mature enough to be included as part of the TIA process. AD essentially looks at the past history of decks that have been built in Core, and assigns archetype-scores to each card in the format based on how often it shows up in each kind of deck (aggro, control, combo, farm, or midrange). Then, upon seeing a new deck, it adds up the individual scores for each card in the deck, producing a histogram that hints at which archetypes the deck is most likely to exist in.
For example, here’s what the current version of Archetype Determinant thought of the decks from the most recent OnlineCon:
Thorax Blue (#1 & 2)
------------------------
Aggro: 19.38 | ===============
Control: 1.17 | =
Farm: 2.97 | ===
Midrange: 2.63 | ===
Combo: 0.86 | =
Cadance Blue (#3)
------------------------
Aggro: 11.77 | ===============
Control: 3.51 | =====
Farm: 2.96 | ====
Midrange: 3.07 | ====
Combo: 3.70 | =====
Leaders Shy Blue (#4)
------------------------
Aggro: 22.61 | ===============
Control: 0.88 | =
Farm: 1.76 | ==
Midrange: 2.00 | ==
Combo: 0.75 | =
Celaeno White (#5)
------------------------
Aggro: 5.57 | =========
Control: 4.70 | =======
Farm: 10.20 | ===============
Midrange: 3.24 | =====
Combo: 1.28 | ==
Cadance Orange (#6 & #13)
------------------------
Aggro: 15.90 | ===============
Control: 4.08 | ====
Farm: 2.04 | ==
Midrange: 2.62 | ===
Combo: 2.35 | ===
Defenders Shy Mono (#7)
------------------------
Aggro: 13.94 | ===============
Control: 2.18 | ===
Farm: 3.27 | ====
Midrange: 6.78 | ========
Combo: 0.82 | =
Leaders Applejack Mono (#8)
------------------------
Aggro: 5.58 | ==========
Control: 8.72 | ===============
Farm: 8.22 | ===============
Midrange: 3.38 | ======
Combo: 1.10 | ==
Gallus Pink (#9)
------------------------
Aggro: 4.67 | =======
Control: 10.98 | ===============
Farm: 4.91 | =======
Midrange: 3.64 | =====
Combo: 3.81 | ======
Celaeno Pink (#10)
------------------------
Aggro: 8.99 | ===============
Control: 5.20 | =========
Farm: 7.74 | =============
Midrange: 2.54 | =====
Combo: 3.53 | ======
Novo Pink (#11)
------------------------
Aggro: 10.93 | ===============
Control: 6.42 | =========
Farm: 1.79 | ===
Midrange: 6.24 | =========
Combo: 2.62 | ====
Grogar Mono (#12)
------------------------
Aggro: 4.64 | =======
Control: 4.54 | =======
Farm: 11.17 | ===============
Midrange: 3.09 | =====
Combo: 0.56 | =
When I was doing ad hoc classification of decks, I found that when the answer wasn’t immediately obvious, I would often look at what you might call “signpost” cards, cards in the deck that usually only show up in particular archetypes. For example, when trying to differentiate between a Blue Aggro and a Blue Farm, seeing a card like Prince Rutherford, Yak Smash is a hard signal that it’s Farm. Similarly, if it’s a bit of a toss-up between Farm and Control, there are certain Troublemakers that tend to mostly show up in Farm like Pony of Shadows or the Storm King.
This is also true of Problem Decks, which tend to be fairly similar across archetypes, as generally aggro prioritizes low confront requirements and/or high bonuses, while control will prioritize the opposite: high confront requirements and low bonuses. So there’s often a lot of clues to be found in looking there too.
Seeing these obvious clues, though, really made me wonder if there may be more subtle signals hidden within the data. For example, we can look at a really innocuous card like Join The Herd. It’s quite a generic effect, and you may perhaps have a notion of the kinds of decks that it would normally go in. As it turns out, it’s used nearly equally in Aggro and Farm decks, with a slight preference to Farm. So it’s a moderate signal for both of those. It’s easy for a human to pick out the strong signals within a decklist, but what about the accumulation of weak signals throughout the list? That’s a problem that’s easier for an algorithm to solve.
Archetype Determinant starts with the TIA database of decklists which have been played in competitive events. The database isn’t enormous, but it’s not small either; it’s 226 decks as of this writing. Obviously, more data is always better, and one of the interesting things about this tool is that it doesn’t need to be limited to decks that saw competitive play. In theory, any constructed decklist could be a data point to interpret, though it is likely that an advanced system should give extra weight to competitive decks. For now, though, AD is only looking at the TIA database.
Over those 226 decks, every card that shows up at least once receives 5 scores reflecting the proportion of each archetype the card shows up in. So, for example, if a card was used half the time in control and half the time in aggro (which usually doesn’t happen, but this is just an example) the card’s individual scores would be (aggro: 0.5, control: 0.5, farm: 0.0, midrange: 0.0, combo: 0.0).
Once every card has a score, rating a deck is as simple as reading the decklist and accumulating the 5 numbers over the whole decklist, creating the 5 scores you see output in the histograms at the top.
So what do the numbers mean? Obviously, if one bar is way ahead of the others, it’s a good signal of which archetype probably sits in. Based on my experience, aggro decks tend to be the ones that most often fit this category, which makes sense as a lot of the cards that tend to show up in aggro decks are usually only in aggro decks, and so tend to be strong signals. Think about Dilemmas, Events, and the Problems which aggro decks tend to use.
More interesting are decks that have two prominent bars in their histogram. When one remains dominant and the other is secondary, that would seemingly indicate a deck that leans in one direction but also has strong affinity for the play-style indicated by its secondary archetype. Consider, for example, a Farming deck which can lean on playing Aggro if it needs to, like many Blue Farming decks do. Or a Control deck which sometimes employs lines of play similar to what a Farming deck would do.
Finally, there are decks out there which the system really can’t properly interpret at the moment, with three or more prominent bars in their histogram. It should be stressed that AD is still a system in its infancy, and so obviously there are going to be decks out that aren’t properly classified for a variety of reasons. And that brings me to:
Shortcomings And Improvements
While the eventual end goal is for Archetype Determinant to form a stage in the TIA pipeline, the system as currently implemented should be considered as essentially an alpha release. I think that it’s got enough features to be useful, but there are many areas it could use improvements in. I also should perhaps stress that the goal of this project isn’t to create an algorithm which can decide for me what archetypes to assign decks into. Even in its final form, AD will be merely a tool that is considered alongside other variables and intuition.
One obvious shortcoming of this system comes to a simple and valuable adage: "The map is not the territory." This system relies on ways that people have used cards in the past to predict how they will be used in the future. Thus, by definition, a new deck that uses cards in ways they haven't been used before is going to be misclassified. This is a big reason why AD in its current form can never be the one final word on which archetype to classify a deck as.
At the same time, there are still improvements I want to make. A change likely to make a significant improvement in the accuracy of the result is to also consider pairs of cards that show up in decks in addition to single cards. After all, many cards exist in loosely-defined “packages” that get added to a deck together in order to augment a certain intended playstyle for that deck. Admittedly, I am having a bit of a tricky time coming up with examples right now, but even if there aren’t any obvious strong signals, there may be weak signals out there of sets of two or more cards which tend to occur together only in decks of a certain archetype.
One issue with doing this, of course, is that we start to look at a lot of data points. The card data for just this version of AD covers 651 cards. If we start covering arbitrary pairs, that number would rise to more than 420 thousand (as there would be 650 possible pairs for every card, so 650x651=423,150). That’s probably still okay to handle, but it’s starting to get high, especially since the obvious next step is to look at 3-tuples, 4-tuples, etc.
All of that may make you wonder: why not just go whole-hog and send a Neural Net after the problem? After all, classifying objects into groups is a pretty textbook application for machine learning. And yeah, at some point in the future I might do that, if I can find some time to learn how.
For the time being, my major areas of work on the system are going to be on adding diagnostic elements, so that hopefully I can come to a greater understanding of how best to interpret its histograms. Another point of work will be to find more decks to feed into the database. And naturally I’ll want to run it on the Harmony and Adventure TIA pools as well at some point in the future. For now, though, it’s simply going to be another tool in the box when it comes to classifying decks and considering archetypes in TIA.