Announcement

Collapse
No announcement yet.

World War Matchmaking - Design Spotlight 2019

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • TinSoldier
    started a topic World War Matchmaking - Design Spotlight 2019

    World War Matchmaking - Design Spotlight 2019

    Hello DomiNations fans! Joe “Muet” Grubb here with Big Huge Games. This spotlight covers a long awaited change to DomiNations: World War Matchmaking.

    Why Change Matchmaking? What are the Problems We’re Trying to Solve?

    The high level goal is twofold: Improve the competitive integrity of World War and improve the match quality.

    Bear with me. I’m trying something new with this Spotlight; we’re going to go deep. We’re going to talk about how the old system worked and how the new system is attempting to solve problems. If that’s not appealing, there’s a tl;dr “what are the changes?” if you skip to the end. It’ll be clearly marked and I won’t fault you for skipping forward, after all you’ve got a Nation to run and World Wars to fight.

    Didn’t skip forward? You sure? Cool. Let’s take a look at the problems in the current system and break down each of them:
    1. Method for comparing the Offensive and Defensive Ratings of a base
    2. Sandbagging/Imbalance in Alliance compositions
    3. The Leaderboard was not part of the evaluation
    4. Queue functionality
    5. Is this match fair to my Alliance?
    6. Sparse queues
    Method for Comparing the Offensive and Defensive Ratings of a Base

    Did you know that the matchmaking system creates a Rating of your base’s offensive and defensive capabilities? It does. Everything on your base contributes to these Ratings. However, the method that the old system (the current matching system on live) used to compare the final Ratings with other bases wasn’t a great predictor of win rate. There was some directional value in its ability to predict win rate. But, we knew this was an area where we could improve match quality.

    That old system compared the absolute value of your Alliance’s Offensive/Defensive (O/D) Ratings with other potential Alliances. We ran some tests internally and discovered that when we match based on a percentage delta between those same Alliance’s O/D Ratings, we actually get a much stronger predictor for winning. So, we’re updating the system to compare Ratings by the percentage delta rather than absolute values.

    Fun fact: If your Alliance has a ~10% higher Defensive Rating in the new system, your Alliance’s chances of winning are increased to 60% (Yes, defense matters. Start upgrading your neglected Catapults!).
    Additional fun fact: The new system would not consider a 10% delta a “good” match. So don’t worry.

    Sandbagging/Imbalance in Alliance Compositions

    As with other team-based games, there is an ability to “carry” in DomiNations. If we only checked the percentage delta between two Alliance’s O/D Ratings, we may end up making the following match:
    Alliance A: 5 Cold War, 5 Medieval
    Alliance B: 1 Cold War with a mix of Enlightenment+.
    Seem familiar? I’m sure all of us have been in Alliance B’s position before. The O/D Ratings may look fair, but this is not a fair fight; Alliance B is going to lose this War.

    We don’t want to limit Alliances to bringing specific compositions to War. Sometimes you just don’t have enough players at close Ages. But we do want to address the sandbagging issue. So, we’re adding a second comparison to the Ratings formula. The new system now also compares the delta within an Alliance and tries to find other Alliances with similar deltas. The result? The system now knows that Alliance A and B are wildly inappropriate to match. It’ll find Alliance A another Alliance with a large Ratings delta in its players while protecting B from an unfair fight.

    Ready for another fun fact?: If your Alliance is sandbagging in the new system, it may result in increased queue times as the system waits to find another sandbagging Alliance for you.

    The Leaderboard was not Part of the Evaluation

    The old system did not use Glory as part of its calculation. In order to preserve the competitive integrity of our Leaderboards, we felt that it was important to start using Glory. In order to do that, Glory gained/lost needed to be updated.

    We had several discussions about potential directions to take Glory and what systems to use.
    A system like Elo (initially popularized as a chess ranking system) provides a single rating that can be used to predict the winner between opponents. Since Glory is serving as our Leaderboards, using a system that would suggest the higher ranking Alliances are more likely to win made sense. Afterall, if you’re Rank 1 on the Leaderboards, you should have earned it. Elo also does some other great things for Glory like awarding you more Glory if you happen to beat an Alliance that is higher ranked than you. That was a tough War and you deserve it.

    Elo is somewhat of a barebones rating system, however. Even chess has been modifying parts of the formula based on player ratings. So, we also considered more robust systems like Glicko-2, a more complicated evolution on the Elo formula that introduces a confidence interval. This confidence interval is a way for the system to determine how accurate it thinks an Alliance’s rating is. The interval can then modify points gained/lost in the system based on this confidence. For example, if your Alliance is 20-0 in War and the system says you have a rating of 18,000 Glory, what are the odds that your Alliance is probably better than 18,000? Probably high. Glicko-2 would take this into account and likely accelerate your subsequent Glory gained.

    While more complex systems may have offered some extra bells and whistles, we ultimately decided that we could get most of what was important to DomiNations by using a modified version of Elo we’re calling DOM MMR (DomiNations Matchmaking Rating). It’s pretty close to standard Elo with some targeted modifications to K-factor and streaking.

    It’s a bit of an oversimplification, but you can think of K-factor as a constant in the Elo formula that sets a maximum on the number of points that can be won or lost from a given match. There’s a handful of problems with this value being constant. Even chess has been modifying the K-factor of players based on their rating.

    So what modifications are we making? DOM MMR will dynamically change an Alliance’s K-factor based on the following:
    1. New Alliance - New combatants in any ranking system are hard to properly evaluate. It is near impossible to properly evaluate a new Alliance with only 2 Wars compared to an Alliance that has 300 under their belt. Therefore, new Alliances will have a higher K-factor for their first several Wars. The result should help get sharks out of the kiddie pool and help new Alliances find a win faster.
    2. Streaking - Winning or losing several Wars in a row will also cause the system to modify your K-factor. Sometimes you’re just not as good as you used to be, you know? Or maybe your Alliance recruited some top-tier players and it’s time to step into the big leagues.
    3. High Glory Alliances - K-factor modifications sound great but K-factor can break down at the extreme ends of a ranking system. Therefore, Alliances with particularly high Glory will have their K-factor treated differently. The main difference is that streaking won’t apply here.
    Now that we’re modifying how Glory works, we’re going to need to crunch the Leaderboards back toward 12,000 Glory. When the new matchmaking system goes live, all Alliances will be pulled toward 12,000 Glory (the strength of the pull being relative to your Alliance’s current Glory). This shouldn’t affect standings much. But, it will bring everyone closer together in Glory for a fresh start.

    Think of it as a season reset.

    Queue Functionality

    The old queue would search the list of available Alliances at the moment your Alliance entered queue. If you were not matched, you were then put in the bucket for other Alliances to potentially pair with when they entered queue.

    In theory this system sounds fine. And it worked okay for the most part. But, this created a rigid queue. Your potential pool of matches never expanded except when other Alliances entered the queue, regardless of how long you waited.

    This is changing in the new system to better incorporate industry best practices of time vs. match quality. The result is a system that is constantly evaluating the entire pool of Alliances against each other. The longer you wait, the less restrictive the system will be.

    Is this Match Fair to My Alliance?

    Some of you that are Alliance leaders may have noticed that your Alliance would sometimes instantly get matched with what appeared to be a poor match. What gives? Why? To the forums! The matchmaking is awful! Well, not quite. This was because the old system didn’t have a concept of handshaking. It compared Ratings at the time your Alliance entered queue and if they were within an acceptable range of another Alliance, bam. Matched.

    Moving forward, the new system will require the match to be acceptable from the perspective of both Alliances. This isn’t an in-game option or agreement that is made. Instead, you can think of it as the system viewing the potential match from both Alliance’s perspective to see if it thinks it is fair.

    What is fair to my Alliance after being in queue for only 10 seconds may be different from what another Alliance is looking for after 5 minutes.

    Sparse Queues

    I know that this will be one of the least popular changes. I apologize in advance for killing your Alliance’s favorite queue. The reality is that DomiNations can’t—and has never truly been able to—support seven different queues. Think of it this way: Each queue added to DomiNations splits the player base. Each of those splits greatly reduces the number of potential matches your Alliance can make and reduces the quality of those matches. Theoretically, the most optimal system would be to force everyone into a single XvX queue. We can’t really do that and aren’t entertaining the idea. But four is better than seven.

    We’re reducing the queues to the following:
    10v10
    20v20
    30v30
    50v50

    We feel that this division of queues will flow more Alliances into more pools and increase the odds of finding your Alliance a better match. It should be noted that 50v50 is not and has never been a popular queue. But, we felt it was important to preserve 50v50 for community events like the Continental World Cup (https://continentalwar.wordpress.com/). If you find your Alliance jumping into 50v50 expect long wait times and extremely relaxed matching requirements.

    Conclusion (“What are the Changes?” tl;dr is here!)

    In short, this is a complete overhaul to World War matchmaking. Standard PvP battle matching is not being touched. The changes are as follows:
    1. The matching logic has changed how it compares base power between Alliances. The new logic is proving to be a strong predictor of win rate and should allow us to make more fair Wars.
    2. The matching logic now understands imbalances within an Alliance’s composition. It understands “sandbagging” and will try to find Alliances opponents with similar compositions.
    3. How Glory is calculated has been redesigned.
    4. All Alliances are being pulled back toward 12,000 Glory when the update goes live. The pull is relative based on your Alliance’s Glory and it shouldn’t affect the standings much. Think of it as a season reset.
    5. Reduced the number of queues from seven to four:
      1. 10v10, 20v20, 30v30 and 50v50
    6. The behind-the-scenes functionality of the queue itself is changing to constantly evaluate the entire pool of Alliances in queue.
    7. The longer you wait in the queue, the more relaxed your matching requirements become.
    Finally, the entire system has been built with iteration in mind. Every piece of the system I explained has a knob that can be turned. We don’t expect the system to be perfect when it first launches but we’ll be actively studying it and tweaking behind the scenes to improve the quality of your Wars. We built these changes based on a lot of data from our internal systems and tools, but a major factor comes through player comments and discussions from all of our community pages. We'd love to get more feedback on this blog and the upcoming changes, so let us know what you think!

    Thanks for reading!
    - Muet

  • No Angel
    replied
    Omg we just got matched with VN Generals in 10v war. They're all 10 SA! And we only got 2 SA. How the heck that supposed to happen TinSoldier BHG_Muet 😓
    ridiculously unfair we got zero chance to win.

    Leave a comment:


  • Rachel
    replied
    I feel if a base is not active or doesn’t log in the 24 hour planning day that particular base forfeits this will stop sandbagging

    Leave a comment:


  • Bootney Lee Fonsworth
    replied
    You're the one who's complaining. I'm perfectly fine in my alliance. What's your idea then?

    Sounds to me like you're you're in the top 0.01% but you're upset that you're consistently matched against the top 0.005%. Congratulations! You've reached the pinacle of the game that few others have. Yet you seem mystified that you encounter the same opponents repeatedly. At least my idea would possibly give you more teams to fight against.
    Last edited by Bootney Lee Fonsworth; 09-10-2019, 10:02 AM.

    Leave a comment:


  • Chadwicke
    replied
    Your solution is for people to leave their friends Bootney Lee Fonsworth ?

    Leave a comment:


  • Bootney Lee Fonsworth
    replied
    I'm still baffled why they don't just weigh the top half of a lineup twice as much as the bottom. It's not perfect but it would probably lead to a reverse sandbagging situation where all the high space agers try to consolidate into a few mega alliances to 'hide' in the lower weighted halves of alliances. That actually seems far preferable to the trickle down sandbagging we've had since day one. Let all the 300+ folks fight over the top 100 and expand the leaderboard to top 500 so there's some sort of measuring stick. Or just have alliance leagues.

    I'd say matchmaking is better from where my alliance sits, but it has more to do with finding the right formula to get mostly favorable matchups than us winning a series of highly competitive wars. Can't really disagree that 95% of wars are still decided as soon as the match is made. According to dominations.pl we're in the 800 to 1000 range though, so your mileage may vary.

    Leave a comment:


  • Chadwicke
    replied
    All the people who were like thank you so so much for this new system still Loving it??? It complete garbage as I said it would be get to face the same teams over and over always one side with an advantage over the other. Most war is won or lost in planning day now easy wars and impossible war is alll this system knows. Time to take it behind the barn and shoot it .

    Leave a comment:


  • Aspergaon
    replied
    Guys. Keep it simple! Because simple is better. WW are mostly defined by the top half of players. If it’s a 20 players war, the top 10 will define it with both attacks. So those first half top players must be matched the most possible: same number of players on each age! If it’s 3 SA in one side the same number must be on the other side. Regarding levels would be great to make them match more or less too! With that solved the most of us will stop complaining. My war right now is 8 SA vs. 4 on my side! My top play rated as 1 is SA level 268. My opponents 11th player is SA level 300!!! Absurd!!!

    Leave a comment:


  • joelegiao
    commented on 's reply
    Yaaa let's hope the fix fixes the fix that they put on to try to fix the problems we had with matchmaking and wich already better than now lol

  • Aspergaon
    replied
    Wow great news!! I can’t wait to try the fix of the fix of the fix of the fix of WW matching “system”!!!!

    Leave a comment:


  • pckrn
    replied
    our 10v10 match

    us: 2 sa, 3 cwa, 3 aa, 1 ga, 1 ia (total of 1 rocket arsenal)
    them: 7 sa, 1 ga, 2 ia (6 rocket arsenals)

    hahaha

    but hang in there people, tin has already mentioned they have a fix coming in the next update.

    https://forum.nexonm.com/forum/nexon...724#post658724

    they just like to see us suffer in between updates is all

    Leave a comment:


  • Aspergaon
    replied
    Still your matching system for world war is absurd and unfair. Look at our current war. Enemy 8 space age and 5 Cold War players. Us: 4 space ages and 1 Cold War. They have it almost guaranteed with their 2 attacks or those 13 players. You need to understand that the first thing to match is same number of players in the same age. Everything else is secondary! Look their 11 th player got 5 stars from our 2nd player!!!! Absurd!!!!! Our top player is SA level 268. The opponents 11th player is SA level 300!!!!

    Leave a comment:


  • SomeRandomPlayer
    commented on 's reply
    That's not so bad of a match-up. Granted the "many defenses are not being accounted" part has merit, and is unfortunate, but this match-up itself is not terrible especially considering there's a lot more to a person's value in war than simply Age.
    Last edited by TinSoldier; 08-12-2019, 12:35 PM.

  • Mountainking
    replied
    US: 6 SA, 3 CWA, 5 AA, 2 IA, 1 EA, 1 Classical, 2 Irons.
    THEM: 12 SA, 6 CWA, 1 IA, 1 MA

    BHG_Muet So you guys still "suspect" what could be causing this? The ridiculousness of this statement is BEWILDERING. EVERY half decent player knows many defenses are not being accounted in war weight.

    Concretely, WTF are you people doing to fix it and HOW SOON? Oh btw GP split: 32/-288

    Leave a comment:


  • Rachel
    replied
    So in short you allow cheating- AKA sandbagging their bases haven’t been active for months even years yet troops can be donated to them and an active player cant until we update our game. Thanks for cheating players

    Leave a comment:

Working...
X