Call of Duty's Skill-Based Matchmaking: An In-Depth Analysis

Call of Duty has recently released an extensive document detailing the implementation of skill-based matchmaking (SBMM) in their games. This article provides a thorough analysis of the key points from this document, breaking down the complex information into digestible sections for players and enthusiasts alike.

Understanding Skill-Based Matchmaking

Skill-based matchmaking is a system designed to create balanced and competitive matches by grouping players of similar skill levels together. The document released by Call of Duty developers offers unprecedented insight into how this system works, what factors are considered, and the goals behind its implementation.

Key Terminology

Before diving into the specifics, it's important to understand some key terms used throughout the document:

• Raw Skill: A single value representing a player's performance relative to the entire player population.
• Skill Percentile: A value indicating where a player's raw skill lies within the population.
• Skill Disparity: The difference between the highest and lowest skilled players in a party or lobby.
• Key Performance Indicator (KPI): Metrics tracked against objectives, such as playtime, lobby retention, and quit rate.

Calculating Skill in Call of Duty

The document provides an interesting example of how skill is calculated in a Team Deathmatch game mode. Three different metrics are considered, each with its own benefits and drawbacks:

1. Match Total Kills: This metric shows how well a player performed in achieving the main objective but doesn't account for deaths or survival ability.

2. Kill/Death Ratio: A more common metric that balances kills against deaths, but it can be manipulated through reverse boosting.

3. Kills Divided by Deaths Per Enemy: This new metric was introduced to counter reverse boosting and provide a more accurate representation of skill.

Dynamic Skill Evaluation

The developers acknowledge that player skill can change over time due to various factors such as fatigue, distractions, or lack of recent play. To account for this, they update player skill values on an ongoing basis, aiming to find an equilibrium quickly without overcorrecting in either direction.

The Importance of Skill Tracking

According to the document, one of the core design principles of Call of Duty is to be a "player-first" game. This means ensuring that players of all skill levels can have a fun and competitive experience. Team balance plays a crucial role in achieving this goal.

The Feedback Loop

One of the main reasons for implementing SBMM is to avoid a negative feedback loop where lower-skilled players continually leave the game as the average skill of the population rises. This phenomenon is illustrated in the document with a simple but effective diagram.

The feedback loop works as follows:

1. As the average skill percentile disparity rises, matches become tougher for lower-skilled players.
2. More lower-skilled players quit or play less frequently.
3. As lower-skilled players leave, the remaining players are shifted down the skill hierarchy.
4. This process repeats, making the game increasingly "sweaty" for all players over time.

By catering to lower-skilled players, the developers argue that they actually help the top 10-20% of players as well. If lower-skilled players disengage from the game, higher-skilled players become the new "low-skilled" players relatively speaking, creating a continuous cycle of increasing difficulty.

Team Balance and Party Matching

The document goes into detail about how team balance is achieved and how parties with varying skill levels are matched. Even small changes in team balance can yield unexpected results, such as a single high-skilled player dominating a match full of average players.

Party Matching Heuristics

When matching parties with different skill levels, the system uses a complex heuristic selection process. This process considers:

• Individual player skills within the party
• The party's average skill
• Acceptable skill matching range
• Skill disparity within the party

The goal is to find other parties or players with similar skill compositions to create balanced matches.

Experimental Results

The document reveals that Activision has been experimenting with different SBMM implementations across various Call of Duty titles. In early 2024, they ran a test in Call of Duty: Modern Warfare 3 where they loosened the constraints on skill in matchmaking for 50% of the population.

The results were significant:

• 90% of players in the experimental group were less likely to return within 14 days.
• Only the top 10% of players showed an increased likelihood of returning.
• Quit rates increased dramatically for all but the top two skill brackets.

These findings suggest that reducing SBMM could lead to a shrinking player base as lower-skilled players find the game less enjoyable and quit more frequently.

Impact on High-Skill Players

While the document acknowledges that SBMM can make matches more challenging for high-skill players, it argues that maintaining a healthy low and medium-skill population actually benefits high-skill players in the long run. This is because:

1. The skill population is highly asymmetric, with most parties sitting in a higher skill distribution.
2. Matchmaking at higher skill levels requires more population to form equitable matches.
3. If lower-skilled players leave due to poor experiences, it becomes increasingly difficult to matchmake for higher-skilled players.

Conclusion

The Call of Duty developers have provided an unprecedented look into the complexities of skill-based matchmaking in their games. While the system may be controversial among some players, particularly those at higher skill levels, the data presented suggests that SBMM plays a crucial role in maintaining a healthy and engaged player base across all skill levels.

By balancing the needs of lower-skilled players with the challenge desired by higher-skilled players, the developers aim to create a more enjoyable and sustainable gaming experience for the entire Call of Duty community. As the game continues to evolve, it's likely that further refinements to the SBMM system will be made based on ongoing data analysis and player feedback.

Ultimately, the goal of skill-based matchmaking in Call of Duty is to ensure that players of all skill levels can find enjoyable and competitive matches, fostering a thriving and diverse player community for years to come.

Article created from: https://www.youtube.com/watch?v=1sA7nq6xnlM