The second white paper in Activision’s "Matchmaking Series" provides extremely interesting insights in the role of skill in Call of Duty's core multiplayer matchmaking system. A hotly debated topics, it offers insights into how skill is calculated, its impact on team balance, and the player experience in arguably the world’s most popular first-person shooter game. Insights that game devs working on multiplayer can use to improve their own matchmaker's rules.

More importantly, it highlights that while skill has an impact on the perceived quality of the matchmaking experience, matchmaking should prioritize latency first and foremost.

How Does Skill-Based Matchmaking Work?

Skill-based matchmaking (SBMM) is a system designed to pair players of similar skill levels in multiplayer games. The idea is to create balanced matches where players have a fair chance of winning, regardless of their skill tier. SBMM in Call of Duty works by tracking each player's performance in games and assigning them a skill rating based on metrics like kills, deaths, objectives completed, and other in-game achievements.

After each match, a player’s performance is evaluated, and their skill level is adjusted accordingly. When a player searches for a new match, the matchmaking system uses this skill level to place them in a game where most players have similar ratings. However, CoD doesn’t solely rely on SBMM; it also factors in things like ping (connection quality) and time spent in the matchmaking queue to ensure smooth and timely matches.

In addition to skill, Call of Duty’s matchmaking algorithm considers several other factors:

• Connection (Ping): This remains the top priority in matchmaking, as a strong connection is crucial for a smooth gaming experience.

• Time to Match: Ensuring players spend more time playing rather than waiting for a match.

• Playlist Diversity and Recent Maps/Modes: These ensure players aren't stuck in repetitive game modes or maps.

• Input Device and Platform: Whether the player uses a controller or keyboard and mouse is also taken into account.

• Voice Chat: This can influence match balancing based on whether it’s enabled or disabled.

Skill plays a significant role, but it's balanced alongside these factors to deliver an engaging multiplayer experience.

Defining Skill

Skill, in the context of Call of Duty, is defined as how well a player is expected to perform against the rest of the population, based on their previous gameplay. The developers aim for a skill metric with the following properties:

1. Predictive: The player’s past performance should predict future outcomes.

2. Summable: The average skill of a team should reflect its combined potential performance.

3. Adaptable: The system should quickly account for changes in a player’s performance, such as experimenting with new loadouts.

4. Stable: The system should remain accurate and resilient to sudden population changes, like an influx of new players.

How Skill is Calculated

Skill is calculated based on relative performance in each match. After a match, player performance is compared across all players in the game, and each player's skill rating is adjusted. These adjustments depend on the likelihood of a player's performance relative to others.

Different performance metrics, like kills or kill/death ratios, were explored, but they each have drawbacks. For example, simply counting total kills doesn't consider how many times a player dies or if they joined the match late. The current algorithm ensures resilience against these limitations.

How Skill Changes Over Time

Player skill can fluctuate due to various factors, such as experimenting with new strategies or simply having an off day. The matchmaking system adjusts skill values dynamically to account for these changes without overcorrecting. Rapid fluctuations in skill levels can lead to unstable matchmaking, so a balance is sought between stability and flexibility.

Why Track Skill?

Tracking skill is critical for maintaining balanced and competitive games. Without skill tracking, matches are more likely to be blowouts, leading to a negative experience for the losing team. Activision has found that integrating skill into matchmaking improves player retention, with 80-90% of players experiencing a better match outcome.

Skill tracking also helps prevent a negative feedback loop where lower-skill players leave the game, which could eventually lead to a population dominated by only the highest-skilled players. This would create an unwelcoming environment for new players and harm the overall player base.

• Team Balance: Team balance aims to make match outcomes unpredictable and reduce blowouts, which are linked to reduced player enjoyment. Call of Duty uses a system where skill differences between teams are minimized to create fairer matches. Even small disparities in skill between teams can significantly affect win rates.

• Testing Skill Matching Approaches: Activision regularly tests different approaches to skill matching. For example, in early 2024, a "Deprioritize Skill Test" was conducted where skill was given less weight in matchmaking. The results showed that although higher-skilled players returned to the game in larger numbers, 90% of players across the lower skill tiers were less likely to return after two weeks. This suggests that loosening skill constraints has a long-term negative impact on the player base as lower-skilled players become discouraged. On the other hand, tightening skill constraints too much can negatively affect higher-skilled players, who may become frustrated with excessively balanced matches. Activision strives for a balance between these two approaches.

Which Call of Duty Games Have Skill-Based Matchmaking?

Skill-based matchmaking has been a part of the Call of Duty franchise for many years, though its implementation has evolved over time. SBMM was first introduced in Call of Duty 4: Modern Warfare but gained more prominence with recent entries in the franchise, particularly starting from Call of Duty: Modern Warfare (2019).

CoD Titles with Significant SBMM Implementation:

• Modern Warfare (2019): One of the first games to make SBMM a focal point of its matchmaking system.

• Black Ops Cold War: Similarly, this entry continued the trend of using SBMM extensively, even drawing debates within the community.

• Warzone: SBMM in Warzone has been a key factor in its player base, balancing the casual experience with competitive dynamics in a battle royale setting.

• Modern Warfare II (2022): Recent CoD titles have further refined SBMM, often tested with different levels of skill looseness to maintain fairness without overly restricting matchmaking options.

Every new CoD title since Modern Warfare (2019) has fine-tuned the use of SBMM, leading to ongoing experimentation and balancing of the system to accommodate different types of players.

The effects of skill matching vary across different Call of Duty titles. For example, in titles with tighter skill matching, lower-skill players tend to have a more balanced experience, whereas looser skill matching leads to greater variance in match outcomes.

A comparison between two Call of Duty titles showed that players with low skill placed in the bottom half of the leaderboard 90% of the time in a title with looser skill matching. In a title with tighter skill matching, this dropped to 75%. However, even with tighter skill matching, higher-skilled players still outperform lower-skilled players significantly.

Will Call of Duty Remove Skill-Based Matchmaking?

A recurring question from the Call of Duty community is whether SBMM will ever be removed from the game. While some players argue that SBMM can make casual gameplay feel too competitive, there are no current plans to remove SBMM entirely. On the contrary, Activision has been refining and testing different levels of SBMM through various experiments to find the best balance between competitive fairness and fun.

For instance, in early 2024, a "Deprioritize Skill Test" was conducted in Call of Duty: Modern Warfare III, loosening the SBMM constraints for a portion of the player base. The results showed that while high-skill players experienced increased satisfaction, 90% of lower-skill players returned less frequently, leading to a significant concern for player retention. This indicates that removing or excessively loosening SBMM may harm the game’s overall player population by discouraging less-skilled players from continuing to play.

While SBMM may be adjusted or refined, it is unlikely to be removed completely, as it plays a vital role in maintaining player engagement and preventing the negative feedback loop of less-skilled players abandoning the game.

Importance of Latency in Matchmaking Rules

Latency, or connection quality, is of paramount importance in Call of Duty's matchmaking system, surpassing even skill considerations. The phrase "Ping is King" reflects how the matchmaking algorithm prioritizes a player’s connection to the server above all else. Here's a breakdown of its role and why it's so crucial:

1. Primary Factor in Matchmaking: Connection quality, measured by ping, is the most heavily weighted factor in Call of Duty’s matchmaking system. Players are matched based on their proximity to a data center (server), with the goal of minimizing latency to ensure smooth gameplay. A low-latency connection reduces lag, prevents delays in player actions, and ensures that matches are more responsive and fair.

2. Connection Quality Overrides Other Factors: While skill-based matchmaking (SBMM) is important, it is given less weight than ping. In the event of a conflict between a player's skill level and connection quality, the matchmaking system will always prioritize a better connection over creating a perfectly skill-balanced match. This is particularly true for players at the higher and lower ends of the skill spectrum, who may be paired with opponents of varying skill to maintain low ping.

3. Delta Ping: Call of Duty uses a concept called "Delta Ping," which measures the difference between a player's best possible ping and their ping to other data centers. The system loosens skill constraints faster than it loosens delta ping constraints, meaning that it is more flexible in finding a match with an acceptable skill difference than it is in compromising on connection quality. This highlights the importance of providing a smooth gameplay experience even if it comes at the cost of less precise skill matching.

4. Backoff Mechanism: As matchmaking time increases, the system begins to "back off" on strict matchmaking rules, including skill and connection requirements. However, it still prioritizes ping to maintain a playable experience. Even as the system loosens skill-based constraints, it is careful about ping, relaxing this constraint more slowly to ensure that players have an optimal connection.

Conclusion

Call of Duty’s experiments emphasize the trade-offs between connection quality and skill-based matchmaking. Developers should recognize that while SBMM can enhance competitive fairness, prioritizing low-latency connections is key to providing a responsive and enjoyable gameplay experience. Players are more likely to tolerate mismatches in skill than poor connections that lead to lag and stuttering gameplay.

Latency, or ping, is the most critical factor in Call of Duty’s matchmaking, with Activision's extensive testing emphasizing its importance for providing a smooth gaming experience. While skill-based matchmaking improves fairness, prioritizing connection quality ensures better player retention and satisfaction.

Game developers can gain significant insights from Call of Duty's experiments, especially regarding the balance between latency and skill, and the effects of matchmaking on long-term player engagement. By incorporating dynamic systems that adjust matchmaking rules based on latency and other factors, developers can create more enjoyable, competitive environments for their players.

Fortunately, Edgegap’s matchmaker is the sole widely-available matchmaker with integrated latency rules. As it’s fully integrated in Edgegap’s orchestrator, it also taps in the world’s largest edge network – which allows your multiplayer to deploy its game server nearest users. Reducing latency by 58% on average and delivers sub-50ms latency to 78% of your playerbase.