Winning determinants between top and second division in Chinese professional football leagues: an interpretable machine learning approach

Abstract

This study investigated the impact of winning determinants in two professional soccer leagues. The sample was composed of 1,440 Chinese Super Football League (CSL) and Chinese Football Association China League (CFACL) matches (CSL matches = 720; CFACL matches = 720) during the 2017–2019 seasons. The study employed eXtreme Gradient Boosting (XGBoost) to assess the importance of 25 indicators exhibiting significant differences (p < 0.05) in their association with match outcomes, and the SHapley Additive explanations (SHAP) was utilized to interpret these findings. The results showed that scoring performance indicators, such as Shots On Target Inside Box (SOTIB), Shots, and Shots On Target (SOT), significantly influenced outcomes in both the CSL ($\:{\varvec{S}}_{\varvec{G}}$=37.854%) and CFACL ($\:{\varvec{S}}_{\varvec{G}}$=38.934%), with SOTIB being the most impactful. Additionally, this study found that defensive feature clearances were highly influential in both leagues, ranking second only to SOTIB of variable importance. Meanwhile, defensive feature fouls were a more significant factor in determining match outcomes in the CFACL than in the CSL. In both the CSL and CFACL, players must prioritize precision in shooting within the penalty area rather than merely increasing the frequency of shots. For CFACL teams, if consistent high-quality passing is unattainable, effective use of set pieces (e.g., free kicks) could serve as an alternative strategy to organize attacks. These findings can assist coaches in formulating tailored tactical strategies suited to the distinct demands of each league level.

Introduction

For a long time, the number of goals scored is crucial for determining match outcomes, but the factors that influence the match outcomes are complex. The performance indicators that predict success in soccer matches have long been a focus of international soccer research [1]. Previous research on soccer match-winning factors has preliminarily analyzed the impact on match outcomes from various perspectives, such as running distance [2], possession [3], pass success ratio, and shooting efficiency [4]. Recent research has emphasized the integration of multiple methods and variables for more comparative analysis, rather than isolating a single factor in determining match outcomes [5, 6]. Previous studies have classified team performance indicators into three distinct categories: variables related to goal scoring, variables related to passing and organizing, and variables related to defending [5,6,7]. This tripartite framework provides a more precise and granular set of descriptors for analyzing match performance dynamics. For instance, one study noted that technical performance is crucial to match outcomes [8, 9]. Another investigation related to winning discriminants in the Bundesliga found that defensive errors and goal efficiency are both significant factors, and cross is a negative indicator [10].

Furthermore, disparities in match performance may exist across different leagues and teams, reflecting the distinct playing styles and strategic priorities inherent in each league [11]. For instance, some researchers investigated the technical tactical and running performance in Span’s first and second divisions league and found that technical-tactical performance variables have a greater influence on a team’s success than running performance [12]. Another research on the Spanish soccer leagues over eight seasons found that technical-tactical variables, like passes and successful passes in the first division, are more than in the second division [13].

Additionally, several studies have investigated match performance in European leagues, researchers have a common view that soccer leagues with different cultural backgrounds displayed different playing characteristics, as the English Premier League showed more tackles, and long passes than Italian Serie A, and Spanish La Liga showed more passes than English Premier League [14]. Previous research on CSL indeed found disparities compared with the literature of European leagues. For instance, some researchers compared the actual playing time between the CSL and the English Premier League and suggested that while non-anthropogenic factors influenced both leagues, the actual playing time in the CSL was more influenced by anthropogenic factors [15]. Therefore, it is crucial to compare the relationships and impacts of winning determinants across different leagues. Machine learning algorithms offer the capability to process intricate datasets and generate real-time insights [16]. For example, XGBoost is excellent at handling diverse and complex data structures to make accurate predictions, while SHAP can directly identify the most critical features from matches [17]. In recent years, the SHapley Additive exPlanations (SHAP) model has been applied to soccer match performance analysis [17,18,19], demonstrating strong predictive capabilities in assessing team performance, such as predicting match outcomes [17, 18] and forecasting player injuries [17, 20].Furthermore, the winning determinants in soccer across different leagues are inherently complex, requiring consideration of multiple dimensions [5, 21], including technical-tactical performance, running performance, physical conditioning, and external factors. As each of these factors interacts in intricate ways, it is essential to employ machine learning to uncover the patterns and insights that contribute to winning outcomes.

In summary, previous studies have compared differences in match performance indicators across teams of different competitive levels. However, these indicators may not fully capture the critical factors influencing team success, as teams at distinct levels face opponents of varying strengths. This necessitates a focused comparison of performance metrics among teams within the same competitive tier. Furthermore, while certain performance indicators exhibit significant differences in winning matches for teams at the same level, their relative importance to match outcomes remains unclear. In football, the relationship between performance data and match results is not strictly linear. Conventional linear regression may lack precision in analyzing such relationships. In contrast, the XGBoost and SHAP models enable robust analysis and prediction of non-linear data, effectively identifying the significance of performance indicators and elucidating complex data relationships [19]. Additionally, we have incorporated considerations of cultural differences and disparities in league competitiveness. By conducting comparative analyses within Chinese football leagues and contextualizing findings against China’s football development stage and top-tier league performance, this study provides a more nuanced discussion of the results.

Based on the above-mentioned context, the aims of this study were to: [1] analyze the variables influencing the outcomes of games between the Chinese Football League (CFL) and Chinese Football Association Champions League (CFACL) teams; and [2] integrate XGBoost and SHAP to assess the contributions of goal scoring performance, passing and organizing performance, and defending performance to the success of teams in both leagues.

Method

Sample and data

In this study, the data consisted of 1,440 matches from CSL and CFACL during three seasons (from 2017 to 2019). Each division includes 720 matches (CSL matches = 720; CFACL matches = 720). All the data were obtained from Champdas Football Big Data Company (http://www.champdas.com), and a semi-automatic soccer match analysis system was developed by Champdas Soccer Big Data Company. The reliability and validity of the Champdas Football Data collection and analysis system have been validated by other researchers [22]. The dataset covers comprehensive match statistics that allowed for in-depth performance analysis between winning, losing and drawing teams in both divisions.

Variables and procedures

Based on prior studies, the data was initially processed using Excel and subsequently imported into R-Studio for further cleaning and analysis. Twenty-five variables were selected as performance data variables and they were divided into three groups according to the available literature: Variables related to goal scoring, Variables related to passing and organizing, and Variables related to defending (Table 1) [7, 23,24,25]. Afterward, One-way ANOVA was conducted to assess the differences between winning, losing teams and drawing teams. For each indicator, the mean and standard deviation (Mean ± SD) were calculated to determine whether there were statistically significant differences between the three groups (p < 0.05). Additionally, effect sizes were computed to measure the magnitude of the difference between the performance of winning, losing and drawing teams. The effect sizes were also reported as partial eta-squared (ηp²) [23].All variables exhibiting significant differences (p < 0.05) in their association with match outcomes will undergo further analysis. Data was analysed with IBM SPSS Statistics. The significance level was set to p < 0.05 for all statistical analyses.

Table 1 Definitions of the variables for each dimension

Fig. 1

Procedure of research

Machine learning

The XGBoost algorithm, developed by Chen et al. in 2016 [26], is an enhanced version of the Gradient Boosting Decision Tree(GBDT). It improves the boosting algorithm by combining multiple weak classifiers to form a strong classifier, using a series of optimizations such as a novel sparsity-aware algorithm for handling sparse data, and a weighted quantile sketch for approximate tree learning. These improvements enable XGBoost to achieve high prediction accuracy and efficient computation, making it a popular choice for machine learning tasks involving large-scale datasets.

The XGBoost algorithm is an optimized gradient-boosting framework that combines weak classifiers into a strong learner. Its objective function minimizes both prediction error and model complexity through regularization. The model prediction for $\:\widehat{y}$ input $\:{x}_{i}$ is expressed as:

$$\hat y = \mathop \sum \limits_{k = 1}^K {f_k}\left( {{x_i}} \right),{f_k}F\# 1$$

To address inconsistencies in feature importance evaluation challenges, SHAP, a game theoretic approach, was introduced to interpret the model output [27]. SHAP combines global and local interpretations to provide insights into the ‘black box’ models [28]. Equation (2) represents the SHAP formula

$${\phi _j} = \mathop \sum \limits_{S \subseteq {\cal M}\left\{ j \right\}} {{\left| S \right|!\,\left( {M - \left| S \right| - 1} \right)!} \over {M!}}\left[ {f\left( {S \cup \left\{ j \right\}} \right) - f\left( S \right)} \right]\;\# 2$$

This study integrates XGBoost and SHAP to analyze the winning factors in two divisions. The target variable of the model is the match outcomes, and the feature set includes 19 indicators, such as Goals, Clearances, and others. The model parameters are optimized to improve performance through training. Subsequently, the SHAP model is introduced to interpret the results of the XGBoost model.

Construction of group influence formula

This study analyzes 1,102 matches from CSL and CFACL using XGBoost and SHAP models. By formulating Eq. (3), we further investigate the winning factors categorized into goal scoring performance, passing and organizing performance, and defending performance. This approach enables a deeper understanding of the factors influencing outcomes in both levels of competition.

$$\:\begin{array}{c}{S}_{x}=\frac{{SHAP}_{x}}{{SHAP}_{total}}\times\:100\%,x\in\:\left\{G,O,D\right\}\#\left(3\right)\end{array}$$

Here, $\:{S}_{x}\:$represents the SHAP proportion of each category including goal scoring performance ($\:{S}_{G}$), passing and organizing performance ($\:{S}_{O}$), and defending performance($\:{S}_{D}$). The $\:{SHAP}_{x}$ value for category $\:x\:$(where $\:x\:$can be goal scoring performance, passing and organizing performance, or defending performance). And $\:{SHAP}_{total}$ indicates the sum of SHAP values for all categories.

Results

According to the results displayed in Table 2, in terms of Variables related to goal scoring, the winning teams in CSL showed significantly differences in SOTIB (p < 0.001, ES = 0.163), SOT (p < 0.001, ES = 0.136), SIPA (p < 0.001, ES = 0.058), Shots (p < 0.001, ES = 0.033), Penalties (p < 0.001, ES = 0.022), and SOTOB (p < 0.001, ES = 0.014) compared to the drawing teams and losing teams. In CFACL, winning teams also showed significantly differences than drawing teams and losing teams in SOTIB (p < 0.001, ES = 0.161), SOT (p < 0.001, ES = 0.154), SIPA (p < 0.001, ES = 0.054), Shots (p < 0.001, ES = 0.04), SOTOB (p < 0.001, ES = 0.024), Penalties (p < 0.001, ES = 0.015).

With regard to Variables related to passing and organizing, winning teams in CSL exhibited a significantly higher values in ATPA (p < 0.001, ES = 0.024), Passes (p < 0.001, ES = 0.013), as well as in ATP (p < 0.001, ES = 0.011) and Corners (p < 0.001, ES = 0.005) compared to drawing teams and losing teams. In CFACL, the winning teams also surpassed drawing teams and losing teams in Passes (p < 0.001, ES = 0.022), and ATPA (p < 0.001, ES = 0.015), with similar trends observed in Key Passes (p < 0.001, ES = 0.01), ATP (p < 0.001, ES = 0.01), Break Throws (p < 0.001, ES = 0.01). In addition, in CSL, winning teams committed fewer Crosses (p < 0.001, ES = 0.009), Free Kicks (p < 0.001, ES = 0.008), Front Free Kicks ((p < 0.001, ES = 0.005). Similarly, CFACL winning teams committed fewer Crosses (p < 0.001, ES = 0.01), Free Kicks (p < 0.001, ES = 0.016), Front Free Kicks ((p < 0.001, ES = 0.012).

Regarding Variables related to defending, winning teams in CSL displayed significantly higher values in Clearances (p < 0.001, ES = 0.023), Tackles (p < 0.001, ES = 0.013), Red Card (p < 0.001, ES = 0.013), Pass Blocks (p < 0.001, ES = 0.007) and Interceptions (p < 0.001, ES = 0.007) compared to drawing teams and losing teams. Similarly, CFACL winning teams showed better performance in Clearances (p < 0.001, ES = 0.016), Foul (p < 0.001, ES = 0.015), and Shots Blocked (p < 0.001, ES = 0.004) than losing teams. Furthermore, in CFACL, drawing teams received more Red Cards (p < 0.001, ES = 0.003), indicating lower discipline compared to winning teams and losing teams. Similarly, CSL drawing teams showed a higher incidence of Red Cards (p < 0.001, ES = 0.013).

Table 2 Descriptive statistics of match performance indicators among (mean ± SD), One-way ANOVA, P, ES

Figure 2 is a typical SHAP Summary plot to explain the results of XGBoost, which displays the impact of various features on the model’s prediction outcomes. This chart includes two sections, displaying the feature impacts in both CSL and the CFACL. The X-axis (SHAPE Value) represents the extent to which each feature affects the model’s predictions. The Y-axis (feature) represents all the features for each league, ranked by importance. The color represents different values of the feature, with yellow indicating lower feature values and purple indicating higher feature values.

Fig. 2

SHAP Summary Plot. Abbreviations: 1st league (CSL), Chinese Super Football League; 2nd league (CFACL), Chinese Football Association China League; ATPA, attacking third successful pass accuracy; SOTIB, shots on target inside box; BT, Breakthrough; SOT, shots on target; SOTOB, shots in target outside box; SIPA, shots inside penalty area

In the CSL, SOTIB (v = 0.272) showed the highest SHAP value, identifying them as the primary predictor for match outcomes. Other features such as Clearances (v = 0.145), Key Passes(v = 0.096), and ATPA (v = 0.073) also have important effects on match outcomes. Clearances, in particular, indicate a strong defensive contribution, where increased clearance frequency shifts the model’s predictions positively. Meanwhile, Cross (v = 0.069), Red Card (v = 0.031), Foul (v = 0.027), and PSR(v = 0.027) exhibit highly levels of importance, their SHAP value on the match outcomes vary. For instance, PSR exert a stronger positive influence on the outcome compared to Red Card. Moreover, while Tackles (v = 0.012), Free Kicks (v = 0.012), Front Free Kicks (v = 0.01), Shots Blocks (v = 0.008), Yellow Card (v = 0.007), and SOTOB (v = 0.007) rank lower on the Y-axis in terms of importance, the significance of the research is also important. Notably, for disciplinary metrics, Red Card and Yellow Card have opposing SHAP values, suggesting that these variables may have contrasting effects on match outcomes under different conditions. Similarly, in the CFACL, SOTOB (v = 0.174) remain the most influential feature affecting match outcomes. Additionally, defensive performance metrics such as Clearances (v = 0.106), Foul (v = 0.054), and as Interceptions (v = 0.013) have a more significant impact compared to their rankings in the CSL, suggesting that defense plays a crucial role in this secondary league. Specifically, the greater influence of Foul in the CFACL, compared to the CSL, indicates that Foul may be a more critical factor in determining match winners in this league.

Finally, this study, utilizing SHAP values and the group influence model, reveals that the values of $\:{S}_{G}$、$\:{S}_{p}$ and $\:{S}_{D}$ in the context of CSL are 37.854%, 35.425%, and 26.721%, In contrast, within CFACL, the corresponding values of $\:{S}_{G}$, $\:{S}_{p}$ and $\:{S}_{D}$ are 38.934%, 32.377% and 32.377%, respectively. Figure 3 provides a clearer representation of the results.

Fig. 3

Group proportion. Abbreviations: CSL, Chinese Super Football League; CFACL, Chinese Football Association China League

Discussion

The purpose of this study was to explore the key factors that differentiate winning teams between the CSL and CFACL. While previous research has examined a range of discriminants related to match success [8, 29], few studies have focused on the comparative analysis of winning factors between different divisions, particularly within the context of China’s two-tier league. Utilizing XGBoost and SHAP, we uncovered significant differences in the impact of various features on match outcome predictions across the two divisions leagues.

This research explores the perspective of indicator groups. Through the data obtained from the constructed indicator group model based on SHAP and XGBoost, it is found that, first of all, the score indicators have a significant impact on match outcomes in both CSL ($\:{S}_{G}$=37.854%) and CFACL ($\:{S}_{G}$=38.934%). This finding is consistent with previous research [12, 30,31,32], highlighting that scoring indicators remain the most critical factors for winning across different soccer leagues. Additionally, although the influence of other indicators on match results is relatively minor, a comparison between the two leagues shows that defensive indicators in CFACL ($\:{S}_{D}$=32.377%) perform better than those in CSL ($\:{S}_{D}$=26.721%). This may be due to several factors, including the strength disparities in lower leagues that prompt coaches to emphasize defensive strategies against stronger opponents [33], the slower match pace that highlights the importance of defense [34], and the differing playing styles in lower-tier competitions where teams, with fewer top players, rely more on coordinated team defense to secure victories [35].

Goal scoring performance

Additionally, previous studies have further identified that the most critical variables influencing match victories include ball possession, shots on target, and successful passes [36]. A comparison of the top ten key performance indicators impacting match outcomes in this study supports this perspective. The results reveal that metrics such as SOT and ATPA rank highly in both CSL and CFACL matches. However, it is noteworthy that in the CSL, the importance of SIPA’s ranking is moderate, whereas in the CFACL, it ranks near the bottom. This discrepancy suggests that, although variables such as total shots play significant roles in both leagues, for the CSL, the number of shots within the penalty area appears to be more crucial for securing victory in matches. This discrepancy may stem from variations in shot quality between the leagues [37], as CSL players generally exhibit higher shot accuracy, leading to more goals from shots inside the penalty area, which consequently has a stronger impact on match outcomes.

Additionally, the study found that SOTIB ranks first in importance among all variables related to goal scoring in both the CSL and CFACL. Research on the German First Bundesliga suggests that shot quality is more important than shot quantity for winning matches [38]. This research also supports this conclusion, indicating that whether in the CSL or CFACL, players need to have precise shots within the penalty area rather than merely increasing the number of shots. This highlights that in top-tier leagues, match outcomes are more influenced by the quality of shots, as shots from different positions are affected by factors such as technical and tactical performance, defensive organization, and coaching strategies [39]. As the level of competition increases in top-tier leagues, the higher standard of defensive organization makes shots within the box more critical and threatening. This finding suggests a potential need for CFACL teams to focus on enhancing shot quality in their training programs. This observation aligns with studies on the first La Liga and the FIFA World Cup [12, 32], which also emphasize the significant impact of shots inside the box on match victories.

Passing and organizing performance

The studies of the UEFA Champions League [40] and the Spanish Professional Football League [1] have found that variables such as ball possession, passes, successful passes, and crosses have a significant impact on match outcomes. Although this research only selected two offensive-related variables, ATPA and Crosses, when identifying data relevant to match results, the SHAP analysis revealed that both variables had high importance in CSL and CFACL. Further analysis of the 2014 FIFA World Cup in Brazil [32]emphasized that, for close games in the same leagues, long passes and crosses—aerial duels—are not always effective. The results of this research also indicate that, although crosses ranked highly in terms of their impact on match outcomes in both the CSL and CFACL, their effect was primarily negative, especially in CSL matches. Therefore, when developing match strategies, coaches should be more cautious in implementing crossing tactics, taking into account the opponent and the league level. This indicates that for CFACL teams, if they are unable to achieve higher-quality passing, they might still be able to organize effective attacks through set-pieces such as free kicks.

Defending performance

Defensive performance plays a decisive role in soccer matches. This study also found some points of interest regarding the defensive metrics of the two-tier leagues. First, in the CSL, the defensive feature Clearances ranked second only to SOTIB in terms of variable importance. This result is consistent with previous research [41], indicating that clearances, as a defensive action, play a pivotal role in maintaining defensive stability and mitigating high-intensity offensive plays in top-tier competitions. Meanwhile, in the CFACL, clearances were equally critical, ranking first among all defensive metrics. This suggests that despite differences in team strength, technical and tactical performance, and match context between leagues of different levels [37], consistent Clearance continues to exert a significant influence on match outcomes in high-level competitions. Furthermore, the research results show that the importance of Clearances ranks below SOTIB in both levels of the league, indirectly indicating that SOTIB consistently have a direct impact on match outcomes across all league levels. Clearances serve as a defensive response to opponents’ shots or attacks, primarily aimed at preventing goals through technical handling [31, 32]. While defensive actions can effectively avert crises, they do not directly influence the score. By contrast, SOTIB is a direct determinant of match outcomes. Representing the conversion of offensive plays such as shooting into tangible scores, which are immediately quantified as match points and ultimately exert a decisive influence on the outcome [42]. Consequently, the importance of the clearance metric ranks consistently after SOTIB across both league levels. Additionally, by comparing defensive metrics between the CSL and CFACL, the study found that although the overall importance of defensive metrics is relatively lower in CFACL, other defensive metrics such as Foul and Interceptions have a more significant impact on match outcomes than in CSL games, with the importance of Foul being particularly higher than that of corresponding metrics in CSL matches. This finding aligns with previous studies that have explored the impact of Foul on match outcomes [12, 32, 42]. For example, research on the first and second divisions of LaLiga [12]has similarly indicated that foul play a crucial role in the second divisions of LaLiga. This phenomenon is influenced not only by the physical performance of players and the tactical strategies employed by coaches but also by the match context [12]. Therefore, for CFACL teams, if they are unable to organize an effective defense in a timely manner, employing tactical fouls in appropriate areas might be a crucial factor in reducing the likelihood of losing. As the level of competition increases, referees tend to adopt stricter standards in the assessment of fouls, with a more diversified foul detection system, thereby reducing the frequency of fouls. Supporting this, studies on the 2014 FIFA World Cup have shown that compared to 2006 and 2010, referees applied stricter foul enforcement, leading to a noticeable decrease in serious fouls and injuries [43].

This study acknowledges several limitations. For example, contextual variables such as home vs. away status, team strength, opponent quality, and match dynamics were not considered. Furthermore, the analysis lacked control for covariates. Future research should incorporate these factors to enable a more nuanced exploration of determinants of success across professional football leagues at different competitive tiers.

Conclusion

This study explores the factors influencing match outcomes in the CSL and CFACL, utilizing XGBoost and SHAP to reveal the varying importance of these discriminants across two divisions. The results show that while Shots On Target Inside Box are consistently the key determinant of match outcomes, other factors exhibit significant differences between the two leagues, particularly Penalties. Defensively, Clearances are crucial in both leagues, and Shots Blocks has a more pronounced impact in the CFACL, potentially due to contextual differences. Therefore, in daily training and matches, coaches and players should prioritize improving shot quality and accuracy within the penalty area over merely increasing shot frequency. For CFACL teams, capitalizing on set-piece opportunities (e.g., free kicks and corners) through dedicated training is critical for securing victories. Additionally, adopting strategic defensive measures—such as tactical fouling in key zones and implementing shot-limiting tactics—can significantly enhance the likelihood of success for CFACL teams. These findings offer valuable insights for coaches in tailoring strategies to different league contexts and emphasize the need for tactical adjustments based on league characteristics and opponent conditions. Future research could further explore the influence of home and away factors, and individual player performances.