Olympic rankings based on objective weighting schemes

Abstract

In this paper, we propose an objective principal components weighting scheme for all-time Winter Olympic gold, silver and bronze medals based solely on the number of medals won. Our results suggest that the approximately equal weights be assigned (or the total medal counts be used regardless of color) if all of the three medal types are retained for ranking purposes. When the proposed methodology is tested against five alternative weighting schemes that have been suggested in the literature using the results for the 2010 Vancouver Winter Olympics, we find a significant agreement in the country rankings. Furthermore, our implementation of principal components variable reduction strategy results in the identification of silver as the best representative medal count for parsimonious Winter Olympics rankings.

1. Introduction

One of the most widely monitored aspects of every summer or winter Olympics is the overall ranking of countries on the basis of the achievements of their athletes. There are two main reasons why the overall ranking of countries attracts considerable interest despite the International Olympic Committee’s historical emphasis on individual competition. First, it provides a measurable yardstick for determining relative Olympic success and the factors that determine such success. Second, as pointed out by Flegl and Andrade [7], Hoffmann et al. [9], Humphreys et al. [10], and Van Hilvoorde et al. [17], among others, the rankings have sociological benefits including promoting the spirit of national pride.

The issue of how best to rank countries on the basis of Olympic Games performance is, however, controversial. In general, there are two main approaches to the ranking of countries. The first approach to the Olympic rankings, which seems popular in academic circles as evidenced by the relatively large number of recent papers, is the Olympic production function approach, for which the various countries participating in the Olympics are viewed as ‘factories’ producing gold, silver and bronze medals. The inputs into ‘medal production’ may include wealth, population and other factors (e.g. host country benefits, culture, politics, etc.) which could affect the potential to win medals. Using a linear programming technique called Data Envelopment Analysis (DEA), countries are ranked on the basis of their so-called technical efficiency scores that reflect how efficiently they have used their inputs to ‘produce’ their medals hauls. A partial list of papers that have followed the DEA approach include Ball [1], Lozano et al. [13], Lins et al. [12], Bernard and Bussee [2], Churilov and Flitman [4], Flegl and Andrade [7], Li et al. [11], Wu et al. [19] and Saaty [14], among many others. Clearly, an important characteristic of the production function approach to Olympic rankings is that country specific advantages with respect to Olympic inputs are taken into account in the rankings. The second approach to the Olympic rankings, which seems popular among the media and the general public, is the ranking of countries based solely on the number of medals won. The popularity of this approach stems from the simplicity of many of the associated ranking rules. With respect to the second approach, the lexicographic or gold first rule, which first orders countries according to the number of gold medals won and breaks ties with the number of silver medals won then by the number of bronze medals won, seems to be the most popular. A drawback of the lexicographic approach is that ambiguous rankings might arise when comparing two countries one of which has won more medals of one color but fewer medals of another color than the other country. Another popular ranking rule is based on the total number of medals won regardless of color. This practice, which amounts to assigning equal weights to gold, silver and bronze medals, is problematic since a gold medal is undervalued and a bronze medal is overvalued under this scheme. Bian [3] rationalizes the total medals approach to Olympic rankings on the grounds that the gap between first, second and third medaling positions in elite competition such as the Olympics is usually so minute that luck plays an important role in determining who ends up in each of the top medal positions. An even simpler ranking rule is the gold only rule for which countries are ranked solely on the number of gold medals won. The problem with the gold only rule is that zero weights are assigned to both silver and bronze medals.

A number of recent academic papers have proposed alternative rankings of countries based solely on the number of medal won. For example, Sitarz [15] proposed two alternative ranking methods, namely, the weighted mean value approach and the volume-based sensitivity approach. More recently, Sitarz [16] proposed the so-called in-center approach which entails optimization with certain constraints imposed on the medal characteristics. The three aforementioned ranking methods proposed by Sitarz [15,16] suffer from two limitations. First, subjective judgements are made along the way. Second, the methods cannot be conveniently implemented using widely-available statistical software packages.

Generally speaking, ranking countries based solely on the number of medals won entails devising weighting schemes for aggregating gold, silver and bronze medal hauls in order to come up with an overall ranking of countries. In this regard, there is no doubt that the issue of appropriate weights for Olympic gold, silver and bronze requires very careful consideration. A common problem with the aforementioned ranking rules is that the weights attached to gold, silver and bronze medals are chosen arbitrarily. To circumvent this problem, it makes sense to devise objective weighting schemes for Olympic gold, silver and bronze medals by taking a closer look at the internal structures of the data.

A comprehensive review of the literature on the ranking of countries by Olympic medal counts reveals the widespread usage of linear programming as the preferred analytical tool, resulting in a disproportionately large number of papers being published in operations research related journals. For the sake of thoroughness, it makes sense to approach the rankings from other perspectives as well.

An inspection of the medal counts for several recent winter Olympics reveals strong positive correlations among the gold, silver and bronze medal counts. In other words, countries that do well in the gold medal counts generally do well on silver and bronze medal counts as well. We believe that these strong positive correlations should be exploited for purposes of devising objective weighting schemes for Olympic medal counts. This exploitation could be achieved using principal components methodology. We are not aware of any published papers that have so far exploited the high correlation among the medal counts to construct a composite medal count variable which can be used for ranking purposes.

The purpose of this paper is to fill in the gap by exploiting the high medal count correlations to devise objective weighting schemes for gold, silver and bronze medals based solely on the number of medals won. To this end, we employ principal components approach to develop objective weighting schemes for gold, silver and bronze medals won at all-time Winter Olympics. We believe that the objectivity entailed in the principal components approach to Olympic ranking constitutes an important advantage of the approach over the alternative approaches mentioned above. Another important advantage of the principal components approach to ranking is that it is easy to implement using widely-available statistical software packages. Furthermore, the intuition that underlies our principal components approach is quite simple.

The format of the rest of the paper is as follows. In Section 2 we provide a brief description of the principal components methodology that forms the foundation for our rankings. In Section 3, the proposed method is applied to all-time Winter Olympic gold, silver and bronze medals and the empirical results implying an appropriate weighting scheme are presented. A comparison of our principal components rankings with five other rankings based on the results for the 2010 Vancouver Winter Olympics is made in Section 4. In the same section, we also report the results of Kendall’s coefficient of concordance test of the degree of agreement among all the six ranking methods we compared and those of applying a principal components variable reduction method to identify the best representative medal type. The final section contains some concluding remarks.

2. Methodology and data

2.1. Principal components analysis

Principal components analysis (PCA) is a popular statistical tool for reducing a set of highly correlated variables, which is certainly the case with recent Olympic medal count data as mentioned above, to a few new principal component variables (PCVs) that account for much of the variation in the original set of variables. For a set of three variables representing country gold, silver and bronze medal counts a maximum of three PCVs can be constructed, each of which is a linear combination of the original variables. Each PCV is a composite variable with characteristic vectors of the 3 × 3 correlation matrix of the variables representing the 3 medal counts as weights. Should the variables be highly correlated then only one PCV, namely, the first PCV which explains the largest percentage of variation in the three original variables, would suffice. Under these circumstances, the country Olympic rankings would be based solely on the first PCV. Another relevant aspect of PCA as far as the present research is concerned is variable reduction, which entails the identification of a few variables that best represent a set of highly correlated variables. As will be seen below, the variable reduction approach has the advantage of providing parsimonious and cost-effective rankings.

For the sake of brevity, the technical details surrounding PCA, including variable reduction, are not provided in this paper but can be found in many multivariate statistics textbooks, including Dunteman [5]. However, for the sake of facilitating exposition highlights of the actual empirical implementation in this paper are provided below.

2.2. Medal data

The all-time Winter Olympics medal table, which is based on information provided by the International Olympic Committee (IOC), is available on the Wikipedia website [18]. The data consists of the number of medal counts for the three variables (i.e. gold, silver and bronze) for the 46 countries that won at least one medal at all-time Winter Olympics from 1896 to 2018. Since the number of Winter Olympic games participated in by each country during this time period varies, we use the average gold, silver and bronze medal counts per game for each country for the purpose of fair comparison (by dividing all-time medal counts for each country by the number of Winter Olympic games that the country has participated in). With respect to the all-time winter medal counts, the break-ups of some countries (e.g. Czechoslovakia into Czech Republic and Slovakia in 1993 and Yugoslavia into 7 different countries in 1992) or country mergers (e.g. East Germany and West Germany into Germany in 1990) were handled by treating the merged and broken-down individual countries involved as separate countries for purposes of calculating the average medal counts taking into account the number of Olympics each country participated in.

Table 1 reports the pairwise correlations among the three variables (i.e. gold, silver and bronze medal counts). It is apparent from the table that the correlations are strongly positive, which provides justification for constructing the principal components rankings.

Table 1. All-time winter medal count correlation matrix.

	Gold	Silver	Bronze
Gold	1.000	0.931	0.859
Silver	0.931	1.000	0.948
Bronze	0.859	0.948	1.000

3. Analysis

3.1. Analysis for all-time Winter Olympics

We employ PCA to propose an objective principal components weighting scheme for all-time Winter Olympics. Before applying PCA, it is important to assess the overall significance of the correlation matrix of the three variables (i.e. Gold, Silver and Bronze) using Bartlett’s sphericity test (see, for example, Field [6]). It is also important to test for factorability of the three variables collectively and individually using the Kaiser-Meyer-Olkin Measure of Sampling Adequacy (MSA) (see, for example, Hair et al. [8]). Bartlett’s test, which indicates the presence of nonzero correlations, is significant at the 0.01 level (χ²= 187.13; p-value = 0.000), indicating significant correlations. The overall MSA value, which looks both the correlations between variables and their patterns, is 0.680, which falls in the acceptable range (i.e. greater than 0.5). The individual MSA values are 0.744 for Gold, 0.608 for Silver, and 0.711 for Bronze, all of which lie in the acceptable range.

Table 2 shows the core principal components results, including the associated characteristic vectors and the proportion of variation in the three medal counts explained by each principal component. It is apparent from the table that the first principal component accounts for more than 94% of the variation of the three variables representing the three medal counts and the first 2 principal components explain almost 99% of the same variation. There is no doubt that the first principal component explains a very large percent of the variation in the all-time Winter Olympic medal counts. Hence, it makes sense to rank countries on the basis of the first principal component composite medal count variable.

Table 2. Principal Components Analysis results using the all-time winter medal count correlation matrix.

	First	Second	Third
Characteristic vector
Gold	0.339	1.970	1.948
Silver	0.350	−0.185	−4.459
Bronze	0.341	−1.768	2.638
Characteristic root:	2.825	0.142	0.033
%variance explained individually:	94.177	4.735	1.088
%variance explained cumulatively:	94.177	98.912	100.00

The magnitudes of the characteristic vector associated with the first principal component, reported in Table 2, suggest that the first principal component composite medal count variable should be computed with weights of 0.339, 0.350 and 0.341 for gold, silver and bronze, respectively (i.e. the highest weight should be assigned to silver and the lowest weight to gold). The process of normalizing the weights such that they add up to one resulted in the first principal component (FPC) composite medal count, $\text{FP}{\text{C}}_i$, which is used for ranking purposes, being computed as indicated in Equation (1) below

$$\text{FP}{\text{C}}_i=0.3291\cdot Gol{d}_i+0.3398\cdot Silve{r}_i+0.3311\cdot Bronz{e}_i$$ (1)

where the subscript i denotes the i-th country and Gold, Silver and Bronze denote the gold, silver and bronze medal count, respectively.

It may seem odd that the principal components rankings assign the highest weight to silver and the lowest weight to gold. However, as noted in Section 2, the purpose of PCA is to create a composite medal count variable that maximizes variation in all the three medal counts based purely on the analyses of the internal structures of the data. We note that a higher weight assigned to silver than gold does not mean that the former is more important than the latter in terms of the performance level. It simply means that silver has a higher level of information explaining the overall picture than gold. That being said, we also note that the principal components weights indicated in Equation (1) are roughly equal, which is somewhat consistent with the assignments of equal weights to the three medal categories. Hence, we would expect our principal components rankings using Equation (1) to be close to the ranking based on total medal counts regardless of the medal color. As is already mentioned above, Bian [3] provides justification for weighting the medals equally in the case of elite competition such as the Olympics.

3.2. Analysis for the 2010 Vancouver Winter Olympics

In order to compare the results of our approach to those of other approaches suggested in the literature, we employ PCA to rank countries on the basis of their achievements at the 2010 Vancouver Winter Olympics. The choice of the 2010 Vancouver Winter Olympics was dictated by the availability of the results of other relevant studies in the literature for direct comparison purposes. Focusing on the Vancouver Winter Olympics medal counts allows us to compare our principal components rankings with several other published rankings based on the same data, including the two methods proposed by Sitarz [15] and the in-center rankings proposed by Sitarz [16].

At the 2010 Vancouver Winter Olympics, 26 countries obtained at least one of the gold, silver and bronze medals. Having checked and met all the assumptions underlying the principal components method (as mentioned in Section 3.1), our analysis resulted in the following first principal component composite medal count $\text{FP}{\text{C}}_i$, which is used for ranking purposes, being computed as indicated in Equation (2) below

$$\text{FP}{\text{C}}_i=0.3219\cdot Gol{d}_i+0.3445\cdot Silve{r}_i+0.3336\cdot Bronz{e}_i$$ (2)

where the subscript i denotes the i-th country and Gold, Silver and Bronze denote the gold, silver and bronze medal count, respectively. It is noteworthy that this objective weighting scheme is very similar to the one in Equation (1) and resulted in the same strategy that the highest principal components weight be assigned to silver and the lowest weight be assigned to gold.

4. Results

4.1. Principal components rankings

Applying Equation (2) weights resulted in the principal components rankings which are reported in the fifth column of Table 3. For the sake of comparisons, we also report the rankings based on five other popular methods in separate columns of Table 3.

Table 3. Comparing different ranking methods for the 2010 Vancouver Winter Olympics.

	Medal Count			Rankings**
Country	Gold	Silver	Bronze	FPC	IC	GF	TOTAL	MEAN	SA
Canada	14	7	5	3	1	1	3	2	2
Germany	10	13	7	2	3	2	2	3	3
United States	9	15	13	1	2	3	1	1	1
Norway	9	8	6	4	4	4	4	4	4
South Korea	6	6	2	7	5	5	7	5	5
Switzerland	6	0	3	11	7	6	11	9	8
China	5	2	4	9	8	7	8	7	9
Sweden	5	2	4	9	8	7	8	7	9
Austria	4	6	6	5	6	9	5	6	6
Netherlands	4	1	3	12	11	10	12	11	11
Russia	3	5	7	6	10	11	6	9	7
France	2	3	6	8	12	12	8	12	12
Australia	2	1	0	22	15	13	18	15	15
Czech Republic*	2	0	4	14	13	14	13	13	13
Poland	1	3	2	13	14	15	13	13	14
Italy	1	1	3	17	16	16	15	16	16
Belarus	1	1	1	20	17	17	18	18	18
Slovakia*	1	1	1	20	17	17	18	18	18
Great Britain	1	0	0	26	20	19	24	23	23
Japan	0	3	2	15	19	20	15	17	17
Croatia*	0	2	1	18	22	21	18	21	21
Slovenia*	0	2	1	18	22	21	18	21	21
Latvia	0	2	0	23	24	23	23	24	24
Finland	0	1	4	16	21	24	15	20	20
Estonia	0	1	0	24	25	25	24	25	25
Kazakhstan	0	1	0	24	25	25	24	25	25

*Czechoslovakia became two independent states in 1993, Czech Republic and Slovakia, which competed in the 2010 Vancouver Winter Olympics as two independent countries. Yugoslavia was dissolved in 1992 and broke off into seven different countries, including Croatia and Slovenia, which competed in the 2010 Vancouver Winter Olympics as independent countries.

**FPC is our principal components ranking as developed in this paper using Equation (2); IC is the ranking based on the so-called in-center scores as developed by Sitarz [16]; GF is the lexicographic ranking based on the gold first principle; TOTAL is the ranking based on the total number of medals won; MEAN is the ranking of weighted mean value as proposed by Sitarz [15]; and SA is the ranking based on volume-based sensitivity analysis as proposed by Sitarz [15] as well.

Several other features of Table 3 are worthy of mention. First, our principal components rankings are closer to the rankings based on the total number of medals won regardless of color, which is not surprising given that the values of the resulting characteristic vector associated with the first principal component also turn out to be roughly equal (same situation as all-time medal counts which has already been alluded to above). Second, Norway is consistently ranked fourth across all six rankings, whereas Canada, Germany and USA alternate in the top 3 rankings depending on the ranking method considered. Third, for 15 of the 24 medal winning countries the rankings across the 6 categories are within 4 spots. Fourth, Australia and Great Britain would suffer the biggest drop in the rankings if our principal components approach was used.

To provide some insights into the magnitudes of the pairwise correlations among the six alternative rankings (including ours) reported in Table 3, we also report the correlation matrix of the six rankings in Table 4. The high positive pairwise correlations among the rankings are apparent from the table, with the lowest pairwise correlation being 0.871 (between gold first ranking and our principal components ranking) and the highest pairwise correlation being 0.995 (between the weighted mean value ranking and the volume based sensitivity ranking both of which were proposed by Sitarz [15]). A closer scrutiny of our rankings reveals the lowest pairwise correlation of 0.871 with the gold first ranking, as already alluded to above, and the highest pairwise correlation of 0.992 with total ranking. As noted above, the fact that the values of the characteristic vector associated with our first principal components ranking turn out to be roughly equal suggests a high degree of agreement between our rankings and the rankings based on total medal counts.

Table 4. Correlation analysis between different ranking methods.

	FPC	IC	GF	TOTAL	MEAN	SA
FPC	1
IC	0.914259	1
GF	0.870867	0.98736	1
TOTAL	0.991854	0.93978	0.899867	1
MEAN	0.946116	0.990245	0.971354	0.967461	1
SA	0.949364	0.98874	0.966971	0.968705	0.995151	1

Given that there are six alternative rankings to choose from in Table 3, it also makes sense to conduct formal tests for the degree of agreement among all the six rankings instead of pairwise. This is done in the next section.

4.2. Are the agreements in the rankings due to chance?

Our goal here is to assess the degree of agreement among the six alternative rankings whose results are reported in Table 3. To this end, we compute the usual Kendall’s coefficient of concordance (KCC) the results of which were used to test the null hypothesis that the agreements in the rankings are due to chance against the alternative that the agreements are systematic. KCC lies between 0 and 1 with a higher value indicating a stronger degree of agreement. The computed value of KCC turns out to be 0.9614, which is very close to 1, with a corresponding p-value of 0.000. On the basis of these results, we reject the null hypothesis at the conventional 5% level of significance and conclude that there is a strong agreement in the rankings. Clearly, these results suggest that the choice of ranking method may not matter that much in a statistical sense at least in the case of the 2010 Vancouver Winter Olympics.

4.3. Which of the three types of medal best represents Olympics rankings?

As mentioned above, one of the nice aspects of PCA is that it can be used to determine the variable(s) that best represent a set of highly correlated variables. This is the problem of variable reduction which is commonly discussed in multivariate statistics textbooks including Dunteman [5]. There are several principal components variable reduction strategies. One popular variable reduction strategy entails first choosing the variable which has the highest correlation with the first principal component variable. Of the remaining variables, we then choose the variable with the highest correlation with the second principal component as the second best representative variable and keep on going until the desired number of representative variables is selected. With respect to the number of representative variables to be retained a common rule of thumb is for the number of retained variables to be equal to the number of characteristic roots of the correlation matrix that are greater than 0.7.

As indicated in Table 2, there is only one characteristic root which is greater than 0.7, the threshold suggested in the aforementioned literature for determining the number of variables to be retained in PCA variable reduction. These results suggest that only one of the three variables be retained for purposes of computing a Simpler FPC (SFPC). If we apply one variable retention strategy to the all-time Winter Olympics medal counts, therefore, we find that only the silver medal count be retained since it has the largest value in the characteristic vector associated with the first principal component. Application of the one variable retention strategy also results in the identification of silver as the best representative medal count for the 2010 Vancouver Winter Olympics. As mentioned in Section 3.1, however, the choice of silver in such a case does not mean that silver has a higher value than gold in terms of athletic performance. It simply suggests that silver has a higher level of information explaining the overall picture than the other two types of medal based purely on the analyses of the internal structures of the data.

5. Concluding remarks

Our results, based on the principal component approach to Olympic rankings, suggest that approximately equal weights be assigned to the three types of all-time Winter Olympics medals. If the goal is to settle for one medal type for ranking purposes then silver could be better than gold for the all-time Winter Olympics data, as well as in the case of the 2010 Vancouver Winter Olympics, as silver turned out to be the variable which provides the highest level of representation. Whether the choice of silver over gold is also appropriate in the case of other Olympic games (e.g. Summer Games) is an open question which could be explored in future research.

Even though we have developed objective weighting schemes for gold, silver and bronze medals won at all-time Winter Olympics, we acknowledge that the principal components approach to Olympic rankings proposed in this paper, like every other ranking methods proposed to date, is not a perfect way to compare countries’ performances. Thus, it is probably a good idea to publish several alternative rankings from which the various stakeholders can choose their preferred rankings. Better still the overall ranking could be based on the average of all available rankings. The high level of agreement in the rankings uncovered in this paper, as evidenced by the KCC results, suggests that we should not be too concerned about the choice of ranking method since each method has its pros and cons.

One limitation of the principal components approach is that information on country-specific advantages with respect to Olympic medal success is not utilized. In other words, our principal components ranking relies strictly on the information provided by the internal structures of the data with absolutely no consideration of any other criteria. Another limitation of the principal components approach may stem from its surprising results calling for the assignment of equal weights to the three types of medals because a gold medal is perceived by the public to be more valuable than a silver medal, which is perceived to be more valuable than a bronze medal. This limitation may call for different methodologies to be used for analyzing the relative importance of the three medal types in future works.

Finally, as pointed out by a referee, the ranking of countries based solely on their medal hauls is less favorable to smaller countries with fewer resources compared to the rankings based on medal hauls but taking into account each country’s advantages with respect to Olympic inputs. In this regard, one way of making the rankings more favorable to smaller countries with fewer resources is by awarding more medals given that for some smaller countries finishing in, say, the top 5 might be seen as a success. We believe that the idea of increasing the number of medal positions, which is a radical departure from the norm and has received some support in the literature (e.g. Flegl and Andrade [7]), makes sense. Hence, in terms of future research, it would be interesting to examine the impact of increasing the number of medal positions on past and future Olympic rankings subject to data availability. Another interesting area of future research is the use of regression methods to investigate the socioeconomic and demographic determinants of Olympic success as measured by the principal components composite medal counts as proposed in this paper.

Disclosure statement

No potential conflict of interest was reported by the author(s).

ORCID

Danny I. Cho http://orcid.org/0000-0003-2751-8232