Flawed global ranking

Discussion in 'General Hockey Chit-Chat' started by Ernst, Nov 6, 2017.

  1. sanabas

    sanabas FHF All Time Great

    Joined:
    Feb 21, 2007
    Messages:
    2,821
    Likes Received:
    1,089
    Yes, you are. You've said it again in this very post:


    Right. So the exact thing you say makes the current system unusable (EUR teams face a harder challenge but don't get higher rewards) also applies to your proposed system. Except in your proposed system the consequences of those discrepancies will be even bigger. If Ireland just fail to make the EUR semis, they still get most of the points, 500 or 550 vs 600. They miss out on 5-10% of their total. But if they just fail to make the olympics, they get 0 instead of 200, miss out on 40% of their total. NZ men only got to go when SA pulled out, they'd have doubled their total points as a result. How's that fair?

    And so how do you rank teams outside the top 15, the ones who don't go to world cups, don't go to the olympics? You're going to use the HWL and only the HWL? The one that starts off with rounds that are area-based and so not a level playing field?

    That's the same NZ men that finished 2nd in the 2013 HWL, where BEL, GER, ENG finished 5th, 7th, 3rd? The same NZ that had a draw with NED at the last HWC? That finished ahead of GB in Rio?
    The same NZ women that finished 2nd in the 2015 HWL, where BEL, GER, ENG finished 13th, 3rd, 7th (as GB)? The same NZ that beat AUS, drew with NED in the semis, finished ahead of BEL at the current HWL semis?

    Even if you concede the merit of winning the various continental tournaments aren't equal, there is a practical reason for the points to be equal as well.

    But if it's so unfair that it must be replaced, surely you'd want to replace it with something that is fair? But your proposed system isn't, by your own standards. And your proposed system introduces some added practical issues, which you refuse to address, other than to say you don't care about the flaw that'd put the olympic host well inside the top 20 overall.

    I don't care about the current system's 'flaw' of EUR champs being harder to win than OCE champs, because it doesn't skew the rankings, and the results are fair. Every top team gets 500+ points for turning up to them, every top team has the chance to increase that to 750 if they win.

    You haven't addressed the existing ones. You say olympic qualification is fair, because everyone has a chance to do so. Even though some teams have to beat nobody (Brazil), some teams have to beat hardly anybody (South Africa), some teams have a very tough path to try and qualify (Ireland). Well, every team has a chance to win a continental championship, too. Every team outside the top 20 has a chance to win their continental qualifier and go to HWL rd 2.

    You won't tell us which rankings produced by the current system are clearly wrong, which team or teams are ranked much higher or lower than they should be.
     
  2. Nij

    Nij FHF All Time Great

    Joined:
    May 22, 2012
    Messages:
    1,745
    Likes Received:
    659
    Hardly complicated. You get some calculator in Excel, plug in the current ranking score, plug in the result of the match, and it gives you the new ranking score.

    At the end of the tournament you plug in all the ranking scores, plug in the results for the tournament, and it gives you the final ranking score from that tournament.

    Individual matches are even easier: you only need the first one. With this in place, world rankings could be updated literally at the final whistle of every single match.

    As for the supposed problem with domestic availability, I don't see it. You're not adding any new matches or tournaments. One-offs and tri-series and quadrangulars and invitationals already happen every month. The only difference is that now they could be counted for more than caps and bragging rights.
     
  3. sanabas

    sanabas FHF All Time Great

    Joined:
    Feb 21, 2007
    Messages:
    2,821
    Likes Received:
    1,089
    Easy to calculate using a tool and complicated to understand the underlying formula aren't mutually exclusive. ;)
     
  4. Nij

    Nij FHF All Time Great

    Joined:
    May 22, 2012
    Messages:
    1,745
    Likes Received:
    659
    To be fair, the formulae for Elo ratings are not that much worse than the formulae for taxes, and most people can follow those.
     
  5. Ernst

    Ernst FHF Starter

    Joined:
    Oct 13, 2017
    Messages:
    22
    Likes Received:
    3
    Location:
    Antwerpen, Belgium
    No I'm not.... ;)

    I never claimed the system I proposed was the ideal system. I clearly state in my blog post the way of allocating points is very much up for debate. The only thing I consider not to be acceptable is earning points from events where others are excluded. So no invitationals such as the Champions Trophy or the future Pro League, nor the continentals.

    Yes those teams that do not make the WC or the Olympics would only be earning points from their participation in the HWL, because that is the only event all countries have access to. We could very well think of a system where those countries that earn their spot in the WC or Olympics would not get their HWL points but only those from WC or Olympics to avoid making the gap seem extra big.

    Sure I will. From my rankings it was already clear in 2013 ARG and BEL where among the top 5 countries instead of at the bottom of the top 10 according to the FIH. My rankings shows years earlier the downfall of countries like KOR and PAK who were highly overrated in the FIH ranking.

    Anyway... like I said the way to calculate the points is very much up for debate for me... But the fundamentals should be you can only get points for events where all have the possibility to qualify for ;)
     
  6. nerd_is_the_word

    nerd_is_the_word FHF All Time Great

    Joined:
    Apr 25, 2006
    Messages:
    2,234
    Likes Received:
    96
    Location:
    Queensland, AUS
    How was argentina in the top 5 countries? They finished 10th are the world cup and 8th at HWL? If they were in the top 5 that is a major flaw in the system.

    Can you give me an idea of when korea was highly overrated? Because i cant see when that happened?

    Those fundamentals dont work, we can't have a system where only one tournament per year is considered for ranking points. It will result in many more errors in world rankings than it currently solves.
     
  7. Nij

    Nij FHF All Time Great

    Joined:
    May 22, 2012
    Messages:
    1,745
    Likes Received:
    659
    So, using a modified Elo that accounts for the actual goals scored, the margin of victory, and the proportion of goals, I ran the upcoming HWL17 Finals.
    A geometric descending score starting at 1000 and adjusted to give a mean of 500 for all 91 currently ranked teams, meant that the range of initial ratings went from 867 to 1000.
    The rating changes account for whether a team scored as many goals as they should, won by as much as they should, and had as many of the goals in their favour as they should. Changes are in proportion to the difference between performance and par: better means going up, worse means going down.
    Each game was entered with current rating, the most likely score taken from the prediction and entered as the result, and the rating updated after each match. Note that because of a two-decimal-point rounding, the predicted scores are almost never a whole number, so even if the prediction is as close as possible, ratings will almost surely still change.

    Results were interesting:
    • The Netherlands win the tournament, beating Argentina in the shootout after a 1-1 drawn final.
    • England comes third, a comfortable 2-1 winner over Germany, who themselves scraped past the USA in a shootout after drawing 1-1.
    • New Zealand lose their quarterfinal 2-1 to England, but eventually place fifth, with the USA very tired of this "1-1* loss" business now.
    • China beats Korea 2-1 for seventh place. Before that, they have a combined eight losses and GD of -10.
    Ranking changes are also very neat to observe:
    • The Netherlands consolidate their top ranking, increasing from 1000 to 1024.
    • Argentina has a major gain of 1005 (from 965) and ranking 2, overtaking England with their minor drop of 982 to 976.
    • New Zealand slips from 931 to 916, and Germany from 915 to 910. That lets the USA take ranking 6 with a good gain of 899 to 915.
    • China is completely unmoved with their ranking still at 8 and their rating still at 883.
    • Korea do the worst. Goals +5/-15, every game a loss, they slide from 867 to 821 and from ranking 9 to ranking 12.
    • Australia holds ranking 4 with 948.
    • India (852), Spain (837) and Japan (822) all rise a ranking together, holding 10-11-12.
     
  8. sanabas

    sanabas FHF All Time Great

    Joined:
    Feb 21, 2007
    Messages:
    2,821
    Likes Received:
    1,089
    Can you post where ARG & BEL got their points from to be ranked top 5 by you? Argentina finished 10th out of 12 at London Olympics, had a single win against South Africa. They made the HWL final, but finished last.

    Belgium ran 5th in both, but wiki also tells me they were seeded 5th prior to the 2012-13 HWL, not in the bottom of the top 10.

    I found the July 2013 rankings, just before the Euros: http://www.fih.ch/files/Sport/World Ranking/FIH Men's World Ranking July 2013.pdf Belgium are 9th. However, there are less than 50 points between NZ in 5th and Belgium in 9th. 5th place NZ are 300 points behind 4th place ENG. Belgium are 9th rather than 5th because they finished 13th at the 2010 world cup, if they'd even managed 11th they'd have been 5th ranked. If your 2013 rankings only include olympics (5th), WC (13th) and HWL (only top 8 at that point, as the finals were yet to be played), does that really give a result which puts Belgium clearly top 5?
     
  9. sanabas

    sanabas FHF All Time Great

    Joined:
    Feb 21, 2007
    Messages:
    2,821
    Likes Received:
    1,089
    Think you'd see much more interesting numbers & an instructive example by running a completed tournament, e.g. the HWL semis, rather than assuming an upcoming tournament would go as predicted. I assume most games predicted 1-1 or 2-1 results?
     
  10. Nij

    Nij FHF All Time Great

    Joined:
    May 22, 2012
    Messages:
    1,745
    Likes Received:
    659
    A lot of 1-1 and 2-1, but of 22 matches, about half were 3-1, 3-2, 4-1 and even a 5-1 and a 5-2.
    I tried it on the Oceania, it expected very close to the actual results (but with four matches and extreme scores, that could mean anything).
    Going to run it on Rio 2016 shortly. Takes a while as I haven't automated the tournament simulation.
     
  11. Nij

    Nij FHF All Time Great

    Joined:
    May 22, 2012
    Messages:
    1,745
    Likes Received:
    659
    About two-thirds of the games are predicted as 1-1, 1-2, 2-2. Others include several 3-1or 3-2, and examples of 4-0, 4-1, 5-2, 6-1.

    In the simulation, with the Netherlands forcing out Australia in the semifinal, they took down Argentina easily, who'd beaten New Zealand in a shootout. Oceania bronze match went to New Zealand by a small margin. Great Britain, China, USA, South Korea all lost in quarterfinals. Spain, Germany, Japan, India were eliminated in groups.

    Given that simulation pool play put Great Britain in the same quarterfinal as the Netherlands (i.e. the real fate of Argentina), and had dumped Germany completely (the real fate of China and Korea), the top end placing matches the expectation of the betting agencies that I could find pre-tournament odds recorded for.

    All in all, the predictions were pretty good. Picked two out of four ties and gave five false positives. However, of the seven errors, all of them were only one goal away from the correct result.
    Picked the winner in 17 of 26 matches. Of those it got one score correct in six matches, and the complete score in two. The mean error in score prediction was 1.70 goals per match (including ties and including correct scores) with a median of 2.00 and a standard deviation of 0.69, so it is statistically never worse than 3.77 goals wrong altogether.

    19 out of 30 correct results. Only 16 different goals prevented it from being 30 of 30 results, less than 1.5 goals in each wrong game. Not a bad afternoon with statistics, is it?
     
    #31 Nij, Nov 12, 2017
    Last edited: Nov 12, 2017
  12. sanabas

    sanabas FHF All Time Great

    Joined:
    Feb 21, 2007
    Messages:
    2,821
    Likes Received:
    1,089
    I may be confused. Didn't you say you were generating the expected results from the elo difference, and then assigning that expected result, rounded to whole goals, as the actual result? How do you get errors, and not the 'correct' result then?

    Which is why I think the HWL semis would be more interesting, because then you've got real life results to compare to the expected, and can look at how those actual results would have changed things.

    :D
     
  13. Nij

    Nij FHF All Time Great

    Joined:
    May 22, 2012
    Messages:
    1,745
    Likes Received:
    659
    This time I did both. Simulate through, and then run it with the actual game results. After pool play they had diverged so the knockout rounds couldn't be compared the same way.
     
  14. sanabas

    sanabas FHF All Time Great

    Joined:
    Feb 21, 2007
    Messages:
    2,821
    Likes Received:
    1,089
    Wonder what the rankings would look like with the actual results for both men & women. NZ women losing 4 games out of 6 to get a silver medal & Argentina winning 4 out of 5 to place 5th would show a dramatic difference between ELO and points based purely on tournament placing. Likewise the Indian men's only win being the bronze medal match, and the gold medal match being between two teams who went winless in the pool stage.
     
    Theory likes this.
  15. Nij

    Nij FHF All Time Great

    Joined:
    May 22, 2012
    Messages:
    1,745
    Likes Received:
    659
    That's why I prefer Elo instead of placings, or at most an Elo with minor bonuses for placing.
    Rating and ranking teams should be based on quality of play throughout the quantity of play, not a couple of flukes due to a quirky tournament structure.
     

Share This Page