Evaluating my simulation model

This is something that I like to do at the end of each season before I decide if there are any tweaks that I want to make to my simulation model.

Pre-Season Odds

The first place to start I suppose is looking at where things started before the season.

Looking at this it might look like a miss giving Manchester City such overwhelming odds of winning the League but I think that turned out pretty reasonable considering it was below the betting market but also because they ended up with some of the best underlying numbers even if that didn't turn into points.

I also did take a (very modest) punt on Liverpool winning the title based on this because my model was much more bullish on them.

Switching over to points:

My biggest miss was Norwich, I did expect them to be in the relegation fight but I didn't expect them to be one of the worst teams in Premier League history. Second was Sheffield United, who I expected to be solidly in the relegation fight but they instead challenged for the European spaces.

Overall I think that this points out that it is really quite hard to predict what will happen when teams come up from the Championship.

I also missed calling Tottenham the third best team to start the season, not many people could have seen the collapse of Maurico Potchitinho coming.

The misses that with hindsight I think should have maybe been more obvious was that Bournmouth were not nearly as good as people thought, I didn't rate them terribly high and I did give them an almost 15% chance of going down Pre-Season. The other one was Leicester, I think that with hindsight maybe they were a better theam that my inital ratings suggested but their collapse at the end made things look closer than they truley were.

Biggest Upset: Norwich City vs Manchester City

The biggest upset looking back was Norwich's 3-2 victory over Manchester City. This looks even more impressive considering that Norwich would only get 18 more points beyond these 3 all season.

Overall Evaluation

Overall I pretty closely matched what would be expected.

At the high end, perhaps I could have been more aggressive. My highest expected win probability was just 76% and all 7 times that I rated something as a 70% or greater chance the favorite won. What is more concerning is that when I gave a team 55% or higher (83 predictions) they won 63 times. It is still a pretty small sample but that is something that I may consider tweaking.

Overall I think that the model does exactly what I want it to do, it gives me practice in building models and helps to give me a tool to play around with the play "what if" to run different scenarios. I wouldn't use this as a betting model because it far from incorporates all of the information that is possibly available.