FanPost

The #HP29RMNWild and You: A Lesson in How to be Wrong

As the Wild clinched their second consecutive playoff berth (hey, we celebrate streaks here in Minnesota) last night, the vast majority of the hockey world was completely un-surprised. Picking the Wild to completely miss the playoffs with a roster sporting Zach Parise, Ryan Suter, Mikko Koivu, and Jason Pominville would seem madness to most. Alas, when you have a "projection system" as reliable as the VOKUTA system over at Hockey Prospectus, nothing is beyond imagination.

Before the season started, not only did HP pick the Wild to miss the playoffs, they picked the Wild to finish 29th over all in the league. Of course, that prediction is hidden behind ESPN's "Insider" wall now, but thankfully, our friends at ESPN 1500 saved the juicy tidbits for us.

But first, here's what they had to say about the Wild, who ranked 29th and are seen as a "draft lottery contender":

Minnesota Wild (83 points): Although most experts still see the Wild as a playoff team, there's significant risk of a disappointing season in 2013-14. Minnesota has a lot of star power on the top lines but doesn't have a lot of depth built around those stars. Ryan Suter, for example, is paid more than all its other six defensemen combined. If the goaltending also becomes an issue, the Wild could easily find themselves drafting first.

Some more on their projection system:

How did we come to these conclusions? Using a statistical projection engine called VUKOTA, which makes use of the goals versus threshold (GVT) metric. Described by creator Tom Awad as "GVT's most powerful descendant," VUKOTA projects a player's likely future based on the scoring of historically comparable players. It is named after former fringe NHL player Mick Vukota as a play on words of baseball's similar PECOTA system.

This is something the folks at HP charge people for. You can buy their yearly book for $14.95, and get a subscription to the site for $14.99. This info will be helpful later.

For the record, I saved the overall rankings in one of those handy-dandy software programs from the makers of Windows.

Here is HP's over all "projections" for the 2013-14 season:

Team

Pts

Pittsburgh

101

New York Rangers

97

Boston

97

Chicago Blackhawks

97

Ottawa

96

St. Louis

96

Edmonton

95

Montreal

95

Philly

94

Toronto

93

Dallas

92

Tampa Bay

92

Phoenix

91

Washington

91

Carolina

91

LA

91

Columbus

91

Anaheim

91

NJ

90

San Jose

90

Vancouver

89

NYI

89

Detroit

88

Winnipeg

88

Colorado

88

Nashville

87

Buffalo

86

Calgary

83

Minnesota

83

Florida

80

There they are. Right there at 29. Your Minnesota Wild.

I'll give you a minute to absorb the rest of the list. Things like Edmonton being ranked 7th overall. The Ducks being picked to miss the playoffs. Only one team in the NHL finishing over 100 points.

We all on the same page? OK. Onward.

I brought up the fact that HP's "projections" were incorrect several times this year. It should have been clear at that point that a regime built around stealing people's money (not literally... more... theft by swindle) wouldn't ever admit their system is terrible. So, I decided to track it over the course of the year.

Here's where we were a quarter of the way through the season:

Real (1/4)

Pts

Diff

Absolute

Anaheim

32

18

18

SJ

31

19

19

Chicago

30

1

1

Phoenix

29

9

9

Colorado

28

20

20

Tampa Bay

28

6

6

Minnesota

28

22

22

St. Louis

27

-1

1

Los Angeles

27

7

7

Boston

25

-7

7

Vancouver

25

10

10

Pittsburgh

24

-11

11

Detroit

24

10

10

Toronto

23

-4

4

Washington

23

-1

1

Dallas

22

-5

5

Montreal

22

-9

9

Winnipeg

22

6

6

Ottawa

20

-14

14

Carolina

20

-5

5

NJ

19

-2

2

NYR

18

-20

20

Nashville

18

3

3

NYI

17

-2

2

Philly

16

-16

16

Calgary

15

2

2

Columbus

15

-10

10

Florida

12

2

2

Buffalo

11

-2

2

Edmonton

10

-23

23

AVG

8.9

That "AVG" at the bottom is the average number of places the "projections" were off by.

Here's where we were half way through:

Real 1/2 Pts Diff Absolute
Anaheim 63 18 18
Chicago 63 3 3
St. Louis 59 3 3
Pittsburgh 59 -3 3
San Jose 56 15 15
Boston 54 -3 3
Los Angeles 54 9 9
Vancouver 53 13 13
Colorado 52 16 16
Tampa Bay 52 2 2
Montreal 50 -3 3
Phoenix 49 1 1
Dallas 47 -2 2
Toronto 47 -4 4
Detroit 46 8 8
Washington 45 -2 2
Minnesota 45 12 12
Philadelphia 44 -9 9
Winnipeg 43 5 5
New Jersey 42 -1 1
NY Rangers 42 -19 19
Ottawa 41 -17 17
Nashville 40 3 3
Carolina 39 -9 9
Columbus 38 -8 8
Florida 36 4 4
Calgary 34 1 1
NY Islanders 33 -6 6
Edmonton 31 -22 22
Buffalo 26 -3 3
AVG 7.5

Here's where we are as of last night:

Last Night

Pts

Diff

Absolute

Boston

113

3

3

St. Louis

111

5

5

Anaheim

110

16

16

Colorado

107

22

22

San Jose

107

16

16

Pittsburgh

105

-5

5

Chicago

105

-2

2

Montreal

97

0

0

LA

96

9

9

Minnesota

94

19

19

Tampa

93

1

1

NY Rangers

93

-10

10

Detroit

90

10

10

Philadelphia

89

-5

5

Columbus

89

2

2

Dallas

87

-5

5

Phoenix

87

-4

4

New Jersey

84

-1

1

Toronto

84

-9

9

Washington

83

-6

6

Nashville

81

5

5

Vancouver

81

-1

1

Ottawa

80

-18

18

Winnipeg

80

0

0

Carolina

79

-10

10

Calgary

75

2

2

NY Islanders

73

-5

5

Edmonton

65

-21

21

Florida

64

1

1

Buffalo

51

-3

3

AVG

7.2

While the "projections" are getting a bit closer, they are still more than 7 places off, on average. Nailed it on Montreal, though. Great job on that one. Impressive that even one team out of 30 can be predicted with that much accuracy. We'll get to that.

Look at how many teams they predicted to finish with 80 or more points. All of them. Every single team in the NHL. Want to know how many times that has happened? Never.

Bullying

When I brought this up on Twitter last night, the HP account decried my "harassment" and said I clearly didn't want explanations or discussion. They said I was "angry" and that their system "doesn't hate [my] favorite team."

You can see the entire conversation on Twitter, should you choose to.

While we can't be 100% sure, it appears that the man behind the curtain last night was HP's Matthew Coller. To be fair, he is not the one who made the original "projections," but it sure seems he was quick to the defense of the ill informed person who did.

In his post last night, Coller calls me a bully, and then tries his best to explain why "projections" aren't something that can be done accurately with so many variables. Why they even try, and why they sell people this system (literally) if it can't be trusted is something I wonder about, but they seem comfortable with it, so good on 'em.

The question came in ragey, bullying form via Twitter...and as frustrating as that may be to deal with...a good question is a good question.

Before we start, let's be clear... bullying is a real problem in the world. I'm not convinced it is as big a problem as we allow it to be made out to be, and a great deal of it seems to me to be easily solved with a good old fashioned knuckle sandwich, but it is, indeed, a real problem.

However, when people like Mr. Coller call people bullies simply for disagreeing and asking for an explanation as to how you can justify taking money for a projection system that is so very wrong so very often... the term bullying losing emphasis, and the people trying to fight it lose credibility.

The example? The Minnesota Wild. Hockey Prospectus's VUKOTA projection system was not a fan. The system, based on past statistical performances - not human instincts or hunches or even things we might infer such as Player X will get top line minutes in Minnesota when he received fourth-line minutes in New York.

No, sir. Actually, the "example" was the entire standings. The Wild were simply one example given, and the only one you seem to have bothered to concoct an excuse for. The key here, as pointed out on Twitter, is that the VOKUTA "system" only takes into account past performance. It leaves no room for improvement... since, you know, no player ever improves their game.

Now, the HP write-ups in the book cover these things. They point out the issues with the projection and what might go differently from what the numbers say. But some can't get past the number.

What's the point of putting out a number if the number isn't what it is about? And if we want the explanation, we have to buy the book? Pass. Prove to me the thing has value, then we'll talk about giving you money.

VUKOTA knew Minnesota could not score goals. It projected them to finish 29th in goal scoring. They are currently 26th. Why? There were very few proven goal scorers present outside of Jason Pominville and Zack Parise. The projection does not favor aging players like Dany Heatley and Mikko Koivu because players past their prime tend to see a reduction in points. In this case, Koivu beat the clock. Most players do not.

So much wrong here. Brain... cramping.

  1. It's Zach. With an H.
  2. Dany Heatley hasn't been relied on to be a goal scoring in three seasons.
  3. Mikko Koivu is "past his prime." There just aren't even words to describe how asinine this is. I'm just going to walk away from that and let you fans have at it.

Also... VOKUTA can't "know" anything. It isn't a person. It's a statistical analysis system. The people behind it "know" things, which creates bias. Own it.

The Wild also relied on young, unproven players. Ones that made projecting them statistically even more difficult. For example, after scoring one NHL goal in 55 games in 2011-12 and spending 2012-13 in the minors, Nino Niederreiter has posted 34 points. He spent a good deal of ice time with Parise. Maybe if you covered the team from training camp on, you would have known from Day 1 he was going to get chances, but by statistical measures, even factoring for AHL production and age, the numbers would never say a John Scott-like year would turn into 34 points.

Yes. It would be painful to attempt to do actual research and know that a player is being under utilized in their current role and might be poised to have a break out year given time in the top 6 on a new team. If only there were a system of "websites" where one might find this information.

And Nino spent "a good deal of ice time with Parise?" False, as proven by Hockey Wilderness' own Ger Devine. Nice try, though.

Is it a shortcoming? Of course. Statistics done a month before the season can not predict circumstances. There are just far too many options. It was just as likely Nino would score 13 goals as he would end up back in the AHL. A projection system designed to gave credit to players like that would be put together illogically. In that case, you'd have a lot of 4th liners projecting to be big time producers. Instead, you deal with the outlier.

Statistics done a month before the season can't predict circumstances? Isn't that the entire point? It was absolutely not as likely he would score 13 goals or be in the minors. Again, research would have helped here.

Mikael Granlund also out-performed his projection, despite VUKOTA factoring his age and coming out with a double-up in production from last season. Same with Charlie Coyle.

Historically, players take a step from age 20 to 21. These two took leaps. And good for them. But if the numbers say that most players only take a step, how would you ever guess these two fellows would leap? Only by trying to arbitrarily weigh the numbers, which would be messy and involve bias.

If you couldn't predict Granlund and Coyle would have good seasons, you not only didn't do basic research, you're admitting you didn't even bother to try. Two of the top prospects in the league, both slated to play in the top 6. All of this written by many people locally and nationally. You are flat out admitting you just didn't care enough to look into it.

But... there's no bias in the system, right? Just a complete lack of willingness to do the research.

There were defenseman who out-performed their past too, namely Marco Scandella, who was an AHL player for all but six games in 2013, yet has turned into a very solid defenseman. Last time he was in the NHL full time, he was minus-22, this year he is plus-10. (Yes, we know about the shortcomings of plus-minus). The point is, that makes four players who out-performed their past by a mile.

Trading for Matt Moulson is super obvious. A projection system can not guess trades that will happen in March. Moulson is an impact player and has scored big goals for the playoff-bound Wild. If you want a system to guess trades, there are other places on the internet for that.

Scandella was a wild card. I'll give you that. However, he is not outperforming anything he was expected to do. The fact that +/- is the only thing you have to go on suggests you threw this in just to shore up your weak argument with an even weaker one.

Moulson has played 18 games for the Wild. Did he improve the team? Absolutely. Did he move them from 29th to 10th overall? Not a freaking chance.

Nice that you hint at HockeyBuzz, though. We'll show just how poetic that is momentarily.

And the biggest factor: Goaltending.

Based on the past numbers of the Wild netminders, VUKOTA said they would end up 20th in the NHL in goals against. Instead, they are fifth.

Why? One is a trade for Ilya Bryzgalov. The other is that Josh Harding performed far, far above his career statistics.

Harding has a .933 save percentage, while his career save percentage is .918. The system takes a big sample to make a prediction. If it randomly predicted an above average goalie to score higher than most Vezina Trophy winners, something would be seriously wrong.

And .933 and .918 are no small difference. The Wild are plus-3 in goal differential. Over 1,000 shots, Harding's hot streak would be a gap of 15 goals. Slip him back to his career average and - BAM - you have the Wild at minus-12...or almost exactly the same as Winnipeg, who is in last place in the Central.

It's not just Harding, either. Bryz has a .923 save percentage - far above what you would expect from a backup - and Darcy Kuemper was above average at .915. Would any system project an AHL goalie from the year before to be above average? Sure, it happens, but you wouldn't expect it.

The trade for Bryzgalov likely saved the Wild's season. That said, Kuemper was playing as well as he needed to, and Harding's play was outstanding. Can't predict it? Sure, maybe not, but you're the one making the... wait for it... predictions.

Bryzgalov isn't a backup. He's a starter. Want to know how I know? He has been the de facto #1 everywhere he has gone, except Philly. Philly is your precious "outlier." Good work on the research there, too.

Something is seriously wrong, but it isn't with goaltending. With the Wild starting four different goalies, and dressing seven, I'm not thinking goaltending is the magic.

Should we have known that some of these things would be different than the statistical projections. Yes. Oh, wait, we did!

From the book:

"Minnesota is a bubble team and a healthy Josh Harding might just be what ultimately makes the difference," - Robert Vollman.

Harding wasn't healthy the whole time, but he was a big difference maker. And they are a wild card team...which sounds like a bubble team to me.

This is wishy washy. They predicted the Wild to finish 29th. That isn't the bubble. And no... a wild card team is not, in fact, a bubble team. If your definition of a bubble team is a team that has locked up their ticket with a week left to play, you're simply not paying attention, or just stretching to make the facts fit your story line. Likely both.

So, what we have here is a perfect storm for a projection to short sell a team because of past performances and goaltending. VUKOTA's 2014 projections had several teams out-perform the stats it spit out and others land right in the same ballpark.

A "perfect storm." Right. Got it. For one team. Now explain Edmonton. Another "perfect storm?" Anaheim? San Jose? Ottawa? Carolina? Detroit? The Rangers? Colorado? LA?

All "perfect storms" for sure.

And, good job Minnesota Wild. Could be an upset team. I'd predict it, but that might just be asking for more bullying.

You keep using that word. I do not think you know what it means. Of course, you've probably been handed everything and told what a winner you are your whole life, so the first sign of adversity is bullying. We'll move on.

Other Systems

HP charges people to sell them guesses. The VOKUTA "system" is no more than a glorified guess, with numbers manipulated to support the guess. It has been just as inaccurate since its inception, yet HP continues to sell us on how accurate it is every fall during their sales push.

It is truly theft by swindle, but one willing undertaken by the same people who believe Eklund and HockeyInsiderr have actual sources. These frauds have been proven with prediction models that show that inserting random names to random teams is more accurate that the BS these two frauds spew out.

Is there a system that could be more accurate than HP's vaunted VOKUTA? I have, in all of my spare time, invented a new prediction system. I will use all caps to give it weight, and make it look like an acronym, giving it an air of mystery and professionalism.

I call it THE STANDINGS.

It's incredible, that if you were to compare last year's final results, you get an average deviation of placement of teams of just 5.9 spots.

Here are the standings from last year:

Last Year

Diff

Absolute

Chicago

-7

7

Pittsburgh

-5

5

Anaheim

0

0

Montreal

-4

4

Boston

4

4

St. Louis

4

4

LA

2

2

Vancouver

-14

14

Toronto

-10

10

Washington

-10

10

San Jose

6

6

NY Rangers

0

0

Detroit

0

0

Ottawa

-9

9

Minnesota

5

5

NY Islanders

-11

11

Columbus

2

2

Winnipeg

-6

6

Phoenix

2

2

Philadelphia

6

6

Dallas

5

5

New Jersey

4

4

Buffalo

-7

7

Edmonton

-4

4

Calgary

-1

1

Carolina

1

1

Nashville

6

6

Tampa

17

17

Colorado

25

25

Florida

1

1

AVG

5.933333

Remember how impressive it was that HP predicted the one team exactly where they sit? THE STANDINGS nailed three.

The Point of All This

It's doubtful that many of you made it through the 3000 plus words here, but if you did, good for you. The point here isn't to bully anyone. The point here is to call a fraud a fraud, and let people know they are flat out giving their money to people with less predictive ability than the standings page on NHL.com.

To suggest that VOKUTA has any value as a predictive model is to say that ice doesn't melt when it is hot. The system is terrible, and anyone who defends it is misinformed and likely lazy.

Of course, that's just bullying, so it likely doesn't matter.

The opinions posted here are not those of Hockey Wilderness

X
Log In Sign Up

forgot?
Log In Sign Up

Forgot password?

We'll email you a reset link.

If you signed up using a 3rd party account like Facebook or Twitter, please login with it instead.

Forgot password?

Try another email?

Almost done,

By becoming a registered user, you are also agreeing to our Terms and confirming that you have read our Privacy Policy.

Join Hockey Wilderness

You must be a member of Hockey Wilderness to participate.

We have our own Community Guidelines at Hockey Wilderness. You should read them.

Join Hockey Wilderness

You must be a member of Hockey Wilderness to participate.

We have our own Community Guidelines at Hockey Wilderness. You should read them.

Spinner.vc97ec6e

Authenticating

Great!

Choose an available username to complete sign up.

In order to provide our users with a better overall experience, we ask for more information from Facebook when using it to login so that we can learn more about our audience and provide you with the best possible experience. We do not store specific user data and the sharing of it is not required to login with Facebook.

tracking_pixel_9355_tracker