“This illustrates the large problems I have with the gospel of advanced stats and prediction systems. #Bluejays rotation ranked 16th in #MLB” – Mike Wilner
This was Mike’s initial reaction this week to the release of the first half of the starting pitching segment of FanGraphs’ positional power rankings. The Blue Jays’ group of starting options did indeed rank 16th, and that certainly is jarring given their quality one through five. But does that make it a moment to rail against a straw man, or might there be something we can actually learn from this?
The whole point of the statistical revolution in baseball — which Wilner, among broadcasters, was a very early proponent of — was to let numbers challenge our assumptions. We don’t recoil when we see numbers we don’t like, we investigate. Either the numbers are trying to tell us something we’re missing, or we can use our understanding to make the next set of numbers better and the current set make more sense.
In this case it’s all of the above.
Projections, as I’ve written about already this off-season, don’t like certain members of the Jays rotation very much. I think we pretty much understand that by now. Marco Estrada induces bad contact with an elite-spin “rising” fastball, a cutter, and one of the best change-ups in baseball, but projection systems generally see the BABIP he induces as wholly unsustainable. They probably have a little bit of a point, even — Estrada’s skillset likely shines a little brighter than maybe it should because of a touch of luck and a very good defence behind him — but the ERA in his projection in FanGraphs’ piece is more than a full run worse than what Marco has actually allowed over the last two seasons.
This is not a player that the projection system handles well, in other words. But a rare case like that, to me, is almost — alllllmost — a feature and not a bug. This is a pitcher who is producing value in an atypical way, and if a team can find out what makes him successful, maybe they can find other guys who are being overlooked that can be successful in a similar way. His ability to seemingly buck his projections leads us into new and different ways of thinking about pitching and how to get batters out. That’s one of the most interesting things about this stuff. Nobody worth listening to is looking at that number and accepting it as gospel.
The other guys in the Jays’ rotation are a little less interesting in this regard:
- Marcus Stroman’s 3.2 WAR projection is fine.
- Francisco Liriano gets dinged because of age and because of the fact that he had an abysmal 2016 season (provided you don’t do what most Jays fans have been doing and ignore everything he did up to the point where he was traded).
- J.A. Happ has a little too much J.A. Happ in his track record to make the system full-on believe he’s really a the guy we’ve seen since he got Searage’d after a mid-2015 trade to Pittsburgh (though his 2.6 WAR projection is hardly outlandish).
- Aaron Sanchez doesn’t have enough in his track record for his projection to line up with his 2016 performance (though his is still among the top 20 projections for any starter, and right in line with Cleveland’s Carlos Carrasco and St. Louis’s Carlos Martinez, which are very reasonable comps.).
In fact, other than being especially light on Estrada (1.9 proj. WAR, compared to 3.0 fWAR in 2016 and 4.0 RA9-WAR), the projections for the Jays’ five starters really aren’t so bad. They’re probably even pretty good!
It’s after we look at those guys where this all starts to go sideways for the Jays. And rather than grandiose dismissals, it’s here where we should probably be paying attention to what the numbers are actually saying. Because the numbers are definitely saying something. They’re saying the same thing that Mark Shapiro and Ross Atkins have been saying about the club’s pitching situation since the day they arrived. They’re saying that depth matters.
We generally refer to a team’s “rotation” as its best five pitchers, but we all know that those aren’t the only guys who’ll make starts for a club over the course of a season. And because they’re not just fucking around out here, FanGraphs’ rankings aren’t just about projecting a team’s front five, they’re about projecting the entire starting staff, using “manually curated playing-time forecasts,” along with “projections for rate stats from ZIPS and Steamer.” (They do this because, as Dave Cameron explains in his intro, “while forecasting systems have been shown to do better than most humans at forecasting production, humans win out when it comes to allocating playing time.”)
The funny thing about projections so apparently awful as to make one question the whole enterprise of advanced stats itself is that, according to the numbers they’re giving us, the Jays’ top five pitchers (by innings pitched) actually stack up rather well with the rest of the league. At 12.9 projected WAR, they’re just a hair behind the Astros (13.1), which would make them the ninth best team in the majors, as far as front fives go.
As much as we may want to slather these five Jays starters in praise, that’s probably exactly around where they should be, because the rotations ahead of them, while maybe not having the Best Fifth Starter In Baseball (TM), sure as hell ain’t bad: the Nationals (17.4), the Mets (17.1), Cleveland (16.2), the Cubs (16.0), the Dodgers (15.9), the Red Sox (15.6), and the Giants (15.4).
Hell, if you really think Estrada is alone in getting screwed and want to bump up the Jays’ total by a couple wins, you just put them right up there with the best of the best!
Ahh, but this. This is why the Jays are ranked 16th by these projections — the ranking of all MLB teams by projected WAR from their sixth starter and beyond:
LAD 4.1
PIT 2.7
CIN 2.5
NYY 2.2
SDP 2.2
TBR 2.1
HOU 2.0
LAA 1.9
MIL 1.8
TEX 1.8
MIN 1.7
BOS 1.6
BAL 1.5
CLE 1.5
NYM 1.5
STL 1.5
MIA 1.4
DET 1.3
PHI 1.3
ARI 1.2
KCR 1.2
CHC 1.1
COL 1.1
OAK 1.1
ATL 1.0
SFG 1.0
SEA 1.0
WAS 1.0
CWS 0.9
TOR 0.6
The Jays lose a half win to damn near everybody because of their lack of depth. They lose a win-and-a-half to the deepest teams. And when we consider that the difference between the top ranked team overall and the 20th ranked team is just eight wins, how they fell so far here starts to make sense.
But then when we remind ourselves that the projections for guys beyond the top five are a understandably a little shakier and more prone to rounding errors, and that the playing time adjustments maybe aren’t quite so flawless, and that the Jays’ outstanding 2016 rotation ranked 25th according to this exercise last season, we can maybe even feel a little bit of comfort about the whole thing.
If they’re healthy, the Jays’ starters can pitch as well as anybody. If their depth gets tested things could get difficult for them. I think we probably knew that already, but it’s nice to see it borne out by the projections. Hey, and we maybe learned a little along the way, too!
No.
I just wanted to see what it would be like to be Stoeten for a second.
If the projection system is forecasting Mat Latos to be the Jays 6th starter this year and pitch 40 innings…maybe don’t worry too much about the projection system.
Who’s the 6th in your mind? and how many innings does that person get?
Andrew makes a good point regarding the depth issue and it is certainly an issue if health becomes a problem. However, I think there is a larger issue he touches upon but doesn’t flush out, which is the efficacy of fWAR for pitchers who have a skill set/arsenal focused on inducing weak contact. The Jays may have identified a market inefficiency in assembling a “FIP buster” rotation, with Estrada and Happ in particular. Sanchez has also outperformed his FIP’y metrics. The Jays rotation actually posted a better bWAR (assuming full season of Liriano) in 2016 (16.1) than is projected in fWAR for the Red Sox’s top 5 projected starters in 2017 (15.5). This disparity in the metrics may best be expressed with looking at bWAR/fWAR for Sanchez and Stroman for the 2016 season, where fWAR had them at 3.9/3.6 while bWAR at 4.8/1.4. No single metric is “perfect,” and there has been a growing tendency for some (not Andrew) to ascribe “fact” to things like Fangraph’s version of WAR. I think that example highlights how fWAR may inherently be a less-than-ideal metric for a big part of the Jays’ outlier rotation.
Good stuff.
bWAR for pitchers is more results based as opposed to fWAR which judges pitchers on Fangraph’s idea of what should have happened. That approach suggests we understand way more about baseball than we actually do and it strips all value from skills that may exist but we can’t yet quantify.
Now imagine an injury or two to that rotation. The wheels would fall off quickly.
Name a team that would be fine if a couple of starters go down.
Dodgers maybe?
this is fine
FG projects Wade Miley to be worth 2.0 WAR compared to 1.9 WAR for Estrada. Would anyone but Miley family members bet on that?
i’ll bet on that, but i have issues
I’m making a simple point – you are not factoring starter health or projected IPs into your article, despite them being an important part of the Fangraphs article – aka, cherry picking.
Trolling refers to your intentional snark, you know “inflammatory comments designed to provoke a response”, as per the dictionary. I sort of think that’s part of your raison d’etre, no? Your ‘maybe we learned a little along the way’ comment is one of your frequent digs against people who don’t celebrate every move made by the FO.
I think your opinion that the AA regime decimated the upper-minors starting depth has been pretty clear for a while now, so you aren’t really being too critical of the current FO by discussing our lack of depth.
If your definition of logical is ‘someone had a reason for something’, it’s likely you that needs to look the word up. It’s not even worth noting that the FO had a reason for signing Smoak and trading for Chavez unless you are making a larger point. Both moves were widely panned at the time, and the results have borne those initial criticisms out.
I like that Shapiro and co will go against the grain at times – Happ being the best example of an unpopular move that worked out. But I feel that their fixation on floor and depth has not proven fruitful, and would prefer genuine discussion around the topic.
Sorry – responding to Stoeten, posted in wrong thread. Is there an edit feature round here?
In what way am I not factoring innings pitched and starter health in? Innings pitched are components of the projected WAR totals I use throughout. I am taking the FanGraphs playing time projections to be sound (even though the reality is that the Jays will be lucky to get as many innings out of their five starters as they suggest — depth is probably *more* important than I’m stating). So… what exactly am I “cherry picking”? What do you think is missing? Please try harder to make your “simple point,” because there still isn’t one here.
Furthermore, you seem very confused about what trolling is and about what I meant by having us having learned something — we learned how better to parse statistics and a ranking that many were simply looking at the whole of; it is not a comment on the front office, and if there is any comment on the front office in this piece it’s that their depth is too thin. And while yes, partly that’s on AA, it’s not entirely on him, and you’re trying awfully hard to make my criticism not a criticism so that you can shit on me for not criticizing. You also seem to think you’re doing this so brilliantly that I’m not going to notice that shit. LOL. Try harder, my man. (Using “raison d’etre” to make yourself sound smarter doesn’t quite cut it — hilarious as that was.)
And then we get “their fixation on floor and depth has not proven fruitful”. So… wait… we should just not care about depth? We should forget that this front office only been here a year and act like it’s not utterly ridiculous to comment on what has and hasn’t borne fruit? We should ignore that they did manage to go back to the playoffs last year? I’m sorry, which of these dumb things would you like me to think?
As for Smoak and Chavez — two small moves that have hardly hurt them (apologies to Liam Hendriks truthers) and you’re trying to hang way too much significance on — though there were indeed actual serious people who grasped the logic and still didn’t like them (based on everything you’ve been writing here, you don’t seem to fall into that category), both of those moves were widely panned at the time largely by people more interested in hearing the sound of their own whining than attempting to understand what the front office was doing. If you want to take the fact that they haven’t worked out well as validation of your thought processes here, that’s probably just going to lead you to more nonsense like thinking depth is bad.
well, I’m not suggesting we should not care about depth – that’s a straw man. I can critique the FO on moves while acknowledging that they have been good so far on the whole. and a year and a half is plenty of time to start assessing the new FO, although it is clearly a SSS. I don’t think I really implied otherwise though. Indeed, two cautious moves designed to raise the performance floor aren’t that significant in the grand scheme of things, but this is internet fandom – everything we are discussing is pretty insignificant.
You wrote: “both of those moves were widely panned at the time largely by people more interested in hearing the sound of their own whining” – this is the attitude that has me calling you out for trolling. I think plenty of intelligent people criticize this FO along with people who maybe haven’t thought about things in much depth – but if you make this sort of blanket condemnation, it sure sounds like trolling to those of us with a more nuanced understanding of the FO.
I’ll be the first to admit that they have grown on me, and I was wrong about the Happ deal, as an example.
Your data table focuses on the WAR generated by depth starters, and this is where I think your explanation falls short. You are explicit in your conclusion: “this is why the Jays are ranked 16th by these projections — the ranking of all MLB teams by projected WAR from their sixth starter and beyond”. Perhaps my term cherry picking is the wrong choice, but I reject your conclusion, and the implied agreement with the ranking. It’s also worth noting just how darn close our ranking is to 10th place – we are 0.7 WAR behind the Cards. Since your thesis is, I believe, that a 16th place ranking is justifiable, leaving that detail out seems like, uhh, cherry picking.
For what it’s worth, my initial post was in the spirit of your site – so a critical attempt to be caustic. I still disagree with you, and still think there’s more than a little troll in you, but I do appreciate the back and forth.
Learn what? how to cherry-pick data to support your narrative? This idea was debunked recently on Batter’s Box. To quote a poster there, “as for the depth issue, that’s not it. Fangraphs depth charts project only 102 innings for starters other than the big 5. These innings are not projected to be any better or worse than typical for your 6-9 starters.”
Look at the Mets – they beat us by 0.9 fWAR from depth starters, but are ranked second over all. They, like us, won’t need those starters the way the Dodgers- ranked first but with an injury-prone front five – will.
LOL
I understand that you prefer trolling to discourse, but the floor is yours for a rebuttal, Stoeten. ‘LOL’ just makes it look like you don’t have one.
Genuine question – is there anything you don’t like that the current FO does? If you want to buy their argument that we need greater starting depth, fine, they’ve been saying that since they got here. Last year, that logic cost us 5 years of Liam Hendricks and 4 million bucks on a replacement level Jesse Chavez.
You also argued for the Smoak deal being logical, if I recall correctly. Those two moves are the worst from the current FO thus far.
First off, I don’t think you know what the word trolling means. Second, what am I to rebut? You accused me of cherry-picking… something, then said that this whole piece was “debunked” because some commenter at Batter’s Box waved their hand and said “that’s not it.” Your second paragraph didn’t make whatever point it is you’re trying to make any clearer, and so I replied with the seriousness I felt you deserved.
And now seeing your reply here, I’m feeling well justified. Is your point that they *don’t* need greater depth??? Are you suggesting I defend everything the front office does even though *this very piece* highlights that their depth is very bad and their low ranking is justifiable???
Perhaps I’m missing your work subtle genius here, but I’m still mostly just thinking: LOL.
Also, the fact that Jesse Chavez didn’t work out and that the club’s starting depth wasn’t tested (because of exceptional health) doesn’t mean that there wasn’t sound logic behind that acquisition. There was also absolutely clear and understandable logic behind the Smoak deal. Those moves have worked out very badly, yes. Was there too much obvious risk in the first place that they might work out this way? Maybe. But there was logic, too. Maybe look up “logical” after you look up “trolling.”