Another look at St. Louis Cardinals Draft-and-Development
Spring Training is underway! There is new stuff coming out of camps everyday now, a respite from the off-season doldrums where we hashed and re-hashed the same few unresolved debates. Who knew that they were running a special on hamate bone repair this month?
As is my want, I tend to zig on writing topics when everyone else zags. New and fresh spring training camp content is hard to come by, so I’m going sojourn off to previously plowed fields on drafting (and signing IFAs) and developing minor league players. As discussed in other venues, I have been working on some methodological changes to my series on draft-and-development. A lot of focus has been in the arena of trying to better discriminate the draft from the development part (difficult to do!). I have some preliminary results. Needs more work, which will be greatly improved by your contributions in the comments sections.
In pondering how to discriminate development from drafting, I decided the most intuitive way would be to evaluate how players improved (or not) from their first (draft) ranking (in FV terms) to where they have ended up. To be certain, one can’t actually determine if a single player improving from a 40+ to a 50 outcome is because of development or because the rating was artificially low. My sense was that although individual outcomes would be difficult to tease apart, I could look at organization-wide outcomes and see if there were trends that transcend individual rating misses.
For example, I might (in theory) see a team like the Cardinals consistently turn a higher percentage of 45 FV prospects into average (50 FV) MLB players. A single under-rated player is probably not uncommon. A repeating theme of same could suggest the fine hand of player development, no?
So, I endeavored to acquire prospect ranking data. I got as far back as 2017 from Fangraphs Prospect Board, so that defined my population (and timeline). This dataset contains nearly 19,000 prospect evaluations, so it is a deep set, albeit error riddled. Ugh! Lots of data wrangling with this set. More to come, too.
You’ve heard me say that it is difficult to evaluate a draft before 7-10 years have passed, and yet I only have a nine year data set to analyze. So, I also endeavored to try to shorten that window a bit, borrowing an idea from Ben Clemens of FG here, (https://blogs.fangraphs.com/how-do-prospect-grades-translate-to-future-outcomes/) who proposed it is rational to take Zips 3-year forward projections and append them to young players short history to develop a more comprehensive view of said player/prospect. So, I joined 5,700 Zips projections that came out a few weeks ago with the 19,000 prospect ratings, covering some 4,300 different prospects. No easy feat. Been working on this all winter, and not done yet.
In essence, as I describe players/prospects, I am describing what Zip+DC thinks this player is today and will be 3 years hence. If you accept projections as a reasonable source of analysis, then I’ve shortened my window to 4-7 years, which gets me inside that 2017 cut-off (I can find no reliable electronic data source of prospect grades prior to that year).
I had to make a couple other methodological choices which I invite you to comment on. One is, I’ve calculated each player/prospects’ actual+projected WAR value and divided that value by that player’s MLB seasons – 1 ti create an “average WAR”. Zips appears to forecast everyone 3 years out, including young-ish prospects such as Raniel Rodriguez, which I found handy. Thus, every player has a minimum of 3 seasons of data, more if they’ve made their MLB debut. I used that AverageWAR to assign an FV value of what that player is today and expected to be in the future, as compared to his prospect peers (not all players). This value is completely driven by Zips projections plus actual production and stands in contrast to the scouting grades I compared them with.
Then, I distributed that players Average WAR along the 20-80 scale, using the guidance that each 10 places is one standard deviation. Ergo, 68% of all prospects will have what I term “Adjusted FV” between 40 and 60 and 98% will fall between FV 30 and FV 70 and the remaining 5% will occupy the nether regions 20 and 80 FV. In practice, I ended up with more 20’s and 30’s because many prospects don’t make an MLB debut, don’t achieve and 35 or higher FV and have no actual production nor any Zips ratings, so they go into the waste bin.
My first test was to evaluate the prospect/players who grade out 80 from their performance and projection. A total of 5. The rarest of the rare, top .3%. You can see the list below. Definitely performances that are outlier (beyond 2 standard deviations from average). The list passes the eye test, no?
Prospects who perform at 80 grade (2 or more SD from average)
PlayerName
primarypositionname
pitcher.type
careerWAR
projectedWAR
firstFV
lastFV
Adjusted.FV
Shohei Ohtani Two-Way Player Starter 49.6779 18.82581 70 70 80
Shohei Ohtani Two-Way Player Starter 49.6779 18.82581 70 70 80
Tarik Skubal Pitcher Starter 19.2694 17.80266 45 60 80
Bobby Witt Jr. Shortstop Starter 26.7328 17.60179 55 65 80
Garrett Crochet Pitcher Reliever 11.8924 15.56059 45 50 80
Paul Skenes Pitcher Starter 10.77 15.51154 60 65 80
Data courtesy of Fangraphs | Zips
I scaled the AdjustedFV value by starters, relievers, and position players. In the list above, you are seeing the top .3% of each group. No relievers performed at 80 FV, FG tends to scale all pitchers to WAR per 200 IP for comparison purposes, but I found the 200 IP limit a bit anachronistic (and this is a modern data set and who pitches 200 IP anymore?) and leverage varies a lot between starters and reliever, so I chose to scale within a like cohort. Tell me if you agree with a list that shows Devin Williams as more valuable since he broke in than say, Dakota Hudson.
If this passes the eye test, then the whole data produces some MLB-wide averages we can begin to compare.
MLB-wide prospect outcomes since 2017
GroupTotal
MultipleRankings
TrendDown
TrendDownPct
TrendUp
TrendUpPct
Traded
TradedPct
NoChgPct
BeatProjection
BeatProjectionPct
UndershotProjection
UndershotPct
HitProjection
HitProjectionPct
4297 3710 920 25% 910 25% 713 19% 51% 1159 27% 2846 66% 292 7%