Rookie Rankings
Settings, Methodology, Improvements (Looking Forward)
Settings
I have always argued that IDP writers should lay out their settings before posting rankings. Granted within-position rankings are useful regardless of number of positions used or scoring system but as soon as you try to evaluate LB3 against S1 or WR12, clarity is crucial. In the rankings you will find I am assuming:
1) Balanced starters (i.e. 11OP/11IDP)
2) Superflex
3) TE Premium
4) True Position
5) 5 IDP Positions (DT/DE/LB/CB/S)
6) No Position Left Behind Scoring
Methodology
I've always been curious how rankers come to their evaluations. I understand the hesitancy to give up the secret sauce but, as an academic, I am an advocate of transparency so (without boring you with formulas) here are the criteria used in my rankings and a brief note as to why they are included in the final model. Note: I am still in the process of collecting data to empirically validate my model but initial results are good.
Starting Point: Consensus Scout Evaluation. I do not watch tape or analyze college stats, I let the professionals do this for me. I take the consensus of dozens of analysts who rely primarily on measurables (rather than those who rely on tape) to get a starting point on player talent. I scale these evaluations using a 0-100 system to allow for year-to-year comparisons.
Modifier (+5% to -5%): Team Fit. This is admittedly a subjective metric but I use post-draft depth charts to determine the likelihood that the player will see the field in Year 1. Playing time is crucial for NFL and fantasy success and so this is the biggest swing.
Modifier (+5.9% to -3%): Draft Capital. Earlier NFL draft picks tend to get more chances at playing time and snap counts are the top predictor of fantasy performance (at least for IDP).
Modifier (+3.03% to -4.67%): Scout Evaluation Variance. I want as much certainty as possible around the quality of a player and so if there is significant agreement/disagreement around who a player is, I bake this in.
Modifier (+2% to -2%): Fantasy relevant metrics. Again there is some subjectivity here but I look for one key data point by position and add or subtract value based on this (i.e. tacking grade for LBs or route running metrics for TEs).
Additional Modifiers (Vary): Actual Positional Value, Perceived Positional Value, IDP Discount, CB Discount. If I only used the 5 criteria above, you would be drafting CB1 from a class near the top of your draft. To try and make my rankings more practically useable I make a fantasy VORP-style adjustment, an adjustment based on historical data on combined drafts, a blanket (negative) IDP adjustment, and a blanket (negative) CB adjustment.
Improvements (Looking Forward)
The 2021 rookie rankings (and all previous unpublished iterations) were relatively static. Starting in 2022, I will publish some pre-NFL Draft rankings (for those of you who draft before the NFL Draft) and then do periodic updates where I shift the weight of the rankings to be reflect Team Fit changes (moves up or down the depth chart) as well as a new subjective factor based on training camp reports. I also hope to collect some data to map my accuracy in one, two, and three year rolling windows.
