This got discussed recently at Baseball Think Factory after Sean Forman wrote Pettitte Falls Short for the Hall of Fame for the NY Times. So here is my take on it.
I first looked at where he ranked all time in RSAA. That stat is from the Lee Sinins Complete Baseball Encyclopedia. It is "RSAA--Runs saved against average. It's the amount of runs that a pitcher saved vs. what an average pitcher would have allowed," including park adjustments. Pettitte now has 204 RSAA. That ranks him 77th all-time. Seems like too low of a rank to make the Hall. But he is 18th among lefites. Maybe left-handed pitchers have a tougher time than righties, so maybe the bar should be a little lower for them. It is not anyone's fault if they are left-handed. They could not have simply worked hard to become a righty. Of course, it also is the case that lefties simply have less value since there are many more right-handed batters. And maybe the Hall has to recognize how much value a pitcher had. But I will continue to show where Pettitte ranks among lefties.
Next I found the RSAA per IP for all pitchers with 2000+ career IP. Pettitte had .0697 (or . 63 runs per 9 IP). That was good enough for 58th. But among lefties, he was 13th. Then I found each pitcher's expected winning percentage using the Bill James pythagorean formula and assumed a league average of 4.5 runs per game. Each pitcher was given a number of games equal to his IP/9. That was multiplied by the expected winning percentage to get projected wins. I then subtracted from that the number of wins a replacement pitcher would have won. For that, I assumed a .400 winning pct. This process predicted that Pettitte would win 186.8 games while the replacement would win 130.06. So that gives him 56.74 WARP or wins above replacement pitcher. He ranks 87th in this WARP measure but is 20th among lefties.
But runs saved is partly determined by the fielders. So I created simple fielding independent ERA. I looked at all all pitchers with 2000+ career IP and used the following stats, all relative to the league average: ERA, HR, SO and BB. 100 is average. A number over 100 means better than average. I ran a regression with ERA as the dependent variable and the others as the independent variables. Here is the equation
ERA = 37.96 + .187*BB + .262*SO + .202*HR
Here are Pettitte's numbers:
So, for example, he gave up 44% fewer HRs than the average pitcher (this comes from Lee Sinins Complete Baseball Encyclopedia). Plugging these numbers into the equation, Pettitte gets 116.85, meaning his projected ERA based on fielding independent stats is 16.85% better than the league average. But his actual ERA is 17% better, so he just happens to project well. Anwyay, he ranks 69th overall but is 20th among lefties.
I also computed a WARP using this predicted ERA in the manner described above. Pettitte ranks 89th while being 21st among lefties with 57.62.
The biggest think in his favor is ranking 13th in RSAA/IP for lefties. But some of his other ranks are pretty low. I think the Hall of Fame has about 219 players, of whom 71 are pitcers or 32.4%. If a team has 25 players and pitchers are 40% of the team, then the Hall should have pitchers (about 87). But if all the position players are deserving (not likely, but I will play along anyway), then about 38 more pitchers need to be in (109/257 is about .4). Only 15 of the pitchers were lefties and Pettitte does rank fairly high among lefties. And if there should be 109 pitchers, he seems to be in the top 109 all-time. Even if there should be 87, even his worst rank that I found is close to that. Of course, all of this assumes that there are no undeserving players or pitchers in right now.
A couple of other things. I thought maybe Pettitte got an advantage pitching at Yankee stadium above the normal park adjustments since he is a lefty and might face alot of righties there where they have a harder time hitting HRs. But from Retrosheet, he gave up a HR% (based on batters faced) at home of 1.94%. On the road it was 2.01%. That does not seem to out of the ordinary. But his HR% (based on ABs) vs. righties has been 2.18% while vs. lefties it has been 2.18% as well. It seems like it should be higher against righties because over the last three years in MLB left-handed pitchers have allowed a HR% of about .5 percentage points higher against right-handed batters. That points to him getting an advantage from Yankee Stadium, but then his home HR% does not seem to give him much of an edge. So I don't know what to conclude from that.
I also once created what I called the Pitcher’s Homerun/Walk Rating. It combined a pitcher's ability to prevent both HRs and BBs into one index rating. Pettitte was 23rd among pitchers with 2000+ IP from 1920-2006. Now it looks like he has slipped to 32nd. But that is out of 277 pitchers. Pretty darn good.