He was not in the top 10 in WAR (from Baseball Reference) and he was only 9th in WAR for pitchers, as the two tables below show. 18 players and pitchers who received MVP votes had a higher WAR than he did.
He did pitch 152 innings, all in relief. He led the league in games (74, a post 1900 record at that time), games finished (62) and saves (22). But only 7 of those saves came in 1-run victories for the Phillies. He also had a 16-7 record. So although he trailed others by quite a bit in WAR, maybe he pitched well in key situations, and that enhanced his value, at least in the minds of the writers. Being on the pennant winning Phillies certainly helped.
Did he do exceptionally well when it mattered? He did finish high in Win Probability Added (WPA) for pitchers, which adds up all the changes in a team's chance of winning after each event (if he pitched in key situations alot, then he had the opportunity to help his team's chances more than other guys). It looks like he came in 2nd, only trailing his teammate Robin Roberts. The two were very close, maybe with just a slight difference that is not seen due to rounding. Musial led batters with 5.6.
If WPA is divided by the average Leverage Index (LI), it removes context, so that a guy that gets to be in alot of key situations does not get an advantage. Musial led batters with 5.4. Six batters had a total of 3.4 or higher (none of them were position players on the Phillies), so that would leave Konstanty 9th in the league.
So he still ranks high, but he does slip just a bit. Maybe the writers accurately sensed that he performed well and often in key situations, although there were others who did better. Again, being on the first place team helps.
One thing the writers might have noticed is how well and often he pitched in extra innings. He had 25.2 extra IP. That was 1.2 more IP than the next two highest in all of MLB combined. He allowed an AVG-OBP-SLG of .140-.253-.174 and OPS of .427. The league average was .723, so Konstanty beat that by a wide margin. His BAbip in extra innings was just .149 (batting average on balls in play).
For the whole season, his BAbip was .208 while his overall AVG allowed was .202. For the whole NL the AVG was .261 and the BAbip was .275. So he had a somewhat smaller overall difference than the rest of the league. The Phillies staff as a whole allowed a .248 AVG and a .261 BAbip. So his differential was about half what it was for the rest of the team. Maybe he had a bit of luck.
The year before his AVG allowed was .264 and his BAbip was .279. The next year those were .281 and .284. For his career, they were .265 & .266. So maybe 1950 was normal for him, having these two numbers be close. Maybe he was a bit better than other pitchers on balls in play.
But his FIP ERA (3.77) was much higher than his overall ERA (2.66). I found all the pitchers who had 150+ IP from 1946-60 (902) and ranked them by their difference between FIP ERA and regular ERA. Konstanty was 13th. So that indicates some luck for him. Here is the top 20.
His career ERA was 3.46 while his career FIP ERA was 4.01, a difference of .055, only half of the 1950 difference.
He did pitch better as the leverage went up. His OPS in low leverage situations was .644. Medium was .599 and high was .544. So he improved as the leverage increased, but it was nothing earth shattering.
But compared to everyone else in MLB that year, his High Leverage performance was great. Here is the top 20 in lowest OPS allowed in High Leverage situations for guys who had 25+ IP overall and faced at least 100 batters in High Leverage Situations.
Although he is 2nd to Maglie, he had many more batters faced and the next guy behind him, Reynolds, is pretty far back. So again, maybe the writers picked up on his outstanding performance in key situations.
But he did not do that well in September, while the Phillies were in the midst of a tough pennant race (although they were up by 7.5 games with 11 left to play, they entered the last day of the season with just a 1 game lead as they had lost 8 out of 10 games and had to beat the Dodgers in extra innings to avoid a tie). In 36.2 IP, his ERA was 3.44 and he had just one save with a 3-3 record. His OPS allowed was .680. The league average was .694 in Sept/Oct.
While the Phillies lead fell from 7.5 games to just 1 in 10 days and 10 games, Konstanty pitched in relief 6 times. His record was 0-2, 0 saves and a blown save. His ERA in 13 IP was 6.23 with an OPS allowed of .915. He allowed 14 hits and 9 BBs (4 IBBs) while striking out just 2. 2 of the hits were HRs. So when it perhaps counted the most, Konstanty was not effective. It seems like that is something the voters would have noticed.
His teammate Robin Roberts had 70 IP in Sept/Oct (yes, 70) with a 2.44 ERA. But his record was just 2-5, maybe due to a lack of run support. He did pitch a 10 inning complete game on the last day of the season, allowing just 1 run. In 6 of those 9 starts, the Phillies scored 2 or fewer runs.
Del Ennis, who led the NL with 126 RBIs that year, a Phillie teammate, had an OPS of .908 in Sept/Oct (.333-.403-.505). His full season OPS was .923 and his WAR was 4.9. So he, along with Roberts, would have been a worthy choice for MVP if being on the first place team was important. Ennis was 4th in the voting while Roberts was 7th.
The New York Times article about the award does not discuss him pitching well in key situations, just his 16-7 record and suggests he saved at least 25 games. Konstanty said he could not have won the award without the help of his teammates.
The Sporting News did not go into too much detail either. They emphasized his total games pitched and one time he came in against the Giants with a 1 run lead in the bottom of the 9th and bases loaded and got the next three hitters out. It also mentioned the confidence that Phillie manager Eddie Sawyer had in him, often bringing him in with several innings left in the game. He had a 10 inning relief appearance (on Sept. 15-maybe it wore him out and that explains his poor performance in late Sept), a 9 inning one and 15 others of 3+ innings.