Matt Olson has 51 HRs this year but never had a 40+ season before. Is that unusual?

No, it does not seem like it. Table below lists all the guys who hit 50+ HRs in a season. Then it shows how many they hit in their first 50+ season and what their season high was before that year. There are many cases like Olson's.

In some cases, there is more to the story. Like Ruth. His 29 was in 1919, still in the so called "dead ball era." Then he hit 54 in 1920. There was also the issue of park effects in Boston. He played there before the bullpen in RF, so it was hard to homer there for lefties. He hit all 11 of his HRs in 1918 on the road and 20 on the road in 1919. And the seasons were shorter in 1918-19 due to the war.

Belle hit his 38 in 1994 when teams only played about 115 games due to the strike. He probably would have hit 40+ without the strike.

Player	HR in  1st Yr.	Previous High
Babe Ruth	54	29
Hack Wilson	56	39
Jimmie Foxx	58	37
Hank Greenberg	58	40
Johnny Mize	51	43
Ralph Kiner	51	23
Willie Mays	51	41
Mickey Mantle	52	37
Roger Maris	61	39
George Foster	52	29
Cecil Fielder	51	14
Albert Belle	50	38
Brady Anderson	50	21
Mark McGwire	52	49
Ken Griffey Jr.	56	49
Greg Vaughn	50	41
Sammy Sosa	66	40
Álex Rodriguez	52	42
Luis Gonzalez	57	31
Barry Bonds	73	49
Jim Thome	52	49
Andruw Jones	51	36
David Ortiz	54	47
Ryan Howard	58	22
Prince Fielder	50	28
Jose Bautista	54	16
Chris Davis	53	33
Aaron Judge	52	Rookie
Giancarlo Stanton	59	37
Pete Alonso	53	Rookie
Matt Olson	51	39