The only real difference between the 4 races is you get Racial Frigate 2 (saving you about 90m training for that race) and the start location.
I always roll Caldari, these days, as you start very close to Jita, so it's easy to pick up all the books you need.

One other tiny difference is the 'free' noob ship you get when docking without a ship - Caldari get the Ibis, which has a 125m3 cargo bay; Amarr get the Impairor, with 115m3; Gallente get the Velator, with 135m3; Minmatar get the Reaper, with 120m3. When you want a Cyno alt, you can pop a cyno using 450 fuel (180m3) with level 1 cyno theory but train that to L4 and you only need 300 fuel (120m3) - so you can use the free noob ships with Cyno L4 as long as you're not Amarr.

Oh, and Amarr don't get the option to wear glasses - had a 3 day back-and-from with a GM trying to work out why one of my original chars couldn't pick shades to wear!