There is a weekly guild XP cap. You get GXP from questing, dungeons, PvP... etc. You also have to get rep with your guild to get access to the guild items once they are unlocked. I am not sure what the weekly cap is, as on beta the GXP is boosted to allow people to test. I think there are some posts out there on it though, but it all could change in the next 2 months.

Guild perks unlock at each guild level, things like +5% XP, more rep, 15m cooldown on hearth. A lot of the perks are tied to achievements, some of which are simply insane.