Answer :

Decline of Roads, Different unfriendly hordes held different regions  instead of one giant peaceful empire also disease was rampant and lack of surplus basically meant people didn't produce enough to go on a trip.
MasterMiyagi
People felt much less safe, with small countries and bands of barbarians all over, that and the decline of upkeep of roads