I don't think there should be a set limit, but some people listen better to people close to their age.
Older folks may listen better to pastors closer to their age or to someone who has shared similar experiences with them. The same goes for kids ministries. A lot of kids seem to take to younger pastors — like high school seniors — because they are closer in age group.