Why do people label you for what you study?

I am a "Civil War" buff, that does not make me want to go out and start or encourage one. People have no place to criticize what or how you study. So long as you are learning, the material does not mean that you approve, or disapprove of it, only that your thirst for knowledge is being satisfied. Pity the people that wrote history books, their endeavors, did not make them all specialist on the subject, or encourage them to start it up again.... I admire, and think that if you are seeking answers, good for you, your mind is at least active, not stagnated by video games...knowledge replaces fear, and I do think many fear that which they do not understand....thank you