While some men may not care about their clothing, others thrive in men's designer shirts. For most, it is simply a matter of priorities. Men's designer shirts are more expensive than others, but the quality is often worth it.
The real difference is not the label, but the material and cut. Designer shirts often last longer and cut to allow more freedom of movement. Wear a shirt in which you can truly feel comfortable.