Is it rude to ask a man is he well endowed?

I think inappropriate. Your brother should have corrected her but he felt comfortable speaking with her about it apparently.