Would you be hesitant of dating a woman who has had weight problems much of her life?

I know in an ideal world, physical attraction shouldn't matter at all, but the fact it that it does. A woman wouldn't date the hunchback of Notre Dame because he's nice. Some might be hesitant because she might gain the weight back that she lost, and then the physical attraction would be gone.