It will mostly depend on the thickness and drape of the fabric, but for a really crisp result you will likely need some interfacing on the collar and button placket (maybe some others). What kind of fabric do you have?
It's just a generic cotton fabric that you find in the solid color cottons at joann's (not quilting cotton). It definitely isn't thick or maintains its shape. I am planning to use interfacing on the collar and sleeves, but I'm not sure if I need it on the torso area. I'm worried about it looking more shirt like than vest like.