Turns out that mashed potato with brown sugar basically just tastes like brown sugar. It's actually pretty good, which makes sense as it's a neutral flavoured carbohydrate with a sweetener.
I tried it last night with a little bit of the mash I was making for dinner - took a small bit out and added caster sugar & molasses (because I don't actually have brown sugar, but that's functionally the same thing). It was a bit over sweet, but I think a better ratio of potato to sugar would improve it.