Milk is so calorie dense, I stopped to drink coffee after measuring it all out
Tracking calories for weight loss, I realized that milk is "heavy", and I poured a lot of it into my coffee. Now I can think of better use for these calories (in solid form) in my <2000kcal deficit plan.
It does depend on how much what milk you use, how much milk you use and how many cups of coffee you drink. I think 50 ml of semi-skimmed milk (1.5% fat) is around 22 calories. Now if you keep it to one or two cups that is pretty doable I think. But it does depend on you budget.
I try to eat between 1200 and 1500 so I do think every bit helps. Plus I only drink one cup at work so it is doable.
But I'll be honest I mostly drink no milk in my coffee anymore and just use some artificial sweetener. But I might splurge on one with frothed milk sometimes.