Romanes eunt domus
What's this then? People called 'Romanes' they go the house?!
It says Romans go home
Just to ruin anyone's plans in case they were thinking about time traveling back and being a roman, those were the well-preserved richest people's houses. I'm certain most people lived in hovels made with mud clay and straw.
You're correct about this being rich people's houses - I note it's an upper class house in the title - but a poor Roman citizen would be more likely to live in an Insula, an apartment made of wood or brick. If in a rural area, small wood cottages were common for poor farmers, though in some particularly rocky areas stone is known to be used. The kind of hovels you're thinking of would have been something more common in unromanized areas of Spain or Britain.
If you feel a compulsion to lose yourself in a rabbit hole of ancient Roman domestic architecture, Khan Academy may satisfy that need: https://www.khanacademy.org/humanities/ancient-art-civilizations/roman/x7e914f5b:beginner-guides-to-roman-architecture/a/roman-domestic-architecture-domus
