I could probably build a computer from scratch, but it's not gonna be a modern one with impossibly small microchips and bajillions of transistors. It's gonna be a room-sized behemoth with only like 8bits of memory that takes 24 hours to compute 1+1.
I was going to add that disclaimer, but it's also a step closer and interesting nonetheless.
I'd also recommend Breaking Taps, while he does amazing stuff in a home lab it also has the disclaimer that he's able to get it already has some super interesting but rare stuff. Electron microscope etc.
I thought I saw someone making homemade, low power processors but for the life of me I can't remember who or where.
If you only care about adding numbers, you can e.g. do that using water or marbles. You only need to build an XOR gate, an AND gate and an OR gate.
In case of water, the gate will have two inputs as water streams. They should be aligned so that when the streams hit each other, the water will flow into a cup in the center of the apparatus. When the streams don't hit each other, the water passes the first cup and flows into another cup on the bottom. Carrying the water out at the bottom is the XOR gate, carrying the water out at the center is the AND gate and both cups together are the OR gate.
Then it's just about setting up the circuit and that would be a full adder without electronics.