A base 4 number system has 4 numbers in it: 0, 1, 2, and 3.
After 3 the next numbers are 10, 11, 12, 13, 20, 21, etc.
A base 10 number system has 10 numbers in it, 0 through 9.
Hexadecimal is base 16: 0-9 followed by A-F
The alien using a "base 4" number system does not have a single digit for the number "4". They represent four of something with the number "10", so for them, their four digit number system is "base 10".
Yes, https://www.mathsisfun.com/numbers/bases.html has a bunch more examples to show why the base of the number system is always represented by 10, because 10 is a short hand we use for d1*b^1 + d2*b^0 where d is a digit between 0 and base-1, and b is the base. b^0 is always one and represents the first digit at the first position. b^1 is the base, so 1*b^1 = the base. And since 10 is 1*b^1 + 0*b^0 it represents the base in any number base system.
We assume a 1 in the 10's place and a 0 in the 1's represents 1,2,3,...,10 of something instead of 0,1,2,3,10 of something because from our perspective we learned numbers in base 10 with 9 digits, but the alien learned 10 means 4 of something in base 4.
Assuming they use the same numbers then yes, 10 is 10 to us but it represents the separator of orders of magnitude. 10 in base 4 would from our perspective be 4, but ignoring the specific numbers and adapting them for the sake of the joke, the alien would write it as 10 and "base 10" would just be base 4. It's like how hexadecimal uses 0-9 and a-f to represent 16 numbers even though we don't have that many in decimal systems. The joke is a translation issue. I feel like I'm over explaining now though...
These people are explaining it well, but the things that helps the most to me is consciously de-coupling your internalized knowledge of human Numbers from what they actually represent abstractly. The numbers represent an absolute quantity.
If you notice on the comic, the alien has 4 digits on it's "hands". That's the root cause for their base number system. Ours is that way because we we have 10 digits on our hands (most of the time, some cultures do it different, but for simplicity, that's what the comic is implying).
Abstractly, adding another digit represents starting over and keeping track of how many times you've done that. So the 1 in ten says you've counted through all the digits 1 time. And the 0 means you're on the first digit of the new sequence. So all they are doing is applying that logic to a base 4 system. So a collection of 4 things is therefore represented by "one zero," 10, in base 4 because they have to, there are no more characters in that system that could do that by itself.
The weird part about it that can trip people up is that they are applying some human conventions to the comic like our script for writing numbers, which obviously an alien wouldn't do, but we have so much inherent ingrained knowledge about what a number means that it's hard to remember that.