I mean, you probably could eventually to some extent... definitely not enough to have a conversation, but you might be able to vaguely understand someone saying something to you.
I’m to the point where I can tell when some things are poorly translated in the subs—i.e. how they could better be translated to english to convey their original meaning. And if I close my eyes I can definitely understand bits and pieces of the conversation. Anime re-uses lots of phrases and expressions, and some words are very distinctive or even happen to sound like an english counterpart of similar meaning. So I’ve learned a good amount of them from sheer repetitive exposure.