Out of sight, out of mind

Undoubtedly, but it won't be public.

Depends on how you define VPN.

First, lets assume they have internet there. Not satellite internet. Literally connected permanent internet connection. I don't think this is a safe assumption because it's a literal island. Second assumption, we're talking about IP geolocation and only that. Not other methods of geolocation because then VPN is irrelevant.

Can you setup a VPN on Little Saint James island, connect to it, and use it? Yes.

Can you get an IP that when you geolocate, it will say: Little Saint James island? Extremely unlikely. So much so that I would say no.

Would an ISP actually assign a block and geolocate that to that specific island and nothing else? Probably not. What does that mean? It means the IP address that gets assigned to a more general purpose "Virgin Islands" kind of geolocation rather than a specific place.

Take this example. Houses with street number 1 to 100 is in queens. And 101 to 200 is in Brooklyn. And ISP decides, we have blocks of 64 IP assignments. So it gives houses 1-64 a Queens geolocated IP. Then houses 65-128 also Queens geolocated IP. And the rest is geolocated to Brooklyn. Then if you lived in #120, your IP will say Queens even though you live in Brooklyn.

That island is TINY. And it's likely a single subscriber of internet. It's a single owner and might get a single IP address, why would you even assign multiple? There's no need. And there's virtually no chance any ISP is gonna bother to geolocate a single IP. Ain't nobody got time for that. Even my example of 64 is just an example and is incredibly high fidelity that's rarely done.

IP geolocations could even be entirely wrong and say it's Chicago when it's Queens. This kind of stuff happens all the time for various reasons.

IP geolocation is not an accurate thing to begin with. ISPs will assign large blocks of IP to a specific geolocation. It will be shared and it doesn't even technically have to do with physically same region. It doesn't have to be correct. They just want it somewhat accurate.

Can you ask the ISP to give you a larger block of IP and have it geolocated specifically to that island and do everything OP is implying? Yes. That's exactly what businesses like data centers do. But this is just an island. There's really not much reason to do this.

So to answer the question more simply. Even if you set up a VPN there, it won't say it's there.

Who the heck is that? I thought this was a meme of world famous pop sensation Joji who definitely never made edgy 2010s YouTube videos