What's the best way to install a linux based OS alongside os x (mac)?

Any reason why you wouldn't go the usual route for PC (put the ISO on a thumb drive and boot from it)?

AFAIK macOS doesn't care if you add another partition in diskutil and install Linux on that through the usual live ISO's. But please make sure you:

• Don't delete your main OS (make a Backup!)
• Install Linux with (U)EFI compatibility
• Don't touch any recovery partitions
  • Don't mess with the Mac's EFI/ESP partition

You can use boot camp to dual boot. VMware fusion in macOS will even let you boot up your other OS partition as a VM from within macOS. I used to do this with windows and macOS, which was great for installing and managing steam games before booting into windows to play them.

I've been working through this on my old 2012 Mac Pro. My issue has been my graphics card. In that era any non-mac graphics card won't give you the boot-screen you need to choose which OS to use, or even to choose a USB boot for installing the Linux os. I got Refind (a super light bootloader) to work with a bit of extra tooling. You can also use opencore, but that is more challenging and makes a lot more changes.

Make a USB that you can "live boot" from, so you can test out how they work with your hardware. Generally speaking, Mint works better with AMD, and Pop! works better with Nvidea. Once you have one you like, you can use the installer on USB to set things up.

Here’s the official basic guide for Mint:
https://linuxmint-installation-guide.readthedocs.io/en/latest/

And here’s the official basic guide for Pop!:
https://support.system76.com/articles/install-pop/