Configuring and building the Linux kernel is actually pretty nice and easy. The main thing that’s frustrating in my experience is just if you need to add a kernel module for a new device or something. It’s not really a big deal, just like “oh, I need to enable the drivers for this”.
I often look around the kernel config, maybe I'm there to add support for something or whatever. Then I start reading the help snippet, and I understand next to nothing. Ok... "If in doubt, select 'n'". What? No, I am in doubt, but I want the feature! Don't try to scare me away :-)