Accidental touch with always on display

Has anyone here experienced issues with accidental touch not working while the phone is in a pocket. The settings I think are causing this are double tap to wake set to on and having the always on display set to always display. Those settings and putting the phone in my pocket screen facing my leg causes the screen to wake inside of my pocket and jump around to different time stamps in the music or podcast I am listening to, sometimes pausing.