My answer's going to be different depending on if you're asking for barriers to use or policy issues.
As far as use goes, transit needs to be frequent enough to be useful and go to where people need it to. It sounds simple put like that but you hear a lot about once daily bus service to some of these towns and then everyone wonders why we're so car dependent.
Access is also important and often overlooked in these discussions. Because of course disabled people who have no ability to drive will use transit. Para-transit style services have such heavy limitations that in many areas it can't be used for work and sometimes not even appointments. It keeps people who otherwise might be working out of work which benefits no one.
Policy wise, funding. It's a chicken and egg situation. No one wants to fund poorly used service, and no one uses the poorly funded service because it isn't fit for purpose.