Because it's toxic for the type of brexiteers who objected to the EU because it meant there were more foreigners in the country. Even "worse", these people will mostly be visible minorities.
This is a crucial constituency for the Conservative party.
People were warned in the run up that we'd have to bring in workers from somewhere and if not the EU then probably the commonwealth. I personally wasn't bothered as I feel more kinship with West Indians and Indians than most people in the EU based on my life experience.
We were relying on medical staff from India and the West Indies long before we joined the EU and freedom of movement began. Should we not have good relations with our own commonwealth?
Why does India want this? Surely skilled workers leaving India for the UK is bad for India - far from being a price for the UK, this would enormously benefit the UK. The UK will be getting skilled workers whose education is mostly at India's expense.
I think if might be about fairness.... probably still a sore point for India, especially with Britian.
Also, skilled workers may not always stay, and bring back what they learnt. Also, many send money home to relatives. We get another supply of skilled workers. It's win win.
Getting visas are easy but citizenship is hard and when you think about the workers family it's really rare that they leave but instead they bring remittance to india