There is a fallacy here. You are not making a specific person's labour a right, but making a type of labour a right. That labour can be provided by many people, each of whom could theoretically have the right to refuse. That labour being a right means that there is some mechanism that ensures a person gets it. E.g., right to an attorney.
Also, many many existing rights require other people's labours. The right to a fair trial in the US would require a judge, a lawyer, and 12 members of a jury.