I don't think you realize how slow Bluetooth is, and how little delay it takes to really throw one off. You're probably right that a millisecond would be manageable. But Bluetooth audio can easily be a half second off as well as not being consistently off by the same amount of time.