About a quarter of cowboys were black. Which makes sense. After the Civil War there were probably a lot of free black men with experience riding and herding cattle, and they'd head west to where the cattle trade was located.
It's something I had heard before but didn't see an attribution (though the perjorative "boy" referring to Black men vs "hand" for white men seemed to me to give credence to the idea). The Rancho Los Cerritos Historic Site makes the same claim:
Originally, White cowboys were called cowhands, and African Americans were pejoratively referred to as “cowboys.” African American men being called “boy” regardless of their age stems from slavery and the plantation era in the South. Many southerners moved to the West and westerners would have been familiar with southern racial etiquette. So, it is no surprise that the racial issues prevalent in the North and South were also impacting the American West.
While they do provides sources for that blog post, they don't directly attribute that statement to a particular work. However, this clip from PBS is an interview with Tyree A. Boyd-Pates, curator at the Autry Museum of the American West and he also states that "cowboy" would have referred to a Black cowhand.
“"Cowboy" was first used in print in 1725, and was used in the British Isles from 1820 to 1850 to describe young boys who tended the family or community cows.”
not to say it might have taken on some racial connotations later (or, in fact, if we believe swift’s words literally), but i’m not sure it’s 100%.