Black people in America during the 50's were not treated well. They were hung, beaten, had their homes and communities destroyed often. There were also separate facilities such as schools, restaurants, and fountains. Many places were specifically "White Only." There were Sundown Towns that pretty much made it illegal to be black after sundown. There were books (green book) written and circulated to make it easier for POC to find hospitable places to travel.
This is not to say everything is all fine and dandy, but POC are usually not the ones dreaming about "the good old days."