I guess the evidence would come in the history of areas or sites where one group displaced another, perhaps leaving signs of a takeover. I have seen documentaries discussing the differences of the species, and how ours wasn't the physically stronger, but our brain enabled us to plan and communicate better in a conflict or attack. I don't know if that was based on evidence or just speculation using the characteristics we know of the two species.
Nothing concrete I don't think. But we do have many thousands of years of racial violence in our collective history so it's not a huge leap of a guess.