There should be a semicolon punctuation for exclamation and question marks.
A difficult part of writing for me is when a single sentence--especially dialogue--contains two tones. It sounds best as a single sentence, but ending with a period, or alternative punctuation, looks wrong. As well as this, using two sentences also looks wrong.
I can't think of a great example right now, but I know I've wanted punctuation that doesn't exist before. I've had moments where it would have been so useful to have a ";!" and a ";?" mark.
I use these often, but it doesn't solve a sentence being split into two distinct tones. A "!?" mark is useful for when you're effectively shouting a question, but it's not the instance I'm referring to.