Video containers don't determine codec. You can use any video codec as well as any audio codec. Which means the audio inside an mp4 can be mp3, or flac, or aac, or wav...
If you're just storing audio, just use an audio file directly. Putting the same exact audio stream inside a video container just makes it slightly more difficult to play, because them you can't use a lot of audio players to play the audio.