Is there any way to see the amount of upvotes and downvotes a comment/post has, and if not, can this be a feature request?

Do you mean the individual upvotes or downvotes, or the current score of the post? For me I can see the current number of upvotes just fine in the app (top right corner of the comment).

If you want to see the individual upvotes and downvotes, I think this is not possible, but not because of Boost. I think this is a limitation of Lemmy. So in this case you would have to request the feature in the Lemmy development channel :)