One thing I’ve never understood is I can only upvote, but some comments are greyed out as if they’ve been downvoted. How does this work?

You need a higher level of karma to downvote.