I'm unsure and confused about him?

Give it longer than a few days to decide what to do in the relationship. Talk to him directly and ask what's wrong.