If I am 5'6'' and 165 is the S too small?...
If I am 5'6'' and 165 is the S too small? This looks like a fairly large hoodie from the picture
I would probably go medium. However this depends on your style. You could pull off a small I just think the medium will be a better fit.