It matches with almost anything. Jeans, shorts, long sleeves, short sleeves, almost any color, I wouldn't suggest wearing Khaki pants with them however. They're great for spring fall and summer but you're feet will get cold in the winter.
Sperry's are super cute with several casual outfits. I wouldn't go so far to say they go with anything though. If you are wearing a professional blazer/suit, sperry's would not match the look. You wouldn't wear sperry's with a nice dress for prom or a some other special occasion either.