Look up pictures online of carpenter ants and termites - I can't tell by the picture, but those are the two big possibilities, and you should make sure to act quickly if you have insects that are eating your structure. If you are renting, it's the landlord's responsibility to make sure this is taken care of. Presumably it's not negative temps inside your house- they've wintered over along with you.

One night I was in my bedroom and I felt something tap my head. So I look up and I see a black bug about the size of a small hornet or a long skinny ant flying around. So I hit it with my slipper and took it downstairs to examine it. I have no idea what it is, but it is black, with curly antennae like a butterfly, 2 body segments and a stinger! It's march in Wisconsin so we still have negative degree temperatures here. Why would these bugs be in my room and what are they? I've encountered 3 of them already.