Where is the property line, is your building right on it so that it would be considered something like a shared fence ???

Absolutely not. You may have to sue him to make him remove it, but he can't use your building without your permission. If your wall sits directly on the property line, he would legally have to build a wall of his own next to your wall with zero clearance.

uh no. does that make any sense? think about it. and we need more details.