A suit is the way to go, then. Besides, it is waaay better to show up overdressed on your first day than underdressed. First impressions are really hard to erase, and showing up looking like a slob gets your career off to a really bad start.. Congratulations on landing a job!
Yes, people who work in professional offices must dress professionally every day. An adjudicator is a position of authority so you must look authoritative to clients and like someone who is serious about the rules and regulations required by the job. You can't expect others to take you seriously about regulations when you can't follow a simple dress code yourself! You want to be taken seriously so you must dress in a serious way that tells the world you are serious. If you dress in t-shirts and cargo pants or dockers or jeans no one will take you seriously and people won't see you as an authority figure.