Brandon,
Once you have set IC up, most configuration changes don't require you to log into your servers at all! Sure, updates to the software, but... Even then, I believe you simply need an account with local admin rights, not necessarily ICAdmin (would need to verify that.)
I'm not sure who told you that doing this stuff from the server was recommended, or why they said that!
Anyway, I recommend adding your account to the Administrators Role and then running IA, Attendant etc. on your local PC (logged in as you.) Same for other people who administer the system. Once you are doing this, then AFAIK, the ICAdmin account no longer needs interactive login rights, which is one way of getting the security folks to agree to exempt it from password change policies. (Changing the ICAdmin's password is a whole ton of work you really could do without!)
This also has the side-benefit that all Logs and audit trails will show who made the change and from where, rather than just "ICAdmin from the Server". This can help when troubleshooting.
HTH