I'm currently trying to set up specific shifts for our team where a small number of agents work in a remote office and aren't scheduled to take any on queue interactions for that day, but I can't make shifts which don't have on queue as the default. has anyone else been able to schedule anything similar?

It's not currently possible to change the default activity in a work plan, but you can configure a work plan such that schedules generated using it won't have on queue activities by having activities cover the entire length of the shift in the work plan. These two screenshots show an example of a simple off queue work plan and the resulting schedule.