That is exactly correct, yes.
Regarding it being part of a bigger app, I would still try to do as much as I can in Attendant and only call Handlers for specific things. Of course, it's your app! Take, for example, getting data from a caller. In Attendant, that's a single operation that can play the prompt, verify the data entered and assign the value to a Call Attribute. In a Handler, you are going to have half a page of code to do the same thing. In your case, you are playing a prompt and then capturing a key press, checking for validity, re-prompt if there is no, valid entry (but count the number of re-prompts so we don't have an endless loop) and so it goes on. Attendant, it's a single "Play Menu"....
If I were writing something like this, I would probably end up with many separate handlers, all being called from different parts of my Attendant IVR. This may
seem more complex at first, but in the long run it will probably be easier to maintain. Also remember that Attendant often gets enhanced in new releases so, down the line, you may be able to remove some of those handlers if Attendant gains the functionality you had to use the handler for.
Anyway, like I said, you app, so your call - this is just my opinion and others may well disagree with me. (Of course, they would be wrong, but that's another matter! :p)