Hello Darryn Chang!
When creating the contact list under the Advanced tab there is a property you can set called "Unique Identifier Column." If you add a column to your contact list that has a unique number for each contact, and then use that column in the Unique Identifier Column, then when you append to the list it will match the contacts and update the existing ones and not copy them to the bottom of the list. There is a little more information about this field in this docs link for bullet point 14.
Let me know if this doesn't solve what you are trying to do.
Have a good day!