2003 Chevy Side Mirrors

I replaced both the driver's side and passenger side mirrors on my 2003 Chevy Tahoe but can't figure out how to transfer the mirror backing to the new mirror. Can anyone help? Are you just replacing the glass or the assy? Just the glass isn't that hard to do,

I replaced the whole side mirror unit but need to take the covers/backing off of the old unit and put them on the new unit. I can't figure out how to get the cover off of the old unit without damaging them.