Same answer 
You can start by taking an inventory
Get-VM | Get-VIPermission
and then use the Remove-VIPermission to remove the obselete permission, followed by a New-VIPermission to set the new permission.
Watch out not to cut the branch on which you are sitting !
Make sure the account you use to run the script doesn't belong to the group you remove.