You have the option to remove any Voices that you have added to your workspace from the marketplace. Simply click on the Voice tile of the desired Voice, and you will be taken to its details page where you can remove it from your workspace.
Please navigate to the Market Place section and expand it. If none of the AIs in the workspace are utilizing this particular voice, the 'Remove From Workspace' button will be activated. By clicking on this button, the voice will be removed from the workspace and returned to the Voice List page.