throw the baby out with the bathwater
US /θroʊ ðə ˈbeɪbi aʊt wɪθ ðə ˈbæθˌwɔtər/
UK /θroʊ ðə ˈbeɪbi aʊt wɪθ ðə ˈbæθˌwɔtər/

1.
to discard something valuable along with something undesirable
:
•
We need to reform the system, but be careful not to throw the baby out with the bathwater.
•
In an effort to cut costs, they might throw the baby out with the bathwater by eliminating essential services.