The compte de Montcorps’ only child Emma married the comte de Savigny, who subsequently added Montcorps onto his own title. Two generations of the de Savigny family enjoyed the estate, but the inheritance was complicated, and during the 1950s the property passed to a d'Armaillé godson of the Countess, who died, seemingly in a car accident in New York, before he ever moved into the house. An insight into the lives of the Savigny de Montcorps' lives in the 19th century can be read in the biography La Reine des Fleurs by Alain de Savigny about a relation who used to visit Fertot at the height of its glory in 1880s, when she was developing a large perfume factory in the south.

The chateau was abandoned in the early 1960s and most of the things which were left there were stolen. However, several photographs and documents, as a testament to the previous ownership, were found on the domain.