The early history is not clear, but by 1700 Fertot was an oratory, maintained by the owners of le Marais. Razed to the ground after 1790, the house was rebuilt in a different style on the original foundations some time during the 1840s by the comte de Moncorps, who also commissioned a local but highly regarded paysagiste, Paul de Lavenne, comte de Choulot, to re-work the landscaping.

Paul de Lavenne, comte de Choulot, was born on the 31st of January 1794 in Nevers, but spent most of his youth as an exile in London. He introduced the concept of the parc agricole to France in the 1840s and during his career designed about 280 parks in Europe, mostly in France. By 1858 he was designing the garden-city of le Vésinet, to the west of Paris. More information about the comte de Choulot can be found in the book Paul de Lavenne comte de Choulot Paysagiste (1794 – 1864) editions du CAUE de la Niévre.

The present château was rebuilt in the 1840s on the foundations of the previous house, which is thought to have been an oratory or a monastery. Some of the farm buildings date from the 17th century.

L'art des Jardins, ou etudes theoriques pratiques sur l'arrangement exterieur des habitations; was an essay published in 1863 authored by the comte de Choulot mentioning the family de Moncorps as owners of the chateau de Fertot at the time.