The house was formerly known as Berrystead Manor House, and the alternative form of Pavenham Bury was adopted after the house was extensively remodelled in 1842 as a square neo-Jacobean three-by-three bay house with two prominent two-storey bay windows on each front. It was extended by the addition of an extra bay with a matching bay window, probably in the early 20th century. However, there was an earlier house at the core, and it retained an 18th century staircase, perhaps dating from the short period of ownership by John Franklyn, until the end. The house was badly damaged by fire in 1960 and demolished the following year; some of the fixtures and fittings were re-used to fit out the former stables and coach house as a new house, now known as Pavenham Court.
