North Salem's roots reach back to the 1730s, when European settlers began building a farming community out of the forested wilderness of the Titicus River Valley. The area was originally part of a larger settlement called Salem, itself divided between the disputed "Oblong" territory along the Connecticut border and Delanceytown, part of the Van Cortlandt Manor land grant. Salem eventually split into Upper and Lower sections, and in 1788, the Town of North Salem was formally incorporated, combining Delanceytown with Upper Salem.

The Revolutionary War left its mark here too: North Salem played a role in the capture of British spy Major John André, whose exposure helped foil Benedict Arnold's plot to surrender West Point to the British.

By the 1840s, the railroad had reached North Salem, and the hamlets of Purdys and Croton Falls grew up around new train stations, giving dairy farmers a faster way to reach market. The town's entrepreneurial streak showed up early: in 1830, three North Salem men founded one of the first circus syndicates in America, inspired by a neighboring showman's success touring an elephant. Decades later, when New York City expanded its water supply in the 1890s, new reservoirs displaced entire communities in town, including much of the hamlet of Purdys, which was relocated or demolished to make way for the rising water.