Following the arrival of Presbyterian missionaries in 1801, Joseph Bullen and James Smylie organized the Bayou Pierre Church at this site in 1807. After part of the congregation formed the Bethel Church southwest of here in 1824, the remaining members moved to Port Gibson. The church was renamed First Presbyterian Church of Port Gibson in 1828. During the Battle of Port Gibson, fought on May 1, 1863, the 20th Alabama Infantry was posted here, anchoring the right flank of Confederate Brig. General Edward D. Tracy's Brigade.