"THE STATE CAPITOL BECAME THE CHURCH"
WIKIPEDIA: Early religious use 
For several decades, beginning when the federal government moved to Washington in the fall of 1800, the Capitol building was used for Sunday religious services as well as for governmental functions.
The first services were conducted in the "hall" of the House in the north wing of the building. In 1801 the House moved to temporary quarters in the south wing, called the "Oven," which it vacated in 1804, returning to the north wing for three years. Then, from 1807 to 1857, they were held in the then-House Chamber (now called Statuary Hall).
When held in the House chamber, the Speaker's podium was used as the preacher's pulpit.
According to the U.S. Library of Congress exhibit Religion and the Founding of the American Republic:
It is no exaggeration to say that on Sundays in Washington during the administrations of Thomas Jefferson (1801-1809) and of James Madison (1809-1817) the state became the church.
Within a year of his inauguration, Jefferson began attending church services in the chamber of the House of Representatives. Madison followed Jefferson's example, although unlike Jefferson, who rode on horseback to church in the Capitol, Madison came in a coach and four. Worship services in the House—a practice that continued until after the Civil War—were acceptable to Jefferson because they were nondiscriminatory and voluntary.
Preachers of every Protestant denomination appeared. (Catholic priests began officiating in 1826.) As early as January 1806 a female evangelist, Dorothy Ripley, delivered a camp meeting-style exhortation in the House to Jefferson, Vice President Aaron Burr, and a "crowded audience."