People traveled the Bowery to get in and out of town during the Dutch and English days. The Bowery, Dutch for “farm” (or “road to the farm”), is how Peter Stuyvesant went to his uptown estate (around 15th Street), and it's the route George Washington took on Evacuation Day in 1784 to see the British off.
Below is the Maerschalck Plan from 1754, zooming in on the area we’re interested in. Broadway is the wide road that ends at what looks like an upright mailbox flag. The "flag staff" was a short road to Anthony Rutgers' farm (one of them), and the square “flag” was a small patch of high ground surrounded by marsh. The land was granted to Rutgers by King George II, and it was just west of a 70 acre-wide fresh water pond called the Collect. On the other side of the Collect (where the heart of today's Chinatown is located) were slaughterhouses, butchers and tanners (the people who prepared hides).
Stokes Iconography (v.3 p. 540) says,
In 1730 Anthony Rutgers, who already owned the land west of the Collect, including the "Kolchhook," petitioned for a grant of the swamp and pond, which was given him, in 1733, on condition that he drain off the swamp within a year's time. This was accomplished so successfully that the tanners about the pond complained that the water was lowered so as to interfere with their supply, and Rutgers was ordered to close up the drain for thirty feet from the Collect. The swamp lands were, however, drained and turned into meadows.
Stokes, I. N. Phelps The iconography of Manhattan Island, 1498-1909 New York : Robert H. Dodd, 1915-1928.Electronic reproduction. v. 1-4. New York, N.Y. : Columbia University Libraries, 2008. JPEG use copy available via the World Wide Web. Master copy stored locally on  DVDs#: ldpd_5800727_001 01-13 ; ldpd_5800727_002 01-19 ; ldpd_5800727_003 01-16 ; ldpd_5800727_004 01-16.. Columbia University Libraries Electronic Books. 2006. Vol 1, pl 34.
Here’s the same map with some highlighted details. The yellow line is Broadway and the green triangle is today’s City Hall Park (known as The Commons back in 1754—City Hall was on Wall Street at the time). The white dots show the “High Road to Boston” (or the Bowery, which it directly fed into it). The purple stripes to the left of Collect Pond were the marsh that Rutgers turned into meadow (which nonetheless flooded whenever it rained). The area was named Lispenard’s Meadow after Leonard Lispenard, who married Anthony Rutgers' daughter, Alice. Three streets: Leonard, Anthony (now Worth) and Thomas were all named for Lispenard's sons (though some accounts say his grandsons). For reference, the black outline shows the World Trade Center site.
Not only did Broadway end just above The Commons, it wasn't even the main route to the Bowery. The roads in blue show, from left to right: Broad Street, William Street and Pearl Street. All three crossed Wall Street (in green), and led more directly to the Bowery.
In 1765, Anthony Rutgers' mansion (in the “mailbox flag”) became the Ranelagh pleasure garden, where citizens came to escape the downtown city, enjoy fireworks and take in the beautiful landscaped gardens. (There would actually be a few pleasure gardens by that same name).
Here's near the spot today, on the high ground surrounded by the sometime-swamp-sometime-meadow. The view is looking south on Broadway, a few blocks north of City Hall, with some buildings identified. Here, Broadway divides TriBeCa to the west (right side) and the Civic Center/Court District to the east (left side).
Sidenote: What I love most about Manhattan is how history layers itself over an area, leaving calling cards in the building facades from different eras. Above are: AT Stewart’s Marble Palace, the country’s very first "department store” built in 1846 (added 3/8/2011: Actually, the Marble Palace was more of a glorified dry goods' store. It may be a question of when does a sapling become a tree, but The Historical Atlas of New York City says that "by the time of Macy's death in 1877 his store was New York's first fully-fledged department store."); the Woolworth Building, 1913 (and the tallest building in the world until 1928), constructed when lower Manhattan was transforming into the nation's corporate command center; and federal buildings for Homeland Security, Immigration, etc, built in the last 40 years.
But in the 1700s, before Stewart, Woolworth and Homeland Security, when this was a suburb of the city (barely a mile downtown), Rutgers' farm, and later the Ranelagh Gardens, were just down Thomas Street to the right. When an eastern section of Federal Plaza (to the left) was built in the 1990s, contractors uncovered the African American Burial Ground from the 1600-1700s. The burial grounds must have been evident, and perhaps even still in use, when Anthony Rutgers, literally across the street, was just starting his farm.
Here’s another view of the area from the Montresor map of 1766 (a year after Rutger’s farm became Ranelagh’s garden). It was made for military purposes--the English side. You can see the gardens, and you can barely discern the line of the “mailbox flag” from the previous map. But this map shows clearly how Broadway dead-ended just beyond City Hall.
Stokes. Vol 1, pl 40.
And the Collect Pond drained in the opposite direction too, making a virtual border of water about two miles north from the tip of the island. The words in the black circle at the right read: “This overflow is constantly filling up in order to build on.”
Edwin Burrows and Mike Wallace's Gotham describes the scene in the early 1800s this way (p. 359):
Landed and mercantile interests meanwhile complained that runoffs from the pond fed a stretch of marshes and swamps between modern Chambers and Canal that nearly cut the island in two, blocking the northward flow of population. One outlet, a sluggish stream, ran along modern Canal Street before losing itself in the swampy wooded salt marshes known as Lispenard’s Meadow, where for decades gentlemen had taken guns and dogs to shoot woodcock and snipe. To the southeast, a second outlet ran through a smaller tidal marsh, still known as “the Swamp,” and along the course of Roosevelt Street, a foul muddy alley, to the East River.Legend has it that after a downpour Native American could canoe from the East River to the Hudson, right across this channel. And in winter, if the Hudson froze, young boys would ice skate from the East River to the Hudson, and even up the Hudson into Greenwich Village by way of Minetta Brook.
Here’s another view of the area using the Viele map, showing 1600s terrain beneath modern streets. The green lines show Broadway and Canal Street, crossing at a narrow section of Lispenard's Meadow (outlined in yellow). The red roads show William and Pearl Streets leading to the Bowery (also in red), the quickest, easiest way in and out of town.
Below are the traditional neighborhoods overlaying the area on the Viele map. “Traditional” because over the past decades Chinatown has expanded to overtake much of Little Italy and the Jewish Lower East Side. Another important historic district was Five Points, which would have been centered where the blue line crosses Worth Street (under the word “District”).
Courtesy of http://kottke.org
Now for the all important crossing of Broadway beyond Canal Street as a main street (it had been a wagon road that could be used when it wasn't flooded). Stephen Jenkins, in The World’s Greatest Street (The Story of Broadway), 1911, describes the development of Broadway in this part of town. It begins with a description of work being done adjacent City Hall Park. Note: Anthony Street is today’s Worth Street. (p. 134)
(Note there was no Thomas Street between Duane and Worth (Anthony) when the Viele map was made in 1865, which begs the question, did it disappear and return? Or did someone, so many years later, think to name it for a Rutger's family member?)
You can still see this old terrain in the streets today!
This is looking east along Leonard Street from Broadway (right on the map above, down hill to the Collect Pond). The scaffolding shows the decline halfway down the street, just where the Viele map says it should be. At the bottom of the small hill is the Court District (remember when Martha Stewart and Rosie O'Donnell went to court?). The Court District was built over the Collect Pond--and Five Points.
Here’s the opposite direction looking down Leonard Street from Broadway, the edge of TriBeCa. Across a mini-plateau to a slighter decline a block ahead.
A few blocks north on Broadway there was literally a canal--a ditch--running down the center of Canal Street until the mid 1800s, when it was finally covered over. The bridge in the image below is still there, just a few feet under the asphalt! It’s speculated that the British built the bridge during the Revolutionary War; Stokes could not find any records of it actually being built by colonists or Americans!
The first image is from Stokes, which estimates the date as 1812. The second is from Valentine's Manuel, 1857 (not the year of the image). It's kind of cool to see two different versions of the same thing.
There were actually two roads along Canal Street, one on either side of the ditch.
Below is the southwest corner of Broadway and Canal in 1928, showing the National City Bank of New York. (Same era as the Woolworth Building--the city as corporate command center). The photographer is across Broadway; Lispinard Street is to the left of the bank, Canal Street is to the right (it's a very short block).
Spinning around and looking down Broadway today.
And from about the same angle as the drawings above.
The facade of the National City Bank of New York would be about here…
Today, Canal Street links the Manhattan Bridge with the Holland Tunnel, that's why there's such a disproportianate amount of heavy industrial traffic along it--most are just passing through. It's a fascinating area, marking the boundaries of a number of neighborhoods.
The Bowery has been so re-worked below Chinatown as to be effectively obliterated. A one-way road leading downtown, sunken beneath the Brooklyn Bridge ramps and running along One Police Plaza, approximates the old route, the red dotted line below.
But that's how people moved in and out of town for about the first 200 years, before Broadway crossed the canal, and when a virtual border of water linked the East River and the Hudson across the island, above the population center below.