The town is made up of three major villages: South Yarmouth, West Yarmouth, and Yarmouth Port.
According to the United States Census Bureau, the town has a total area of 28.2 square miles (73.1 km2), of which 24.1 square miles (62.5 km2) is land and 4.1 square miles (10.5 km2), or 14.40%, is water. Yarmouth is bordered by Cape Cod Bay to the north, Dennis to the east, Nantucket Sound to the south, and Barnstable to the west. Yarmouth is approximately 75 miles (121 km) southeast of Boston.
The Bass River and its tributaries make up the majority of the town's border with Dennis. There are also several smaller ponds throughout the town. The Cape Cod Bay shore is marshy, with several creeks flowing into it. The town's southern shore is known for its beaches, from the west jetty of the Bass River to Great Island, on the east and south sides of Hyannis Harbor. The town has no state forests or wildlife management areas, although there are many such natural, unprotected areas in town. The town is also the site of four golf courses, a rod and gun club, and a Boy Scout camp.