American witchhazel (Hamamelis virginiana) is native to all of Eastern North America. Although considered a shrub, the straight species witchhazel can reach up to 20' tall and wide.
It blooms in fall - usually with yellow flowers, but sometimes orange or red if you have a cultivar. They should be grown in full sun or part shade, preferring moist, acidic, rich soil. The fruits will last through the winter and will burst open the following fall. Witchhazel would be a nice addition to a woodland garden and can be used as an informal screen.
These guys support at least 69 species of caterpillars here in SC PA.
Sources: Missouri Botanical Garden, USDA, National Wildlife Federation
Photo with thanks: @tulsabotanic Tulsa Botanic Garden - cultivar "Harvest Moon"