A light-skinned African-American family are "passing" in an all-white New England town. When the truth comes out, the more prejudiced neighbors demand their expulsion from the community.

Lawyer Joe Morse wants to consolidate all the small-time numbers racket operators into one big powerful operation. But his elder brother Leo is one of these small-time operators who wants to stay that way, preferring not to deal with the gangsters who dominate the big-time.