The wide concern may help explain slipping ratings for Republican presidential contender Herman Cain, who is fighting allegations that he made unwanted sexual advances when he ran the National Restaurant Association in the 1990s.
Overall, about one in six Americans say they have been sexually harassed at work, including 24 percent of women and 9 percent of men.
The percentage of women saying they have been sexually harassed has fallen from 32 percent in a 1994 ABC News poll, a shift driven by younger and more highly educated women.
Seventeen years ago, nearly four in 10 women ages 18 to 49 said they had been sexually harassed at some point. Now, one in four says so. Similarly, 25 percent of college-educated women in the new survey report experiencing harassment, compared with 42 percent in 1994.
There has been a corresponding dip in the number of men younger than 50 expressing concern about being falsely accused and a big drop in the number who think they may have harassed unintentionally. Only 7 percent of men younger than 50 report acting in a potentially inappropriate way, down from 27 percent in 1994.
One number that has tilted in the opposite direction since the 1994 survey is the percentage of people experiencing harassment who say they reported it.
Still, nearly six in 10 women who said they have been harassed said they never reported it. Asked why, 29 percent said they didn't think it was important, 22 percent said they were concerned about the consequences and 19 percent said they didn't think it would do any good. About three in 10 said the reason was something else.
The poll was conducted Nov. 9-13 among a random national sample of 1,018 adults on conventional and cellphones. It has a margin of sampling error of 3.5 percentage points.