"No one should be duped into believing they are mainstream conservatives," Jay Kaiman, the league's Southeast director, said of the Council of Conservative Citizens.
The league said Monday that Buddy Witherspoon of Lexington should resign from the council. Last week, RNC Chairman Jim Nicholson urged Republicans to withdraw. He directly appealed to Witherspoon.
Witherspoon, an orthodontist from Columbia, refused the league's request just as he refused Nicholson.
He said group members are concerned with conservative and Southern causes, especially the right to display the Confederate flag.
The Anti-Defamation League, founded in 1913, fights anti-Semitism.
Senate Majority Leader Trent Lott, R-Miss., and Rep. Bob Barr, R-Ga., have both distanced themselves from the group and its views after reports that they spoke at council meetings.