Speaking at the Israeli-American Council National Summit in Washington, the former president expressed his concern and made controversial remarks