The Tory peer said her party needs to make it clear that ‘racist rhetoric and rabble rousing’ is not a strategy