Give Clarke poll role, say Tories