Actor 'had affair with Diana'