The 37-year-old actor - who has three-year-old son Flynn with Miranda - was later spotted dancing, though tried to go unnoticed in a large fedora hat.
The source added: "He was breaking it down dancing."
Orlando was also overheard telling pals he is planning to move out of his apartment in trendy Tribeca.
Following their separation after three years of marriage, Orlando and Miranda - who is rumoured to have started dating Australian businessman James Packer - have pledged to remain friends for the sake of their son.