Spurs were also aggrieved by two appeals for handball against Lee Bowyer inside the penalty area which were not given.
Gardner said: "I didn't think it was a push. The ball came over, Shola went underneath it, but I backed off and headed it. Then he blew the whistle.
"I thought he had blown for handball on the line, but he said I had pushed Shola. My hands were nowhere near him.
"The decisions just didn't go our way. There were a couple of handball shouts as well. The close decisions are everything in these type of big games and, if you don't get them, you end up losing.
"It just seemed like everything was going their way. It's hard enough coming to these grounds as the away team, but to not get any decisions as well, it hurts a lot. This was our big chance to get to a final."