Wales continue their World Cup qualifiers at home to Northern Ireland on Wednesday before facing Sven-Goran Eriksson's side at Old Trafford on 9 October.
Hartson, who scored in Wales's 2-0 victory in a Euro 2004 qualifier in Baku two years ago, said: "England are three games away but we have to concentrate on the next one. We can't even think about Northern Ireland next week because we have to get the job done in Baku.
"We are expecting a tough game tomorrow and we do not want to be playing catch-up on other countries. We have to get off to a flying start, like we did last time in our Euro 2004 group games.
"I remember scoring there last time. Gary Speed got the other and Paul Jones made a very important save early on. We won and really dominated the game - that's what we must aim for this time.
"But we must not forget that when we last played Azerbaijan their football was in a mess with internal problems and no domestic matches.
Now they have sorted that out, got a new manager and will be a lot more difficult to beat."
Hartson has vowed to make negotiations over a new contract with Celtic a priority on his return from international duty.
He added: "I want to stay. I love it at Celtic and now it all depends on making sure that everything is right for both parties. The manager wanted to get the transfer window out of the way before we talked about me."
