Am I wrong for going out tonight with his daughter coming?

WELL depends on the situation. If he needs the help because he works too then your being selfish. If he didn't say anything then you didn't do anything wrong except prove to your kids that your more important to yourself then your family I guess.