If he understand the no and leave it, then that's a good start. I think you need to go beyond that and redirect too. Get him completely away from the situation altogether. Don't give him the chance to stare and growl. Remove the dog completely from the room if you have to. Change what he's focusing on.
A stare and growl can turn into an attack very quickly with an unstable dog and the wrong move. Not saying your dog is completely unstable, but it's not a good sign when he's growling at family members either.
How old is he by the way? Has this ever happened before with anyone else?
And if he doesn't want to be friends with your father, that's fine. The staring and the growling is completely unacceptable though. It shouldn't be allowed. You need to keep his focus off your dad and on you.