He cries when you go inside because he wants to be with you. You should let him in the house, but you will need to housetrain him, he won't know that he must only eliminate outdoors, and how to alert you that he needs to go out if you don't teach him. And yes, he probably is bored out there by himself, an hour a day of human interaction is not nearly enough, which is also probably why he's excited and rough when he does get to be with you.
Forever would have been too short
NAFA: FM, MBDCh-S