I had kind of the opposite happen to me (but still very similar)
I grew up in the suburbs right by a large city. I hated it growing up. I spent as much time as I could away from it by spending all my time hiking, kayaking, and at the barn with the horses. I was "weird" to all my fellow classmates, and felt like an outcast.
When college started, I moved to the northern part of the state. Other then the school... it was all farm land, and forests. A very quiet, clean, and a traditional southern town. I fell in love. I was surrounded by the horse world.... western, trail, eventing, jumpers, hunters, fox hunting at huge estates, racing, etc. I mean, literally everything you could think of. I was at an awesome university there, had a great job at the best equestrian center I've ever been to... working with imported warmbloods, wild mustangs for the Mustang Makeover, learning about the breeding world of horses, also helping with sales horses. In return, I had one of the best mentors and trainers here in the US. I felt like I was in the right place... I fit in. People were actually like me. A fun time for them was a bonfire out in the woods with a few good friends... or a trail ride down at the national forest. Not shopping at the mall, or going to get drunk at the clubs downtown. I was living life very comfortable and felt accomplished.
Well.... a few years later... I got hit with the "home" bug. Felt I needed to move back. I missed my family, friends, my boyfriend at the time was back home, I had my dogs at home.... and I felt like I missed the town. I spent 18 years there. No doubt it was "home". My yearning to be "home" again made this city seem WAY more then it really was to me. I remembered all the good (magnified them), and forgot the bad.
So I packed my stuff and moved back in a hurry... leaving everything I had worked so hard for behind. It was great for about a year. After that, I would have given anything to go back to my college town. I visited every few months, and literally would break down watching the town pass me as I traveled back home. I felt like part of me was still there, and every time I left it was being ripped away from me again. I hate this city, I don't like the people, the traffic, the crime (even though we live in a great area... there are still people who cause trouble), and the "go go go!" lifestyle makes me miserable. I can't stop to enjoy life, or a car ride... there's barely any land here for trails or farms, owning horses is nearly impossible here... and yet again, I'm the "outcast" at work and with friends. However, between the house here that we own, and my SO's career.... we are stuck here. So I'm trying to just learn to live with it.
With your situation, I would definitely say make sure it's not just the "home" bug getting you, and that you truly want to be there for your happiness. From what it sounds though, you're like me..... I know I am happier in a small town out in the country.... even if it would mean I would lose some money and take a cut in my job, I would. In a heartbeat! If I was in your situation, I would move back and leave the city life behind. Happiness is the most important IMO.
Good luck with your decision!