I would ditch the prong and the head halter (honestly I'd use the prong over the head halter in this scenario but neither are really helpful here). Back up several steps, he sounds like he's just not ready for walks of this nature quite yet. I would take him somewhere where there will be other people, but he's not walking. Like a baseball field with a game going on, a college campus. Just stand/sit there and don't walk. Do the LAT game as Danielle suggests. He needs to know that it's OK, you have things under control and make a more neutral or even positive association with people walking around him.
The prong was mainly so that he can't drag me away, he weighs 10lbs less than me and is very strong. Off leash, we don't have this problem at all. He's right by my side, focusing on me and will do everything I ask him to. Unfortunately, the minute he has a leash attached to him, we have this problem. We've tried with a regular flat collar (when he was about 60lbs) and he nearly pulled me down the street trying to get away from something. Having the prong or the head halter on seems to make him think he can't get away, which is probably part of the problem but I don't know what else to do to make sure that I can control him.
He spends a lot of time in the warmer weather in a park close to our house - it's very similar to what I think you're describing as a college campus, lots going on but still quiet areas. He's fine there, watches people and sleeps usually, he'll even eat there (we split a poutine quiet often, lol), but the second we get up to go somewhere (different group of people, to the bathroom, going home), he's hyper aware of where people are and is constantly looking behind him. It's like he can't relax on a walk if he's on his leash.
I guess I could start using a command like look at that (I don't want to use that because we already use look at me) at home and then follow up outside once the weather gets a bit better and we can be out longer at places like the park. Maybe I'll also see if I can get a lighter, longer leash and see if that helps too?