I second the switching off of grain-free food. Once you're feeding a limited ingredient diet, see what works and what doesn't in terms of protein. It is so often the food...and so often due to grain. If he has allergies this severe, I would also highly recommend seeing a dermatologist and having an allergy test done. It might not give you anything to work with, but if it does it is often valuable information.
My guy has SEVERE grain allergies. He's also allergic to chicken, salmon, and lamb. I feed Pinnacle Limited Ingredient and it has worked well for us. When his food issues were under control and he continued having seasonal issues, we had the allergy test done and determined that he was also very allergic to the fungus that is frequently found in grass in the spring and rotting leaves in the fall. There was no cure for this obviously, but it made management a lot easier.
My mother ran out of his food and didn't realize I had left her another bag...so she started feeding him her dog's grain-based food. Within a couple of days he started developing severe rashes and bleeding sores on his lips. It really can be that severe. Unfortunately, most vets aren't really equipped for dealing with it besides saying, "give them benadryl."
In the mean time, try claratin OTC-- it's more expensive, but it has less of a drowsy effect. Oatmeal baths are also very helpful. Veterycin has been very helpful for dealing with the sores that arise, and septiderm or the doggie hydro-cortisone lotions also help with irritation. I've had great luck with emu oil as a general skin conditioner. Also keep an eye out for yeast infections, especially in the skin between the limbs and body (if you ever see what looks like dirt crusted in the fur, or smell a funky odor, it is often that). I use a benzoyl peroxide and antifungal combo shampoo when he starts to get welts, and it also has soothing ingredients (hydro-cortisone and oatmeal) in it. A cider-vinegar rinse can also be helpful for yeast issues, especially in the ears. Don't over-bathe! You could easily cause more irritation, especially with chemical shampoos.
Also keep track of his ears and eyes. Whenever Echo is having an allergy attack, it very frequently cascades to goopy, irritated eyes and/or goopy, irritated, yeasty ears.
Good luck! It's a tough, often very frustrating road... but don't just settle at "give him benadryl" and assume there's nothing else to be done
Out of curiosity... you said he's not itching much... did they do a skin scrape for demodex?