This dog has been failed by everyone. You aren't failing him -- you're giving him his best hope. You might have taken on more than you thought, but instead of feeling defeated, just decide that it's going to force you to expand your skill set!
First, shelter research has shown that it takes at least 3 days for the cortisol to come down after the shift to a new environment -- so he's chemically been in fight or flight mode the whole time you've had him. You can't train through that. You can't even do a fair temperament assessment in that period.
It's not too late to start him in a 2-week shutdown either -- I would!
Dogs like this just take time. Years of living in the horror of a hoarding situation means that you've got a lot of learned fear -- he expects bad stuff to happen, constantly. Counter conditioning is a hugely important technique to learn about -- Patricia McConnell's stuff on her website and in her books is all very approachable (her 30-page booklet called "The Cautious Canine" should be mandatory reading for dogs like this).
Her blog posts about fear are collected here:
I would also recommend reviewing the resources at Fear Free Pets -- an organization that trains vets, techs, groomers, and other professionals techniques to minimize stress in frightened pets. They have a site just for pet owners too:
I'm also a big proponent of clickers for dogs like this -- they haven't ever experienced them, so there's no negative association, so it's a blank slate for training. Reaching for a dog that's been abused, even with a treat, can be overwhelming, so learning to mark positively without reaching for the dog is sometimes helpful, especially in the beginning. It's a way of communicating within the dog's narrow limits right now. A clicker-based trainer could be very helpful in getting you started.
We have 2 in the rescue now that flailed like fishes out of water the first time we leashed them, they busted out of crates, destroyed the foster's home, and got into everything because they had never been inside a house. They were sweet but WILD. One of them has come so far in 3 months that he's a candidate for a PTSD service dog program right now! They were out at a pet adoption festival last weekend and were well behaved, awesome dogs -- one was used to demonstrate an obedience drill at the festival. Their foster dad works full time, is single, and is fairly young -- he just figured it out, using the resources available to him.
The rescue has another dog right now that was terrified of all humans (severely abused). I wasn't even sure we'd be able to get him to an adoptable point when I pulled him, but there was a foster home with experience with these dogs that promised they'd be in it for as long as it takes because they like helping this kind of dog. So I pulled him out of the shelter and hoped for the best. After 4 months, he's become a dog that just graduated from a group obedience class where he successfully now works around other dogs and strange people! For him, just being in a group class was impossible at first -- he had to attend class a parking lot away because the people scared him and he'd lie flat on the ground and shut down. He's made phenomenal progress under this trainer, and he's now participating in class. Again, it took about about 4 months to get him here. He's still a shy dog, but he's not debilitated by fear. He's looking to his person for a cue on what's safe and okay, and he is transforming into a lovely companion.
One thing we added for him that seems to help is a prescription for a new anti-anxiety probiotic called Calming Care. It is a proprietary strain that you can't purchase in regular supplements. It has to be purchased through a vet. It's not going to hurt, and the research on it is very interesting.
If you have access to a board-certified veterinary behaviorist, I would encourage you to consider a consultation. There may be some meds to take the edge off his fear to help get access to his focus for more successful training work, but I wouldn't trust a regular vet for that. (The board-certified vet behaviorists have a DVM plus years of additional training, research, and residency, and earned diplomat status in the ACVB -- but there aren't very many of them.)
You're not going to solve this in a week, or a month. You'll make some progress in 3 months, some more in 6 months, and in a year, if you are lucky, he may finally feel like a dog you're really proud of. Someday you won't even recognize it's the same dog. The timetable though has to be set by the dog, not by your expectations.
Working with dogs like this is a challenge and also a huge opportunity to grow in your ability to communicate with dogs. If you let him, he will teach you a lot about yourself too. It's humbling, involves lots of days that feel like set backs, and lots of self-doubt sometimes -- but when you get there eventually, the feeling of pride is incomparable because you earned it together. I think dogs like this make us better people by forcing us to set our own egos aside and listen to the dog.
One of the secrets of rescue is that it's hard, full of disappointments, and often soul-crushing. I feel like you aren't really a "rescuer" until you've had a dog like this make you sob like a baby at how much you don't know -- and then we put on our "big kid pants" and find ourselves trainers and mentors to teach us what we need to learn to help the dog. Being a rescuer isn't posting "save this dog" on Facebook when there's a sad thread about a hording case...lots of useless people do that...the real work is what you are doing, and it's HARD. Most people who've been active in rescue have been EXACTLY where you are, either with a foster dog or a personal dog. All those experienced rescuers who seem to know so much only know so much because they went through this same thing years ago and let it be a chance to learn and grow, rather than an insurmountable obstacle.
This dog has no one else -- he's with you because he was in a shelter languishing. There's no magical pool of people out there waiting to help or adopt dogs like this: if you give up on him, who better than you is he really going to find? People take dogs like this and put them on chains to guard property, but not many people want to bring them in their homes like this. (If you must rehome him, please at least try rehabilitate him for 3-6 months to get him to a more adoptable state so that he can attract more potential interest so he'll have better odds of getting a worthy home.)
You haven't failed until you give up -- and you haven't given up. Vent. Cry. Laugh. Pray. Whatever gets you through. You will be tested, you will struggle, you will learn, but it *will* get better over time if you have patience, set your ego aside, find local resources to support you, and ask him only to do only as much as he can do right now. Tomorrow it might be a little more than he can do today. With enough tomorrows, you'll get there together.