OP, these people are talking loud enough for you to hear them through the walls, but they stop whenever someone who might be a witness to the harassment comes to your apartment? How do they know when you have a visitor?
Since you like the apartment, but the landlord won't get involved, you need to contact your case manager and have them advocate for you. I know it can be a pain to get other people involved. But you need help and your case manager knows your history and the particulars of your situation. It isn't fair to you and your desire to live independently to have no help, and to end up having an episode because of it.
Contact your case manager.