Maybe it comes from the same pot as Trad Wives and plantation weddings. What people picture is someone walking a mentally-ill person around well-tended grounds and gently insisting that they take their meds, which is pretty much what happened, never, and is still pretty much ripe for abuse...literally from where I sit. Even people who might seem pretty radical say "Well, it's okay if *some* people get locked away forever," more than I would like.
The "good old days" notion is probably the bulk of it, yeah. Its hard to see people on the street who should be somewhere quiet and safe and cared for - but that never actually was an option, now was it? Even when that was the initial intent funding was shit and the need outweighed means just about immediately just about always.
And the abuses when no one even tried to do well or were just fucking greedy evil sadistic bastards... a boyfriend of mine worked with former Willowbrook residents. Enough said there.