why the hell did Mal entrust the unconscious Simon and River to Jayne?
Yeah, Morgana, that is one of those moments that require a fair bit of handwavium to get through, because the real answer to your question is, "Because otherwise the action would be over in 20 minutes, and the rest of the episode would consist of the crew spending their ill-gotten gains at some resort moon. And while everyone in bathing suits would be very scenic, probably would not result in the drama and character development of Mal deciding not to space Jayne because he showed a semblance of shame for his actions.
Alternatively, it could be presumed that Jayne's reading level would make him a poor choice for identifying the most profitable drugs in a timely fashion - Mal and Zoe were working fast there, while he would have still been trying to sound things out - making it necessary to take the chance on him because he couldn't get the other part of the job done, and there was no one else to fill the spot of watching out for the Tams.