There was a definite jealousy thing going on and why wouldn't there be? He said it later. He never had the crusts cut off of his PB&J.
This specific memory of Dean's was new information to Sam, as affirmed by Sam's observation that Dean had been cleaning up Dad's messes prior to Mary's death and Dean's comment that their marriage was perfect only after Mary died. Up until now, canon seems to suggest that Dean could never talk to Sam about Mary without getting upset. So I don't know that Sam's had an opportunity to build up resentments of the specifics of what Dean had with their mom so much as he was trying placate Dean's pissiness over what he observed to be Sam's "happy" memories.
No doubt Sam has had an overarching longing for the "normal" life he thought Dean had for four+ years, so it's not a wonder to me that Sam's memories would be those times when Sam made or sought out his own "normal". However, Sam has some new information on Dean and John that he never had before.
Interestingly, to me, this exercise in heaven was more about seeing who would overcome and push through and who would break. Initially, Dean seemed to be the object of emotional abuse (exposure to what he'd had and lost in innocent wee!Sam and presumably his own moments of youthful innocence; his loving memories of Mary tainted by the memory of his father having left them temporarily; Sam's "happy" memories that did not involve Dean; and of course, Zachariah's psychic torture using Mary). And yet, it appears to me God had made a clean slate of the earthly trespasses of both boys. Now it's up to them to either push forward and save the world or not.
In the aftermath, Dean and Castiel appear lost, while for the last two episodes, Sam seems to be asking what's left of his family to not give up. In DMDWP, he says to Bobby, who just stares at him, "But you're going to be all right. Right, Bobby?". In this last episode, he entreats Cas to "Wait!" before a despairing Cas disappears. He tells a despondent Dean they'll find a way to stop all this. "You and me, we'll find it." And Dean walks away tossing Sam's gift to him into the trash.
Maybe it is Sam's responsibility now.