Chill a mixing bowl and beaters. Whip cream, powdered sugar, and vanilla to soft peaks. Refrigerate.
Whisk jam with orange juice and lemon juice until smooth and pourable.
Toss 3 cups raspberries with granulated sugar and 1 tablespoon of the jam mixture. Let stand briefly.
Whisk pudding mix with cold milk until thickened. Let it set a few minutes.
Layer half the cake cubes in a trifle bowl. Drizzle with half the jam mixture to moisten.
Spoon on half the sugared berries, then half the pudding. Add a third of the whipped cream.
Repeat layers with remaining cake, jam mixture, berries, and pudding. Finish with the rest of the whipped cream.
Top with reserved raspberries and toasted almonds. Cover and chill until cold.
Garnish with mint just before serving. Scoop into bowls.