Line a 9x13-inch baking dish with a single layer of graham crackers, breaking pieces to fit snugly.
Beat cream cheese until smooth. Whisk in pudding mix and cold milk until thick, about 2 minutes.
Fold in whipped topping, vanilla, and salt until creamy and light.
Stir apple pie filling with cinnamon and lemon juice. Chop any large slices for easier layering.
Spread half the cream layer over crackers. Spoon on half the apples and drizzle with some caramel.
Add another snug layer of graham crackers. Top with remaining cream, then the rest of the apples.
Finish with a final cracker layer. Warm caramel slightly and pour over the top; spread gently.
Cover and chill at least 6 hours, preferably overnight, until crackers soften into cake-like layers.
Sprinkle with peanuts or toffee if using. Slice into squares and serve cold.