Heat oven to 350°F. Line a 10x15-inch rimmed sheet pan with foil, then parchment.
Arrange crackers in a snug single layer on the lined pan.
Melt butter and brown sugar in a saucepan over medium heat. Bring to a gentle boil and cook 3 minutes, stirring.
Stir in vanilla and kosher salt, then immediately pour the toffee over the crackers. Spread to cover.
Bake until bubbling and slightly darkened, 8–10 minutes.
Scatter chocolate chips over hot toffee. Let sit 2 minutes, then spread into a smooth layer.
Sprinkle with toasted almonds and flaky salt. Cool 10 minutes, then chill until set, 30–45 minutes.
Remove from pan and cut into 24 bars. Serve chilled or at room temperature.