Make the filling: In a medium saucepan, combine 1½ cups of the blueberries with the granulated sugar, lemon juice, lemon zest, and a pinch of salt. Set over medium heat and cook, stirring, until the berries start to release their juices, 3–4 minutes; lightly mash some of the berries with a spoon for a jammy base.
Stir the cornstarch and water together in a small bowl to make a smooth slurry. Pour the slurry into the saucepan, stir constantly until thick and glossy, 1–2 minutes. Stir in the remaining 1½ cups blueberries and simmer 1 minute more. Scrape the filling onto a shallow plate to cool quickly to room temperature, then chill 10 minutes while you prepare the pastry.
Prepare the pastry: On a lightly floured surface, roll each sheet of thawed puff pastry to about a 10×12-inch rectangle. Using a sharp knife or pizza wheel, cut each sheet into 4 equal rectangles (8 total). Beat the egg with the milk to make an egg wash.
Assemble: Working one piece at a time, spoon about 2 tablespoons of filling in a strip along the center of the short half of each rectangle, leaving at least a 1/2-inch border. Brush the exposed borders with egg wash, fold the empty half over the filling to form a rectangle, and press firmly to seal. Crimp edges with a fork.
Transfer turnovers to a parchment-lined baking sheet, spacing them about 1 inch apart. Cut two small slits on top of each for steam. Brush lightly with more egg wash and sprinkle with coarse sugar. Freeze the tray for 10 minutes to firm up while you preheat the oven.
Bake: Preheat the oven to 410°F with a rack in the center. Bake the turnovers for 18–22 minutes, or until puffed and deep golden brown. Let cool on the pan for 10 minutes, then transfer to a rack.
Glaze and serve: Whisk the powdered sugar, milk, vanilla, and optional lemon zest until smooth and drizzly. Adjust with a few drops of milk if needed. Drizzle over the warm (not hot) turnovers and serve.