Line two baking sheets with parchment. Set oven to 340°F (170°C). Pop the marshmallows into the freezer to firm up while you make the dough; this slows their melting.
In a medium bowl, whisk together the flour, baking soda, baking powder, and fine sea salt until evenly combined.
In a large bowl with a hand mixer (or stand mixer), beat the softened butter, creamy peanut butter, and honey on medium-high until fluffy, about 2 minutes.
Add the brown sugar and granulated sugar and beat 1 minute more. Mix in the egg and egg yolk just until smooth, then blend in the milk and vanilla on low speed.
Add the dry ingredients to the bowl and mix on low until the flour is just incorporated. Cover and let the dough rest for 15 minutes to hydrate; it will thicken slightly and scoop more cleanly.
Scoop heaping tablespoonfuls (about 1 1/2 tablespoons each) onto the prepared sheets, spacing 2 inches apart. Gently press each mound to about 3/4-inch thick. Par-bake for 6 minutes to set the edges.
Remove the pans from the oven. Working quickly, press one cold marshmallow into the center of each cookie. Return to the oven and bake 6 to 7 minutes more, until the edges are set and the marshmallows show light golden spots.
Cool on the baking sheet for 8 to 10 minutes to firm up, then transfer to a rack. While still warm, finish with a tiny pinch of flaky sea salt on each cookie.