Scrub potatoes and cut larger ones in half. Place in a pot, cover with cold water, and season the water generously.
Bring to a boil, then simmer until just tender, 12–15 minutes. Drain well and return to the warm pot to steam-dry.
Toss warm potatoes with red wine vinegar, 1 tbsp olive oil, and a pinch of salt. Let cool 10 minutes.
Grate the cucumber on the large holes. Squeeze out excess liquid with a clean towel until nearly dry.
Whisk yogurt, remaining olive oil, lemon juice, lemon zest, garlic, dill, mint, salt, and pepper in a bowl.
Fold the drained cucumber into the yogurt mixture to make the tzatziki dressing.
Add red onion and capers to the cooled potatoes. Pour over the tzatziki and gently toss to coat.
Chill 30 minutes to meld flavors. Taste and adjust salt, pepper, or lemon. Garnish with extra dill to serve.