Preheat the oven to 350°F. Line muffin pans with 18 cupcake liners.
In a large bowl, whisk together the flour, baking powder, baking soda, and salt. Set aside.
In a separate bowl, whisk together the vegetable oil, peanut butter, brown sugar, egg, sour cream, milk, and vanilla until smooth.
Pour the wet ingredients into the dry ingredients and stir just until combined. Do not overmix.
Divide the batter evenly among the cupcake liners, filling each about 2/3 full.
Bake for 20–23 minutes, or until a toothpick inserted into the center comes out clean.
Transfer the cupcakes to a wire rack and cool completely before frosting.
For the Frosting:
Beat the softened butter and peanut butter with a hand or stand mixer until smooth and fluffy.
Gradually add the powdered sugar, alternating with 2 tablespoons of milk. Beat until smooth. Add additional milk as needed to reach the desired consistency.
Pipe the frosting onto the cooled cupcakes.
Cut the Reese's Peanut Butter Cups in half and top each cupcake with one half.
Serve and enjoy.
Notes
Refrigerate leftover cupcakes in a covered container for up to 5 days.