Preheat oven to 425 degrees. Lightly brush donut pan with butter.
In a medium bowl, whisk flour, baking powder, baking soda, nutmeg, and salt. Set aside.
In the bowl of a stand mixer, beat butter and sugar on high until light and fluffy. Reduce mixer to medium speed. Add eggs and vanilla, beating until combined. Reduce speed to low. Add flour mixture in two batches, alternating with milk, and beat until combined.
Transfer batter to a piping bag or ziploc with a hole in the corner. Divide mixture evenly between prepared pans. Bang pan several times on counter to release air bubbles.
Bake until tops of donuts spring back when lightly pressed, 12 to 14 minutes. Let cool 10 minutes, then transfer to a wire rack to cool.
To make the glaze, whisk the milk, icing sugar, and vanilla in a bowl. Add a few drops of blue food coloring and swirl around using a knife. Don’t overmix—it should look like marble.
Once cool, take the donuts and dip the tops into the glaze. Let excess drip off a bit, then set aside. Add sprinkles if desired. Enjoy!