Place milk in a small pot and heat just shy of simmering. Add tea bags and steep for 5 minutes. Remove tea bags and squeeze as much of the milk out of them as you can. Remeasure milk and add additional milk to get back to 1 cup. Let cool to room temperature.
Using an electric mixer, beat together butter and brown sugar.
Add eggs one at a time, stopping to scrape down sides of the bowl. Add vegetable oil and vanilla extract.
In a medium bowl, combine flour with baking powder, salt, nutmeg, cinnamon, and baking soda.
Add flour mixture to mixing bowl, alternating with milk. Do not overmix.
Scoop batter into 12 muffin cups lined with cupcake liners. Bake at 350 degrees for about 15 minutes.
To make glaze, beat cream cheese using electric mixer. Add 1/2 cup confectioners sugar and 1 tablespoon sweet tea. Mix until smooth. Add another 1/2 cup confectioners' sugar and 2 tablespoons sweet tea. Beat until smooth. Add more sugar if you want a sweeter glaze and more tea for a thinner glaze.
Spread glaze on muffins. Glaze should be refrigerated. If you are not going to eat the muffins all at once, do not glaze the ones you are going to save so you do not have to refrigerate them.