In a medium bowl, combine Graham-cracker crumbs, gingersnap crumbs, chopped pecans, and butter. Press into the bottom of a 9x13-inch baking pan.
Place in oven and bake for 10 minutes. Let cool.
In a medium saucepan, combine 2 3/4 cups milk, mashed sweet potato, and 3/4 cup brown sugar. Cook over medium heat, whisking frequently, until mixture boils. Remove from heat and stir in vanilla extract.
In a small bowl, combine remaining 1/4 cup milk and gelatin. Let sit 5 minutes.
Meanwhile, whisk egg yolks, flour, and remaining 1/4 cup brown sugar in a medium bowl until smooth. Mixture will be thick.
Add gelatin mixture to milk mixture and whisk to combine. Use a large measuring cup to scoop about 1 1/2 cups of the hot milk mixture. Gradually, while whisking constantly, add this hot milk mixture to the egg mixture. Pour egg mixture back into milk mixture while whisking. Stir in spices.
Cook over medium-high heat while stirring continuously, until mixture begins to thicken. Gently boil for 2 minutes.
Spread mixture onto prepared crust. Cool to room temperature and refrigerate for at least several hours.
In a chilled mixing bowl, beat heavy cream with an electric mixer on medium-high speed until it starts to thicken. Turn speed down to medium-low and gradually add confectioners' sugar. Turn speed back to medium-high until medium peaks form.
Spread whipped cream on top of chilled custard. Store covered in refrigerator.
Notes
To make mashed sweet potatoes, bake 3-4 sweet potatoes in a 400 degree oven until very soft, about 1 hour. Let cool slightly. Peel skins off and discard. Mash the pulp with a fork until fairly smooth, like you would do with a banana for banana bread. Due to the sweet potato's high moisture content, there's no need to add any liquid.