Preheat your oven to 375°F (190°C). Line a baking sheet with foil or parchment for easy cleanup and set aside.
Peel and chop the sweet potatoes into even pieces. Boil or steam until very tender, about 12-15 minutes. Drain well.
In a large bowl, mash the potatoes until smooth. Stir in the butter, brown sugar, milk, eggs, vanilla, cinnamon, salt, and nutmeg if using. Mix until creamy and well blended.
Transfer the potato mixture to a lightly greased 9x13-inch baking dish, spreading it into an even layer.
Make the butter pecan crumble: in a separate bowl, combine chopped pecans, flour, melted butter, and extra brown sugar. Mix until a crumbly, sandy texture forms.
Sprinkle the crumble evenly over the potato mixture. Bake for 25-30 minutes, until the topping is golden brown and the center is set.
If you like a crisper top, broil for 1-2 minutes at the end, watching closely to prevent burning.
Remove from oven and let rest for about 5-10 minutes before serving. This helps the casserole set and makes slicing easier.