Mix sugar, cornstarch, and salt together thoroughly in a medium heavy-bottomed saucepan.
Gradually stir in the milk, making sure to dissolve the cornstarch completely.
Whisk in the egg yolks thoroughly, and add in cut pieces of butter, stirring constantly heat over medium heat until the mixture just comes to a simmer. Reduce the heat to low, stirring briskly, bring to a simmer and cook for 2 minutes (Pudding will be thick and smooth).
Remove the saucepan from heat and stir in the vanilla extract.
Press plastic wrap directly onto the surface of the pudding and set aside.
Have the vanilla wafers ready, peel and slice (¼ inch thick) 3 to 4 bananas. Line the bottom of a 2 to 2 ½-quart dish with the wafers. Top the wafers with banana slices and cover with half of the pudding. Arrange a layer of wafers over the top of the pudding, then cover it with the remaining banana slices, and pudding.
Be sure to spoon pudding over any exposed bananas to prevent browning. Press plastic wrap directly onto the surface of the pudding (or cover) and refrigerate for at least 2 hours.
Just before serving cover with vanilla wafer crumbs, whipped topping, or both. Enjoy!