I don't wash my fabrics either. The only fabrics that I prewash is flannel. Flannel is a must because it will shrink more.
For regular fabrics it is a preference. If you don't prewash, you will get what I call the "quilt crinkle" after you wash and dry your quilt, which I love cause it makes the quilt look homey, and cozy.......But, if you do not like that look, pre washing your fabrics will lessen that effect.