I also like to use flannel - but I wash it first to get it to shrink as much as possible.
Would you pre-wash your flannel even if you haven't pre-washed your quilt top?