I came up with slightly more than 3 yards of background needed for a 4x5 setting without any additional borders and with very careful cutting.

I considered the width of fabric to be 40-42" after selvedges are removed. A 5" wide strip across the width of fabric should yield the four 4 1/2" and four 5" squares needed for 1 block (and up to 4 of the 1 1/2" squares depending on width of fabric). For 20 blocks (4x5 setting), that would require 100" of fabric length. A 1 1/2" wide strip across the width of fabric should yield 27-28 of the small squares. For 20 blocks you need 240 small squares. If you can get four 1 1/2" squares at the end of each 5" strip that leaves 160 squares to be cut. 160 divided by 27 squares per strip = 6 strips. 6 x 1 1/2" = 9" of fabric length. 100" + 9" = 109" or 3.03 yards.