This reminds me of a phrase by Dylan Thomas - 'the spaces between the houses' - from a love letter to Caitlin.

It turns out that the rest of the passage also talks of 'non-music':

I want you to be with me; you can have all the spaces between the houses, and I can have a room with no windows; we’ll make a halfway house; you can teach me to walk in the air and I’ll teach you to make nice noises on the piano without any music; we’ll have a bed in a bar, as we said we would, and we shan’t have any money at all and we’ll live on other people’s, which they won’t like a bit.