As T.S.Eliot, so often the voice of frustration and regret, wrote:

Footfalls echo in the memory

Down the passage which we did not take

Towards the door we never opened

Into the rose-garden.

It's always best at least to venture down the passage and open the door, even if after a while you might decide the rose garden isn't all that great after all, and choose to head back.

Or find the door has locked behind you, and just make the most of it anyway, secure in the knowledge that at least you dared, and that daring expanded your life.