Exactly. It makes each word feel like it is there purely for its initial letter, rather than its specific meaning, which undermines the effect.

I think that with both prose and poetry, it has to be the right word, first and foremost. If there are several candidates, it makes sense to choose the alliterative option if that is the desired effect, but not to shoehorn a word with slightly the wrong meaning, or a sequence of too many alliterative words, just for the sake of lining up ducks in a row.