Jamieson, Fausset and Brown on Numbers 33:55

55. But if ye will not drive out the inhabitants of the land from before you--No associations were to be formed with the inhabitants; otherwise, "if ye let remain, they will be pricks in your eyes, and thorns in your sides"--that is, they would prove troublesome and dangerous neighbors, enticing to idolatry, and consequently depriving you of the divine favor and blessing. The neglect of the counsel against union with the idolatrous inhabitants became fatal to them. This earnest admonition given to the Israelites in their peculiar circumstances conveys a salutary lesson to us to allow no lurking habits of sin to remain in us. That spiritual enemy must be eradicated from our nature; otherwise it will be ruinous to our present peace and future salvation.