Jamieson, Fausset and Brown on Deuteronomy 4:2

2. Ye shall not add unto the word which I command you--by the introduction of any heathen superstition or forms of worship different from those which I have appointed (De 12:32; Nu 15:39; Mt 15:9).

neither shall ye diminish aught from it--by the neglect or omission of any of the observances, however trivial or irksome, which I have prescribed. The character and provisions of the ancient dispensation were adapted with divine wisdom to the instruction of that infant state of the church. But it was only a temporary economy; and although God here authorizes Moses to command that all its institutions should be honored with unfailing observance, this did not prevent Him from commissioning other prophets to alter or abrogate them when the end of that dispensation was attained.