My covenant will I not break - literally, I will not pollute, defile, profane. See the notes at Psalms 89:31, where the same word is used. God says that he will not do in regard to the covenant as they had done.
Nor alter the thing ... - The promise which I have made. I will not make it a different thing. I will not modify its conditions, or withdraw it. It shall stand precisely as it was when I uttered it. What God promises will be exactly performed.