I will be glad - I will rejoice, and will express my joy.
And rejoice in thee - I will exult; I will triumph. That is, he would express his joy in God - in knowing that there was such a Being; in all that he had done for him; in all the evidences of his favor and friendship.
Will sing praise to thy name - To thee; the name often being put for the person.
O thou Most High - Thou who art supreme - the God over all. See the notes at Psalms 7:17.