Thus my heart was grieved - literally, and more expressively, “was soured.” The meaning is, that his heart was grieved, pained, dissatisfied. His mind was embittered, and he was rendered unhappy, by the views which he cherished about God, as doubting the wisdom and justice of his dealings with people - and about people, as being envious at their prosperity.
And I was pricked in my reins - The reins are often in the Scriptures represented as the seat of the thoughts or affections. See the notes at Psalms 7:9. The word rendered “pricked” means to sharpen, as a sword; and then, to pierce and penetrate as a sword does. The idea is, that these thoughts, so distressing and painful, seemed to be like a sharp sword penetrating to the seat of life.