Other lords beside thee have had dominion - The allusion here is to the kings of Babylon who had subdued and oppressed them, and who in their long captivity had held them in subjection to their laws.
But by thee only will we make mention of thy name - This may be better rendered, ‘but only thee, thy name will we henceforward commemorate.’ The words ‘by thee,’ and ‘thy name,’ are put in apposition, and denote the same thing. The word ‘make mention’ (נזכיר nazekiyr) means literally to cause to be remembered; to commemorate; to celebrate. The idea is, that during their long captivity they had been subject to the dominion of other lords than Yahweh; but now that they were restored to their own land, they would acknowledge only Yahweh as their Lord, and would henceforward celebrate only his name.