… is from page 207 of the 1992 collection of some of William Graham Sumner’s best essays, On Liberty, Society, and Politics (Roger C. Bannister, ed.); specifically, this quotation is from Sumner’s famous 1883 essay “The Forgotten Man“:
If I want to be free from any other man’s dictation, I must understand that I can have no other man under my control.