The supposition is, that absolute power, in the hands of an eminent individual, would insure a virtuous and intelligent performance of all the duties of government. Good laws would be established and enforced, bad laws would be reformed; the best men would be placed in all situations of trust; justice would be as well administered, the public burdens would be as light and as judiciously imposed, every branch of administration would be as purely and as intelligently conducted as the circumstances of the country and its degree of intellectual and moral cultivation would admit. I am willing, for the sake of the argument, to concede all this, but I must point out how great the concession is, how much more is needed to produce even an approximation to these results than is conveyed in the simple expression, a good despot. Their realization would in fact imply, not merely a good monarch, but an all-seeing one. He must be at all times informed correctly, in considerable detail, of the conduct and working of every branch of administration, in every district of the country, and must be able, in the twenty-four hours per day, which are all that is granted to a king as to the humblest laborer, to give an effective share of attention and superintendence to all parts of this vast field; or he must at least be capable of discerning and choosing out, from among the mass of his subjects, not only a large abundance of honest and able men, fit to conduct every branch of public administration under supervision and control, but also the small number of men of eminent virtues and talents who can be trusted not only to do without that supervision, but to exercise it themselves over others. So extraordinary are the faculties and energies required for performing this task in any supportable manner, that the good despot whom we are supposing can hardly be imagined as consenting to undertake it unless as a refuge from intolerable evils, and a transitional preparation for something beyond. But the argument can do without even this immense item in the account. Suppose the difficulty vanquished. What should we then have? One man of superhuman mental activity managing the entire affairs of a mentally passive people. Their passivity is implied in the very idea of absolute power. The nation as a whole, and every individual composing it, are without any potential voice in their own destiny. They exercise no will in respect to their collective interests. All is decided for them by a will not their own, which it is legally a crime for them to disobey. What sort of human beings can be formed under such a regimen?...
A good despotism means a government in which, so far as depends on the despot, there is no positive oppression by officers of state, but in which all the collective interests of the people are managed for them, all the thinking that has relation to collective interests done for them, and in which their minds are formed by, and consenting to, this abdication of their own energies. Leaving things to the government, like leaving them to Providence, is synonymous with caring nothing about them, and accepting their results, when disagreeable, as visitations of Nature.-- J.S. Mill (1861) Considerations on Representative Government, Chapter 3.
In Against Democracy, Jason Brennan suggests that Mill's argument for representative democracy is based on the idea that democratic participation would make people smarter, more concerned with the public good, and nobler. In Brennan's memorable words, Mill hoped to turn Hooligan/Hobbit-like people into Vulcans by widening the franchise. That is, one way to understand this interpretation of Mill, is to see in democratic participation a potentially politically transformative experience (PTE recall). Reflecting on the reality of partisanship, Brennan finds Mill's argument wanting empirically. Let's grant, for the sake of argument, Brennan's interpretation of the empirical facts here. Let's also grant that sometimes Mill has some such hopes for the effects of democracy.
As Brennan correctly observes, Mill's actual (plural) voting design reflects an elitist strain--it lets the aristocratic (in the intellectual sense) elite have more weight. This suggests that Mill was not especially optimistic about the transformative effects. This is no surprise: Mill has a healthy distrusts of crowds, and especially the effects of public opinion (which, recall, he thinks is conformist and homogenizing). As he notes, in reflecting on these, "that does not hinder the government of mediocrity from being mediocre government. No government by a democracy or a numerous aristocracy, either in its political acts or in the opinions, qualities, and tone of mind which it fosters, ever did or could rise above mediocrity." (On Liberty, Chapter III). Unlike contemporary democrats, Mill does not expect public reason to be developed in mass democratic society. That is, he shares with Brennan a skepticism about any public reason project (if that project is not guided by experts.) Mill's distrust of public opinion, which also informs his analysis of free speech -- he is extremely alert to the fact how majorities silence minorities -- should alert us that Brennan may be missing something about the underlying sensibility of Mill's position.
Now, Mill has lots of arguments for widening the democratic franchise as much as possible, and the present post only reflects what I take to be ones that may engage with premises that Brennan would share. I see in the quoted passage two kinds of arguments against epistemocratic, benevolent dictatorship. To be sure, that's not Jason's position because he wants voting by the informed (a position that is, in fact, Mill-ian in spirit), but it's useful to reflect on these anyway (in part, this is not required for the rest of the post, but I say it anyway, because it's easy to see how Jason's arguments would slide into abolishing elections altogether).
First, Mill articulates the proto-Hayekian argument that benevolent/epistemocratic rule faces a calculation problem. (I think the argument is indebted to Smith.) The problem is really two-fold: one is purely epistemic -- 'the conduct and working of every branch of administration, in every district of the country, and must be able, in the twenty-four hours per day, which are all that is granted to a king as to the humblest laborer, to give an effective share of attention and superintendence to all parts of this vast field" --, but the other is more moral: namely the ruling elite(s) must be able to judge character ("honest and able"). The first of these arguments, Brennan is happy to concede -- for Brennan, I suspect, markets > experts > government --, but the second is a real problem for any elite rule.
There are two versions of the moral version of the proto-Hayekian argument. Both are explicitly recognized by Brennan (although I am unsure he recognizes these are Mill's arguments): in the real world people with power can't be trusted to use the means of exclusion in fair and impartial ways. This is true both at the level of procedure, but also at the level of what will count as the relevant knowledge to be selected for--this will be essentially contested. (Brennan recognizes this, but shrugs it off.) The second is that to be a responsible elite does not just require epistemic skills, but also the right moral dispositions (this falls under what Brennan calls the competence principle). And according to Mill, a self-selecting epistemocratic elite lacks access to the relevant information about the moral competence of their peers. I think we can strengthen Mill's argument, again on grounds that Brennan would accept, that cognitive elites will neglect the relevant information about the lack of moral fitness of their peers due to familiar group biases (reinforced by their self-interest). That is, once knowledge is a means to rule, knowledge becomes a source of faction (with all the dangers attending it).
The upshot of the proto-Hayekian argument by Mill is an argument against elite rule and for small government (and, as Mill noted, lots of worker managed cooperatives). But it's also meant to be an argument for a wider franchise, because it suggests that when all things are equal, wider participation by the ignorant is better than narrow participation of the smart because the narrow participation will always lead to expert overconfidence and to self-reinforcing biases. Note that this is essentially negative argument: Mill completely allows that wider participation is going to be oppressive (because anything the masses will do is oppressive).
The second argument in the quoted passage suggests that elite rule will make the people passive (in the way that certain forms of Christianity make folk quietist). That is to say, Brennan is right to be alert to the fact that many forms of political participation makes things objectively worse; Mill grants that. But creating a form of government in which mobilization and participation is not encouraged also makes things worse: it reinforces human passivity.
Notice, again, that the argument is a negative one. Mill knows that active human beings are capable of very bad things. (His cult of genius allows that genius-gone-bad -- Alcibiades -- is very bad.) But we may say that Mill has high tolerance for risk here. For he thinks that active human beings are an intrinsic good worth aiming at. And we should never opt for a social structure that reinforces passivity in the population as such. (Mill thinks that not just boring, but (a) ultimately not dynamic economically, and (b) intrinsically oppressive--because a passive population will be homogeneous, etc.)
One can grant, thus, Brennan that political mobilization does not turn us into enlightened folk (I really doubt Mill expected that), but Brennan seems to overlook Mill's (secularized Christian) point that freedom involves the freedom to make one's own mistakes. This is not ad hoc in Mill: for Mill political freedom involves an "experiment of living" for each of us. Brennan thinks allowing such experiments means, in practice, to have the unjust create bad effects for all of us. Mill does not deny this; he thinks that just is a consequence of any political life. (His is not Bentham's technocratic hope of getting it right.) But in Brennan's risk aversion and noble outrage at injustice, he misses (and so does not account for) the unexpected goods that come from allowing folk to mess up, even mess up big time.*