LEO Comes to Donklephant
By Jonathon York | Related entries in NewsFirst of all I wish to thank Justin Gardner for the opportunity to participate in the Donklephant experience, hopefully as a regular contributor. In particular it is my sincere desire that these contributions will be fruitful and beneficial for overall political discourse.
Before we get into the meat of this next series of articles an introduction is in order. I am a instructor and student of politics. My educational background is largely in political theory and philosophy and I am fortunate enough to be working in my field. I currently teach government survey courses full-time at Mountain View College in Dallas TX, and because of this responsibility finding the time for independent research has been a challenge. However, this challenge is one I have chosen, for better or worse, to accept.
Over the past four months I have maintained a record of my research from the last year-and-a-half, and have published some of the findings on a weblog I call The LEO Test. The object of this research has been to develop a rubric whereby one may submit various political actors to a test of ideological preference. At the outset, the demands of this test were that it be:
1.accurate enough to serve as a “yardstick� by those interested in discerning political ideologies,
2.reliable enough to provide a means by which interested observers may make an informed political decision regarding political candidacies or political speech, and to advise caution when an extremist ideological position is indicated,
3.flexible enough to apply to a broad range of individuals who publish their political opinions, and
4.simple enough to be applied and interpreted by a non-academic audience. (This requirement is a particular challenge for many in the academic world.)
A fifth possible requirement for this rubric is that it be unencumbered by one’s own political or ideological bias. As a professional student of politics, this is perhaps the most significant caveat, and indeed one of the most difficult to overcome. However, in the interest of advancing genuine knowledge of political things it is perhaps the most crucial demand.
After a careful and patient reading of some of the major ideological literature which has influenced the American political experience one way or another, a descriptive model emerged with three distinct preferences: liberty, equality, and order (LEO). Once this preliminary model had presented itself, it was necessary to discern a means for testing and refining the model using individual samples of political language from known ideologues, and thereafter applied to other political actors.
That said, here is the method of observation:
I intend to count references to liberty, equality and order in each subject put to the test to determine the subject’s primary concern. In future analyses I may require that both positive and negative references to liberty, equality and order be included to get a clearer picture of the subjects’ core ideological focus, but for the time being, simple reference to these three principles will have to suffice.
A qualitative measurement would include a consideration of the context of the terms used, and the force with which the ideas of liberty, equality, or order are either defended or disparaged. A somewhat more quantitative approach would involve a simple gathering of references to liberty, equality, or order from a sample of the subject’s rhetoric.
For most American politicians on the national scene, a ready resource for measuring the degree to which a subject addresses liberty, equality, or order is available via their internet presences, i.e., their official webpages or weblogs. Otherwise, I may find collections of speeches or writings available from a variety of print sources.
To provide for consistent application of this yardstick, one must define references to “liberty,� “equality,� and “order.� One must also ensure that each measuring tool included the same number of discrete terms in order to avoid an unbalanced or skewed measurement. Reference to liberty should include synonyms and ideas or phrases that are unequivocally connected with the idea of “liberty� and a similar model would apply to measuring tools for “equality� and “order.�
For a preliminary measuring tool, I have chosen specific words or simple phrases to use which are commonly associated with “liberty,â€Â? “equalityâ€Â? and “order”. Some of these terms may only indirectly point to the root concept, such as “ownerâ€Â? and “optionâ€Â? for liberty, “accessâ€Â? or “fairnessâ€Â? for equality, or “traditionâ€Â? and “safeâ€Â? for order. However, at the heart of each of these terms a connection to their associated concepts may be clearly discerned. A more rigorous method of identifying rhetorical flags for liberty, equality, or order must be developed through an analysis of texts whose express subjects are liberty, equality, or order. In the meantime, however, I am using four separate but linked word lists or ‘dictionaries’ to measure quickly the frequency of ideological indicators in each sample.
After all, politics waits for no one.
So who can be put to the test? Just about anyone who has published a political opinion.
Who will be put to this test? There are the usual suspectsâ€â€?congressional leaders, presidential hopefuls, the traditional “talking headsâ€Â? of the mainstream media. But then there are those who up to now have avoided this kind of scrutiny: perhaps online news sites, perhaps our own neighbors in the blogosphere… perhaps even ourselves.
Stay tuned.
This entry was posted on Monday, October 3rd, 2005 and is filed under News. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
October 3rd, 2005 at 12:51 pm
Welcome aboard, Jonathon. (I must resist the urge to call you “Leo”.) I think this will be interesting.
Out of curiosity: Do you have an automated program that you feed text into to count the words in your dictionaries? Or do you do so manually?
October 3rd, 2005 at 3:47 pm
He’ Montag; thanks for the welcome. I originally did hand-counts, and still have to do that for hard copy. Thanks to a colleague I have a standalone executable that serves as proof-of-concept. It counts target text from a predefined dictionary file. It’s a command-line affair, it only reads text files, and the dictionary is limited to single words, but there are plans for new development.
October 3rd, 2005 at 4:41 pm
I’m sorry, but I am suspicious of analyses that judge content by word repetition. I tend to use the word “godd—n” on a fairly frequent basis despite that I believe in neither God nor hell.
On a somewhat more serious note, I am very devoted to the concept of liberty but almost never use the word. I just don’t like the word much. In fact, writing in support of liberty I’m more likely to attack those I see as opponents of same and consequently any screed of mine on the topic is likely to use the words “religion,” “religious,” “right wing,” and some other words I know you guys don’t approve of on a PG rated site. I wonder if the preference of some writers for defending a position by attacking its opposite won’t cause problems in your model.
October 3rd, 2005 at 5:27 pm
Michael,
First off I appreciate your objection, and understand your skepticism. I am generally suspicious of them as well. Consider the objection below:
http://theleotest.typepad.com/the_leo_test/2005/08/on_judicial_dec.html
The analysis does not simply rely on the repetition of just one word per principle. Part of the difficulty in developing a reasonable test table is that one must find concepts closely identified with the principle in question, and weed out those terms that are either ambiguous or ambivalent.
On the lighter side of your objection, since a word like “godd–n” is used so frequently beyond the context of religious discussion, it does not qualify as an indicator of a religious or anti-religious position. ;)
Similiarly, some words or phrases one would think would otherwise qualify under one or another of the principles measured actually cannot, either because they stretch across more than one principle or because they are skewed through grammatical or usage conventions.
As far as negative references are concerned, there is indeed a difficulty, but it appears as if some ideological positions are simply negatively defined. Hence, some people who think they are either ‘liberal’ or ‘libertarian’ may actually be ‘anti-conservatives’ instead. I submit that there is a marked difference.
At this stage distinguishing a preference or an anti-preference still requires manual text analysis. This can serve as a means of error-correction.
Also, measurements of an egalitarian, establishmentarian, or libertarian position are not merely a matter of counting only references to one principle. Statements of moderate ideological preferences of any stripe will–indeed must– contain a variety of indicators to liberty, equality, and order.
October 3rd, 2005 at 5:57 pm
Let’s give it a little time Michael and then we can call the methodology into question. Remember, ever methodology that has changed things has essentially been roundly ignored at first and then widely accepted later on.
I’m not saying this describes Jonathon’s, but it very well may. I’ll willing to take a chance and find out over a year’s time, which is how long he’s agreed to blog with us.
October 3rd, 2005 at 8:09 pm
Justin:
I was just hazing the new kid.
I’m always drawn to metrics. Don’t always believe them, don’t always believe particular interpretations of them, but I enjoy them just the same. There’s never anything wrong with pulling out your scalpel and slicing the world up from a new direction, see what’s in there.
Jonathon, you’ve hooked up with a really great site. Welcome to the exciting, entertaining and profitable (ah hah hah hah, oh, godd–n, that’s always funny,) world of blogging.
October 4th, 2005 at 11:25 am
If President Bush starts reading this blog, and learns of this “test”,
then the test goes straight to hell, because he’ll learn what word(s)
he’s not saying often enough. Problem is, you won’t know if/when that has happened. Now that you’ve published results, you can no longer measure in the same manner. The act of observation will now skew the test.
October 4th, 2005 at 2:18 pm
Hehe, well, I’m glad you think we have so much pull in the White House ford.
Simply put, testing methods evolve along with the subjects they test. But even if he does start using certain words more, the test will show trends over time, and therefore will be able to create a broader picture and possibly show a political evolution in its own right.
October 4th, 2005 at 3:21 pm
Hi ford,
That is a hazard with any method of observation, but the difficulty for the subject under scrutiny then comes when he must put his actions where his mouth is. One application of this model involves comparing rhetoric with language of legislation or of executive orders. If the subject appears to betray his own stated principles with his actions, he’ll lose support.
Also, as far as measuring concept frequencies go, who’s to say that they’re not doing this already? A while back I tracked Bush’s highlighted speeches from whitehouse.gov and found that his 2nd Inaugural Address had a signature so different from that of his campaign that it could be attributed either to a calculated move to preserve the Reagan Coalition or to a totally new speechwriter generally unfamiliar with the president’s agenda. He spent the next several months trying to bring those two ends together. Here’s the link:
http://theleotest.typepad.com/the_leo_test/2005/07/george_w_bush_a.html
October 4th, 2005 at 4:02 pm
Justin, stranger things have happened. Maybe Bush really can read on his own! :)
Jonathon,
It’s a good thing that it’s your job to keep up with this kind of stuff, otherwise, I’d say you have far too much time on your hands!
I honestly feel that most politicians just say what they think their constituents want to hear, not what they truly feel. So what is it that
your really measuring?
October 4th, 2005 at 4:05 pm
grrrr… any chance we’ll ever be able to edit our comments (or at least
preview), like we can on some other blogs? I hate when I make
grammatical errors!
October 4th, 2005 at 5:28 pm
If it’s a speech I’m measuring stated ideological indicators. There’s always a measure of difficulty with measuring phenomena of this sort, mainly because we cannot see directly into the human psyche. With such studies there are three different kinds of sources: What people do, what they say, and what they think. The first is readily observable. The second is observable but inspires doubt, and the third is nearly inscrutable…
October 4th, 2005 at 11:34 pm
Sorry ford. Let me look into that and get back to you. Hectic times around the homestead and work