How Bad Is Mark Penn With Numbers?
There are all kinds of problems with the statistical inference Mark Penn reaches in this article but just take a gander at this part of Penn’s response to his critics:
The question of how much traffic it takes to make a living also comes from the Technorati report. We say it takes “about 100,000 unique visitors a month to generate an income of $75,000 a year” and Technorati states those who had 100,000 or more unique visitors the average income is $75,000.
I’m pretty sure you couldn’t get a passing grade on an AP math test making this kind of mistake, much less pass yourself off as a data-crunching expert. Technorati says that the average income of professionals bloggers who have over 100,000 unique visitors per month is $75,000. Penn glosses this as saying that “about 100,000 unique visitors” is enough to earn $75,000 which isn’t even close to being the same thing. The mean earnings of high-traffic bloggers are pretty decent. The median earnings are almost certainly lower. And the earnings of a blogger operating at the low-end of what counts as high-traffic will be lower still.
34 Responses to “How Bad Is Mark Penn With Numbers?”
Register
RSSLogin
Topic Cloud
ArchivesBlog RollAbout Matt YglesiasContact Me
Donate
April 22nd, 2009 at 8:34 am
Wow. Just wow.
April 22nd, 2009 at 8:48 am
I don’t think that’s such an innumerate mistake. What percentile of the distribution for earnings would new media know-it-alls use as the cut-off point for having about enough visitors to earn $75,000 (assuming that detailed enough data exist)? The twenty-fifth percentile? The first percentile? Or would the existence of one blogger with 500,000 visitors per month who loses the checks from his or her advertisers be enough to say “nyaaah, nyaah, it’s not enough to earn $75,000!”
April 22nd, 2009 at 8:50 am
This is the high-paid political consultant who didn’t realize Democratic primaries weren’t winner-take-all, so why would anybody be surprised by more evidence that he’s a moron? The real mystery is why the hell the Clintons have always been convinced he’s a genius.
April 22nd, 2009 at 8:54 am
So if I put 100 teachers and Bill Gates in a room, I could accurately report that the average net worth of individuals in that group is over $100 million.
That does not, however, mean that the average teacher is worth $100 million.
Isn’t that Yglesias’ point?
April 22nd, 2009 at 8:54 am
What the hell are you babbling about, “not Mark Penn”? Do you understand the concept of an arithmetic mean (which is what “average” almost always means when used without qualification)? What the hell does MY’s point have to do with percentiles?
April 22nd, 2009 at 9:00 am
Also, there is a difference between starting a blog and getting 100,000 unique visitors and blogging for an organization where you get 100,000 unique visitors and you happen to make a salary–such as people at the Atlantic, TNR, or CAP do.
April 22nd, 2009 at 9:06 am
On the other hand, My understanding is that taking over a campaign and running it into the ground while sucking its treasury dry can be very lucrative. The Democratic Party alone sustains a thriving cottage industry in that area.
April 22nd, 2009 at 9:08 am
Hey, Hillary, paid off that $6 Million campaign debt yet?
I’m sure that there are any number of foreign powers who would be happy to help.
April 22nd, 2009 at 9:09 am
You know, its amusing how many people just jump to the conclusion that this post is somehow Matt whining about…uh…something.
He’s just pointing out that Penn’s a moron.
April 22nd, 2009 at 9:13 am
The above is what the hell MY’s post has to do with percentiles. But I’m just being difficult. There’s no reason to worry about this. It is true that Mark Penn is very wrong for using the arithmetic mean to suggest professional bloggers as a group (as opposed to the succesful professional bloggers who do reach this plateau) with 100,000 hits have “enough” traffic to earn $75,000 (although he never said that he did not understand what an arithmetic mean is), but the same could be said, in strictly literal terms, about any attempt to define how much work is enough for someone to earn a certain amount of money. The median isn’t good enough because half of the people are still below. The 25th percentile of a class in terms of earnings is not enough because one-quarter are still below. At the 10th percentile, there are still many people below.
At some point, however, this gets ridiculous. Even if the income of those at the fifth percentile does not represent a guaranteed income, it represents an expected income. It is pointless to make such a huge deal out of using the mean to estimate income in this way without suggesting what would be an appropriate statistic or at least giving an indication that you have thought about whether any statistic reflects this fact.
April 22nd, 2009 at 9:25 am
not Mark Penn–
The problem isn’t so much that he used the mean, though I think using the median value is probably a better way of getting at his point. The problem is that he took the mean of a category that includes sites with far more than 100,000 hits, and then stated that the low end used to define the category is sufficient to generate that mean income. That’s just idiotic.
April 22nd, 2009 at 9:28 am
Exactly.
April 22nd, 2009 at 9:28 am
There is nothing arithmetically wrong with what Penn said. First he makes a statement and then backs it up with another statistic which theoretically could mean the same thing.
Anyways for what he said to be demonstrably false it requires more evidence. The median doesn’t have to be lower than the mean. How many millionaire high traffic bloggers are there really? If you have 1 millionaire blogger and only 10 really low paid bloggers with everyone else earning around 100k. The median would be higher. And Penn would be right. And unless we see the income distribution of high traffic bloggers, 100,000 hits could be “about” enough to earn 75k.
April 22nd, 2009 at 9:31 am
nMP, Penn doesn’t treat it as an expected income. He treats 100,000 hits as the threshold to earn $75k. That’s an important difference that is obvious anyone with technical skills.
April 22nd, 2009 at 9:34 am
Unless you think that either no one gets more than 100k unique visitors a month or people are not paid more for more unique visitors over 100k a month, what Penn said is obviously wrong. I would assert that both of those facts are non-controversial.
April 22nd, 2009 at 9:40 am
There is nothing arithmetically wrong with what Penn said.
And yet, the information he provided is at best worthless to the reader and at worst misleading.
One has to wonder if budding economics and finance people justify their terrible models with statements such as GG’s that say, “There’s is nothing arithmetically wrong…”
April 22nd, 2009 at 9:42 am
BP in MN,
Yah that’s a good point but even if Penn is misusing the evidence to make some inference about bloggers pay it requires more proof. It is often impossible to find perfect data to make your point so people make assumptions and simplifications. Like if 95% of bloggers over 100,000 hits receive only slightly more than 100,000 hits maybe using “about” is appropriate. Honestly, i know nothing about bloggers pay so Matt is probably right. I would be interested to see what Matt is using to make his assumptions because while personal experience matters it isn’t perfect.
April 22nd, 2009 at 9:46 am
No, there is no reason from the statistics available (the fact that something is not demonstrably false is not a good reason to believe something, Gordon Gekko) to assert, as Penn does, that 100k visitors per month is the threshhold for $75k per year. I would add for the benefit of eriks that my previous comments make it clear that I realize that Penn is talking about the threshhold even if I did not use that word and inappropriately used the phrase “to estimate income” which btw does not necessarily mean “to estimate expected income.”
I am just being a pointless troll by wondering if there is any statistic (e.g. earnings of 10th percentile within pro bloggers with a certain amount of traffic) that may or may not be available that MY would acknowledge as being useful for discovering the traffic threshhold-guidelines (as opposed to an absolute threshhold) for 75k or if, in the case of Mark Penn at least, everything he could say on this topic would be greeted by “Bwaahaahaa, 100k per month is not enough to earn 75k per year for everyone, so it makes no sense to think of it as a threshhold-guideline!”
April 22nd, 2009 at 9:53 am
Penn is quite obviously wrong in a way that borders on innumeracy. Terrifying in a pollster.
But this does make me think that web advertising is really under-priced, relative to TV advertising (although, as I’m sure Matt will point out, this is economically impossible as we’re all rational).
100,000 is roughly the size of Billings, Montana. If you were to buy television ad time in Billings, you have a total potential audience that’s the same size as a blog reaching 100,000. But I’m certain that the monthly market for TV ads in Billings is at least an order of magnitude higher than $75,000.
I don’t see how you can explain away that kind of difference in any kind of rational economic way.
April 22nd, 2009 at 9:58 am
No wonder he couldn’t count votes.
April 22nd, 2009 at 10:05 am
I think there is a difference between being bad with numbers and being dishonest with numbers. Penn’s analysis is wrong, but being a good pollster doesn’t just involve conducting good scientific polling, it involves twisting and distorting the numbers so that you can convince members of congress that continuing stupid policies will be really good for them politically. Penn doesn’t seem to me to be a great person, but he may not be stupid either.
April 22nd, 2009 at 10:09 am
I wasn’t sure that Yglesias was right about this until I went to the Technorati study itself and lo and behold! The median income of full-time bloggers is $22,000. The difference between $75k and $22k is not just a matter of “quibbling”. Penn is cherry picking his stats to make his point. See the study results here … http://technorati.com/blogging/state-of-the-blogosphere/blogging-for-profit/
April 22nd, 2009 at 10:10 am
@gordon gekko,
The Technorati report Penn uses for his source actually notes that some of the blogs in that category receive over a million hits per month, and also notes that the median income for those bloggers is $22,000; I wish they’d also listed the median number of visitors for those sites as well. So Penn is just wildly off, and either confused by simple statistics or being deliberately misleading.
@not Mark Penn–
I think using either the median or the mean for a better-defined group would actually be defensible, though the median would probably be a better choice if the data are available. If there are a significant number of blogs with, say, between 100,000 and 125,000 visitors with an average income of $30,000, saying that 100,000 visitors is a the threshold for making $30K would be reasonable unless there was some reason to suspect that one of the blogs is a giant outlier. Alternately, something like the average income of the bloggers between the 25th and 75th percentiles might be a reasonable measure, since it would probably measure income for an average amount of commercialization of the blogs as well.
April 22nd, 2009 at 10:18 am
For anyone out there still having trouble understanding why Penn is frightfully wrong here, consider a statement like:
“People earning more than $20,000 a year earn an average of $75,000 a year. Therefore, earning $20,000 is sufficient to earn $75,000.”
This is equivalent to what Penn said, if you assume that hits correlates closely with income, and suppose 100k hits = $20k. (Which is probably not the exact number, but the real one is likely something similarly much lower than $75k.)
April 22nd, 2009 at 10:21 am
During the primaries, every time HIllary Clinton told a heartwarming story about a little girl who sent in the contents of her piggy bank, or an old lady who wrote a $5 check, my mind mentally added “…and then we sent it to Mark Penn” to the end of the sentence.
April 22nd, 2009 at 10:22 am
100,000 is roughly the size of Billings, Montana. If you were to buy television ad time in Billings, you have a total potential audience that’s the same size as a blog reaching 100,000. But I’m certain that the monthly market for TV ads in Billings is at least an order of magnitude higher than $75,000.
I don’t see how you can explain away that kind of difference in any kind of rational economic way.
I think there’s almost certainly a very large difference in the “impression” impact of a tv ad, with sound and video, that you’re forced to sit through, and banner ads on a webpage that most people simply filter out. Web ads are just less effective than tv ads on a per-viewer basis.
That said, I wouldn’t be inclined to argue that the pricing is entirely rational. Nor, I think, would Matt.
April 22nd, 2009 at 11:28 am
It’s really interesting to see the split between those who think that Penn was ‘technically right’ or just a bit approximate and those who understand that his statement was deeply innumerate. Evidently deep innumeracy is common among bloggers as well as the rest of the population. I hope lecou’’s first post can help the former group understand why Penn’s statement was neither formally true nor a reasonable approximation.
Penn may well have had a motivation to distort things, but it would be hard for anybody who understood numbers to make a public display of not knowing his ass from a hole in the ground uless there was a huge immediate payoff. All the evidence suggests that he doesn’t understand numerical data. The Clintons unfortunately are suckers for characters like him, Ira Magaziner, Dick Morris…
April 22nd, 2009 at 12:43 pm
Contra Penn, the median earnings of blogs with over 100,000 visitors per month who are not fundamentally American in their outlook is substantially less than $75,000 per year. You know, Blacks, people from Hawaii, and people with funny names.
April 22nd, 2009 at 1:45 pm
$75,000 for 100,000 a month is something like 6 cents a visitor.
Is this just blogging income, or do they count Dan Drezner’s professorial salary? How many such bloggers are there?
April 22nd, 2009 at 5:49 pm
I thought the problem was that Penn makes no distinction between (1) the average income of people who blog, and (2) the average _income from blogging_ of people who blog.
April 22nd, 2009 at 11:47 pm
I think the point is that 100,000 visitors per month are worth $75,000 per year, and both Technorati and Penn agree. But if you read carefully, there is no time dimension to the Technorati factoid.
April 23rd, 2009 at 8:35 am
The statistic says that the average income for bloggers who have 100,000 or more unique visitors is 75k. Penn then says that 100k unique visitors is a reasonable level of traffic for an expected income of 75k. I see two problems with this: (1) Like Coruscation noted that the Technorati stat doesn’t give a time dimension; I can only hope it’s the same as Penn’s, because otherwise we’re really just cluees, and (2) The groups aren’t the same at all — bloggers with “about 100k in in unique visitors” are not the same group as “bloggers with 100k and more”.
April 23rd, 2009 at 10:40 am
Let’s go over this again.
Penn says:
The problem is NOT the time dimension. True, Penn drops this in the second part of his statement, but you can infer he’s still talking about “per month” because of the initial mention. That’s just English. (Maybe it’s a little sloppy, but then, repetition would have been pedantic.)
Nor is the problem with distinguishing between personal income and blog revenue – as near as I can tell, both Penn and Technorati are talking only about income received from the blog.
Penn also accurately paraphrases the Technorati report, which states:
Where he goes off the rails in an astonishingly innumerate way is where he thinks that sentence from Technorati implies “it takes about 100,000 unique visitors a month to generate an income of $75,000 a year”.
Obviously, it doesn’t. A few bloggers get millions of hits, and hundreds of thousands of dollars, but Technorati says the MEDIAN revenue of the >100k hits bloggers was only $22k. And that means the revenue for someone at the bottom of the category, someone actually getting only around 100k hits, is even lower than that.
Let’s say it’s just a bit lower – say you can expect to get about $20,000 in revenue for running a blog with about 100k monthly unique visitors (modulo other things that might affect ad revenue, like the demographics of you audience).
Then we can restate Technorati’s statement: “average income was $75,000 for those who [had $20,000 or more of income]”
That’s still perfectly sensible. Now let’s run it through Penn logic:
Hopefully the innumeracy is clear.
Penn logic makes nonsense of all kinds of things. You can try it at home. Like, let’s say the average income of people who work more than 1 hour a year is $50,000.
In Penn logic, this implies that working 1 hour a year will earn you $50,000.
(Huh. I guess I can see how Penn might get the impression that Penn logic works – but it doesn’t work that well for the rest of us.)
April 23rd, 2009 at 2:22 pm
I gave Penn’s article a good old fashioned fisking on the VQR blog a couple of days ago, going over the thing point by point. He actually makes a lot of other incredibly foolish claims that demonstrate his utter misunderstanding of statistics, including his apparent belief that most bloggers are children, and that most of them are college graduates. There’s simply nothing about this that makes sense.