If performance differences among money managers are partly due to
skill, one would expect those differences to be correlated to relevant
characteristics of managers, such as intelligence. There is research
showing that intelligence is highly correlated with SAT scores and that
managers who attended high-SAT schools (and probably have higher scores
themselves) outperform other managers on average. If investing in
stocks involved no more skill than coin-flipping, one would not expect
to see such correlations. The references are below.

http://papers.ssrn.com/sol3/papers.c...ract_id=686849
Hedge Fund Performance and Manager Characteristics - Education and Age
Matter When Selecting Your Hedge Fund Managers
HAITAO LI
Cornell University - Samuel Curtis Johnson Graduate School of
Management
RUI ZHAO
Columbia Business School
XIAOYAN ZHANG
Cornell University - Samuel Curtis Johnson Graduate School of
Management
Abstract:
Using a large sample of characteristics of hedge fund managers, we
provide probably the first comprehensive empirical analysis of hedge
fund performance and manager characteristics. We document a strong
relation between hedge fund risk-taking behavior and performance (both
raw and risk-adjusted returns) and manager educational background and
working experience. For example, we find that managers from higher-SAT
undergraduate institute tend to have better performance and take less
risks. We also find that managers with longer working experience tend
to have worse performance. These findings are robust to the many
risk-adjustment models we consider for hedge funds. Our results confirm
the conjecture of Chevalier and Ellison (1999) that certain portfolio
managers are indeed better than others and can be valuable to hedge
fund investors in identifying managers with superior performance.
Keywords: Hedge fund performance, manager characteristics, risk
adjustments, panel-data regression
JEL Classifications: G23, G11, G12

http://papers.ssrn.com/sol3/papers.c...ract_id=225637
Are Some Mutual Funds Managers Better Than Others? Cross-Sectional
Patterns in Behavior and Performance
JUDITH A. CHEVALIER
Yale School of Management; National Bureau of Economic Research (NBER)
GLENN ELLISON
Massachusetts Institute of Technology (MIT) - Department of Economics;
National Bureau of Economic Research (NBER)
December 1996
NBER Working Paper No. W5852
Abstract:
In this paper we explore cross-sectional differences in the behavior
and performance of mutual fund managers. In our simplest regression of
a fund's market excess return on characteristics of its manager we find
that younger managers earn much higher returns than older managers and
that managers who attended colleges with higher average SAT scores earn
much higher returns than do managers from less selective institutions.
These differences appear to derive both from systematic differences in
expense ratios and risk-taking behavior and from additional systematic
differences in performance managers from higher SAT schools have higher
risk-adjusted excess returns. Managers with the paper also presents a
preliminary look at the labor market for mutual fund managers. Our data
suggest that managerial turnover is more performance sensitive for
younger managers.

Elle wrote:

- quote -

Based on the mathematical model used in the paper, I think
"distinguished" managers are defined as those who had positive alpha in
the past.

- quote -

It's true that an investor considering a strategy using actively
managed funds would care about returns after fees. However, according
to Table III in the published paper, managers ranked in the top decile
over the last year according to their own alpha or a weighted alpha
(the measure introduced by the authors) go on to earn an alpha of 3.6%
or 5.5% over the next year, before fees. Since most actively managed
funds have expense ratios in the 0.5% - 2.0% range, the top decile
funds would probably have positive alphas even after expenses.

- quote -

In a tax-deferred account, a long-term market timing strategy that
trades once or twice a year between stock and bond mutual funds
probably would not incur any transaction costs. Even in a taxable
account, if realized gains are long term and taxed at only a 15% rate,
the transaction costs may not be prohibitive.

"Tad Borek" <borekfm[at]pacbell.net> wrote
- quote -

Tad, it is a simple linear regression model that helps identify the exposure
to different asset allocations appropriate to each investor. Being a model,
it necessarily relies on certain assumptions. These assumptions have a
margin of error associated with them, which of course propagate and so
generate a certain margin of error in whatever output one is deriving from
the model.

This is why it has some value but not the precision you seem to persistently
attribute to it.

- quote -

You need a first course in college statistics.

Elle wrote:
- quote -

Elle - we're not talking about asset allocation, but rather some ways
for an individual to assess whether their stock-picking has done any
good. The Fama-French kind of approach is more refined because it
compares your performance to what is arguably a better benchmark than
something like a Vanguard broad-market fund. But to apply it you need
Beliavsky-esque math skills (and interest level). Yes you can do it but
most people aren't going to bother. And at the end of the day, I think
it's just as interesting to just look at the "dollars at the end of the
pipe".

-Tad

Tad Borek wrote:

<snip
- quote -

Maybe you are right, but data on the Fama-French factors is freely
available from Professor French's web site
http://mba.tuck.dartmouth.edu/pages/...a_library.html
, so I think an investor would only need a time series of portfolio
returns and a linear regression package (even LINEST in Excel) to
calculate the alpha and beta's.

"Tad Borek" <borekfm[at]pacbell.net> wrote
- quote -

In what way is it "more refined"?

For example, can you say what the uncertainty is in a portfolio allocation
designed using Fama-French formulations of risk and diversifying? If so, on
what order is the uncertainty? A few percent? 20%?

I believe I asked Beliavsky this not too long ago and got no answer.

I'm wondering if the so-called "precision" in such models is irrelevant
given the high margin of error attaching to many of the assumptions, thus
propagating a similar, high uncertainty with the output.

It's fun math, but I suspect it has limits as to its practicality.

Will Trice wrote:
- quote -

Will,
As a practical matter, given all-US-stocks, I'd suggest just comparing
your results to those of a readily-available alternative investment -
say, Vanguard's total stock market index fund. That's sort of the
default position an extreme "efficient market" guy might take - meaning,
if you didn't believe in stock picking at all, and didn't even think
distinctions between growth/value, small/large matter. You know, if you
were Jack Bogle.

If that comparison shows you well ahead of your imaginary index fund
investment, you might look a little closer and see if, for example, you
had nothing but small-company stocks, and if so look also at an index or
index fund that's representative of that asset class. And ask yourself
whether you'd keep plugging away at small-caps and in that case, a
small-cap benchmark is more appropriate. You can keep picking small
stocks or can put the whole nut in a small-cap index fund. Are you
capturing better performance or just swimming in a rising tide?

There's going to be slop in this of course. The Fama-French stuff is
much more refined but it's just not practical for an individual investor
to do. From a stats perspective you can make an argument for using an
equal-weight index instead of a cap-weight fund for your benchmark. But
at the end of the day you're probably more interested in whether you
ended up with more money, vs. a mutual fund portfolio. If you have some
other mutual fund mix in mind, maybe benchmark against that.


- quote -

Sure it does - let's say you get a fresh $10,000 to invest tomorrow. If
you were really good at analyzing small PC manufacturers it won't do you
any good today, because that business is gone. If you started with $50k
ten years ago and had the good fortune to pick a few companies that were
merged out of existence eight years ago (and you haven't made a trade
since, having banked some well-above-market gains) it's not going to do
you any good going forward. It's important to assess whether your
stock-picking method is likely to be repeatable in the future, and if
you only had a couple good picks over ten years, instead of dozens that
collectively beat the market, it raises the question of whether you'll
have something to do with new money.

-Tad

"Michael Sullivan" <michael[at]bcect.com> wrote
- quote -

I believe there is some kind of simple statistical test that will say
something intelligent about a manager who beats the S&P 500 10/10 years.

- quote -

No, this is an abuse of the statistical term "confidence level."

- quote -

Let's go with this a bit, as it's not a bad model. Suppose there is a 50%
chance of beating the benchmark each year, based on chance alone. The
chances of a person doing this by chance ten years in a row is (1/2)^10, or
about 1 in 1000. So that's some kind of measure.

But don't slip "confidence level" into this approach. That's related to some
of the examples we are examining but you are misusing it in the posts where
I have seen you elaborate. Google for {hypothesis testing confidence level
definition} or similar.

Tad Borek wrote:

- quote -

I'm not sure I'm following you. You previously made the case for a
diversified investor to gauge themselves against their Fama-French risk
bin (suggesting different "markets" for different investors), but then
you acknowledged that this factors out the investor's choice of asset
allocation (I'm making the assumption that an individual investor is
free to choose their asset allocation as opposed to a mutual fund
manager who may be beholden to a style box). So for this investor, why
does a cap-weighted index represent "the market"? It seems that "the
market" really is equally weighted for the individual.

Speaking for myself, I invest primarily in individual U.S. stocks (I
have mutual funds only in 401(k)s), no foreign, no bonds, no shorts, no
futures, no options, no commodities. My portfolio is not intentionally
weighted towards any risk level, capitalization, or valuation measure.
My "market" is equally weighted (across U.S. stocks on major exchanges),
isn't it? If I have $1000 to invest, I'm not going to weight it by
company size. So I think I really am competing against the dart board.
Either that, or I'm competing against index funds, but which index?
Must I beat all indices all the time to rationalize stock picking? I do
not think that's a reasonable hurdle.

- quote -

I agree. But the question I started with was: if I accomplish X > Y,
was it luck?

- quote -

Do you think this applies to the individual as well?

Thanks,
-Will

Tad Borek wrote:
- quote -

Another thought on this. For an individual investor, wouldn't the
appropriate benchmark be a non-weighted total market index?

Will Trice wrote:
- quote -

Will-
I think either could be appropriate..if the question is whether an
individual is better at stock-picking than a dartboard would be, there's
some sense to using an equal weighted index. That type of index deals
with the issue of when, say, a small number of stocks dropped in value
but the majority rose, so the dart-thrower was more likely to hit an
"up" stock than the performance of a cap-weighted index might suggest.

But I think the real question should be: did your efforts beat "the
market"? (and if not, why are you bothering?, just buy the market.) The
starting point a picker is trying to disprove is that markets are
efficient and it's not worth the bother trying to beat them. To make
that comparison it's more appropriate to use a cap-weighted index, which
is representative of "the market". Or better, a corresponding
broad-market index fund which reflects costs. That's what drives it
home: your stock portfolio is worth $X, your Vanguard fund would be
worth $Y given the same starting value, is X> Y? This is the argument
that works very well at convincing some people to be stock pickers and
others (a larger number) to be index fund investors. Forget the academic
arguments and speak in dollar terms.

RE: # of picks vs. time...I think picks are important because when you
hire a new manager or invest in a new fund you hand over cash, and they
need to throw a new dart. Gains that came from old ideas that could no
longer be implemented won't help a new investor. You need someone that
has been able to generate ideas consistently, otherwise you worry about
them having a place to put your latest IRA contribution, 401k deferral,
bonus, etc.

-Tad

Michael Sullivan wrote:



- quote -

This sounds kind of like Heisenberg's Uncertainty Principle.

- quote -

I don't know about Sandy What's-her-name, but Bill Miller is up to 14
years in a row now. Two years ago I bought his fund based partly on his
record. Maybe I collapsed the quantum probability field that allowed
him to continue his run...

Michael Sullivan wrote:
- quote -

This is not in agreement with the results of many papers, for example
the Journal of Finance paper Beliavsky cited. Now whether the
prediction is good enough to cover the costs of rebalancing is another
story...

-Will

Tad Borek wrote:
- quote -

Tad,

Would this really be related to the number of stock picks? It seems
that the decision not to sell is equivalent to the decision to buy. I
would think that the confidence would increase with time independent of
trades.

-Will

Elle <elle_navorski[at]nospam.earthlink.net> wrote:
- quote -

There are precise numbers, given a single result. But we don't have a
single result, we have millions. And the total set of results is well
within the statistical parameters for "all luck".

, the test premise is simple: just figure the chance of said
performance being due to luck. If that's less than 5%, then you are 95%
confident of it not being due to luck.

For something as simple as consecutive year on year benchmark beating,
it's easy to calculate. What's the chance of beating the benchmark due
to variance alone if you are an average performer? Call that p.
the probability of beating it n years in a row is then 1 - (1-p)^n,

But that doesn't tell you much, unless you predicted the outperformance
before the experiement started.

- quote -

That would be a highly anomalous result. Fund managers beat the S & P
about 10-20% of the time, so if you pointed to someone and said "This
person is a skillful investor who beats the market. They will beat the
S & P almost every year for the next 10." -- if they then proceeded to
do so -- that would be *very* good evidence for your hypothesis.

But my point is that this is *NOT* what is happening, when you -- today
-- point to someone who beat the S & P every year for the last 10. That
person is selected out of many millions of investors (or hundreds of
thousands of money managers) who attempted to do the same, and most of
whom failed.

- quote -

There were two big fund managers back when I was paying attention to the
financial press regularly who had stretches of 8-10 years of beating
their benchmark every year. One was Bill Mueller of Legg Mason Value,
and one was less well known, Sandy something from New York something or
other fund. In both cases, the year after they were lauded in _Money_
for their incredible achievements, their funds underperformed.

There have probably been examples since, and I'm sure there are examples
outside the world of public mutual funds. If a competent large-cap
manager will beat the S&P 20% of the time, then out of a million money
managers, you'd expect about 3 on average to beat it for every year of
any given 8 year period. If you assume no management fees or
transaction costs on either side, then a competent manager should have
close to a 50-50 shot on luck alone, which means about 4 in 1000 would
be able to do 8 years in a row, and about 1 in 1000 for 10 years in a
row.

I'm not aware of anybody well-known outperforming consistently year by
year for much longer than that. But going up on the benchmark for, say,
30 out of 35 years would be more impressive than just 10 in a row
without ever necessarily having 10 in a row. That's a somewhat more
complicated formula, although in the 50-50 scenario you can model it
with combinatorics so it isn't too bad. You're looking at (35 choose
30) / 2^35. (35 choose 30) is 324,632, and 2^ 35 is 34,359,738,368, so
the chance of doing this by luck is 1 in 105,842. Very low, but with
the universe of investors out there, you'd expect a number of people to
have done it, and probably at least 1 money manager (before expenses).

So, while these results are outside of the "95% confidence interval" for
any given person -- what you're looking at in a large population is not
"how rare is an individual result due to luck" but "how many people had
this result vs. how rare it should be if it's just luck". Unless the
result is so rare it would be highly unusual for *anyone* to have it.
Find someone who beat the S&P every single year for 40 years in a row,
and that's probably skill.



Michael

"Michael Sullivan" <michael[at]bcect.com> wrote
- quote -

I thought there were some very basic statistical tests that would provide a
precise number for this. I was hoping someone would quickly post a precise
response.

In other words, suppose someone beats the S&P 500 ten years out of ten.
What's the significance level of this? ("95% confidence" isn't quite the
appropriate vocabulary here and is misleading.)

I often use a rough, statistical rule of thumb: X successes out of N
attempts has an approximate margin of error of plus or minus 100% / sqrt(X)
. So we'd say a gal that had beat the S&P 500 ten years out of ten has a
probability of doing this again next year of about 67% to 100%. This has
nothing to do with confidence levels or p-values, of course, but it is a
crude measure of the importance, shall we say, of a given average.

In the vein of Beliavsky's 2005 Journal of Finance citation, what mutual
fund managers have done this? Who is a distinguished mutual fund manager,
and by what numerical measure is he/she so distinguished?

Douglas Johnson <johnson[at]classtech.NOTPARTOFADDRESS.com> wrote:
- quote -

Depends on how good those returns are, relative to relevant benchmarks
and additional volatility accepted.

In my last survey of attempts to determine this, the number of people
with "outside of 95% confidence" results was about what you'd expect
given the size of the investing universe.

That's the problem. If 100 million people all attempt to do something
for 30 years, 5% of them are going to end up with results outside the
95% confidence interval. When it's investing, the 2.5% on the high side
become rich.

If you had picked 1 random investor in 1975 and over the last 30 years
they had achieved a 99th percentile result, you'd figure there was a
good chance that they had, in fact, outperformed due to skill. But if
you just look around for people *today* that have done that well, you
would expect to find a few even if it's all luck, because they have
self-selected.

When researchers have tried in the past to "pick good investors" based
on various ways to measure their performance, and then looked at their
results into the future -- the "good investors" have rarely outperformed
their relevant benchmarks in the future as a group. Nobody has come up
with a measurement of past performance, that has a statistically
significant chance to survive into the future -- Except for *bad*
performance. At least in the universe of public mutual fund managers of
listed stocks and bonds.

I know there are obscure corners of the financial world where people do
really beat the market due to skill (or perhaps just due to better
access to information), but we're talking about things that a typical
individual investor cannot easily do (or buy the services of someone who
does).

It's also true that some segments of the market have outperformed
others, but it's not clear that this isn't due to taking on greater
risk.


Michael

<beliavsky[at]aol.com> wrote
snip
- quote -

I searched this article for the word "distinguished," so I might find out
how the authors define consistent, superior performance by such managers.
I'd sure like to read that these "distinguished" managers have been beating
the S&P 500 index for at least ten years, every year, say, to persuade me
that this research has some credibility.

But not only did I not find the word repeated, I found nothing in the paper
that defined these animals ("managers with distinguished performance
records"). Can someone else find this?

- quote -

I think this quotation from the conclusion is more accurate than your
statement above:

"Our evidence suggests that mutual fund investors could benefit
significantly from investing in funds selected by combining the information
contained in alpha and in our measures, at least before costs and fees."

The qualifier "at least before costs and fees" to me casts enormous doubt on
the real-life usefulness of this method for investing. One would think this
phrase deserved a place in the abstract, but of course then the abstract
would not have such a sensational result to claim (misleadingly, IMO). Or I
guess the regular readership of this journal knows that such claims are to
be read with a grain of salt or not to be taken at face value until the full
article has been read and digested.

I understand there are similar academic studies that demonstrate well that
the market can be timed, at least assuming no transaction costs... If angels
could dance on pinheads?

beliavsky[at]aol.com wrote:

<snip
- quote -

The June 2005 issue of the Journal of Finance has an article on
predicting mutual fund excess returns that uses the alpha method I
described (with the Fama French factors mentioned by Tad) and a new
method of the authors. Here is the citation.

Judging Fund Managers by the Company They Keep
RANDOLPH B. COHEN
JOSHUA D. COVAL
Lubos Pastor
Journal of Finance
Volume 60: Issue 3, June 2005
Abstract: We develop a performance evaluation approach in which a fund
manager's skill is judged by the extent to which the manager's
investment decisions resemble the decisions of managers with
distinguished performance records. The proposed performance measures
use historical returns and holdings of many funds to evaluate the
performance of a single fund. Simulations demonstrate that our
measures are particularly useful in ranking managers. In an
application that relies on such ranking, our measures reveal strong
predictability in the returns of U.S. equity funds. Our measures
provide information about future fund returns that is not contained in
the standard measures.

An earlier version of the paper is at
http://www.people.hbs.edu/rvargas/Randy/judging4.pdf . If the paper is
correct, it IS possible to choose, in real-time, mutual funds that will
on average outperform their benchmarks.

michael[at]bcect.com (Michael Sullivan) wrote:

- quote -

How long do they have to live? In other words, how many years do you need of
good returns to have a 95% confidence it's not luck?

-- Doug