| Friday,
January 2, 2003 It's raining in Los Angeles. It never rains
on the First though.
Column and finishing Burning Tower. More work...
====================
Can someone explain to me just why anyone accepts
a computer model that uses 3 decimal place accuracy on
single place data as the arbiter of who gets to be the Number 1 ranked
football team in the country? I have searched and all I find is a bunch of
mumbo-jumbo about how the computer systems are more "just" and "accurate"
and sometimes an interminable list of variables they different computer
systems use, but nothing about what weights are given to each of the
variables, or even how some of the variables are calculated.
Clearly some iterative process is going on since it is said that these
things take an hour or more to run, but once again I find nothing on how it
is done. Surely if we are to believe in these things they ought not be
secret?
When I took Systems Analysis from Herman Kahn (Boeing paid for him to
come teach a seminar for a couple of weeks back in about 1960; stiincidentally,
does anyone know where I can find a copy of Kahn, Herman. Techniques
of Systems Analysis. Report, Rand Corp,
1956-1957. 161 p. RAND-RM-1829-1.still one of the best
introductions to the techniques ever written? [found the source thanks to
readers]) -- anyway, in those days the
purpose of systems analysis was to make explicit the processes by which
command decisions were made. We could show the general what prompted a given
recommendation. He still had to make his own decisions, but with systems
analysis you could make it clear what factors the advice was based on, and
the weight given to those factors.
Now I don't know if the BCS systems are linear regression models or some
kind of iterative process (I would guess the later simply because of the
reported times they take to run), or how they were validated. I presume they
were validated? But against what? Since one of the BCS computer model system
inventors has taken the trouble to write denouncing the AP and Coaches polls
as "too democratic" while the computer systems are "more just", it's a
matter of curiosity: what did he use to validate his computations? If his
system had consistently spit out a terrible team of losers as Number One, it
would be simple enough to reject it: but how do you validate a 3rd decimal
point calculation ranking one team second and one team third? What
determines if the computer got it right or not?
I would like to see those computer models. I'd like to know how they were
and are validated. And why the New York Times has one that seems way off
compared to the others -- what does it do? Wrong variables? Different
weights? So far I have been unable to find out any of this, which may be a
comment on my Google skills, or may be a comment on just how gullible people
are. But real money rests on these ratings: and to make them in secret,
straining like a gear box to produce a result without showing how the result
was obtained, seems odd, and accepting these things on the grounds that they
are "computer models" puts more faith in these machines than I have.
I could come up with a "computer model" of this stuff, if I had any kind
of validation criterion. The method is relatively simple. List all the
quantifiable variables you think might have a bearing on how to rank order
teams from best to worst; put them into a big matrix of predictors; do
matrix manipulations to get the correlation of each of those variables with
the vector of validation scores; and build a multiple regression equation
from as many of those variables as you need to account for as much of the
variance as you can account for. This isn't very difficult given small
computers, and the model could be written in BASIC or PASCAL or MODULA so
that it would be more or less readable.
But without a vector of valid ratings -- without some validation method
-- the model will do no more than predict things according to the opinion of
the model maker on the relative importance of the variables. Sure, I can
make a model that will get closer and closer to the rank orderings produced
by any other non-random method based on those variables; and so what?
One account of one of the models said it was better because it made use
of the "standard deviation". Standard deviation of what about which mean was
not specified, but apparently using the standard deviation makes that a
better model, at least according to one enthusiast.
Anyone have a source on how to find out more about what is in those
models?
Incidentally, apologies for spending time on this rather than a year end
analysis of technology; I made the mistake of reading the sports page at
lunch, and it was filled with discussions of the worth or lack of worth of
these computer models -- and very little on how the models are constructed.
Here and there were vague lists of variables including "the standing of the
teams played" (another reason for supposed these are iterative models) but
nothing on the weights given the different variables. Ah well. More
substance another time.
====================================
And if one of you knows how I can find a copy of
Kahn, Herman. Techniques of Systems Analysis. Report, Rand
Corp,
1956-1957. 161 p. RAND-RM-1829-1
or
Quade, Edward S. Analysis for Military Decisions,
or
Quade, Edward S. Military Systems Analysis RAND Corp report
RAND-RM-3452-PR
I would greatly appreciate the information, and I'll pay to have copies
made if there is no other way to get a copy of each. Those are old
studies but they are still among the best on both how to make models and the
limits of modeling.
And shortly after
Subject: RAND reports
Available here:
http://www.rand.org/Abstracts/
http://www.rand.org/cgi-bin/Abstracts/ordi/getabbydoc.pl?doc=RM-1829-1
---- Roland Dobbins
Thanks. I have ordered the ones I need
RAND accepted an order but another mail says that Herman's reports are no
longer available. We will see. Quade's lectures are more commonly available,
but Kahn's was a good disquisition on why you do this kind of analysis as
well as some techniques for doing it.
============================
Don't we feel safer now?
http://www.washingtonpost.com/wp-dyn/articles/A48173-2004Jan1.html
==================
I must use the wrong techniques. I tried all kinds of entries in hopes of
finding a way to buy old RAND documents, and got nothing useful; minutes
after I posted the above I get five emails telling me where to go.
Of course it can be said I use the right technique which is to ask
someone who knows. Pournelle's law, if you don't know what you are doing,
work with someone who does...
===================
Our Haiti Intervention:
http://www.nytimes.com/2004/01/02/international/americas/02HAIT.html
==========================
Shout Hallelujah!
Dan scrubbed the Mac and reinstalled OS X 10.3 with all traces of Thursby
AdmitMac gone.
We found that Apple sold me a "Mac with the new operating system" with
10.2 installed, and an UPGRADE version of 10.3 in the box. This made it
harder to do than it ought to have been. They did generously include an
installation version of 10.2 on my brand new system for which I paid full
price at the Apple Store at the Glendale Galleria. Buyer beware.
But with the new version, with the downloaded patch, it was simple enough
to connect to the Active Directory net. There are minor glitches, but
nothing that stops the show. See the column for details, but Active
Directory and Mac with the patched version of 10.3 (10.3.2) work together
quite well. The glitch is that from the Windows side I can see Ariadne by
name, but I have to connect to her by IP address
\\192.168.1.129; if I do that it works
splendidly.
Friday TOP
Current Mail
|
CLICK HERE for
a Special Request. 
