Netscape Navigator and Cookies
by
Robert Bruce Thompson
Revised: December 12, 1998
Other reports by Thompson:
Netscape Navigator and Cookies
December 10, 1998
[The following article is based on material that previously appeared in my Day Notes journal. RBT]
Another data point in the war against cookies. For those who are unfamiliar with them, cookies are one mechanism that can be used to overcome some of the problems associated with the fact that HTTP servers are stateless. Stateless means that every interaction between a client and server is completely independent of other interactions that have occurred between them. The server doesn't keep track of past interactions with the client, even though they may have occurred only a fraction of a second earlier. In other words, no session exists.
For example, if you hit Amazon.com to look for a book, no persistent session is established. Instead, each page and other element you request is delivered to you independently of all other elements served to you during your browser session. The web server has no idea that subsequent requests are related to the original request. It simply delivers individual pages to you as you request them. That makes it hard for the server to do useful things like storing your account information, keeping track of the items in your shopping basket, etc. The kind of things you'd like it to do to improve your browsing experience..
One way the web server can get around this problem is to send your browser a cookie, which is simply a small data file that your browser stores on your local hard disk. In addition to sending you cookies, the web server can also read cookies that it sent to you earlier. For example, when you add a book to your shopping basket, that item is stored in a cookie on your hard drive, allowing the web server to keep an updated list of the current contents of your shopping basket. This is a good and valid use of cookies, as is the use of cookies to store your username and password for sites that require a login. There's a darker side to cookies, however.
Many people think that cookies are no security threat because cookies can only be read by the domain that created them in the first place. That's true as far as it goes, but it doesn't go far enough. A web page can be designed to redirect incoming requests transparently to a different web page, which may reside on that site on on a remote site that uses a different domain name. For example, entering the URL http://www.altavista.com causes the browser to retrieve and display the main AltaVista page. However, that page redirects your browser to a site in the doubleclick.net domain, which records your unintentional visit. Although you never explicitly told your browser to connect to any site in the doubleclick.net domain, this redirection allows doubleclick.net to take control of your browser (unknown to you) and write its own doubleclick.net cookie to your hard disk. If you subsequently visit a different web site that also has an arrangement with doubleclick.net, the doubleclick.net server can read the cookie it wrote during your earlier session and use that information to keep track of where you've been.
In theory--and often in practice--these unauthorized cookies are an aid to tracking ad delivery. But they open the door to abuses of your privacy. Because companies like DoubleClick, Imgis/AdForce, and MatchLogic keep track of which web sites you visit, the potential exists for them to build a profile of your browsing habits. I don't want some faceless company keeping track of what I do on the web, and you probably don't either. Unfortunately, they make it very hard to avoid.
Until recently, both Internet Explorer and Netscape Navigator gave you only three choices about cookies. The default choice, Accept All Cookies, leaves you wide open to cookie abuse. Reject All Cookies keeps you safe from the invasive actions of DoubleClick and their like, but also prevents you from using "good" cookies for their original purpose. The third choice warns you each time a cookie is served to you and allows you to decide individually whether or not to accept each cookie. The problem with that one is that a single web page can deliver many cookies. I have IE setup with the Warn option, and one web page I hit delivered almost 30 cookies. Choosing that option makes the web browser unusable.
There are third-party browser add-ons like CookieCrusher that take a rational approach by allowing you to allow or disallow cookies by domain. Using one of these products, for example, you could allow all cookies except those that originate from DoubleClick, Imgis/AdForce, or MatchLogic. This is the kind of functionality that should be built into IE and Navigator, but isn't, probably because Microsoft and Netscape are implicitly or explicitly in league with those companies.
So when I installed Netscape Navigator 4.05 some time ago, I was pleased to see a new option for handling cookies. The check box is labeled, Accept only cookies that get sent back to the originating server, which sounded exactly what I was looking for. I don't mind accepting cookies from the sites I visit intentionally. It's those stealth cookies delivered by redirection that upset me. So I was heartened to see this new option in Navigator. The trouble is, it doesn't work the way it's supposed to. I don't know if that's because it was never intended to, because it has bugs, or because MatchLogic and other companies have come up with crafty ways to get around it.
What I do know is that I kept a close eye on my cookie file for quite a while after I installed Navigator 4.05, and it appeared to be working as expected. That now turns out not to be the case. I hit my agent's web site last night, and was shocked to see that it's now delivering cookies, including "bad" cookies. I emailed my agent, who told me that he wasn't aware that was going on and that he'd put a stop to it. But getting those cookies from his site motivated me to go out and look at my Netscape cookie file. I wasn't pleased at what I found. Below are some excerpts from it:
# Netscape HTTP Cookie File
# http://www.netscape.com/newsref/std/cookie_spec.html
# This is a generated file! Do not edit.
ads.enliven.com FALSE / FALSE
1893455946
Ogilvy.ngadcenter.net FALSE / FALSE
942189081
.imgis.com TRUE / FALSE
1070371689
.preferences.com TRUE / FALSE
1182140350
I particularly like the gratuitous warning not to edit the file. It's a standard text file, and you can delete anything you please from it. Apparently, they want to dissuade people from deleting "bad" cookies. I greatly resent the way that Microsoft and Netscape appear to actively cooperate with these companies. In fact, I think what all of them are doing is probably illegal under existing law. When I visit a web site, it could be argued that I am implicitly giving that site permission to write its own cookie to my hard disk. But nothing I have done grants permission to these other companies--whose sites I have not voluntarily chosen to visit--to abuse my computer and my hard disk by storing information that is for their own benefit. They are in essence stealing resources from me, and I suspect they could be charged under existing anti-hacking laws if anyone cared to make a point of it. For their defective cookie management mechanisms, Microsoft and Netscape should be charged as accessories before and after the fact.
Dear Jerry,
I have a couple of issues with your article on cookies.
First, let me get the standard disclaimer out of the way: these opinions are mine, and have nothing to do with Netscape. I’m not involved with any of the ad services either.
Point 1: Cookies are not ‘files’ that get sent to the browser - they are just bits of information that get stored in a cookie file. This is a distinction that I think needs to be clear. (I think IE keeps each cookie in it’s own file, but that’s not the way Netscape does it). There are two types of cookies - one type that gets written to the cookie file (a ‘permanent’ cookie that has an expiration date) and a temporary one that goes away when the browser is closed down. The temporary ones never make it into the cookie file - they’re stored in memory.
Point 2: Ad services do not ‘redirect’ the browser to another site - they just reference something that lives on another site. Ad services *never* "take control" of your browser - in fact, no site does this.
Now, a couple of other things:
I did a bit of checking with the cookie settings, and here’s what I think is happening when you have ‘only send back to originating server’ checked. As far as I can tell, this works as advertised - when browsing, if you have this option checked, you don’t get cookies from the ad sites.
The problem shows up when people send you html pages via email. Because there isn’t an ‘originating server’, it seems to make a request for the images in the page, and that allows the ad server cookies to get set.
There are a couple of workarounds for this.
First, do what I do when reading your HTML mail - turn off images.
That will prevent these cookies from getting set.
Another option is to change the permissions on your cookie file to read-only - I haven’t tested this, but it should prevent it from getting updated. You could also create some sort of start-up script that would keep a copy of your ‘approved’ cookie file, and overwrite the existing cookie file before starting up the browser - that way, any unwanted cookies that get saved there during your session will get wiped out next time you start up.
While these things are annoying, keep in mind that as long as content is free, content producers need to have a way to make money from their content and they do this by selling ads.
--
Chris Fullerton
International Web Engineer - http://merchant-int.netscape.com/
Bob Thompson has replied to this; I have inserted this as it was sent, with comments interpolated. I have left the origina above because for myself, I hate reading my own thoughts chopped up with other people's replies interpolated as a gloss, and I expect everyone else does. So here is the letter, with replies. I wish the format were better but I have not time to fix it.
> I have a couple of issues with your article on cookies.
>
> First, let me get the standard disclaimer out of the way: these
> opinions are mine, and have nothing to do with Netscape. I’m not
> involved with any of the ad services either.
>
> Point 1: Cookies are not ‘files’ that get sent to the browser -
> they are just bits of information that get stored in a cookie file.
> This is a distinction that I think needs to be clear. (I think IE
> keeps each cookie in it’s own file, but that’s not the way Netscape
> does it). There are two types of cookies - one type that gets
> written to the cookie file (a ‘permanent’ cookie that has an
> expiration date) and a temporary one that goes away when the browser
> is closed down. The temporary ones never make it into the cookie
> file - they’re stored in memory.
You’re right on all points, of course. I was using the term file loosely. Netscape does indeed store cookie data within one file and IE as separate cookie files. As far as persistent versus non-persistent cookies, the non-persistent ones are useless to the tracking companies, as they could only track your visits within one browser session.
> Point 2: Ad services do not ‘redirect’ the browser to another site -
> they just reference something that lives on another site. Ad
> services *never* "take control" of your browser - in fact, no site
> does this.
I was using the term "redirect" to mean that HTML code on the site that I explicitly visit causes my browser to retrieve data from another site entirely. I call that redirection. If that’s not the correct term technically, I apologize. As far as taking control of my browser, I consider sending code that causes my browser without my knowledge or permission to write data to my hard drive to be taking control of it.
> Now, a couple of other things:
>
> I did a bit of checking with the cookie settings, and here’s what I
> think is happening when you have ‘only send back to originating
> server’ checked. As far as I can tell, this works as advertised -
> when browsing, if you have this option checked, you don’t get
> cookies from the ad sites.
>
> The problem shows up when people send you html pages via email.
> Because there isn’t an ‘originating server’, it seems to make a
> request for the images in the page, and that allows the ad server
> cookies to get set.
Well, no, that can’t be the cause. I say that because I’ve seen this behavior occur on a computer that doesn’t have a mail client installed. It sits behind a proxy server, and POP and SMTP aren’t configured on that client. I’ve also watched it occur on another computer that does have an email client installed. In this case, the mailer wasn’t opened, my cookie file had no "bad" cookies on it. I visited many sites during that browser session, so I can’t say for sure which site did it, but I ended up with a preferences.com cookie in my Netscape cookie file afterwards. I know that that couldn’t have been caused by mail, either, because my POP server happened to be down all that afternoon.
As I mentioned, Netscape’s "Accept only cookies that get sent back to the originating server" seems to work most of the time, so I’m willing to concede that Netscape is at least attempting to address the situation. I’ve had several other people tell me basically the same thing—that this option keeps most but not all bad cookies off their hard drives. If I had to guess, I’d say that preferences.com and other similar companies have come up with a way around this restriction.
> There are a couple of workarounds for this.
>
> First, do what I do when reading your HTML mail - turn off images.
> That will prevent these cookies from getting set.
>
> Another option is to change the permissions on your cookie file to
> read-only - I haven’t tested this, but it should prevent it from
> getting updated. You could also create some sort of start-up script
> that would keep a copy of your ‘approved’ cookie file, and overwrite
> the existing cookie file before starting up the browser - that way,
> any unwanted cookies that get saved there during your session will
> get wiped out next time you start up.
Yes, but again, the point is why should I have to do this? I have not given doubleclick.net, Imgis/AdForce, Preferences.com or any of those companies permission to write cookie data to my hard disk. They are doing this for their own benefit, without my permission, and against my wishes. What you are suggesting is equivalent to letting a burglar go free and blaming the homeowner for not installing better locks.
> While these things are annoying, keep in mind that as long as
> content is free, content producers need to have a way to make money
> from their content and they do this by selling ads.
They’re free to sell all the ads they want. I’ll even look at one once in a great while. But what they aren’t free to do is hijack my hard drive for their own nefarious purposes. What they perceive as their own need gives them no claim whatsoever on my resources.
Regards.
Bob
Copyright © 1998 by Robert Bruce Thompson. All Rights Reserved.
Send feedback to: webmaster@ttgnet.com
—30—
