website traffic analysis, statistics, web tracking, website log file reports & analysisBellaCoola Software Corporation

HOME

CORPORATE
 - Corporate Profile

WEB TRACKING
 
- The Problem With Site Logs
   -
Caching Explained
 -
BellaCoola Tracker Series
 -
Sample Reports
   -
Sample BellaCoola Log
   -
Sample HitList Report
   -
Sample WebTrends Report
 -
Pricing Plans
 -
Reseller / Volume Discounts
 -
Terms and Conditions
 -
Join Now!

CLIENT AREA
 - Your Private Stats
 -
Check your account balance
 -
Add an additional account
 -
What´s New
 -
Help with Installing Trackers
 -
Usage Tips
 -
Known Issues/Bug Reports
 -
Technical Support
 -
Feature Requests
 -
Forget Your Password?

PROGRAMMING
 - Java, JavaScript,
  
CGI services
 -
Web Database Design
 -
Case Study of the Month
 -
Got a Question?

CONTACT US
 - How to Reach Us
 -
Ask a Question

Comments:
 

Industry-Standard BellaCoola Log Files

Overview

This white paper describes the most-commonly used log file format in the industry: NCSA´s “Extended Common Log File Format” and how to read it.

What’s in The BellaCoola Tracker Log File?

The BellaCoola sniffers produce an industry-standard log file (Extended Common Log File Format) that can be analyzed by most commercial and shareware analysis programs.  This format was designed to be machine-readable, but die-hard (and curious) webmasters can learn a lot by reading the log files themselves.

A log file consists of a series of entries, one for each page that is viewed by a visitor.

A typical entry in the log file looks like this (word-wrapped to fit on this screen):

dial1-30-45.nbn.net - bgates [2/Sep/1998:19:54:14 +0000] "GET /html/win95_updates.htm HTTP/1.0" 200 54 "http://www.infoseek.com/Titles?qt=%22OEM+service+release+2%2 2&col=New+Search&oq=%22service+release+2%22&sv=N4&lk=ip-nofra mes&nh=10" "Mozilla/4.01 [en] (Win95; I)"

Let’s break this mumbo-jumbo down into its components:

1) The “internet name” of the user accessing your site:
dial1-30-45.nbn.net
Typically, this name doesn’t identify “who” is accessing your site (nor their email address), just the domain name of their ISP.  In this case, it is a user dialing in through nbn.net

If you’re interested in who owns the name nbn.net, you can always look it up in the InterNIC database.  (Try it yourself) Note that InterNIC only administers .COM, .ORG and .NET domain names.

2) Some (normally) unused fields:
- bgates
The second field will the login ID of users within password-protected parts of your site.  For “public” portions of your site, these fields will be “- -”.  In this example, “bgates” is visiting the password-protected portion of our site.

3) The date and time of the page request (in GMT):
[2/Sep/1998:19:54:14 +0000]
Because your visitors access your site from all over the world, all times are recorded in Greenwich Mean Time (GMT).  To convert GMT to your local time zone, use the following chart:

Pacific Standard Time

subtract 8 hours

Pacific Daylight Time

subtract 7 hours

Eastern Standard Time

subtract 5 hours

Moscow

add 3 hours

  • Because our log files use GMT, they are perfect for any web site, no matter where your visitors are located.

4) The name of the page viewed on your site:
"GET /html/win95_updates.htm HTTP/1.0"
The meat is in the middle (as they say).  Just ignore anything that doesn’t look like a URL.

5) Two more unused fields (in our case) (for the curious: Status Code and Transfer Size):
200 54
All the entries in your BellaCoola log will have these same values.

6) The referer field—perhaps the most important information you can gather:
"http://www.infoseek.com/Titles?qt=%22OEM+service+release+2%2 2&col=New+Search&oq=%22service+release+2%22&sv=N4&lk=ip-nofra mes&nh=10"
This tells you what page the visitor came from.  In this case, the user was searching for the phrase “OEM service release” in InfoSeek.

  • You can view the page that the user came from by typing the referer value into your browser (Try it!)

If the user typed in your URL directly, or called it up as a bookmark (great news!) then this field will be “(none)”.

7) The User Agent field (aka The Browser field):
"Mozilla/4.06 [en] (Win95; I)"
This shows what browser the user was using.  In this case, it’s Netscape (code-named Mozilla) version 4.06, english, International version under Windows95. 

Whew!  That’s a lot of useful information, especially when you’re considering adding “advanced content” that may not be supported by all browsers to your sites (frames, cascading style sheets, Java applets...).

So how do we use all this information?

Let’s take a look at a couple of ways that we can use this information to recreate how a user used our site.

Scenario 1:

a) A visitor arrives from Lycos (where s/he searched for "best page")
207.bridgeton-011.mo.dial-access.att.net - - [10/Sep/1998:07:38:55 +0000] "GET /html/best_of_the_www.htm HTTP/1.0" 200 54 "http://www.lycos.com/cgi-bin/pursuit?query=best+AND+page&bac klink=217&maxhits=10" "Mozilla/3.0Gold (Win95; I)"

b) drills down to the Win95 Updates part of our site (taking 7-1/2 minutes to reach it—must be taking the time to carefully read each page)
207.bridgeton-011.mo.dial-access.att.net - - [10/Sep/1998:07:39:56 +0000] "GET /html/web_tools.htm HTTP/1.0" 200 54 "http://www.bellacoola.com/html/best_of_the_www.htm" "Mozilla/3.0Gold (Win95; I)"

207.bridgeton-011.mo.dial-access.att.net - - [10/Sep/1998:07:41:13 +0000] "GET /html/conferencing.htm HTTP/1.0" 200 54 "http://www.bellacoola.com/html/web_tools.htm" "Mozilla/3.0Gold (Win95; I)"

207.bridgeton-011.mo.dial-access.att.net - - [10/Sep/1998:07:42:04 +0000] "GET /html/web_tools.htm HTTP/1.0" 200 54 "http://www.bellacoola.com/html/best_of_the_www.htm" "Mozilla/3.0Gold (Win95; I)"

207.bridgeton-011.mo.dial-access.att.net - - [10/Sep/1998:07:45:31 +0000] "GET /html/web_tools.htm HTTP/1.0" 200 54 "http://www.bellacoola.com/html/best_of_the_www.htm" "Mozilla/3.0Gold (Win95; I)"

207.bridgeton-011.mo.dial-access.att.net - - [10/Sep/1998:07:45:50 +0000] "GET /html/conferencing.htm HTTP/1.0" 200 54 "http://www.bellacoola.com/html/web_tools.htm" "Mozilla/3.0Gold (Win95; I)"

207.bridgeton-011.mo.dial-access.att.net - - [10/Sep/1998:07:46:19 +0000] "GET /html/win95_updates.htm HTTP/1.0" 200 54 "http://www.bellacoola.com/html/conferencing.htm" "Mozilla/3.0Gold (Win95; I)"

c) then follows our link to a page in the Microsoft site (a download page)
207.bridgeton-011.mo.dial-access.att.net - - [10/Sep/1998:07:47:59 +0000] "GET http://www.microsoft.com/ntserver/info/PPTPdownload1.htm HTTP/1.0" 200 54 "/html/win95_updates.htm" "Mozilla/3.0Gold (Win95; I)"

d) returns 3-1/2 minutes later (yay!) by using the Back Arrow
NOTE: No other web tracking tool will show you this information!  Not even standard web server logs.
207.bridgeton-011.mo.dial-access.att.net - - [10/Sep/1998:07:51:17 +0000] "GET /html/win95_updates.htm HTTP/1.0" 200 54 "http://www.bellacoola.com/html/conferencing.htm" "Mozilla/3.0Gold (Win95; I)"

e) then again follows a link off our site to get the 12+ Win95 updates from Microsoft
207.bridgeton-011.mo.dial-access.att.net - - [10/Sep/1998:07:52:08 +0000] "GET http://www.microsoft.com/windows/software/updates.htm HTTP/1.0" 200 54 "/html/win95_updates.htm" "Mozilla/3.0Gold (Win95; I)"

How to Interpret Scenario 1

These log file entries tell us a lot of valuable information about how well our site is (or isn’t) designed:

  • This visitor took 7-1/2 minutes to read 6 pages (over a minute per page).  It appears that the visitor is taking time to read each page thoroughly, rather than just randomly clicking through the site.
  • The visitor came back after clicking an off-site link!  We find that this is one of the strongest indicators of good site design.
    We know of no other tool that can show you this. 
  • It is completely unaffected by browser caching and proxy servers.  You can completely track your visitors’ sessions, whether they use the Back Arrow or not.
  • By analyzing hundreds of users’ visits with a commercial or shareware log analysis program, you can find out what is and what isn’t working with your site’s design.

It’s just like standing in your shop while shoppers wander in and out of your store.

Scenario 2:

I recently posted an article to the link exchange newsgroup discussing the popularity of various search engines. 

Many web marketers call newsgroup postings one of the most effective marketing tools available today.

If I were using traditional web server logs, I would have absolutely no idea of how many people were actually reading my newsgroup messages.

Standard web server logs cannot capture this valuable marketing information.

However, by simply dropping a BellaCoola sniffer in my newsgroup messages, I now know that >75 people read this post this first day--a lot more eyeballs than saw my postings on other newsgroups. 

With this sniffer, I can gauge:
a) The number of eyeballs reading my postings, and
b) The percentage click-throughs as a result.  i.e. no more guessing which newsgroups are truly effective at reaching your target market.

Here’s what my logs show:  the 2 most recent people to read my posting were:

resh1509.tigernet.trinity.edu - - [10/Sep/1998:20:14:10 +0000] "GET news://news2.linkexchange.com/340DFA12.AE098B48@bellacoola.co m HTTP/1.0" 200 54 "le.discuss.popularity" "Mozilla/3.01Gold (Win95; I)"

204.71.189.74 - - [10/Sep/1998:20:59:27 +0000] "GET news://news2.linkexchange.com/340DFA12.AE098B48@bellacoola.co m HTTP/1.0" 200 54 "le.discuss.popularity" "Mozilla/3.01Gold (Win95; I)"

Notice that they didn’t need to click through to our site to be logged—they only needed to read an article we posted on a newsgroup. 

Armed with this information, I can now track:

  • newsgroup viewership, and
  • click-through rates.

No more second-guessing why your web traffic increased.

  • Which newsgroups are generating the best leads for your business?
  • Which newsgroups are not being read?
  • Which signature slogans have the highest appeal?

We know of no other tool that can reliably tell you this information.

Sign up and start getting the whole picture today!   You´ll see why we´re the professional´s choice for all-in-one web tracking.

BellaCoola Tracker Series  |  Sample Reports  |  Pricing
Join Now!

CorporateReseller  |  Programming  |  Contact Us

BellaCoola®, WebHound® and Adios® are registered trademarks of BellaCoola Software Corporation.

BellaCoola Software Corporation, 2346 Hamiota Street, Victoria BC CANADA V8R 2N2
 Tel: 250/384-6237   email:
Copyright ©1996-2013,BellaCoola Software Corp., All Rights Reserved