Saturday, December 7, 2013

IIS Advanced Logging

I recently needed a solution to reduce the size of extremely large log files created by Internet Information Services (IIS).
We recently added a new feature to our Web server application that required polling the server on a regular basis. Every client browser with a current session was making new server requests about every two minutes. The result was that our log files went from about 200MB per day to over 1GB per day and this caused the server's storage to fill up at least once per week.
After researching this, I found that the Advanced Logging IIS extension would solve this issue at Advanced Logging allows a much higher degree of control over the standard IIS logging that is available:
  • Select which fields to log.
  • Ordering of the fields.
  • Filter only for the desired logs.
  • Choose the name of the log file.
  • Create more than one log file.
  • etc.
Here are the steps I took to install IIS Advanced Logging. This required a simple Microsoft add-on for IIS7 to be installed on that machine. It took just a few minutes.

  1. Download (see link above) and install the Advanced Logging extension.
  2. Ensure normal logging is turned off. Standard logging is unaffected by Advanced Logging. Since we have an overwhelming number of logs coming in, turning this off prevents storage overruns.
  3. Enable Advanced Logging at the server level. By default, this is disabled after installation.
  4. Edit the Log Definition – this gave me control over two things:

  1. I can select which fields get logged out. At first I was trying to match what the standard log fields were but then realized some of these were unnecessary; i.e. server IP.
  2. I can set a filter to ignore the those excessive client requests here.

  1. Recycle the Portal’s App Pool. Apparently the logging doesn’t start until then.

Making Adjustments
To tune Advanced Logging, open the IIS Manager Console, select the server and the see the Features View. You should see the Advanced Logging icon under the IIS grouping.

Open Advanced Logging, select the only Log Definition listed (in the list-view), and click the Edit Log Definition... link button. You’ll see the Selected Fields that are logged to the output file. These fields can be reordered, added, or removed. Below the Selected Fields group, there is the Filter group where I filter only (allow logging for) requests that do NOT contain the ‘/signalr/poll’ in the URI-Stem field.

A few things to keep in mind:
  • Changing these settings will recycle the app pool and reset everyone’s session.
  • Adding another Log Definition will create another log file output.
  • Logging is output to ...\inetpub\logs\AdvancedLogs (by default)  instead of the standard logging location of ...\inetpub\logs.