Log Parser Lizard creating custom regex format

I first heard about Log Parser Lizard (LPL) when Scot Hanselman blogged about it back in 2011 and since then its been part of my tool belt.

LPL makes analysing logs easy enabling you to point it at single log file, or even a directory of files, it will parse them and then allow you to query the data using SQL syntax.

LPL comes with a lot of predefined searches from Active Directory, IIS, event logs, etc and has the ability to allow you to define your own custom RegEx format for parsing text files.

Recently I needed to use the custom RegEx format, its not the easiest thing to do and as I couldn’t find any examples I thought I’d write up what I did to help me, and possibly anybody else that needs to do it, in the future.

Format

The custom format is stored in an xml file and consists of:

• Regular Expression
• Fields for LPL to use

Fields

I’m covering fields first as they’re the easy part to understand, each field is a xml element with name and data type <field name="Priority" type="Integer"/>.

LPL will only give you columns based on the fields that you define and as I understand it the types are standard .Net types.

Regular Expression

First you need to create a RegEx that will correctly parse the line, so if a line in your text file contains a number, date and text such as:

1 2014-10-31 Message text

This RegEx will parse that line:

^(\d)\s(\d{4}-\d{2}-\d{2})\s(.*)$

And will give you 3 capture groups:

1. The numnber: 1
2. The date: 2014-10-31
3. The text: Message text

In LPL is you create a RegEx Import Format query and configure it to use this RegEx it will successfully execute the query but you’ll get no results in the grid.

The thing that I was missing and couldn’t find info on is that you have to add additional capture groups to the RegEx, wrapping the existing capture groups, to allow LPL to understand which capture group belongs to which field.

The format for the LPL capture group is (?&lt;’Field Name’&gt;(\d)) where ‘Field Name’ is the name of the field that you want to store the value in. So our previous RegEx would be need to be altered to:

^(?&lt;Number&gt;(\d))\s(?&lt;Date&gt;(\d{4}-\d{2}-\d{2}))\s(?&lt;Text&gt;(.*))$

Sample Files

I’ve created a sample config and corresponding log file for the regex above which you can find here to allow you to try this out and see it working.