On logs that have several header lines prior to the actual columnar data i usually have to manually pare the file down for the column data to work properly. The other issue is multiple separators. Variable separators (for space formatted logs) the option to treat multiple consecutive separators as a single separator would enable logs that have variable separators between columns of data of one or more separators to more cleaning parse otherwise its just better to load the file without column structuring on these types of files in my experience.

Oct 29, 2011  • #1

I've added the option for treating multiple consecutive separators as one column to our feature request list.

As for the header lines before the columnar data, could you provide an example log file that has this issue?

Thanks!

Keith Lammers
Binary Fortress Software

Nov 3, 2011  • #2

This is only an example but occurs throughout much of I.T. to where a log file attempts to explain the subsequent lines of data



When the column feature attempts to structure this file into columns it gets caught up on the initial lines being set into columns and the subsequent lines not being seperated correctly.

Nov 4, 2011 (modified Nov 4, 2011)  • #3

Awesome, thanks! I'll check it out and report back.

Keith Lammers
Binary Fortress Software

Nov 4, 2011  • #4

I just noticed that this forum software doesnt allow tabs to be inserted into the post text. Im not sure how to properly represent the file. I'll grab another one and see if i can just attach it.. If you want to replicate what i did, I just exported the windows event log.

Nov 5, 2011  • #5

It seems to work ok for me. I exported the System Event Log as a Tab Separated text file. Opened it in LogFusion and added 6 tab separated columns. See the attached screenshot for the output.

Perhaps I'm testing it differently than you are? If you could attach a log that's broken, along with a screenshot of the column configuration in LogFusion, that would be fantastic.

Thanks!

Keith Lammers
Binary Fortress Software

• Attachment: System Event Log with Columns.jpg [426,043 bytes]

Nov 8, 2011 (modified Nov 8, 2011)  • #6

Could you also provide a sample log for the variable separators, and let me know how you currently configure the columns, and how you would like to be able to configure them? There are a few different ways we could handle the variable separators, so we just want to make sure we're understanding your request fully.

Thanks!

Keith Lammers
Binary Fortress Software

Nov 8, 2011  • #7

Below is what i call a dirty log where the separator ISN'T the space in between the different columns of data. Rather its parsed via regex where the DATE is filtered into a column follow by a single space and then a single digit followed by a single space followed by a right justified (Variable number of white-space) number of variable length followed by a right justified number of variable length again followed by variable length text. This is a nightmare to attempt any kind of separator defined column separation.

• Attachment: example_log_lines.png [60,577 bytes]

Nov 9, 2011 (modified Nov 9, 2011)  • #8

Thanks David! We'll see what we can do to make this easier to setup

Keith Lammers
Binary Fortress Software

Nov 11, 2011  • #9

In Beta 3 we've added an option to strip out multiple consecutive separators and treat them as one.

As for parsing columns using RegEx, we won't be able to include it in this version, but we'll work on it for LogFusion 2.1.

Thanks!

Keith Lammers
Binary Fortress Software

Nov 15, 2011  • #10

Great this makes the columns very readable. I have a few more features beyond the basics but this makes the feature very workable.

When possible can we still have "Remove header line(s) from beginning of log" as an option on the column configuration page?

Nov 16, 2011  • #11

Glad to hear it! I had missed the request about removing the column headers from the log but I've added it to our list now. We'll see what we can do

Keith Lammers
Binary Fortress Software

Nov 16, 2011  • #12