Unified2 File Reading

idstools provides unified2 readers for reading unified2 records.

Reader Objects

RecordReader

class idstools.unified2.RecordReader(fileobj)

RecordReader reads and decodes unified2 records from a file-like object.

Parameters:fileobj – The file-like object to read from.

Example:

fileobj = open("/var/log/snort/merged.log.1382627987", "rb")
reader = RecordReader(fileobj):
for record in reader:
    print(record)

next()

Return the next record or None if EOF.

Records returned will be one of the types Event, Packet, ExtraData or Unknown if the record is of an unknown type.

tell()

Get the current offset in the underlying file object.

FileRecordReader

class idstools.unified2.FileRecordReader(*files)

FileRecordReader reads and decodes unified2 records from one or more files supplied by filename.

Parameters:files... – One or more filenames to read records from.

Example:

reader = unified2.RecordReader("unified2.log.1382627941",
                               "unified2.log.1382627966)
for record in reader:
    print(record)

next()

Return the next record or None if EOF.

Records returned will be one of the types Event, Packet, ExtraData or Unknown if the record is of an unknown type.

tell()

Returns the current filename and offset.

SpoolRecordReader

class idstools.unified2.SpoolRecordReader(directory, prefix, init_filename=None, init_offset=None, follow=False, rollover_hook=None)

SpoolRecordReader reads and decodes records from a unified2 spool directory.

Required parameters:

Parameters:

• directory – Path to unified2 spool directory.
• prefix – Filename prefix for unified2 log files.

Optional parameters:

Parameters:

• init_filename – Filename open on initialization.
• init_offset – Offset to seek to on initialization.
• follow – Set to true if reading should wait for the next record to become available.
• rollover_hook – Function to call on rollover of log file, the first parameter being the filename being closed, the second being the filename being opened.

Example with following and rollover deletion:

def rollover_hook(closed, opened):
    os.unlink(closed)

reader = unified2.SpoolRecordReader("/var/log/snort",
    "unified2.log", rollover_hook = rollover_hook,
    follow = True)
for record in reader:
    print(record)

next()

Return the next decoded unified2 record from the spool directory.

tell()

Return a tuple containing the filename and offset of the file currently being processed.

Record Types

Event

class idstools.unified2.Event(event)

Event represents a unified2 event record with a dict-like interface. The unified2 file format specifies multiple types of event records, idstools normalizes them into a single type.

Fields:

• sensor-id
• event-id
• event-second
• event-microsecond
• signature-id
• generator-id
• signature-revision
• classification-id
• priority
• source-ip
• destination-ip
• sport-itype
• dport-icode
• protocol
• impact-flag
• impact
• blocked
• mpls-label
• vlan-id

Deprecated: Methods that return events rather than single records will also populate the fields packets and extra-data. These fields are lists of the Packet and ExtraData records associated with the event.

Packet

class idstools.unified2.Packet(*fields, **kwargs)

Packet represents a unified2 packet record with a dict-like interface.

Fields:

• sensor-id
• event-id
• event-second
• packet-second
• packet-microsecond
• linktype
• length
• data

ExtraData

class idstools.unified2.ExtraData(*fields, **kwargs)

ExtraData represents a unified2 extra-data record with a dict like interface.

Fields:

• event-type
• event-length
• sensor-id
• event-id
• event-second
• type
• data-type
• data-length
• data