u2json - A unified2 to JSON converter

Read unified2 log files and output records as JSON.

Usage

usage: idstools-u2json [-h] [-C <classification.config>] [-S <msg-msg.map>]
                       [-G <gen-msg.map>] [--snort-conf <snort.conf>]
                       [--directory <spool directory>]
                       [--prefix <spool file prefix>] [--bookmark <filename>]
                       [--follow] [--delete] [--output <filename>] [--stdout]
                       [--sort-keys] [--verbose] [--packet-printable]
                       [--packet-hex] [--extra-printable]
                       [filenames ...]

positional arguments:
  filenames

options:
  -h, --help            show this help message and exit
  -C <classification.config>
                        path to classification config
  -S <msg-msg.map>      path to sid-msg.map
  -G <gen-msg.map>      path to gen-msg.map
  --snort-conf <snort.conf>
                        attempt to load classifications and map files based on
                        the location of the snort.conf
  --directory <spool directory>
                        spool directory (eg: /var/log/snort)
  --prefix <spool file prefix>
                        spool filename prefix (eg: unified2.log)
  --bookmark <filename>
                        enable bookmarking
  --follow              follow files/continuous mode (spool mode only)
  --delete              delete spool files
  --output <filename>   output filename (eg: /var/log/snort/alerts.json
  --stdout              also log to stdout if --output is a file
  --sort-keys           the output of dictionaries will be sorted by key
  --verbose             be more verbose
  --packet-printable    output printable packet data in addition to base64
  --packet-hex          output packet data as hex in addition to base64
  --extra-printable     output printable extra data in addition to base64

If --directory and --prefix are provided files will be read from the specified
'spool' directory. Otherwise files on the command line will be processed.

Example - View unified2 File as JSON

idstools-u2json /var/log/snort/unified2.log.1397575268

To resolve alert descriptions and classifications:

idstools-u2json --snort-conf /etc/snort/etc/snort.conf \
    /var/log/snort/unified2.log.1397575268

The above assumes that sid-msg.map, gen-msg.map and classification.config live alongside the specified snort.conf. If they do not, the options to specify each individually may be used:

idstools-u2json -C /etc/snort/etc/classification.config \
    -S /etc/snort/etc/sid-msg.map \
    -G /etc/snort/etc/gen-msg.map \
    /var/log/snort/unified2.log.1397575268

Example - Continuous Conversion to JSON

idstools-u2json --snort.conf /etc/snort/etc/snort.conf \
    --directory /var/log/snort \
    --prefix unified2.log \
    --follow \
    --bookmark \
    --delete \
    --output /var/log/snort/alerts.json \

The above command will operate like barnyard, reading all unified2.log files in /var/log/snort, waiting for new unified2 records when the end of the last file is reached.

Additionally the last read location will be bookmarked to avoid reading events multiple times, the unified2.log files will be deleted once converted to JSON, and JSON events will be written to /var/log/snort/alerts.json.

Configuration File

A configuration file is simply a file containing the command line arguments, one per line with an ‘=’ separating the name from the argument. For example, to save the arguments used in example 2 above:

--snort-conf=/etc/snort/etc/snort.conf
--directory=/var/log/snort
--prefix=unified2.log
--follow
--bookmark
--delete
--output=/var/log/snort/alerts.json

Then call idstools-u2json like:

idstools-u2json @/path/to/config-file

Addtional arguments can also be provided like:

idstools-u2json @/path/to/config-file --stdout