README

This idea was born because of the need for a simple tool in order to automate the execution of simple log analysis obtained from Nokia SROS routers. The tool reads the content of a log in txt or json format, parses only pre defined variables and does the comparison, thus verifying if there are changes in such variables . The logs are generated by the use of taskAutom.

Setup

System Libraries

These libraries have been tested under Ubuntu 20.04 and Python3.8.

sudo pip3 install -r requirements.txt

Compile

You can run logAnalyzer directly from the CLI using Python. However, compiling improves performance.

python3 -m nuitka logAnalizer.py

Compiling has been tested succesfully under Ubuntu. Don't know if this is directly supported under Windows. If it fails, let me know. Nevertheless, as mentioned, you can run logAnalyzer_win directly from the CLI using Python

Usage

The program needs two mandatory inputs:

• a folder which contains the parsing templates, either in textFSM or ttp format;
• a folder, which contains the logs obtained a router, via taskAutom. Though not mandatory, taskAutom is suggested as a way of obtaining the logs, because these can be stored in a json file.

Templates

The templates are stored by default under the Templates/ folder. logAnalyzer reads either the CSV or the template folder to perform the function of parsing.

Inside each template file, one can include control variables. Also, some mandatory comments are needed.

- Mandatory

#Command: /show router bgp summary

The above is neede in each template file to let logAnalyzer know which command we are trying to parse. We could use some variables as well, suche as #Command: /show router \S+ interface.

- Optional

#filterAction:exclude or include-only
#filterColumns:Var1,Var2,Var3
#majorDown:String1,String2

These are control keywords. The control keyword #filterAction allows only the actions exclude or include-only. This will modify the resulting columns of the report. The resulting columns sholud be declared under the control keyword #filterColumns.

The keyword #majorDown allows us to declared a number of template-specific keywords that logAnalyzer will look for when processing the outputs. So, for example, if our output should be considerd as down when the string connect is seen, then connect should be placed in #majorDown.

CSV

It is possible to use a CSV file, that includes the specific templates to be used in the analysys.

show_router_bgp_summary.template
show_router_interface.template
show_service_sdp.template

If omitted, all the templates insisde the templates folder, will be used.

Results

If logAnalyzer is invoked only with option -pre, reads the specific content in the log folder for a given command and then saves the results in an Excel report. We need to specify format of the logs, either -json yes|no and also the parsing engine, either -te textFSM|ttp. Also, the folder where the templates are located, with -tf Folder.

$ python3 logAnalyzer.py -csv templateExample.csv -pre folderLogs/ -json yes -te ttp -tf TemplatesTTP/

<_io.TextIOWrapper name='Templates/nokia_sros_show_service_sdp-using.template' mode='r' encoding='UTF-8'>
#####Plantillas Cargadas Exitosamente#####
#########Logs Cargados Exitosamente#########
ROUTER_EXAMPLE_rx.txt nokia_sros_show_service_sdp-using.template
#
#
Saving Excel
#

On the other hand, if logAnalyzer is invoked with folder -pre and -post, it compares the content of pre and post log folders, such as if we run checks to see the status of the routers before and after a task, and then saves the results in an Excel report.

$ python3 logAnalyzer.py -csv templateExample.csv -pre folderLogsBefore/ -post folderLogsAfter/ -json yes --te textFSM -tf TemplatesFSM/

<_io.TextIOWrapper name='Templates/nokia_sros_show_service_sdp-using.template' mode='r' encoding='UTF-8'>
#####Plantillas Cargadas Exitosamente#####
#########Logs Cargados Exitosamente#########
#########Logs Cargados Exitosamente#########
ROUTER_EXAMPLE_rx.txt nokia_sros_show_service_sdp-using.template
ROUTER_EXAMPLE_rx.txt nokia_sros_show_service_sdp-using.template
#
#
Saving Excel
#

Configuration Options

logAnalyzer can be configured through CLI as shown below.

$ python3 logAnalyzer.py -h
usage: PROG [options]

Log Analysis

optional arguments:
  -h, --help            show this help message and exit
  -pre PREFOLDER, --preFolder PREFOLDER
                        Folder with PRE Logs. Must end in "/"
  -post POSTFOLDER, --postFolder POSTFOLDER
                        Folder with POST Logs. Must end in "/"
  -csv CSVTEMPLATE, --csvTemplate CSVTEMPLATE
                        CSV with list of templates names to be used in parsing. If the file is omitted, then all the templates inside --templateFolder, will be considered for parsing. Default=None.
  -json {yes,no}, --formatJson {yes,no}
                        logs in json format: yes or no. Default=yes.
  -tf TEMPLATEFOLDER, --templateFolder TEMPLATEFOLDER
                        Folder where templates reside. Used both for PRE and POST logs. Default=Templates/
  -tf-post TEMPLATEFOLDERPOST, --templateFolderPost TEMPLATEFOLDERPOST
                        If set, use this folder of templates for POST logs. Default=Templates/
  -te {ttp,textFSM}, --templateEngine {ttp,textFSM}
                        Engine for parsing. Default=textFSM.
  -ri {name,ip,both}, --routerId {name,ip,both}
                        Router Id to be used within the tables in the Excel report. Default=name.
  -v, --version         Version