diff --git a/docs/docs/.pages b/docs/docs/.pages index 9907bbf..ff4d6d7 100644 --- a/docs/docs/.pages +++ b/docs/docs/.pages @@ -7,6 +7,7 @@ nav: - monitoring.md - configuration-cleaning.md - ... + - version_matrix.md - FAQ.md - CMDB - Data-Aggregation-API diff --git a/docs/docs/Data-Aggregation-API/configuration.md b/docs/docs/Data-Aggregation-API/configuration.md new file mode 100644 index 0000000..9178fb3 --- /dev/null +++ b/docs/docs/Data-Aggregation-API/configuration.md @@ -0,0 +1,140 @@ +# Configuration + +## Configuration file + +The Data Aggregation API can be configured with flags, environments variables and configuration file. + +The precedence order for the different methods is: + +!!! info + + * flags + * environment variables + * configuration file (settings.yml) + +```yaml +Datacenter: "europe" + +Log: + Level: "error" + Pretty: true + +API: + ListenAddress: "127.0.0.1" + ListenPort: 1234 + +Authentication: + LDAP: + URL: "ldaps://URL.local" + BindDN: "cn=,OU=,DC=" + BaseDN: "DC=" + Password: "" + WorkersCount: 10 + Timeout: 5s + MaxConnectionLifetime: 1m + InsecureSkipVerify: false + +NetBox: + URL: "https://netbox.local" + APIKey: "" + DatacenterFilterKey: "site" + LimitPerPage: 500 + +Build: + Interval: "30m" + AllDevicesMustBuild: false +``` + +## Global settings + +| Parameter | Default | Description | +|-----------|---------|-------------| +| Datacenter | `` | Value used to filter devices. The key is defined by `DatacenterFilterKey`. | + +## Log settings + +All parameters below are in the `Log` section of the configuration. +This section is optional. + +| Parameter | Default | Description | +|-----------|---------|-------------| +| Level | `info` | Log level. | +| Pretty | `false` | If enabled: human readable logs (with colors). If disabled: structured logs. | + +## API settings + +All parameters below are in the `API` section of the configuration. +This section is optional. + +| Parameter | Default | Description | +|-----------|---------|-------------| +| ListenAddress | `0.0.0.0` | Listening address of the web API. | +| ListenPort | `8080` | Listening port of the web API. | + +## Authentication settings + +All parameters below are in the `Authentication`->`LDAP` section of the configuration. + +LDAP Authentication is only to authenticate users when they try to query the Web API, i.e. when they want to retrieve the built config. + +This section is optional. + +| Parameter | Default | Description | +|-----------|---------|-------------| +| InsecureSkipVerify | `false` | Ignore LDAP TLS warnings. | +| URL | | URL of the LDAP server. | +| BindDN | | Bind used to query the LDAP server. | +| Password | | Password to query the LDAP server. | +| BaseDN | | Only users matching the BaseDN are authorized to query the web API. | +| WorkersCount | `10` | Number of workers to authenticate users concurrently. | +| Timeout | `10s` | Time to wait before considering a LDAP request timed out. | +| MaxConnectionLifetime | `1m` | Lifetime of worker connection to LDAP. Useful to re-use existing LDAP connection. | + + +## NetBox settings + +All parameters below are in the `NetBox` section of the configuration. + +| Parameter | Default | Description | +|-----------|---------|-------------| +| URL | | NetBox base URL. | +| APIKey | | NetBox API token. | +| DatacenterFilterKey | `site` | Key used to filter devices, using `Datacenter` as a value. | +| LimitPerPage | `100` | Number of elements per page when getting data from NetBox API. | + +## Build settings + +All parameters below are in the `NetBox` section of the configuration. +This section is optional. + +| Parameter | Default | Description | +|-----------|---------|-------------| +| Interval | `1m` | Build interval, e.g.: `10m`, `1h` | +| AllDevicesMustBuild | `false` | The build fails if one device has not been built. | + +## Alternative methods + +### Environment variables + +All settings available in the configuration file can be set as environment variables, but: + +* all variables must be prefixed by `DAAPI_` +* uppercase only +* `_` is the level separator + +For example, the equivalent of this config file: + +``` yaml +Datacenter: "europe" +NetBox: + URL: "https://netbox.local" + APIKey: "" +``` + +is: + +```shell +DAAPI_DATACENTER="europe" +DAAPI_NETBOX_URL="https://netbox.local" +DAAPI_NETBOX_APIKEY="" +``` diff --git a/docs/docs/Data-Aggregation-API/design.md b/docs/docs/Data-Aggregation-API/design.md index 6cbccc3..7084475 100644 --- a/docs/docs/Data-Aggregation-API/design.md +++ b/docs/docs/Data-Aggregation-API/design.md @@ -83,12 +83,33 @@ Thanks to [ygot](https://github.com/openconfig/ygot) and [OpenConfig YANG models ## Reporting -!!! warning - - :wrench: This section is under construction. - -## Endpoints - -!!! warning - - :wrench: This section is under construction. +| Endpoint | Description | +|------------|-----------| +| `/v1/report/last` | Last or ongoing build report | +| `/v1/report/last/complete` | Report of the last complete build (whether it failed or not) | +| `/v1/report/last/successful` | Report of the last successful build | + +The reports contain: +* The status of the build. +* When it started and finished. +* Statistics like performance. +* Logs. + +## OpenConfig + +| Endpoint | Description | +|------------|-----------| +| `/v1/devices/[hostname]/openconfig` | Get OpenConfig data for a specific device | +| `/v1/devices/*/openconfig` | Get OpenConfig data for all devices | +| `/v1/devices/[hostname]/afk_enabled` | Indicates if a device has the tag AFK enabled in NetBox | +| `/v1/devices/*/afk_enabled` | Same as above, but for all devices | + +Note: `afk_enabled` can be used to enable or disable AFK schedule in Salt via `afk-enabled` tag in NetBox DCIM. + +# Other endpoints + +| Endpoint | Description | +|------------|-----------| +| `/metrics` | Prometheus metrics | +| `/api/version` | Details about the running version | +| `/api/health` | Dummy endpoint for basic healthcheck of the app | diff --git a/docs/docs/Data-Aggregation-API/installation.md b/docs/docs/Data-Aggregation-API/installation.md index 19d0b4c..14119cb 100644 --- a/docs/docs/Data-Aggregation-API/installation.md +++ b/docs/docs/Data-Aggregation-API/installation.md @@ -1,5 +1,41 @@ # Installation -!!! warning +## Quickstart - :wrench: Open Source version not yet ready. \ No newline at end of file +1. Download the latest release archive. +2. Extract the single binary. +3. Configure the Data Aggregation API accordingly. +4. Run the binary. + +Example of basic configuration (`settings.yml`): +```yaml +Datacenter: "europe" +Log: + Level: "error" + Pretty: true + +API: + ListenAddress: "127.0.0.1" + ListenPort: 1234 + +Log: + Level: "info" + Pretty: true + +NetBox: + URL: "https://netbox.local" + APIKey: "" + DatacenterFilterKey: "site" + +Build: + Interval: "30m" +``` + +See [Configuration](./configuration.md) for more details. + + +## Dependencies + +The only dependency is our [Network CMDB](https://github.com/criteo/netbox-network-cmdb) NetBox plugin. + +The installation guide of this plugin is available in the CMDB section [here](/docs/docs/CMDB/). diff --git a/docs/docs/SONiC-support/overview.md b/docs/docs/SONiC-support/overview.md index f9557e0..5ddff30 100644 --- a/docs/docs/SONiC-support/overview.md +++ b/docs/docs/SONiC-support/overview.md @@ -4,7 +4,7 @@ | SONiC version | Support | |---------------|---------| -| 201911 | :white_check_mark: | +| 201911* | :warning: | | 202205 | :white_check_mark: | !!! info "Legend" @@ -14,6 +14,8 @@ - :warning: deprecated - :no_entry: not supported +* FRR 7.2.1 in SONiC 201911 has a memory leak which can be triggered by AFK + ## Installation | Step | Description | Guide | diff --git a/docs/docs/index.md b/docs/docs/index.md index ae4829c..224b6dd 100644 --- a/docs/docs/index.md +++ b/docs/docs/index.md @@ -22,8 +22,7 @@ It is based on [NetBox](https://netbox.dev), [OpenConfig](https://www.openconfig | Repository | Description | Latest commit | |------------|-------------|---------------| | [Network CMDB](https://github.com/criteo/netbox-network-cmdb) | Network CMDB plugin for Netbox | ![Last commit](https://img.shields.io/github/last-commit/criteo/netbox-network-cmdb/main) | -| Data aggregation API (coming soon) | Aggregate data from CMDB and convert to OpenConfig | | -| [AFK Salt modules](https://github.com/criteo/afk-saltstack) | Salt modules to apply configuration from OpenConfig data | ![Last commit](https://img.shields.io/github/last-commit/criteo/openconfig-SaltStack/main) | +| [Data aggregation API](https://github.com/criteo/data-aggregation-api) | Aggregate data from CMDB and convert to OpenConfig | ![Last commit](https://img.shields.io/github/last-commit/criteo/data-aggregation-api/main) || [AFK Salt modules](https://github.com/criteo/afk-saltstack) | Salt modules to apply configuration from OpenConfig data | ![Last commit](https://img.shields.io/github/last-commit/criteo/openconfig-SaltStack/main) | | [SONiC Salt Deployer](https://github.com/criteo/sonic-salt-deployer) | Tool to deploy and configure salt-minion on SONiC devices | ![Last commit](https://img.shields.io/github/last-commit/criteo/sonic-salt-deployer/main) | | [SONiC SaltStack](https://github.com/criteo/sonic-saltstack) | States/execution modules for SONiC | ![Last commit](https://img.shields.io/github/last-commit/criteo/sonic-SaltStack/main) | | [SONiC utilities](https://github.com/criteo/criteo-sonic-utilities) | SONiC scripts used by some SONiC SaltStack modules | ![Last commit](https://img.shields.io/github/last-commit/criteo/criteo-sonic-utilities/main) | diff --git a/docs/docs/installation.md b/docs/docs/installation.md index 9fd722f..5499611 100644 --- a/docs/docs/installation.md +++ b/docs/docs/installation.md @@ -3,6 +3,7 @@ | Component | How to install | |-----------|----------------| | NetBox | [https://docs.netbox.dev/en/stable/installation/](https://docs.netbox.dev/en/stable/installation/) +| [Network CMDB](https://github.com/criteo/netbox-network-cmdb) | Network CMDB plugin for Netbox | ![Last commit](https://img.shields.io/github/last-commit/criteo/netbox-network-cmdb/main) | | Data Aggregation API | see our [Installation guide](/Data-Aggregation-API/installation/) | SaltStack | [https://docs.saltproject.io/salt/install-guide/en/latest/](https://docs.saltproject.io/salt/install-guide/en/latest/) | SaltStack for JunOS/EOS | [proxy-minion](https://docs.saltproject.io/en/latest/topics/proxyminion/index.html) with [napalm driver](https://github.com/napalm-automation/napalm-salt) diff --git a/docs/docs/version_matrix.md b/docs/docs/version_matrix.md new file mode 100644 index 0000000..2af2079 --- /dev/null +++ b/docs/docs/version_matrix.md @@ -0,0 +1,21 @@ +# Version matrix + +## v0.1-alpha + +|Component|Version| +|---------|-------| +|Network CMDB|| +|Data Aggregration API|v0.0.1-alpha.x| +|AFK SaltStack module|| +|SONiC SaltStack modules|| + +|Dependency|Version| +|----------|-------| +|NetBox|v3.3.10| +|Saltstack|3001.x to 3004.x| + +|Supported Network OS|Supported Versions| +|--------------------|------------------| +|SONiC|202205, 202211| +|Juniper JunOS|>16.1 (tested with up to 22.x)| +|Arista EOS|should all work, but tested up to 4.22.x| \ No newline at end of file