The Netdata registry implements the node menu on the top left corner of the netdata dashboards and enables the Netdata cloud features, such as the node view. The node menu lists the netdata servers you have visited. The node view offers a lot of additional features on top of the menu, with many more to come. To enable the global Netdata registry and the cloud features, you need to Sign In to Netdata cloud. By signing in, you opt in to let the registry receive and store the information described here. You can still get the node menu, but not the cloud features, if you run your own registry.
Netdata provides distributed monitoring.
Traditional monitoring solutions centralize all the data to provide unified dashboards across all servers. Before netdata, this was the standard practice. However it has a few issues:
- due to the resources required, the number of metrics collected is limited.
- for the same reason, the data collection frequency is not that high, at best it will be once every 10 or 15 seconds, at worst every 5 or 10 mins.
- the central monitoring solution needs dedicated resources, thus becoming "another bottleneck" in the whole ecosystem. It also requires maintenance, administration, etc.
- most centralized monitoring solutions are usually only good for presenting statistics of past performance (i.e. cannot be used for real-time performance troubleshooting).
Netdata follows a different approach:
- data collection happens per second
- thousands of metrics per server are collected
- data do not leave the server where they are collected
- netdata servers do not talk to each other
- your browser connects all the netdata servers
Using netdata, your monitoring infrastructure is embedded on each server, limiting significantly the need of additional resources. Netdata is blazingly fast, very resource efficient and utilizes server resources that already exist and are spare (on each server). This allows scaling out the monitoring infrastructure.
However, the netdata approach introduces a few new issues that need to be addressed, one being the list of netdata we have installed, i.e. the URLs our netdata servers are listening.
To solve this, netdata utilizes a central registry. This registry, together with certain browser features, allow netdata to provide unified cross-server dashboards. For example, when you jump from server to server using the node menu, several session settings (like the currently viewed charts, the current zoom and pan operations on the charts, etc.) are propagated to the new server, so that the new dashboard will come with exactly the same view. Netdata cloud has a roadmap to offer many more features over and above the simple node menu.
The registry keeps track of 4 entities:
- machines: i.e. the netdata installations (a random GUID generated by each netdata the first time it starts; we call this machine_guid)
For each netdata installation (each machine_guid) the registry keeps track of the different URLs it is accessed.
- persons: i.e. the web browsers accessing the netdata installations (a random GUID generated by the registry the first time it sees a new web browser; we call this person_guid)
For each person, the registry keeps track of the netdata installations it has accessed and their URLs.
- URLs of netdata installations (as seen by the web browsers)
For each URL, the registry keeps the URL and nothing more. Each URL is linked to persons and machines. The only way to find a URL is to know its machine_guid or have a person_guid it is linked to it.
- accounts: i.e. the information used to sign-in via one of the available sign-in methods. Depending on the method, this may include an email, an email and a profile picture.
For persons/accounts and machines, the registry keeps links to URLs, each link with 2 timestamps (first time seen, last time seen) and a counter (number of times it has been seen). machines, persons and timestamps are stored in the netdata registry regardless of whether you sign in or not.
Your web browser only! If sending this information is against your policies, you can run your own registry
Your netdata servers do not talk to the registry. This is a UML diagram of its operation:
https://registry.my-netdata.io, which is currently served by https://london.my-netdata.io. This registry listens to both HTTP and HTTPS requests but the default is HTTPS.
https://netdata.cloud is the additional registry endpoint, that enables the cloud features. It only accepts HTTPS.
Yeap! The registry can handle 50.000 - 100.000 requests per second per core (depending on the type of CPU, the computer's memory bandwidth, etc). 50.000 is on J1900 (celeron 2Ghz).
We believe, it can do it...
Every netdata can be a registry. Just pick one and configure it.
To turn any netdata into a registry, edit /etc/netdata/netdata.conf and set:
[registry]
enabled = yes
registry to announce = http://your.registry:19999
Restart your netdata to activate it.
Then, you need to tell all your other netdata servers to advertise your registry, instead of the default. To do this, on each of your netdata servers, edit /etc/netdata/netdata.conf and set:
[registry]
enabled = no
registry to announce = http://your.registry:19999
Note that we have not enabled the registry on the other servers. Only one netdata (the registry) needs [registry].enabled = yes.
This is it. You have your registry now.
You may also want to give your server different names under the node menu (i.e. to have them sorted / grouped). You can change its registry name, by setting on each netdata server:
[registry]
registry hostname = Group1 - Master DB
So this server will appear in the node menu as Group1 - Master DB. The max name length is 50 characters.
netdata v1.9+ support limiting access to the registry from given IPs, like this:
[registry]
allow from = *
allow from settings are netdata simple patterns: string matches that use * as wildcard (any number of times) and a ! prefix for a negative match. So: allow from = !10.1.2.3 10.* will allow all IPs in 10.* except 10.1.2.3. The order is important: left to right, the first positive or negative match is used.
Keep in mind that connections to netdata API ports are filtered by [web].allow connections from. So, IPs allowed by [registry].allow from should also be allowed by [web].allow connection from.
/var/lib/netdata/registry/*.db
There can be up to 2 files:
-
registry-log.db, the transaction logall incoming requests that affect the registry are saved in this file in real-time.
-
registry.db, the databaseevery
[registry].registry save db every new entriesentries inregistry-log.db, netdata will save its database toregistry.dband emptyregistry-log.db.
Both files are machine readable text files.
The registry opens a whole world of new possibilities for netdata. Check here what we think: netdata#416
The registry URL should be set to the URL of a netdata dashboard. This server has to have [registry].enabled = yes. So, accessing the registry URL directly with your web browser, should present the dashboard of the netdata operating the registry.
To use the registry, your web browser needs to support third party cookies, since the cookies are set by the registry while you are browsing the dashboard of another netdata server. The registry, the first time it sees a new web browser it tries to figure if the web browser has cookies enabled or not. It does this by setting a cookie and redirecting the browser back to itself hoping that it will receive the cookie. If it does not receive the cookie, the registry will keep redirecting your web browser back to itself, which after a few redirects will fail with an error like this:
ERROR 409: Cannot ACCESS netdata registry: https://registry.my-netdata.io responded with: {"status":"redirect","registry":"https://registry.my-netdata.io"}
This error is printed on your web browser console (press F12 on your browser to see it).