Add guide for how to expand and contract a cluster and considerations

[bruth]

Brief points to be expanded:

• Adding nodes is straightforward
  • The routes should use the list of seed servers
  • There is full mesh communication overhead (and Raft meta for JS) if the cluster grows to large
  • JS-enabled nodes can be added to increase the total storage size and spread out the placement of streams
• Removing nodes requires a few steps to prevent dropped messages and/or data loss
  • Abrupt removal will cause dropped messages, so a server must be put in lame-duck mode to drain and safely force clients to reconnect to another server
  • For JS-enabled nodes, R1 assets need to be explicitly moved to a new server (which assumes server tags are in use)
  • For R3 or R5 streams, the node needs to be removed as a peer for that stream, so the replica is moved elsewhere
  • Of course if R3 or R5 streams are present, the cluster size can't go below the max replica count
  • Likewise, the total available storage on a node needs to be sufficient for all streams placed on it