Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow installing a second agent on the same machine for development #4822

Merged
merged 54 commits into from
Jun 20, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
b48f695
Define development mode, alter install path.
cmacknz May 27, 2024
ceb17e2
Change service name in development mode.
cmacknz May 27, 2024
4b2f1a7
Make shell wrapper path conditional on development mode.
cmacknz May 27, 2024
7ef8e1b
Make control socket run path account for dev mode.
cmacknz May 27, 2024
7d6f153
Development mode automatically binds to port 0.
cmacknz May 27, 2024
06c1335
Refactor dev mode into separate file.
cmacknz May 28, 2024
8e1c177
Shorten command to --develop.
cmacknz May 28, 2024
f20c909
Fix windows build error.
cmacknz May 28, 2024
39bb905
Fix using wrong config option in install.
cmacknz May 28, 2024
16ba140
Add run --develop command.
cmacknz May 29, 2024
1f3ee13
Add initial test for --develop.
cmacknz May 29, 2024
1e38915
Initial version of an integration test for --develop mode.
cmacknz May 29, 2024
347dc65
Wait on the watcher instead of just releasing it.
cmacknz May 30, 2024
959725a
Add --develop test with base path.
cmacknz May 30, 2024
f8b1331
Add privileged install tests with --develop
cmacknz May 30, 2024
ea3999d
Move install tests to the same file.
cmacknz May 30, 2024
3897f40
Refactor develop test into function.
cmacknz May 30, 2024
d373536
Invert condition to match installopts
cmacknz May 30, 2024
b780b9b
Automatically add development tag on enroll.
cmacknz May 30, 2024
2dbd0a4
Change shell wrapper path to development.
cmacknz May 30, 2024
7546d09
Add documentation for --develop.
cmacknz May 31, 2024
3b0d997
Use lowercase for consistency.
cmacknz May 31, 2024
ce929fb
Remove TODO comments.
cmacknz May 31, 2024
03fe19c
Fix README typos.
cmacknz May 31, 2024
3c1ed11
Adjust comments.
cmacknz May 31, 2024
6e36b1a
More typo fixes.
cmacknz May 31, 2024
58afd75
Fix description not to mention beats.
cmacknz Jun 3, 2024
b6bd76a
Change windows service name to avoid collision.
cmacknz Jun 3, 2024
7b8e541
Make service display name unique on Windows.
cmacknz Jun 3, 2024
428e439
Merge branch 'main' into second-agent-same-machine
cmacknz Jun 7, 2024
724d5da
Merge branch 'main' into second-agent-same-machine
cmacknz Jun 10, 2024
18e7e5e
Add concept of an installation namespace.
cmacknz Jun 10, 2024
d340e19
Add nolint directives.
cmacknz Jun 10, 2024
3696791
Switch from strings.Replace to fmt.Sprintf.
cmacknz Jun 11, 2024
8d36a0a
Allow empty nolint directives.
cmacknz Jun 11, 2024
891a9e9
Enforce agent prefix. Add whitespace tests.
cmacknz Jun 11, 2024
ad85971
Fix typo.
cmacknz Jun 11, 2024
fc8e7dd
Remove unnecessary conditional check.
cmacknz Jun 11, 2024
2d38aec
Read the namespace once at startup.
cmacknz Jun 11, 2024
6e98115
Add missing license header.
cmacknz Jun 11, 2024
35bebe8
Also disable staticcheck for Windows lint warnings.
cmacknz Jun 11, 2024
b318abd
Revert "Allow empty nolint directives."
cmacknz Jun 12, 2024
18904f8
Better handling of empty fmt strings on windows.
cmacknz Jun 12, 2024
750983f
Merge branch 'main' into second-agent-same-machine
cmacknz Jun 12, 2024
967aabf
Fix use of hard coded service name.
cmacknz Jun 12, 2024
d5d3e3b
Fix merge errors in integration tests.
cmacknz Jun 12, 2024
5f2eab8
Properly handle empty format string on Windows.
cmacknz Jun 12, 2024
3efe29d
Add Address dropped in merge conflict resolution.
cmacknz Jun 13, 2024
9202c25
Merge branch 'main' into second-agent-same-machine
cmacknz Jun 13, 2024
414ae84
Fix ServiceName uses after merge.
cmacknz Jun 13, 2024
162d430
Merge branch 'main' into second-agent-same-machine
cmacknz Jun 17, 2024
f69ccfe
Add --namespace installation option.
cmacknz Jun 17, 2024
d4efa5f
Use --namespace in integration tests.
cmacknz Jun 17, 2024
051d4a8
Get integration tests to pass.
cmacknz Jun 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add concept of an installation namespace.
Restrict use to only the well known development namespace.
  • Loading branch information
cmacknz committed Jun 10, 2024
commit 18e7e5e1251c20e3b583c733590a2cdae8b412c1
8 changes: 3 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,13 +39,13 @@ sudo ./elastic-agent install --develop
./elastic-agent run -e --develop
```

Using the `--develop` option will install the agent in an isolated `DevelopmentAgent` agent directory in the chosen base path.
Development agents enrolled in Fleet will have the `development` tag added automatically. Using the default base path on MacOS you will see:
Using the `--develop` option will install the agent in an isolated `Agent-Development` agent directory in the chosen base path.
Development agents enrolled in Fleet will have the `Development` tag added automatically. Using the default base path on MacOS you will see:

```sh
sudo ls /Library/Elastic/
Agent
DevelopmentAgent
Agent-Development
```

The `elastic-agent` command in the shell is replaced with `elastic-development-agent` to interact with the development agent:
Expand All @@ -61,8 +61,6 @@ The primary restriction of `--develop` installations is that you cannot run Elas
install Defend twice will fail with resource conflicts. All other integrations should be usable provided conflicting configurations are
changed ahead of time. For example two agents cannot bind to the same `agent.monitoring.http.port` to expose their monitoring servers.

To follow the changes made to support development mode, search for the `IsDevelopmentMode()` function in the source code.

### Test Framework

In addition to standard Go tests, changes to the Elastic Agent are always installed and tested on cross-platform virtual machines.
Expand Down
81 changes: 0 additions & 81 deletions internal/pkg/agent/application/paths/common_development_mode.go

This file was deleted.

131 changes: 131 additions & 0 deletions internal/pkg/agent/application/paths/common_namespace.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
// Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
// or more contributor license agreements. Licensed under the Elastic License;
// you may not use this file except in compliance with the Elastic License.

// This file encapsulates the common paths that need to account for installation namepsaces.
// Installation namespaces allow multiple agents on the same machine.
package paths

import (
"path/filepath"
"strings"
)

// installNamespace is the name of the agent's current installation namepsace.
var installNamespace string

const (
// installDirNamespaceFmt is the format of the directory agent will be installed to within the base path when using an installation namepsace.
// For example it is $BasePath/$DevelopmentInstallDirName, on MacOS it is /Library/Elastic/$DevelopmentInstallDirName.
installDir = "Agent"
installDirNamespaceFmt = "Agent-%s"

// DevelopmentNamespace defines the "well known" development namespace.
DevelopmentNamespace = "Development"

// Service display names. Must be different from the ServiceName() on Windows.
serviceDisplayName = "Elastic Agent"
serviceDisplayNameNamespaceFmt = "Elastic Agent - %s"
)

// SetInstallNamespace sets whether the agent is currently in or is being installed in an installation namespace.
func SetInstallNamespace(namespace string) {
installNamespace = namespace
}

// InstallNamespace returns the name of the current installation namespace. Returns the empty string
// for the default namespace. For installed agents, the namespace is parsed from the installation
// directory name, since a unique directory name is required to avoid collisions between installed
// agents in the same base path. Before installation, the installation namespace must be configured
// using SetInstallNamespace().
func InstallNamespace() string {
if installNamespace != "" {
return installNamespace
}

if RunningInstalled() {
return parseNamespaceFromDir(filepath.Base(Top()))
cmacknz marked this conversation as resolved.
Show resolved Hide resolved
}

return ""
}

func parseNamespaceFromDir(dir string) string {
parts := strings.SplitAfterN(dir, "-", 2)
if len(parts) <= 1 {
cmacknz marked this conversation as resolved.
Show resolved Hide resolved
return ""
}

return parts[1]
}

// InInstallNamespace returns true if the agent is being installed in an installation namespace.
func InInstallNamespace() bool {
return InstallNamespace() != ""
}

// InstallDirNameForNamespace returns the installation directory name for a given namespace.
// The installation directory name with a namespace is $BasePath/InstallDirNameForNamespace().
func InstallDirNameForNamespace(namespace string) string {
if namespace == "" {
return installDir
}

// Use strings.Replace() to avoid having to sanitize format specifiers in the namespace itself.
return strings.Replace(installDirNamespaceFmt, "%s", namespace, 1)
cmacknz marked this conversation as resolved.
Show resolved Hide resolved
}

// InstallPath returns the top level directory Agent will be installed into, accounting for any namespace.
func InstallPath(basePath string) string {
namespace := InstallNamespace()
if namespace == "" {
return filepath.Join(basePath, "Elastic", installDir)
}

return filepath.Join(basePath, "Elastic", InstallDirNameForNamespace(namespace))
}
cmacknz marked this conversation as resolved.
Show resolved Hide resolved

// ServiceName returns the service name accounting for any namespace.
func ServiceName() string {
namespace := InstallNamespace()
if namespace == "" {
return serviceName
}

// Use strings.Replace() to avoid having to sanitize format specifiers in the namespace itself.
return strings.Replace(serviceNameNamespaceFmt, "%s", namespace, 1)
cmacknz marked this conversation as resolved.
Show resolved Hide resolved
}

// ServiceDisplayName returns the service display name accounting for any namespace.
func ServiceDisplayName() string {
namespace := InstallNamespace()
if namespace == "" {
return serviceDisplayName
}

// Use strings.Replace() to avoid having to sanitize format specifiers in the namespace itself.
return strings.Replace(serviceDisplayNameNamespaceFmt, "%s", namespace, 1)
cmacknz marked this conversation as resolved.
Show resolved Hide resolved
}

// ShellWrapperPath returns the shell wrapper path accounting for any namespace.
// The provided namespace is always lowercased for consistency.
func ShellWrapperPath() string {
namespace := InstallNamespace()
if namespace == "" {
return shellWrapperPath
}

// Use strings.Replace() to avoid having to sanitize format specifiers in the namespace itself.
return strings.Replace(shellWrapperPathNamespaceFmt, "%s", strings.ToLower(namespace), 1)
}

// ControlSocketRunSymlink returns the shell wrapper path accounting for any namespace.
// Does not auto detect the namespace because it is used outside of agent itself in the testing framework.
func ControlSocketRunSymlink(namespace string) string {
if namespace == "" {
return controlSocketRunSymlink
}

// Use strings.Replace() to avoid having to sanitize format specifiers in the namespace itself.
return strings.Replace(controlSocketRunSymlinkNamespaceFmt, "%s", namespace, 1)
}
65 changes: 65 additions & 0 deletions internal/pkg/agent/application/paths/common_namespace_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
package paths

import (
"fmt"
"path/filepath"
"testing"

"github.com/stretchr/testify/assert"
)

func TestInstallNamespace(t *testing.T) {
namespace := "testing"
basePath := filepath.Join("base", "path")
SetInstallNamespace(namespace)

assert.Equal(t, namespace, InstallNamespace())
assert.True(t, InInstallNamespace())
assert.Equal(t, filepath.Join(basePath, "Elastic", fmt.Sprintf(installDirNamespaceFmt, namespace)), InstallPath(basePath))
assert.Equal(t, fmt.Sprintf(serviceNameNamespaceFmt, namespace), ServiceName())
assert.Equal(t, fmt.Sprintf(serviceDisplayNameNamespaceFmt, namespace), ServiceDisplayName())
assert.Equal(t, fmt.Sprintf(shellWrapperPathNamespaceFmt, namespace), ShellWrapperPath())

Check failure on line 21 in internal/pkg/agent/application/paths/common_namespace_test.go

View workflow job for this annotation

GitHub Actions / lint (windows-latest)

printf: fmt.Sprintf call has arguments but no formatting directives (govet)
assert.Equal(t, fmt.Sprintf(controlSocketRunSymlinkNamespaceFmt, namespace), ControlSocketRunSymlink(namespace))

Check failure on line 22 in internal/pkg/agent/application/paths/common_namespace_test.go

View workflow job for this annotation

GitHub Actions / lint (windows-latest)

printf: fmt.Sprintf call has arguments but no formatting directives (govet)
}

func TestInstallNoNamespace(t *testing.T) {
namespace := ""
basePath := filepath.Join("base", "path")
SetInstallNamespace(namespace)

assert.Equal(t, namespace, InstallNamespace())
assert.False(t, InInstallNamespace())
assert.Equal(t, filepath.Join(basePath, "Elastic", installDir), InstallPath(basePath))
assert.Equal(t, serviceName, ServiceName())
assert.Equal(t, serviceDisplayName, ServiceDisplayName())
assert.Equal(t, shellWrapperPath, ShellWrapperPath())
assert.Equal(t, controlSocketRunSymlink, ControlSocketRunSymlink(namespace))
}

func TestParseNamespaceFromDirName(t *testing.T) {
testcases := []struct {
name string
dir string
namespace string
}{
{name: "empty", dir: "", namespace: ""},
{name: "none", dir: "Agent", namespace: ""},
{name: "develop", dir: "Agent-Development", namespace: "Development"},
{name: "dashes", dir: "Agent-With-Dashes", namespace: "With-Dashes"},
{name: "special", dir: "Agent-@!$%^&*()-_+=", namespace: "@!$%^&*()-_+="},
{name: "format", dir: "Agent-%s%d%v%t", namespace: "%s%d%v%t"},
}

for _, tc := range testcases {
t.Run(tc.name, func(t *testing.T) {
assert.Equal(t, tc.namespace, parseNamespaceFromDir(tc.dir))

// Special case: if the directory is empty the install dir is the default "Agent" not "Agent-"
wantDir := tc.dir
if wantDir == "" {
wantDir = installDir
}
assert.Equal(t, wantDir, InstallDirNameForNamespace(tc.namespace))
})
}
}
14 changes: 7 additions & 7 deletions internal/pkg/agent/application/paths/paths_darwin.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,20 +16,20 @@ const (

// controlSocketRunSymlink is the path to the symlink that should be
// created to the control socket when Elastic Agent is running with root.
controlSocketRunSymlink = "/var/run/elastic-agent.sock"
controlSocketRunSymlinkDevelopmentMode = "/var/run/elastic-agent-development.sock"
controlSocketRunSymlink = "/var/run/elastic-agent.sock"
controlSocketRunSymlinkNamespaceFmt = "/var/run/elastic-agent-%s.sock"

// serviceName is the service name when installed.
serviceName = "co.elastic.elastic-agent"
serviceNameDevelopmentMode = "co.elastic.elastic-agent-development"
serviceName = "co.elastic.elastic-agent"
serviceNameNamespaceFmt = "co.elastic.elastic-agent-%s"

// shellWrapperPath is the path to the installed shell wrapper.
shellWrapperPath = "/usr/local/bin/elastic-agent"
shellWrapperPathDevelopmentMode = "/usr/local/bin/elastic-development-agent"
shellWrapperPath = "/usr/local/bin/elastic-agent"
shellWrapperPathNamespaceFmt = "/usr/local/bin/elastic-%s-agent"

// ShellWrapper is the wrapper that is installed. The %s must
// be substituted with the appropriate top path.
ShellWrapper = `#!/bin/sh
ShellWrapperFmt = `#!/bin/sh
exec %s/elastic-agent $@
`
)
Expand Down
14 changes: 7 additions & 7 deletions internal/pkg/agent/application/paths/paths_linux.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,23 +15,23 @@ const (
DefaultBasePath = "/opt"

// serviceName is the service name when installed.
serviceName = "elastic-agent"
serviceNameDevelopmentMode = "elastic-agent-development"
serviceName = "elastic-agent"
serviceNameNamespaceFmt = "elastic-agent-%s"

// shellWrapperPath is the path to the installed shell wrapper.
shellWrapperPath = "/usr/bin/elastic-agent"
shellWrapperPathDevelopmentMode = "/usr/bin/elastic-development-agent"
shellWrapperPath = "/usr/bin/elastic-agent"
shellWrapperPathNamespaceFmt = "/usr/bin/elastic-%s-agent"

// ShellWrapper is the wrapper that is installed. The %s must
// be substituted with the appropriate top path.
ShellWrapper = `#!/bin/sh
ShellWrapperFmt = `#!/bin/sh
exec %s/elastic-agent $@
`

// controlSocketRunSymlink is the path to the symlink that should be
// created to the control socket when Elastic Agent is running with root.
controlSocketRunSymlink = "/run/elastic-agent.sock"
controlSocketRunSymlinkDevelopmentMode = "/run/elastic-agent-development.sock"
controlSocketRunSymlink = "/run/elastic-agent.sock"
controlSocketRunSymlinkNamespaceFmt = "/run/elastic-agent-%s.sock"
)

// ArePathsEqual determines whether paths are equal taking case sensitivity of os into account.
Expand Down
14 changes: 7 additions & 7 deletions internal/pkg/agent/application/paths/paths_windows.go
Original file line number Diff line number Diff line change
Expand Up @@ -21,19 +21,19 @@ const (
DefaultBasePath = `C:\Program Files`

// controlSocketRunSymlink is not created on Windows.
controlSocketRunSymlink = ""
controlSocketRunSymlinkDevelopmentMode = ""
controlSocketRunSymlink = ""
controlSocketRunSymlinkNamespaceFmt = ""

// serviceName is the service name when installed.
serviceName = "Elastic Agent"
serviceNameDevelopmentMode = "Elastic Development Agent"
serviceName = "Elastic Agent"
serviceNameNamespaceFmt = "Elastic Agent - %s"

// shellWrapperPath is the path to the installed shell wrapper.
shellWrapperPath = ""
shellWrapperPathDevelopmentMode = ""
shellWrapperPath = ""
shellWrapperPathNamespaceFmt = ""

// ShellWrapper is the wrapper that is installed.
ShellWrapper = "" // no wrapper on Windows
ShellWrapperFmt = "" // no wrapper on Windows
)

// ArePathsEqual determines whether paths are equal taking case sensitivity of os into account.
Expand Down
Loading
Loading