Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add WebSocket support to the existing outline-ss-server #225

Merged
merged 100 commits into from
Jan 30, 2025
Merged
Show file tree
Hide file tree
Changes from 98 commits
Commits
Show all changes
100 commits
Select commit Hold shift + click to select a range
b2aa859
refactor: split UDP serving from handling of packets
sbruens Nov 18, 2024
c0c0e9f
Use a read channel.
sbruens Nov 20, 2024
8d95309
Rename `wrappedPacketConn` to just `packetConn`.
sbruens Nov 20, 2024
51d5c88
Update `shadowsocks_handler`.
sbruens Nov 21, 2024
422542b
Change the `HandlePacket` API to not require the `pkt`.
sbruens Nov 22, 2024
c10176f
Remove the NAT map from the `Handle()` function.
sbruens Nov 25, 2024
004c6c3
Rework the metrics now that the NAT is no longer done by the handler.
sbruens Nov 26, 2024
a91d55b
Move the metrics `AddClosed()` call to after the `timedCopy` returns.
sbruens Nov 26, 2024
1d7200b
Remove the explicit `targetConn.Close()` and let it expire on its own.
sbruens Nov 26, 2024
4b8eeae
Add the NAT metrics back in at the server level.
sbruens Nov 27, 2024
1e89e85
Use buffer pool on `packetHandler` instead of global.
sbruens Nov 27, 2024
b4e7dbb
Revert wrapping the `clientConn` with shadowsocks encryption/decryption.
sbruens Nov 27, 2024
63c2cff
Rename `packet` to `association`.
sbruens Nov 27, 2024
1eeb4d6
Fix tests.
sbruens Nov 28, 2024
08e8bba
Update the docstring for `PacketServe`.
sbruens Nov 28, 2024
34a01d7
Fix comment to refer to "associations".
sbruens Nov 28, 2024
dd5f439
feat: add WebSocket support to the existing `outline-ss-server`
sbruens Nov 28, 2024
e769585
Fix config reloads.
sbruens Dec 3, 2024
033ba9d
Fix metrics test.
sbruens Dec 3, 2024
87a9d23
Use `slicepool`.
sbruens Dec 3, 2024
50feaa2
Let the assocation handler provide the buffer.
sbruens Dec 4, 2024
c0e942b
Merge branch 'sbruens/udp-split-serving' into sbruens/websocket
sbruens Dec 4, 2024
6eb828e
Fix StreamConn wrapper.
sbruens Dec 4, 2024
831371a
Move web server config into its own top-level config structure.
sbruens Dec 10, 2024
db09289
Add a TODO to create a new `ClientConn` struct.
sbruens Dec 10, 2024
b7a4a3c
Remove the `packetConnWrapper` and move the logic into `natconn` inst…
sbruens Dec 10, 2024
f80294c
Fix close while reading of `natconn`.
sbruens Dec 10, 2024
36d4b27
Simplify the natmap a little.
sbruens Dec 10, 2024
f5d9ac3
Refactor `PacketServe` to use events (close and read).
sbruens Dec 13, 2024
e0547f2
Use correct logger.
sbruens Dec 13, 2024
21cae17
Update tests for config validation.
sbruens Dec 10, 2024
e210fb0
Format.
sbruens Dec 13, 2024
3c1277c
Merge branch 'sbruens/udp-split-serving' into sbruens/websocket
sbruens Dec 13, 2024
0c45bad
Close the connection.
sbruens Dec 13, 2024
3dc12d1
Keep `readCh` and `closeCh` unbuffered.
sbruens Dec 16, 2024
afb9cd1
Catch panics in the `ReadFrom` go routine.
sbruens Dec 16, 2024
5d2acc6
Close the `readCh` instead of sending the error on the `readCh`.
sbruens Dec 16, 2024
7cd0f1f
Wrap a logger with the association's client address so we can simplif…
sbruens Dec 16, 2024
078032c
Reference GitHub issue for supporting multiple IPs.
sbruens Dec 16, 2024
2b3f746
Simplify packet handling with a new `association` struct.
sbruens Dec 18, 2024
2f2268b
Add some comments to the `Association` interface.
sbruens Dec 18, 2024
bfe4b35
Consolidate debug logging.
sbruens Dec 18, 2024
f37f23a
Rename some vars.
sbruens Dec 18, 2024
08de4c3
Update doc.
sbruens Dec 18, 2024
418c0e4
Format.
sbruens Dec 19, 2024
98988b0
Do not unpack first packets twice.
sbruens Dec 19, 2024
fffb6a2
Merge branch 'sbruens/udp-split-serving' into sbruens/websocket
sbruens Dec 20, 2024
b9c0286
Move handling into the association.
sbruens Dec 20, 2024
38e2eae
Merge branch 'sbruens/udp-split-serving' into sbruens/websocket
sbruens Dec 20, 2024
d14ea20
Merge branch 'master' into sbruens/udp-split-serving
sbruens Dec 20, 2024
eef630a
Add some comments to the timeout value.
sbruens Dec 20, 2024
3241298
Merge branch 'sbruens/udp-split-serving' into sbruens/websocket
sbruens Dec 20, 2024
1c462b1
Don't set the stream dialer in the old config flow.
sbruens Dec 20, 2024
ed4f4de
Merge branch 'sbruens/udp-split-serving' into sbruens/websocket
sbruens Dec 20, 2024
0918050
Rename `AddAuthentication` and `AddClose`.
sbruens Jan 6, 2025
78af4be
Update comment to reflect it handles packets from both directions.
sbruens Jan 6, 2025
2cf398c
Separate the interfaces for `Handle()` and `HandlePacket()`.
sbruens Jan 6, 2025
1055cb1
Fix typo in `UDPAssocationMetrics`.
sbruens Jan 6, 2025
f8ab81a
Don't pass `conn` to `Handle()`.
sbruens Jan 6, 2025
e6ef2f3
Update comment.
sbruens Jan 6, 2025
2939f8a
Add comments.
sbruens Jan 6, 2025
5b14062
Merge branch 'sbruens/udp-split-serving' into sbruens/websocket
sbruens Jan 6, 2025
ed980ad
Address review comments.
sbruens Jan 7, 2025
23a03e9
Remove ConnAssociation in favor of a `HandleAssociation(Conn, PacketA…
sbruens Jan 8, 2025
aa130f0
Split `Service` interface into outline-ss-server and Caddy interfaces.
sbruens Jan 8, 2025
19fb5f7
Remove unused const.
sbruens Jan 8, 2025
c656b36
Don't require `conn` in `HandleAssociation()`.
sbruens Jan 8, 2025
31cec7c
Exit the loop if the connection is closed.
sbruens Jan 9, 2025
0642698
Re-use global buffer pool.
sbruens Jan 10, 2025
a16c1b3
Remove app-specific interfaces.
sbruens Jan 10, 2025
77983fc
Decouple the association and the packet handling.
sbruens Jan 11, 2025
f3d63ae
Move timedCopy handling to the packet handler.
sbruens Jan 11, 2025
4c4072c
Remove unused property.
sbruens Jan 13, 2025
413a1aa
Decouple shadowsocks from association.
sbruens Jan 13, 2025
db3d0a3
Move authentication into its own function.
sbruens Jan 13, 2025
5f685bb
Remove the `packetMetrics` struct.
sbruens Jan 13, 2025
1dbfa99
Update tests.
sbruens Jan 13, 2025
23050ef
Remove the `Metrics()` method.
sbruens Jan 13, 2025
47222f6
Move variables into the anonymous functions.
sbruens Jan 16, 2025
8e9af05
Rename stream and packet handlers `Handle()` methods.
sbruens Jan 17, 2025
5704718
More `handle` naming clarification.
sbruens Jan 17, 2025
0cbcd61
Refactor to make packet handler an association handler.
sbruens Jan 21, 2025
bcac22b
Only handle the association if it was new.
sbruens Jan 21, 2025
220d1d7
Fix the metric race condition in tests.
sbruens Jan 21, 2025
d81f128
Move `AddNATEntry()` call to new entry only.
sbruens Jan 21, 2025
96de2a6
Format.
sbruens Jan 22, 2025
09e471f
Address review comments.
sbruens Jan 22, 2025
64c48ce
Make `clientConn` an `io.Writer`.
sbruens Jan 22, 2025
589abba
Merge branch 'sbruens/udp-split-serving' into sbruens/websocket
sbruens Jan 23, 2025
b12e0bb
Handle the `EOF` case and stop reading from the connection.
sbruens Jan 23, 2025
354d8a8
Add comment about using localhost listeners for the web config.
sbruens Jan 23, 2025
d37e358
Add comment about support half-closed states.
sbruens Jan 23, 2025
6ee3084
Refactor how we parse the listener configs.
sbruens Jan 29, 2025
f2fda60
Merge branch 'master' into sbruens/websocket
sbruens Jan 29, 2025
8fca003
Add comments.
sbruens Jan 29, 2025
d7e43f7
Align yaml string.
sbruens Jan 29, 2025
775692d
Skip the unnecessary JSON marshalling.
sbruens Jan 29, 2025
c076337
Use reflection and a type map to avoid code duplication.
sbruens Jan 29, 2025
b00ee41
More review comments.
sbruens Jan 29, 2025
aeef1f2
More review comments.
sbruens Jan 29, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
191 changes: 169 additions & 22 deletions cmd/outline-ss-server/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,21 @@
package main

import (
"errors"
"fmt"
"net"
"reflect"
"strings"

"github.com/go-viper/mapstructure/v2"
"gopkg.in/yaml.v3"
)

type Validator interface {
// Validate checks that the type is valid.
validate() error
}

type ServiceConfig struct {
Listeners []ListenerConfig
Keys []KeyConfig
Expand All @@ -29,15 +38,139 @@ type ServiceConfig struct {

type ListenerType string

const listenerTypeTCP ListenerType = "tcp"
const (
TCPListenerType = ListenerType("tcp")
UDPListenerType = ListenerType("udp")
WebsocketStreamListenerType = ListenerType("websocket-stream")
WebsocketPacketListenerType = ListenerType("websocket-packet")
)

type WebServerConfig struct {
sbruens marked this conversation as resolved.
Show resolved Hide resolved
// Unique identifier of the web server to be referenced in Websocket connections.
ID string

const listenerTypeUDP ListenerType = "udp"
// List of listener addresses (e.g., ":8080", "localhost:8081"). Should be localhost for HTTP.
Listeners []string `yaml:"listen"`
}

// ListenerConfig holds the configuration for a listener. It supports different
// listener types, configured via the embedded type and unmarshalled based on
// the "type" field in the YAML/JSON configuration. Only one of the fields will
// be set, corresponding to the listener type.
type ListenerConfig struct {
sbruens marked this conversation as resolved.
Show resolved Hide resolved
Type ListenerType
// TCP configuration for the listener.
TCP *TCPUDPConfig
// UDP configuration for the listener.
UDP *TCPUDPConfig
// Websocket stream configuration for the listener.
WebsocketStream *WebsocketConfig
// Websocket packet configuration for the listener.
WebsocketPacket *WebsocketConfig
}

var _ Validator = (*ListenerConfig)(nil)
var _ yaml.Unmarshaler = (*ListenerConfig)(nil)

// Define a map to associate listener types with [ListenerConfig] field names.
var listenerTypeMap = map[ListenerType]string{
TCPListenerType: "TCP",
UDPListenerType: "UDP",
WebsocketStreamListenerType: "WebsocketStream",
WebsocketPacketListenerType: "WebsocketPacket",
}

func (c *ListenerConfig) UnmarshalYAML(value *yaml.Node) error {
var raw map[string]interface{}
if err := value.Decode(&raw); err != nil {
return err
}

// Remove the "type" field so we can decode directly into the target struct.
rawType, ok := raw["type"]
if !ok {
return errors.New("`type` field required")
}
delete(raw, "type")

lnType := ListenerType(rawType.(string))
sbruens marked this conversation as resolved.
Show resolved Hide resolved
fieldName, ok := listenerTypeMap[lnType]
if !ok {
return fmt.Errorf("invalid listener type: %v", lnType)
}
v := reflect.ValueOf(c).Elem()
field := v.FieldByName(fieldName)
if !field.IsValid() {
return fmt.Errorf("invalid field name: %s for type: %s", fieldName, lnType)
}
fieldType := field.Type()
if fieldType.Kind() != reflect.Ptr || fieldType.Elem().Kind() != reflect.Struct {
return fmt.Errorf("field %s is not a pointer to a struct", fieldName)
}

configValue := reflect.New(fieldType.Elem())
field.Set(configValue)
if err := mapstructure.Decode(raw, configValue.Interface()); err != nil {
return fmt.Errorf("failed to decode map: %w", err)
}
return nil
}

func (c *ListenerConfig) validate() error {
v := reflect.ValueOf(c).Elem()

for i := 0; i < v.NumField(); i++ {
field := v.Field(i)
if field.Kind() == reflect.Ptr && field.IsNil() {
continue
}
if field.Type().Implements(reflect.TypeOf((*Validator)(nil)).Elem()) {
sbruens marked this conversation as resolved.
Show resolved Hide resolved
if err := field.Interface().(Validator).validate(); err != nil {
return fmt.Errorf("invalid config: %v", err)
}
sbruens marked this conversation as resolved.
Show resolved Hide resolved
}
}
return nil
}

type TCPUDPConfig struct {
// Address for the TCP or UDP listener. Should be in the format host:port.
Address string
}

var _ Validator = (*TCPUDPConfig)(nil)

func (c *TCPUDPConfig) validate() error {
if c.Address == "" {
return errors.New("`address` must be specified")
}
if err := validateAddress(c.Address); err != nil {
return fmt.Errorf("invalid address: %v", err)
}
return nil
}

type WebsocketConfig struct {
// Web server unique identifier to use for the websocket connection.
WebServer string `mapstructure:"web_server"`
// Path for the websocket connection.
Path string
}

var _ Validator = (*WebsocketConfig)(nil)

func (c *WebsocketConfig) validate() error {
if c.WebServer == "" {
return errors.New("`web_server` must be specified")
}
if c.Path == "" {
return errors.New("`path` must be specified")
}
if !strings.HasPrefix(c.Path, "/") {
return errors.New("`path` must start with `/`")
}
return nil
}

type DialerConfig struct {
Fwmark uint
}
Expand All @@ -53,40 +186,54 @@ type LegacyKeyServiceConfig struct {
Port int
}

type WebConfig struct {
Servers []WebServerConfig `yaml:"servers"`
}

type Config struct {
Web WebConfig
Services []ServiceConfig

// Deprecated: `keys` exists for backward compatibility. Prefer to configure
// using the newer `services` format.
Keys []LegacyKeyServiceConfig
}

// Validate checks that the config is valid.
func (c *Config) Validate() error {
existingListeners := make(map[string]bool)
for _, serviceConfig := range c.Services {
for _, lnConfig := range serviceConfig.Listeners {
// TODO: Support more listener types.
if lnConfig.Type != listenerTypeTCP && lnConfig.Type != listenerTypeUDP {
return fmt.Errorf("unsupported listener type: %s", lnConfig.Type)
}
host, _, err := net.SplitHostPort(lnConfig.Address)
if err != nil {
return fmt.Errorf("invalid listener address `%s`: %v", lnConfig.Address, err)
}
if ip := net.ParseIP(host); ip == nil {
return fmt.Errorf("address must be IP, found: %s", host)
var _ Validator = (*Config)(nil)

func (c *Config) validate() error {
for _, srv := range c.Web.Servers {
if srv.ID == "" {
return fmt.Errorf("web server must have an ID")
}
for _, addr := range srv.Listeners {
if err := validateAddress(addr); err != nil {
return fmt.Errorf("invalid listener for web server `%s`: %w", srv.ID, err)
}
key := string(lnConfig.Type) + "/" + lnConfig.Address
if _, exists := existingListeners[key]; exists {
return fmt.Errorf("listener of type %s with address %s already exists.", lnConfig.Type, lnConfig.Address)
}
}

for _, service := range c.Services {
for _, ln := range service.Listeners {
if err := ln.validate(); err != nil {
return fmt.Errorf("invalid listener: %v", err)
}
existingListeners[key] = true
}
}
return nil
}

func validateAddress(addr string) error {
host, _, err := net.SplitHostPort(addr)
if err != nil {
return err
}
if ip := net.ParseIP(host); ip == nil {
return fmt.Errorf("address must be IP, found: %s", host)
}
return nil
}

// readConfig attempts to read a config from a filename and parses it as a [Config].
func readConfig(configData []byte) (*Config, error) {
config := Config{}
Expand Down
12 changes: 12 additions & 0 deletions cmd/outline-ss-server/config_example.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,12 @@
# See the License for the specific language governing permissions and
# limitations under the License.

web:
servers:
- id: my_web_server
listen:
- "127.0.0.1:8000"

services:
- listeners:
# TODO(sbruens): Allow a string-based listener config, as a convenient short-form
Expand All @@ -20,6 +26,12 @@ services:
address: "[::]:9000"
- type: udp
address: "[::]:9000"
- type: websocket-stream
web_server: my_web_server
path: "/SECRET/tcp" # Prevent probing by serving under a secret path.
- type: websocket-packet
web_server: my_web_server
path: "/SECRET/udp" # Prevent probing by serving under a secret path.
keys:
- id: user-0
cipher: chacha20-ietf-poly1305
Expand Down
Loading