Skip to content

Commit

Permalink
feat: add configer provider info report interface
Browse files Browse the repository at this point in the history
  • Loading branch information
@bilosikia
bilosikia committed Nov 11, 2024
1 parent fd95ac6 commit b8b7d05
Show file tree
Hide file tree
Showing 2 changed files with 145 additions and 112 deletions.
6 changes: 5 additions & 1 deletion config_server/protocol/v2/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -238,6 +238,10 @@ Client：这个版本的配置状态上报中修改了version的定义，-1仍

Server：这些信息是Agent状态的一部分，可选保存。与通过Event上报可观测信息不同的是，作为状态信息没有时间属性，用户可通过接口可获取即刻状态，而不需要选择时间窗口合并事件。

上报配置状态有两种途径：
1. 在全量心跳或者配置状态变化后的压缩心跳 request 中将配置最新状态带上。
2.【可选补充】通过 ReportConfigRequest 接口上报。适合对状态更新实时性要求比较高，或对心跳服务、配置服务、状态服务拆分以减少故障半径的实现。

### 预定义命令

Client: 通过request的flag传递，定义了FullStatus，表明本条信息为全量状态
Expand All @@ -259,4 +263,4 @@ Client: 当HeartbeatResponse中的code为0时，Agent应该正常处理下发的
### 辅助信息
在command\_info, command\_detail, config\_info, config\_detail中，都预留了extra字段，可以用于传递一些额外的用户自定义的辅助信息。\

注意：extra字段仅作传递辅助信息使用，不会对管控行为造成任何影响。
注意：extra字段仅作传递辅助信息使用，不会对管控行为造成任何影响。
251 changes: 140 additions & 111 deletions config_server/protocol/v2/agentV2.proto
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,171 +3,200 @@ package configserver.proto.v2;
option go_package = "/config_server/service/protov2;configserver_proto_v2";

message AgentGroupTag {
string name = 1;
string value = 2;
string name = 1;
string value = 2;
}

enum ConfigStatus {
// The value of status field is not set.
UNSET = 0;
// Agent is currently applying the remote config that it received earlier.
APPLYING = 1;
// Remote config was successfully applied by the Agent.
APPLIED = 2;
// Agent tried to apply the config received earlier, but it failed.
// See error_message for more details.
FAILED = 3;
// The value of status field is not set.
UNSET = 0;
// Agent is currently applying the remote config that it received earlier.
APPLYING = 1;
// Remote config was successfully applied by the Agent.
APPLIED = 2;
// Agent tried to apply the config received earlier, but it failed.
// See error_message for more details.
FAILED = 3;
}

// Define the Config information carried in the request
message ConfigInfo {
string name = 1; // Required, Config's unique identification
int64 version = 2; // Required, Config's version number or hash code
ConfigStatus status = 3; // Config's status
string message = 4; // Optional error message
map<string, bytes> extra = 5; // Optional extra info
string name = 1; // Required, Config's unique identification
int64 version = 2; // Required, Config's version number or hash code
ConfigStatus status = 3; // Config's status
string message = 4; // Optional error message
map<string, bytes> extra = 5; // Optional extra info
}

// Define the Command information carried in the request
message CommandInfo {
string type = 1; // Command's type
string name = 2; // Required, Command's unique identification
ConfigStatus status = 3; // Command's status
string message = 4; // Optional error message
map<string, bytes> extra = 5; // Optional extra info
string type = 1; // Command's type
string name = 2; // Required, Command's unique identification
ConfigStatus status = 3; // Command's status
string message = 4; // Optional error message
map<string, bytes> extra = 5; // Optional extra info
}

// Define Agent's basic attributes
message AgentAttributes {
bytes version = 1; // Agent's version
bytes ip = 2; // Agent's ip
bytes hostname = 3; // Agent's hostname
bytes hostid = 4; // Agent's hostid https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/
map<string, bytes> extras = 100; // Agent's other attributes
// before 100 (inclusive) are reserved for future official fields
bytes version = 1; // Agent's version
bytes ip = 2; // Agent's ip
bytes hostname = 3; // Agent's hostname
bytes hostid =
4; // Agent's hostid
// https://opentelemetry.io/docs/specs/semconv/attributes-registry/host/
map<string, bytes> extras = 100; // Agent's other attributes
// before 100 (inclusive) are reserved for future official fields
}

enum AgentCapabilities {
// The capabilities field is unspecified.
UnspecifiedAgentCapability = 0;
// The Agent can accept pipeline configuration from the Server.
AcceptsPipelineConfig = 0x00000001;
// The Agent can accept instance configuration from the Server.
AcceptsInstanceConfig = 0x00000002;
// The Agent can accept custom command from the Server.
AcceptsCustomCommand = 0x00000004;

// Add new capabilities here, continuing with the least significant unused bit.
// The capabilities field is unspecified.
UnspecifiedAgentCapability = 0;
// The Agent can accept pipeline configuration from the Server.
AcceptsPipelineConfig = 0x00000001;
// The Agent can accept instance configuration from the Server.
AcceptsInstanceConfig = 0x00000002;
// The Agent can accept custom command from the Server.
AcceptsCustomCommand = 0x00000004;

// Add new capabilities here, continuing with the least significant unused
// bit.
}

enum RequestFlags {
RequestFlagsUnspecified = 0;
RequestFlagsUnspecified = 0;

// Flags is a bit mask. Values below define individual bits.
// Flags is a bit mask. Values below define individual bits.

// Must be set if this request contains full state
FullState = 0x00000001;
// bits before 2^16 (inclusive) are reserved for future official fields
// Must be set if this request contains full state
FullState = 0x00000001;
// bits before 2^16 (inclusive) are reserved for future official fields
}

// API: /Agent/Heartbeat

// Agent sends requests to the ConfigServer to get config updates and receive commands.
// Agent sends requests to the ConfigServer to get config updates and receive
// commands.
message HeartbeatRequest {
bytes request_id = 1;
uint64 sequence_num = 2; // Increment every request, for server to check sync status
uint64 capabilities = 3; // Bitmask of flags defined by AgentCapabilities enum
bytes instance_id = 4; // Required, Agent's unique identification, consistent throughout the process lifecycle
string agent_type = 5; // Required, Agent's type(ilogtail, ..)
AgentAttributes attributes = 6; // Agent's basic attributes
repeated AgentGroupTag tags = 7; // Agent's tags
string running_status = 8; // Human readable running status
int64 startup_time = 9; // Required, Agent's startup time
repeated ConfigInfo pipeline_configs = 10; // Information about the current PIPELINE_CONFIG held by the Agent
repeated ConfigInfo instance_configs = 11; // Information about the current AGENT_CONFIG held by the Agent
repeated CommandInfo custom_commands = 12; // Information about command history
uint64 flags = 13; // Predefined command flag
bytes opaque = 14; // Opaque data for extension
// before 100 (inclusive) are reserved for future official fields
bytes request_id = 1;
uint64 sequence_num =
2; // Increment every request, for server to check sync status
uint64 capabilities = 3; // Bitmask of flags defined by AgentCapabilities enum
bytes instance_id = 4; // Required, Agent's unique identification, consistent
// throughout the process lifecycle
string agent_type = 5; // Required, Agent's type(ilogtail, ..)
AgentAttributes attributes = 6; // Agent's basic attributes
repeated AgentGroupTag tags = 7; // Agent's tags
string running_status = 8; // Human readable running status
int64 startup_time = 9; // Required, Agent's startup time
repeated ConfigInfo pipeline_configs =
10; // Information about the current PIPELINE_CONFIG held by the Agent
repeated ConfigInfo instance_configs =
11; // Information about the current AGENT_CONFIG held by the Agent
repeated CommandInfo custom_commands =
12; // Information about command history
uint64 flags = 13; // Predefined command flag
bytes opaque = 14; // Opaque data for extension
// before 100 (inclusive) are reserved for future official fields
}

// Define Config's detail
message ConfigDetail {
string name = 1; // Required, Config's unique identification
int64 version = 2; // Required, Config's version number or hash code
bytes detail = 3; // Required, Config's detail
map<string, bytes> extra = 4; // Optional extra info
string name = 1; // Required, Config's unique identification
int64 version = 2; // Required, Config's version number or hash code
bytes detail = 3; // Required, Config's detail
map<string, bytes> extra = 4; // Optional extra info
}

message CommandDetail {
string type = 1; // Required, Command type
string name = 2; // Required, Command name
bytes detail = 3; // Required, Command's detail
int64 expire_time = 4; // After which the command can be safely removed from history
map<string, bytes> extra = 5; // Optional extra info
string type = 1; // Required, Command type
string name = 2; // Required, Command name
bytes detail = 3; // Required, Command's detail
int64 expire_time =
4; // After which the command can be safely removed from history
map<string, bytes> extra = 5; // Optional extra info
}

enum ServerCapabilities {
// The capabilities field is unspecified.
UnspecifiedServerCapability = 0;
// The Server can remember agent attributes.
RembersAttribute = 0x00000001;
// The Server can remember pipeline config status.
RembersPipelineConfigStatus = 0x00000002;
// The Server can remember instance config status.
RembersInstanceConfigStatus = 0x00000004;
// The Server can remember custom command status.
RembersCustomCommandStatus = 0x00000008;

// bits before 2^16 (inclusive) are reserved for future official fields
// The capabilities field is unspecified.
UnspecifiedServerCapability = 0;
// The Server can remember agent attributes.
RembersAttribute = 0x00000001;
// The Server can remember pipeline config status.
RembersPipelineConfigStatus = 0x00000002;
// The Server can remember instance config status.
RembersInstanceConfigStatus = 0x00000004;
// The Server can remember custom command status.
RembersCustomCommandStatus = 0x00000008;

// bits before 2^16 (inclusive) are reserved for future official fields
}

enum ResponseFlags {
ResponseFlagsUnspecified = 0;

// Flags is a bit mask. Values below define individual bits.

// ReportFullState flag can be used by the Server if the Client did not include
// some sub-message in the last AgentToServer message (which is an allowed
// optimization) but the Server detects that it does not have it (e.g. was
// restarted and lost state).
ReportFullState = 0x00000001;
FetchPipelineConfigDetail = 0x00000002;
FetchInstanceConfigDetail = 0x00000004;
// bits before 2^16 (inclusive) are reserved for future official fields
ResponseFlagsUnspecified = 0;

// Flags is a bit mask. Values below define individual bits.

// ReportFullState flag can be used by the Server if the Client did not
// include some sub-message in the last AgentToServer message (which is an
// allowed optimization) but the Server detects that it does not have it (e.g.
// was restarted and lost state).
ReportFullState = 0x00000001;
FetchPipelineConfigDetail = 0x00000002;
FetchInstanceConfigDetail = 0x00000004;
// bits before 2^16 (inclusive) are reserved for future official fields
}

// ConfigServer's response to Agent's request
message HeartbeatResponse {
bytes request_id = 1;
CommonResponse commonResponse = 2; // Set common response
uint64 capabilities = 3; // Bitmask of flags defined by ServerCapabilities enum

repeated ConfigDetail pipeline_config_updates = 4; // Agent's pipeline config update status
repeated ConfigDetail instance_config_updates = 5; // Agent's instance config update status
repeated CommandDetail custom_command_updates = 6; // Agent's commands updates
uint64 flags = 7; // Predefined command flag
bytes opaque = 8; // Opaque data for extension
bytes request_id = 1;
CommonResponse commonResponse = 2; // Set common response
uint64 capabilities =
3; // Bitmask of flags defined by ServerCapabilities enum

repeated ConfigDetail pipeline_config_updates =
4; // Agent's pipeline config update status
repeated ConfigDetail instance_config_updates =
5; // Agent's instance config update status
repeated CommandDetail custom_command_updates = 6; // Agent's commands updates
uint64 flags = 7; // Predefined command flag
bytes opaque = 8; // Opaque data for extension
}

// optional api
// API: /Agent/FetchPipelineConfig/
// API: /Agent/FetchInstanceConfig/
// Agent request to ConfigServer, pulling details of the config
message FetchConfigRequest {
bytes request_id = 1;
bytes instance_id = 2; // Agent's unique identification
repeated ConfigInfo req_configs = 3; // Config's name and version/hash
bytes request_id = 1;
bytes instance_id = 2; // Agent's unique identification
repeated ConfigInfo req_configs = 3; // Config's name and version/hash
}

// ConfigServer response to Agent's request
message FetchConfigResponse {
bytes request_id = 1;
CommonResponse commonResponse = 2;
repeated ConfigDetail config_details = 3; // config detail
bytes request_id = 1;
CommonResponse commonResponse = 2;
repeated ConfigDetail config_details = 3; // config detail
}

message CommonResponse
{
int32 status = 1;
bytes errorMessage = 2;
}
// Optional api
// API: /Agent/ReportPipelineConfig/
// API: /Agent/ReportInstanceConfig/
// Agent request to ConfigServer, report status of the config
message ReportConfigRequest {
bytes request_id = 1;
bytes instance_id = 2; // Agent's unique identification
repeated ConfigInfo config_infos = 3; // Config status info
}

// ConfigServer response to Agent's request
message ReportConfigResponse {
bytes request_id = 1;
CommonResponse commonResponse = 2;
}

message CommonResponse {
int32 status = 1;
bytes errorMessage = 2;
}

0 comments on commit b8b7d05

Please sign in to comment.