Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added structs for replication diag #330

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions health.go
Original file line number Diff line number Diff line change
Expand Up @@ -1065,9 +1065,10 @@ type HealthInfo struct {
Version string `json:"version"`
Error string `json:"error,omitempty"`

TimeStamp time.Time `json:"timestamp,omitempty"`
Sys SysInfo `json:"sys,omitempty"`
Minio MinioHealthInfo `json:"minio,omitempty"`
TimeStamp time.Time `json:"timestamp,omitempty"`
Sys SysInfo `json:"sys,omitempty"`
Minio MinioHealthInfo `json:"minio,omitempty"`
Replication ReplicationInfo `json:"replication,omitempty"`
}

func (info HealthInfo) String() string {
Expand Down Expand Up @@ -1123,6 +1124,7 @@ const (
HealthDataTypeSysErrors HealthDataType = "syserrors"
HealthDataTypeSysServices HealthDataType = "sysservices"
HealthDataTypeSysConfig HealthDataType = "sysconfig"
HealthDataTypeReplication HealthDataType = "replication"
)

// HealthDataTypesMap - Map of Health datatypes
Expand All @@ -1140,6 +1142,7 @@ var HealthDataTypesMap = map[string]HealthDataType{
"syserrors": HealthDataTypeSysErrors,
"sysservices": HealthDataTypeSysServices,
"sysconfig": HealthDataTypeSysConfig,
"replication": HealthDataTypeReplication,
}

// HealthDataTypesList - List of health datatypes
Expand All @@ -1157,6 +1160,7 @@ var HealthDataTypesList = []HealthDataType{
HealthDataTypeSysErrors,
HealthDataTypeSysServices,
HealthDataTypeSysConfig,
HealthDataTypeReplication,
}

// HealthInfoVersionStruct - struct for health info version
Expand Down
167 changes: 167 additions & 0 deletions replication.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
//
// Copyright (c) 2015-2025 MinIO, Inc.
//
// This file is part of MinIO Object Storage stack
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as
// published by the Free Software Foundation, either version 3 of the
// License, or (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//

package madmin

import (
"time"

"github.com/minio/minio-go/v7/pkg/replication"
)

type ReplicationInfo struct {
Error string `json:"error,omitempty"`
ActiveWorkers WorkerStat `json:"active_workers,omitempty"`
Queued InQueueMetric `json:"queued,omitempty"`
ReplicaCount int64 `json:"replica_count,omitempty"`
ReplicaSize int64 `json:"replica_size,omitempty"`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do ReplicaCount and ReplicaSize represent?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ReplicaCount = no of objects replicated
ReplicaSize = no of bytes replicated

IIUC

Proxying bool `json:"proxying,omitempty"`
Proxied ReplProxyMetric `json:"proxied,omitempty"`
Sites []ReplicationSite `json:"sites,omitempty"`
}

type ReplicationSite struct {
Addr string `json:"addr,omitempty"`
Info SiteReplicationSiteInfo `json:"info,omitempty"`
}

type SiteReplicationSiteInfo struct {
Nodes []MinIONode `json:"nodes,omitempty"`
LDAPEnabled bool `json:"ldap_enabled,omitempty"`
OpenIDEnabled bool `json:"openid_enabled,omitempty"`
BucketsCount int `json:"buckets_count,omitempty"`
// ReplicationWorkers int `json:"replication_workers"`
// MaxReplicationWorkers int `json:"max_replication_workers"`
Edge bool `json:"edge,omitempty"`
ILMEnabled bool `json:"ilm_enabled,omitempty"`
EncryptionEnabled bool `json:"encryption_enabled,omitempty"`
ILMExpiryReplication bool `json:"ilm_expiry_replication,omitempty"`
Copy link

@dhananjaykrutika dhananjaykrutika Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't lines 46-54 be captured per node, perhaps in MinIONode below, the intention being, to flag any mismatch across nodes in the subsequent reporting step?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thought was that across sites if they mismatch, that itself is issue

ReplicatedBuckets []ReplicatedBucket `json:"replicated_buckets,omitempty"`
ObjetLocking ObjectLockingInfo `json:"object_locking,omitempty"`
Throttle ReplicationThrottle `json:"throttle,omitempty"`
ReplicationTargets []ReplicationTarget `json:"replication_targets,omitempty"`
ReplicatedCount int64 `json:"replicated_count,omitempty"`
ReplicatedSize int64 `json:"replicated_size,omitempty"`
}

type MinIONode struct {
Addr string `json:"addr,omitempty"`
MinIOVersion string `json:"minio_version,omitempty"`
Uptime int64 `json:"uptime,omitempty"`
PoolID int `json:"poolid,omitempty"`
SetID int `json:"setid,omitempty"`
SiteHealingLeader bool `json:"site_healing_leader,omitempty"`
IsScanning bool `json:"is_scanning,omitempty"`
ILMExpiryInProgress bool `json:"ilm_expiry_in_progress,omitempty"`
// Resync BucketResyncInfo `json:"resync"`
}

type ReplicatedBucket struct {
Name string `json:"name,omitempty"`
ReplicationInfo BucketReplicationInfo `json:"replication_info,omitempty"`
}

type MissingReplicationInfo struct {
Bucket string `json:"bucket,omitempty"`
Config map[string]string `json:"config,omitempty"`
}

type ObjectLockingInfo struct {
Enabled bool `json:"enabled,omitempty"`
Buckets []string `json:"buckets,omitempty"`
}

type ReplicationTarget struct {
SourceBucket string `json:"source_bucket,omitempty"`
TargetBucket string `json:"target_bucket,omitempty"`
Addr string `json:"addr,omitempty"`
Online bool `json:"online,omitempty"`
TotalDowntime int64 `json:"total_downtime,omitempty"`
CurrentDowntime int64 `json:"current_downtime,omitempty"`
AdminPermissions bool `json:"admin_permissions,omitempty"`
SyncReplication bool `json:"sync_replication,omitempty"`
HeartbeatErrCount int64 `json:"heartbeat_err_count,omitempty"`
BandwidthLimit uint64 `json:"bandwidth_limit,omitempty"`
Latency TransferLatency `json:"xfer_rate,omitempty"`
Edge bool `json:"edge,omitempty"`
LBEndpoint bool `json:"lb_endpoint,omitempty"`
HeathCheck HeathCheckDetails `json:"heath_check,omitempty"`
}

type TransferLatency struct {
Current time.Duration `json:"current,omitempty"`
Average time.Duration `json:"avg,omitempty"`
Maximum time.Duration `json:"max,omitempty"`
}

type HeathCheckDetails struct {
Timestamp time.Time `json:"timestamp,omitempty"`
Latency string `json:"latency,omitempty"`
Duration time.Duration `json:"duration,omitempty"`
}

type BucketReplicationInfo struct {
VersionEnabled bool `json:"version_enabled,omitempty"`
ObjectLocking bool `json:"object_locking,omitempty"`
Edge bool `json:"edge,omitempty"`
ExcludedPrefixes []string `json:"excluded_prefixes,omitempty"`
DeleteReplication bool `json:"delete_replication,omitempty"`
DeleteMarkerReplication bool `json:"delete_marker_replication,omitempty"`
ILM ReplicationILMInfo `json:"ilm,omitempty"`
Encryption ReplicationEncInfo `json:"encryption,omitempty"`
Config replication.Config `json:"config,omitempty"`
ReplicationPriority int `json:"replication_priority,omitempty"`
Resync BucketResyncInfo `json:"resync,omitempty"`
Throttle ReplicationThrottle `json:"throttle,omitempty"`
}

type ReplicationILMInfo struct {
Enabled bool `json:"enabled,omitempty"`
Rules []ReplicationILMRule `json:"rules,omitempty"`
}

type ReplicationILMRule struct {
ID string `json:"id,omitempty"`
Expiration bool `json:"expiration,omitempty"`
Transition bool `json:"transition,omitempty"`
}

type ReplicationEncInfo struct {
Enabled bool `json:"enabled,omitempty"`
EncRules []BucketEncRule `json:"enc_rules,omitempty"`
}

type BucketEncRule struct {
Algorithm string `json:"algorithm,omitempty"`
EncKey string `json:"enc_key,omitempty"`
}

type BucketResyncInfo struct {
InProgress bool `json:"in_progress,omitempty"`
StartTime time.Time `json:"start_time,omitempty"`
FailedCount int64 `json:"failed_count,omitempty"`
FailedSize int64 `json:"failed_size,omitempty"`
ReplicatedCount int64 `json:"replicated_count,omitempty"`
ReplicatedSize int64 `json:"replicated_size,omitempty"`
}

type ReplicationThrottle struct {
IsSet bool `json:"is_set,omitempty"`
Limit uint64 `json:"limit,omitempty"`
}
Loading