diff --git a/docs/KubectlSplunk.md b/docs/KubectlSplunk.md index d8741ab0c..c691980a8 100644 --- a/docs/KubectlSplunk.md +++ b/docs/KubectlSplunk.md @@ -2,15 +2,17 @@ ## Overview -`kubectl-splunk` is a kubectl plugin that allows you to execute Splunk commands directly within Splunk pods running in a Kubernetes cluster. It simplifies the management and interaction with Splunk instances deployed as StatefulSets or Deployments by providing a convenient command-line interface. +`kubectl-splunk` is a `kubectl` plugin that allows you to execute Splunk commands directly within Splunk pods running in a Kubernetes cluster. It simplifies the management and interaction with Splunk instances deployed as StatefulSets or Deployments by providing a convenient command-line interface. This plugin supports various features such as: - Executing Splunk CLI commands inside Splunk pods. +- Running Splunk REST API calls via port-forwarding. - Specifying pods directly via command line or configuration file. +- Automatic retrieval of Splunk admin credentials from pods. - Interactive shell access to Splunk pods. - Copying files to and from Splunk pods. -- Handling authentication securely. +- Handling authentication securely with credential storage. - Customizable configurations and verbosity levels. - Cross-platform compatibility. - Auto-completion support. @@ -30,8 +32,11 @@ This plugin supports various features such as: - [Configuration File](#configuration-file) - [Environment Variables](#environment-variables) - [Pod Selection Behavior](#pod-selection-behavior) -- [Features](#features) - [Authentication](#authentication) + - [Default Credentials](#default-credentials) + - [Credential Storage](#credential-storage) +- [Features](#features) +- [REST API Mode](#rest-api-mode) - [Copy Mode](#copy-mode) - [Interactive Shell](#interactive-shell) - [Logging and Verbosity](#logging-and-verbosity) @@ -48,6 +53,10 @@ This plugin supports various features such as: - **kubectl**: The Kubernetes command-line tool must be installed and configured. - **Access to Kubernetes Cluster**: You should have access to the Kubernetes cluster where Splunk is deployed. - **Splunk CLI in Pods**: The Splunk Command Line Interface should be available within the Splunk pods. +- **Python Packages**: Install required Python packages: + ```bash + pip install requests argcomplete + ``` --- @@ -94,6 +103,7 @@ kubectl splunk [global options] [mode options] ### Modes of Operation - **exec**: Execute a Splunk command inside a Splunk pod. +- **rest**: Execute a Splunk REST API call via port-forwarding. - **cp**: Copy files to or from a Splunk pod. - **interactive**: Start an interactive shell inside a Splunk pod (use `--interactive` flag). @@ -109,18 +119,41 @@ kubectl splunk [global options] [mode options] - `-P`, `--pod`: Specify the exact pod name to run the command on (can be set in config/env). - `-i`, `--interactive`: Start an interactive shell inside the Splunk pod. - `--splunk-path`: Path to the Splunk CLI inside the container (default: `splunk` or from config/env). +- `--local-port`: Local port for port-forwarding in REST mode (default: `8000` or from config/env). - `-v`: Increase output verbosity (use `-v`, `-vv`, or `-vvv`). - `--version`: Show program version and exit. ### Authentication Options -- `-u`, `--username`: Username for Splunk authentication. -- `-p`, `--password`: Password for Splunk authentication (will prompt if not provided). +- `-u`, `--username`: Username for Splunk authentication (default: `admin`). +- `-p`, `--password`: Password for Splunk authentication (will prompt or auto-detect if not provided). +- `--insecure`: Disable SSL certificate verification (useful for self-signed certificates). +- `--save-credentials`: Save credentials securely for future use. ### Mode Options -- **exec**: Followed by the Splunk command and its arguments. -- **cp**: Requires `src` and `dest` arguments for source and destination paths. +#### **exec** + +- **Usage**: `kubectl splunk exec [splunk_command]` +- **Options**: + - `splunk_command`: Splunk command to execute (e.g., `list user`). + +#### **rest** + +- **Usage**: `kubectl splunk rest METHOD ENDPOINT [options]` +- **Options**: + - `METHOD`: HTTP method (`GET`, `POST`, `PUT`, `DELETE`). + - `ENDPOINT`: Splunk REST API endpoint (e.g., `/services/server/info`). + - `--data`: Data to send with the request (for `POST`/`PUT`). + - `--params`: Query parameters (e.g., `"key1=value1&key2=value2"`). + +#### **cp** + +- **Usage**: `kubectl splunk cp SRC DEST` +- **Options**: + - `SRC`: Source file path. + - `DEST`: Destination file path. + - Use `:` to indicate the remote path in the pod (e.g., `:/path/in/pod`). --- @@ -156,14 +189,6 @@ Or using the short alias: kubectl splunk -P splunk-idxc-indexer-0 exec status ``` -### Use Pod Name from Configuration File - -If you have set the `pod_name` in your configuration file, you can run commands without specifying the pod: - -```bash -kubectl splunk exec status -``` - ### Start an Interactive Shell ```bash @@ -182,15 +207,23 @@ kubectl splunk cp /local/path/file.txt :/remote/path/file.txt kubectl splunk cp :/remote/path/file.txt /local/path/file.txt ``` -**Note**: Use `:` to indicate the remote path in the pod. +### Execute a REST API Call + +```bash +kubectl splunk rest GET /services/server/info --insecure +``` -### Use Authentication +### Create a Search Job (POST Request) ```bash -kubectl splunk -u admin exec list user +kubectl splunk rest POST /services/search/jobs --data "search=search index=_internal | head 10" --insecure ``` -You will be prompted for the password if not provided with `-p` or `--password`. +### Use Authentication and Save Credentials + +```bash +kubectl splunk -u admin --save-credentials exec list user +``` ### Increase Verbosity @@ -222,12 +255,14 @@ namespace = splunk-namespace selector = app=splunk splunk_path = splunk pod_name = splunk-idxc-indexer-0 # Default pod name +local_port = 8000 # Default local port for REST mode ``` - **namespace**: Default Kubernetes namespace. - **selector**: Default label selector to identify Splunk pods. - **splunk_path**: Path to the Splunk CLI inside the container. - **pod_name**: Default pod name to use if not specified via command line. +- **local_port**: Default local port for port-forwarding in REST mode. ### Environment Variables @@ -237,6 +272,7 @@ You can set environment variables to override defaults: - `KUBECTL_SPLUNK_SELECTOR`: Sets the default label selector. - `KUBECTL_SPLUNK_PATH`: Sets the default Splunk CLI path. - `KUBECTL_SPLUNK_POD`: Sets the default pod name. +- `KUBECTL_SPLUNK_LOCAL_PORT`: Sets the default local port for REST mode. **Example**: @@ -244,6 +280,7 @@ You can set environment variables to override defaults: export KUBECTL_SPLUNK_NAMESPACE=splunk-namespace export KUBECTL_SPLUNK_SELECTOR=app=splunk export KUBECTL_SPLUNK_POD=splunk-idxc-indexer-0 +export KUBECTL_SPLUNK_LOCAL_PORT=8000 ``` ### Pod Selection Behavior @@ -258,30 +295,83 @@ The script determines which pod to use based on the following priority: --- +## Authentication + +### Default Credentials + +- **Username**: Defaults to `admin` if not specified. +- **Password**: If not provided, the script attempts to retrieve the password from the pod's `/mnt/splunk-secrets/password` file. +- **Automatic Password Retrieval**: Works if the password file is accessible within the pod. + +**Usage Without Credentials**: + +```bash +kubectl splunk exec list user +``` + +### Credential Storage + +- **Save Credentials**: Use `--save-credentials` to store credentials securely for future use. +- **Credentials File**: Stored in `~/.kubectl_splunk_credentials` with permissions set to `600`. +- **Provide Credentials Once**: + + ```bash + kubectl splunk -u admin --save-credentials exec list user + ``` + +- **Subsequent Commands**: Credentials are used automatically. + +**Note**: Passwords are handled securely and are not exposed in logs or command outputs. + +--- + ## Features - **Execute Splunk Commands**: Run any Splunk CLI command directly within the pod. +- **REST API Support**: Execute Splunk REST API calls via port-forwarding. - **Pod Selection**: Specify a pod directly via command line, environment variable, or configuration file. If not specified, the script will prompt for selection when multiple pods are present. +- **Automatic Credential Retrieval**: Defaults to `admin` user and retrieves the password from the pod if not provided. - **Interactive Shell**: Start a shell session inside the Splunk pod. - **Copy Files**: Transfer files to and from Splunk pods. -- **Authentication Support**: Securely handle Splunk authentication credentials. +- **Authentication Handling**: Securely handle Splunk authentication credentials with options to save them. - **Configuration Flexibility**: Use config files or environment variables for defaults. - **Verbosity Control**: Adjust logging levels for more or less output. - **Caching**: Pod information is cached for improved performance. -- **Cross-Platform**: Works on Linux, macOS, and Windows. - **Auto-Completion**: Supports shell auto-completion for commands and options. +- **Secure Logging**: Sensitive information such as passwords is not logged. --- -## Authentication +## REST API Mode + +Use the `rest` mode to execute Splunk REST API calls via port-forwarding. -If the Splunk CLI requires authentication, you can provide credentials using `-u` and `-p`: +**Usage**: ```bash -kubectl splunk -u admin -p mypassword exec list user +kubectl splunk rest METHOD ENDPOINT [--data DATA] [--params PARAMS] [options] ``` -If you omit `-p`, you will be securely prompted for the password. +- **METHOD**: HTTP method (`GET`, `POST`, `PUT`, `DELETE`). +- **ENDPOINT**: Splunk REST API endpoint (e.g., `/services/server/info`). +- **Options**: + - `--data`: Data to send with the request (for `POST`/`PUT`). + - `--params`: Query parameters (e.g., `"key1=value1&key2=value2"`). + - `--insecure`: Disable SSL certificate verification. + +**Examples**: + +- **Get Server Info**: + + ```bash + kubectl splunk rest GET /services/server/info --insecure + ``` + +- **Create a Search Job**: + + ```bash + kubectl splunk rest POST /services/search/jobs --data "search=search index=_internal | head 10" --insecure + ``` --- @@ -336,6 +426,8 @@ Adjust the logging verbosity using the `-v` flag: kubectl splunk -vv exec status ``` +**Note**: Sensitive information such as passwords is masked in logs. + --- ## Caching @@ -383,7 +475,7 @@ eval "$(register-python-argcomplete kubectl-splunk)" Reload your shell configuration: ```bash -source ~/.bashrc +source ~/.bashrc # or your shell's config file ``` --- @@ -402,6 +494,10 @@ source ~/.bashrc - **Caching Issues**: If you believe the script is using an outdated pod from the cache, clear the cache by deleting the cache file. +- **Password Retrieval Failure**: If the script fails to retrieve the password from the pod, ensure that: + - You have the necessary permissions to execute commands in the pod. + - The password file `/mnt/splunk-secrets/password` exists and is accessible. + --- ## License @@ -416,12 +512,10 @@ Contributions are welcome! Please submit issues and pull requests via the projec --- - ## Feedback -If you have any questions, suggestions, or need assistance, please open an issue on the project's GitHub repository or contact support. +If you have any questions, suggestions, or need assistance, please open an issue on the project's GitHub repository or contact [Your Contact Information]. --- Thank you for using `kubectl-splunk`! We hope this plugin enhances your productivity when managing Splunk deployments on Kubernetes. - diff --git a/tools/kubectl-splunk b/tools/kubectl-splunk index abf43aecf..1dc7dc8de 100755 --- a/tools/kubectl-splunk +++ b/tools/kubectl-splunk @@ -10,9 +10,15 @@ import threading import time import json import getpass +import requests from concurrent.futures import ThreadPoolExecutor from argparse import RawDescriptionHelpFormatter +try: + import argcomplete +except ImportError: + argcomplete = None + def load_config(): """Load configuration from file and environment variables.""" config = configparser.ConfigParser() @@ -23,27 +29,30 @@ def load_config(): selector = config.get('DEFAULT', 'selector', fallback='app=splunk') splunk_path = config.get('DEFAULT', 'splunk_path', fallback='splunk') pod_name = config.get('DEFAULT', 'pod_name', fallback=None) + local_port = config.getint('DEFAULT', 'local_port', fallback=8000) else: namespace = 'default' selector = 'app=splunk' splunk_path = 'splunk' pod_name = None + local_port = 8000 # Override with environment variables if set namespace = os.environ.get('KUBECTL_SPLUNK_NAMESPACE', namespace) selector = os.environ.get('KUBECTL_SPLUNK_SELECTOR', selector) splunk_path = os.environ.get('KUBECTL_SPLUNK_PATH', splunk_path) pod_name = os.environ.get('KUBECTL_SPLUNK_POD', pod_name) + local_port = int(os.environ.get('KUBECTL_SPLUNK_LOCAL_PORT', local_port)) - return namespace, selector, splunk_path, pod_name + return namespace, selector, splunk_path, pod_name, local_port -def parse_args(namespace, selector, splunk_path, pod_name_from_config): +def parse_args(namespace, selector, splunk_path, pod_name_from_config, local_port_from_config): """Parse command-line arguments.""" epilog_text = """ Examples: kubectl splunk exec search "index=_internal | head 10" kubectl splunk -n splunk-namespace -l app=splunk -P splunk-idxc-indexer-0 exec status - kubectl splunk cp /local/path/file.txt :/remote/path/file.txt + kubectl splunk rest GET /services/server/info kubectl splunk --interactive """ parser = argparse.ArgumentParser( @@ -63,14 +72,20 @@ Examples: help='Start an interactive shell inside the Splunk pod') parser.add_argument('--splunk-path', default=splunk_path, help='Path to the Splunk CLI inside the container (default from config/env or "splunk")') + parser.add_argument('--local-port', type=int, default=local_port_from_config, + help='Local port for port-forwarding in REST mode (default from config/env or 8000)') parser.add_argument('-v', '--verbose', action='count', default=0, help='Increase output verbosity (e.g., -v, -vv, -vvv)') - parser.add_argument('--version', action='version', version='kubectl-splunk 1.3', + parser.add_argument('--version', action='version', version='kubectl-splunk 1.6', help='Show program version and exit') auth_group = parser.add_argument_group('Authentication') - auth_group.add_argument('-u', '--username', help='Username for Splunk authentication') - auth_group.add_argument('-p', '--password', help='Password for Splunk authentication (will prompt if not provided)') + auth_group.add_argument('-u', '--username', help='Username for Splunk authentication (default: admin)') + auth_group.add_argument('-p', '--password', help='Password for Splunk authentication (will prompt or auto-detect if not provided)') + auth_group.add_argument('--insecure', action='store_true', + help='Disable SSL certificate verification') + auth_group.add_argument('--save-credentials', action='store_true', + help='Save credentials securely for future use') subparsers = parser.add_subparsers(dest='mode', required=True, help='Available modes') @@ -83,6 +98,16 @@ Examples: cp_parser.add_argument('src', help='Source file path') cp_parser.add_argument('dest', help='Destination file path') + # Subparser for rest mode + rest_parser = subparsers.add_parser('rest', help='Execute a Splunk REST API call') + rest_parser.add_argument('method', choices=['GET', 'POST', 'PUT', 'DELETE'], help='HTTP method') + rest_parser.add_argument('endpoint', help='Splunk REST API endpoint (e.g., /services/server/info)') + rest_parser.add_argument('--data', help='Data to send with the request (for POST/PUT)') + rest_parser.add_argument('--params', help='Query parameters (e.g., "key1=value1&key2=value2")') + + if argcomplete: + argcomplete.autocomplete(parser) + args = parser.parse_args() return args @@ -174,6 +199,66 @@ def load_cached_pod(namespace, selector): return cache['pod_name'] return None +def get_credentials(args, pod_name): + """Retrieve stored credentials, prompt the user, or retrieve from pod.""" + credentials_file = os.path.expanduser('~/.kubectl_splunk_credentials') + username = args.username + password = args.password + + # Default username to 'admin' if not provided + if not username: + username = 'admin' + + # Load credentials from file if available + if not password: + if os.path.exists(credentials_file): + try: + with open(credentials_file, 'r') as f: + creds = json.load(f) + username = creds.get('username', username) + password = creds.get('password', password) + except Exception as e: + logging.warning("Failed to read stored credentials.") + os.remove(credentials_file) # Remove corrupted credentials file + + # Retrieve password from pod if still not available + if not password and username == 'admin': + password = retrieve_password_from_pod(args, pod_name) + if not password: + logging.error("Failed to retrieve password from pod.") + sys.exit(1) + + # Prompt for missing credentials + if not password: + password = getpass.getpass(prompt='Splunk Password: ') + + # Save credentials if requested + if args.save_credentials: + creds = {'username': username, 'password': password} + try: + with open(credentials_file, 'w') as f: + json.dump(creds, f) + os.chmod(credentials_file, 0o600) # Set file permissions to be readable by owner only + except Exception as e: + logging.warning("Failed to save credentials.") + + return username, password + +def retrieve_password_from_pod(args, pod_name): + """Retrieve the admin password from the pod's /mnt/splunk-secrets/password file.""" + kubectl_cmd = ['kubectl'] + if args.context: + kubectl_cmd.extend(['--context', args.context]) + kubectl_cmd.extend(['exec', '-n', args.namespace, pod_name, '--', 'cat', '/mnt/splunk-secrets/password']) + + logging.debug(f"Retrieving password from pod {pod_name}") + try: + password = subprocess.check_output(kubectl_cmd, universal_newlines=True).strip() + return password + except subprocess.CalledProcessError as e: + logging.error(f"Failed to retrieve password from pod {pod_name}: {e}") + return None + def execute_on_pod(args, pod_name): """Execute the Splunk command on a single pod.""" kubectl_cmd = ['kubectl'] @@ -185,24 +270,23 @@ def execute_on_pod(args, pod_name): cmd = kubectl_cmd + [args.splunk_path] cmd += args.splunk_command # Add the Splunk command arguments # Authentication - if args.username: - if not args.password: - args.password = getpass.getpass(prompt='Splunk Password: ') - cmd += ['-auth', f"{args.username}:{args.password}"] # Add auth after command arguments + username, password = get_credentials(args, pod_name) + cmd += ['-auth', f"{username}:{'********'}"] # Masked password for logging + logging.debug(f"Executing command on pod {pod_name}: {' '.join(cmd)}") + # Replace masked password with actual password in the command to be executed + cmd[-1] = f"{username}:{password}" elif args.mode == 'interactive': cmd = kubectl_cmd + ['/bin/bash'] + logging.debug(f"Starting interactive shell on pod {pod_name}") else: return - sanitized_cmd = [arg if '-auth' not in arg else '-auth *****:*****' for arg in cmd] - logging.debug(f"Executing command on pod {pod_name}: {' '.join(sanitized_cmd)}") try: subprocess.run(cmd, check=True) except subprocess.CalledProcessError as e: logging.error(f"Command failed on pod {pod_name} with exit code {e.returncode}") sys.exit(e.returncode) - def copy_to_pod(args, pod_name): """Copy files to/from the Splunk pod.""" kubectl_cmd = ['kubectl'] @@ -224,16 +308,84 @@ def copy_to_pod(args, pod_name): logging.error(f"Copy command failed with exit code {e.returncode}") sys.exit(e.returncode) +def execute_rest_call(args, pod_name): + """Execute a Splunk REST API call.""" + # Start port-forwarding + port = 8089 + local_port = args.local_port # User-specified local port + kubectl_cmd = ['kubectl'] + if args.context: + kubectl_cmd.extend(['--context', args.context]) + kubectl_cmd.extend([ + 'port-forward', '-n', args.namespace, pod_name, + f'{local_port}:{port}' + ]) + + logging.debug(f"Starting port-forward with command: {' '.join(kubectl_cmd)}") + # Start port-forwarding as a background process + port_forward_proc = subprocess.Popen(kubectl_cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + try: + # Wait for port-forwarding to start + for _ in range(10): + if port_forward_proc.poll() is not None: + raise Exception("Port-forward process terminated unexpectedly.") + time.sleep(0.5) + # Build the URL + url = f'https://127.0.0.1:{local_port}{args.endpoint}' + # Handle authentication + username, password = get_credentials(args, pod_name) + auth = (username, password) + + # Handle SSL verification + verify_ssl = not args.insecure + + # Prepare request parameters + method = args.method.upper() + headers = {'Content-Type': 'application/json'} + data = args.data + params = {} + if args.params: + # Parse query parameters + params = dict(param.split('=') for param in args.params.split('&')) + + logging.debug(f"Sending {method} request to {url}") + # Send the request + response = requests.request( + method=method, + url=url, + headers=headers, + params=params, + data=data, + auth=auth, + verify=verify_ssl + ) + # Print the response + print(response.text) + if response.status_code >= 400: + logging.error(f"HTTP {response.status_code}: {response.reason}") + sys.exit(1) + except Exception as e: + logging.error(f"Error during REST API call: {e}") + sys.exit(1) + finally: + # Terminate port-forwarding + port_forward_proc.terminate() + port_forward_proc.wait() + def main(): # Load configuration - namespace, selector, splunk_path, pod_name_from_config = load_config() + namespace, selector, splunk_path, pod_name_from_config, local_port_from_config = load_config() # Parse arguments - args = parse_args(namespace, selector, splunk_path, pod_name_from_config) + args = parse_args(namespace, selector, splunk_path, pod_name_from_config, local_port_from_config) # Set up logging setup_logging(args.verbose) - logging.debug(f"Arguments: {args}") + # Ensure sensitive information is not logged + masked_args = vars(args).copy() + if masked_args.get('password'): + masked_args['password'] = '********' + logging.debug(f"Arguments: {masked_args}") # Handle interactive shell separately if args.interactive: @@ -275,6 +427,12 @@ def main(): logging.error("Copy mode does not support multiple pods. Please specify a single pod.") sys.exit(1) copy_to_pod(args, pods[0]) + elif args.mode == 'rest': + # REST API call (only supports a single pod) + if len(pods) > 1: + logging.error("REST mode does not support multiple pods. Please specify a single pod.") + sys.exit(1) + execute_rest_call(args, pods[0]) else: logging.error(f"Unknown mode: {args.mode}") sys.exit(1)