Huawei Cloud CCE Capacity Trend Forecaster

⚠️ Execution Method (Must Read): This skill executes queries via the local Python dispatcher script. Using hcloud, openstack, or other CLI tools or direct API calls is prohibited.

• The dispatcher script is located at scripts/huawei-cloud.py within the skill directory
• All scripts and environment check scripts are inside the skill package. You must use skill action=exec to execute them. Do not run them directly in a shell.
• Do not attempt hcloud, openstack, curl IAM, or any other CLI/API methods. This skill does not depend on those tools.
• All paths are relative to the skill directory, which is the directory where this SKILL.md is located.

Overview

This skill analyzes CCE capacity trends over a user-selected period (1 hour to 1 month), forecasts resource bottlenecks, simulates elasticity policies, generates curve charts and reports, and compares recurring history records. The core principle is to treat capacity as a time-series problem: rising trends with high p95 signal risk, low utilization with conservative simulation signals cost optimization opportunity.

This skill has read-only tools (collection, analysis, simulation, report generation) and preview tools for HPA and autoscaler configuration. Applying configuration changes requires a two-step confirmation workflow with confirm=true and explicit user approval.

Related Skills

Capabilities

1. Collect node metrics, HPA status, nodepool autoscaling status, and business Deployment inventory
2. Calculate trend statistics (avg, min, max, p95, slope, trend direction) for CPU, memory, and disk
3. Predict bottleneck arrival time against configurable thresholds
4. Simulate recommended node counts and reducible nodes with headroom
5. Generate JSON/Markdown/HTML reports with embedded SVG curve charts
6. Store and compare recurring history records to track before-and-after effects
7. Preview and apply HPA configuration with two-step confirmation workflow
8. Recommend node autoscaler min/max bounds adjustments

Typical Use Cases

• Run a 6-hour capacity trend analysis and identify rising CPU trends approaching bottleneck threshold
• Compare weekly capacity records to validate that a previous optimization reduced utilization
• Simulate node requirements with 60% CPU target and 15% headroom, then preview HPA adjustments
• Forecast when memory will reach 80% bottleneck threshold based on current slope
• Generate a monthly capacity report with curve charts for stakeholder review
• Check HPA coverage percentage and preview HPA manifest for uncovered Deployments

Prerequisites

Python Dependencies

The dispatcher script requires Python >= 3.6 and the following packages:

• huaweicloudsdkcore
• huaweicloudsdkaom
• huaweicloudsdkcce
• huaweicloudsdkiam

Credential Configuration

🚫 Never expose or log AK/SK values. Credentials exist only in the current request call stack and are released after each invocation. Do not write credentials to files, logs, or responses.

✅ Use environment variables HUAWEI_AK / HUAWEI_SK for authentication. The dispatcher reads them automatically.

Security rules for credentials:

1. No persistent storage — never write AK/SK, tokens, or certificates to disk files
2. No long-term memory cache — AK/SK exists only during the current API call and is released afterward
3. Project ID memory cache only — only non-sensitive project IDs may be cached in process memory (never written to disk)
4. No log leakage — never include AK/SK in logs, response output, or error messages
5. Output desensitization — output only capacity, trend, and simulation information; never expose authentication credentials

AK/SK may be provided in two ways:

• Via environment variables HUAWEI_AK / HUAWEI_SK (recommended)
• Via per-call parameters ak and sk (not recommended for production)

IAM Permissions

Core Tools

All actions are invoked via the dispatcher script:

python3 scripts/huawei-cloud.py <action> region=<region> [key=value ...]

Two-Step Confirmation Workflow for Mutation Operations

Mutation operations (apply HPA configuration with confirm=true, change nodepool autoscaling bounds, resize node pools) require confirm=true to execute. Without confirm, the tool returns a preview and confirmation prompt only.

Step 1: Preview — call without confirm:

python3 scripts/huawei-cloud.py huawei_configure_cce_hpa \
  region=cn-north-4 cluster_id=<cluster-id> \
  namespace=default deployment_name=<name> \
  min_replicas=2 max_replicas=10 target_cpu_percent=60

Returns: operation preview, target HPA, configuration fields, and confirmation example. No real modification is performed.

Step 2: Confirm execution — call again with confirm=true:

python3 scripts/huawei-cloud.py huawei_configure_cce_hpa \
  region=cn-north-4 cluster_id=<cluster-id> \
  namespace=default deployment_name=<name> \
  min_replicas=2 max_replicas=10 target_cpu_percent=60 \
  confirm=true

Operations Requiring Confirmation

Prohibited Actions

Capacity Trend Analysis (Read-Only)

# Basic 6-hour analysis
python3 scripts/huawei-cloud.py huawei_analyze_cce_capacity_trend \
  region=cn-north-4 cluster_id=<cluster-id> hours=6

# Weekly analysis with custom targets and output directory
python3 scripts/huawei-cloud.py huawei_analyze_cce_capacity_trend \
  region=cn-north-4 cluster_id=<cluster-id> hours=168 \
  target_cpu_percent=60 target_memory_percent=70 \
  headroom_percent=15 bottleneck_percent=80 \
  output_dir=debug/capacity-trend/<cluster>

# Monthly analysis with business namespace scope and action note
python3 scripts/huawei-cloud.py huawei_analyze_cce_capacity_trend \
  region=cn-north-4 cluster_id=<cluster-id> hours=744 \
  business_namespaces=default,production \
  action_note="Scaled HPA max_replicas from 5 to 10 on 2026-05-28"

# Recurring run for comparison
python3 scripts/huawei-cloud.py huawei_analyze_cce_capacity_trend \
  region=cn-north-4 cluster_id=<cluster-id> hours=6 \
  output_dir=debug/capacity-trend/<cluster> \
  history_dir=debug/capacity-trend/<cluster>/history

Inventory and Context (Read-Only)

# List clusters
python3 scripts/huawei-cloud.py huawei_list_cce_clusters region=cn-north-4

# Get node inventory
python3 scripts/huawei-cloud.py huawei_get_kubernetes_nodes \
  region=cn-north-4 cluster_id=<cluster-id>

# List nodepools
python3 scripts/huawei-cloud.py huawei_list_cce_nodepools \
  region=cn-north-4 cluster_id=<cluster-id>

# Get deployments
python3 scripts/huawei-cloud.py huawei_get_cce_deployments \
  region=cn-north-4 cluster_id=<cluster-id>

# List HPA configurations
python3 scripts/huawei-cloud.py huawei_list_cce_hpas \
  region=cn-north-4 cluster_id=<cluster-id>

Metrics (Read-Only)

# Node metrics TopN
python3 scripts/huawei-cloud.py huawei_get_cce_node_metrics_topN \
  region=cn-north-4 cluster_id=<cluster-id> top_n=10 hours=6

# AOM metrics
python3 scripts/huawei-cloud.py huawei_get_aom_metrics \
  region=cn-north-4 cluster_id=<cluster-id>

HPA Configuration (Preview + Confirm)

# Generate HPA manifest preview (read-only)
python3 scripts/huawei-cloud.py huawei_generate_cce_hpa_manifest \
  region=cn-north-4 cluster_id=<cluster-id> \
  namespace=default deployment_name=<name> \
  min_replicas=2 max_replicas=10 target_cpu_percent=60

# Preview HPA configuration (no execution)
python3 scripts/huawei-cloud.py huawei_configure_cce_hpa \
  region=cn-north-4 cluster_id=<cluster-id> \
  namespace=default deployment_name=<name> \
  min_replicas=2 max_replicas=10 target_cpu_percent=60

# Confirm and apply HPA configuration
python3 scripts/huawei-cloud.py huawei_configure_cce_hpa \
  region=cn-north-4 cluster_id=<cluster-id> \
  namespace=default deployment_name=<name> \
  min_replicas=2 max_replicas=10 target_cpu_percent=60 \
  confirm=true

Parameter Reference

Capacity Trend Analysis Parameters

HPA Configuration Parameters

Output Format

Capacity Trend Report

Each analysis run writes the following files:

Key Output Fields

See references/output-schema.md for the full JSON response schema.

Verification

1. Run the dispatcher with a known region and cluster to confirm connectivity:

   python3 scripts/huawei-cloud.py huawei_analyze_cce_capacity_trend \
     region=cn-north-4 cluster_id=<cluster-id> hours=1
   

2. Verify that capacity_stats includes trend direction and bottleneck prediction for CPU, memory, and disk
3. Verify that simulation.avg_recommended_nodes and max_recommended_nodes are within autoscaler min/max bounds
4. Verify that capacity-trend-report.html contains embedded SVG charts
5. Run a second analysis and verify that history_comparison shows deltas from the first run
6. Test HPA preview workflow: call huawei_configure_cce_hpa without confirm and verify it returns a preview only
7. After a confirmed HPA change, run a new capacity analysis to compare before-and-after effects

Best Practices

1. Always use huawei_analyze_cce_capacity_trend as the main action; it collects, analyzes, simulates, and generates reports in a single call
2. Accept windows from 1 hour to 1 month; common schedules are every 6 hours, daily, weekly, and monthly
3. Do not treat a single period as enough evidence for aggressive downsizing; prefer at least two comparable records before lowering baseline capacity
4. If the simulation recommends lower capacity but p95 or max utilization is close to the bottleneck threshold, keep headroom and propose observation instead
5. For real changes, provide the exact reason, expected effect, rollback path, and validation checks
6. Configuration execution must be user-approved and should be followed by a new capacity record to compare before-and-after effects
7. Use action_note to record optimization actions so the next cycle can compare their effects
8. If AOM metrics are unavailable, report the gap and do not invent trend conclusions
9. Always consider both workload elasticity (HPA) and node elasticity (nodepool autoscaling) when making recommendations

Reference Documents

Notes

1. This skill has read-only tools (analysis, simulation, report generation) and preview tools for HPA configuration — applying configuration changes requires confirm=true two-step confirmation
2. Do not treat a single period as enough evidence for aggressive downsizing; prefer at least two comparable records
3. If simulation recommends lower capacity but p95 is near bottleneck threshold, keep headroom and propose observation
4. Never expose or log AK/SK or environment variable values
5. All actions are executed via python3 scripts/huawei-cloud.py <action>; do not use hcloud CLI or direct API calls
6. Use action_note to record optimization events so subsequent runs can validate their effects
7. If remediation actions are needed (scale, reboot, drain), output recommendations only and hand off to huawei-cloud-cce-auto-remediation-runner

Common Pitfalls