{
  "phases": {
    "hot": {
      "actions": {
        "rollover": { "max_age": "1d", "max_primary_shard_size": "30gb" },
        "set_priority": { "priority": 100 }
      }
    },
    "warm": {
      "min_age": "7d",
      "actions": {
        "shrink": { "number_of_shards": 1 },
        "forcemerge": { "max_num_segments": 1 },
        "allocate": { "include": { "data_tier": "warm" } },
        "set_priority": { "priority": 50 }
      }
    },
    "cold": {
      "min_age": "30d",
      "actions": {
        "searchable_snapshot": { "snapshot_repository": "s3-cold" },
        "set_priority": { "priority": 0 }
      }
    },
    "delete": {
      "min_age": "365d",
      "actions": { "delete": {} }
    }
  }
}

Pattern 3: Use Index Templates with Explicit Mappings

Schemaless ingestion is convenient until it isn't. The day a developer logs response_time: "1.2s" in one service and response_time: 1200 in another, mapping conflicts break dashboards. The fix is index templates with explicit field types and dynamic-mapping disabled for production indices.

PUT _index_template/logs-app
{
  "index_patterns": ["logs-app-*"],
  "template": {
    "mappings": {
      "dynamic": "strict",
      "properties": {
        "@timestamp":   { "type": "date" },
        "service":      { "type": "keyword" },
        "level":        { "type": "keyword" },
        "message":      { "type": "text" },
        "response_time_ms": { "type": "long" },
        "trace_id":     { "type": "keyword" }
      }
    }
  }
}

Strict mappings reject documents with unmapped fields. That sounds harsh; it isn't. Reject-on-unknown is what surfaces accidental field-name drift before it metastasises into a hundred broken dashboards.

Pattern 4: Snapshot Daily and Test Restores

Snapshots to object storage are the only real disaster recovery for an Elasticsearch cluster. Configure them on day one. The minimum standard:

• Daily snapshots to S3 / GCS / Azure blob, retained 30 days
• Weekly snapshots retained 12 weeks
• Monthly snapshots retained 12 months for archival / compliance
• Quarterly restore drill — pull the latest snapshot into a parallel cluster and verify data integrity

The fourth bullet is the one most teams skip and the one that bites them when they need it. Snapshots that have never been restored are not real backups.

Pattern 5: Separate Ingest, Storage, and Compute Costs

Cost optimisation on ELK is rarely about reducing total data — it is about putting each piece of data on the right tier. Three rules of thumb:

1. Hot tier (last 7-14 days) on local SSD — fast indexing, fast search, expensive disk. Optimise for IOPS
2. Warm tier (15-30 days) on slower disk — read-mostly, occasional search. Cheaper IOPS, larger disk
3. Cold / frozen tier (30-365 days) on object storage with searchable snapshots — minimal compute, cheap storage. Slower queries acceptable

For an AWS deployment, gp3 EBS for hot, st1 for warm, S3 for cold typically cuts storage cost by 50-70% versus everything-on-gp3.

Pattern 6: Build a Cluster Monitoring Cluster

Self-monitoring breaks during outages. Run a small dedicated monitoring cluster (3 nodes, modest sizing) and ship metrics from your production cluster to it via Metricbeat with the elasticsearch module. Alert on:

• Cluster status not green for >5 minutes
• Indexing rate sudden drop >50% YoY weekday
• Search latency p95 >2x baseline
• Disk watermark hit on any node
• JVM old-gen pressure >75% sustained
• Pending tasks count >100

These six alerts catch roughly 90% of the meaningful production incidents.

Pattern 7: Apply RBAC, Field Security, and Audit Logging

Elasticsearch security is feature-complete but not on-by-default for self-managed clusters. The minimum production configuration:

• TLS on transport and HTTP layers (mandatory in 8.x)
• Role-based access control with index- and document-level filtering
• Field-level security to redact PII for general users
• Audit logging shipped to a separate cluster or to your soc security delivery SIEM
• SAML / OIDC integration for SSO

Pattern 8: Plan Version Upgrades Quarterly

Elasticsearch ships major versions roughly annually and minor versions quarterly. Skip three minor versions and the upgrade path becomes painful. The discipline that prevents this:

• Quarterly minor-version upgrade rolling through the cluster
• Annual major-version review with parallel-cluster migration plan
• Always read the breaking-changes notes before scheduling

How Opsio Helps

Opsio's end-to-end elk stack service applies these patterns as standard. Customer engagements typically reduce hot-tier disk cost by 30-50% in the first quarter through ILM tuning alone, and reduce shard count by 60-90% on clusters that had drifted into shard-explosion territory. We also handle cloud cost optimization services across the wider data platform, where ELK is one of several systems whose tier and retention policies usually need joint redesign.