fix: OpenSearch creates no logs index before its template and retention policy exist

backend/opensearch/start.sh

@@ -13,8 +13,8 @@ curl -sf -X PUT "$api/_index_template/logs" -H 'Content-Type: application/json'
 curl -sf -o /dev/null "$api/_plugins/_ism/policies/logs-30d" \
   || curl -sf -X PUT "$api/_plugins/_ism/policies/logs-30d" -H 'Content-Type: application/json' -d @/etc/opensearch-setup/logs-retention-policy.json > /dev/null \
   || exit 1
-# An index the collector created before the policy existed gets it too; one that has it already is skipped
-curl -s -X POST "$api/_plugins/_ism/add/logs-*" -H 'Content-Type: application/json' -d '{"policy_id": "logs-30d"}' > /dev/null
+# Compose starts OpenSearch refusing to create logs- indices, so none exists without the template and the policy
+curl -sf -X PUT "$api/_cluster/settings" -H 'Content-Type: application/json' -d '{"persistent": {"action.auto_create_index": "true"}}' > /dev/null || exit 1
 # The first wait returns when the trap runs; the second waits for OpenSearch to finish shutting down
 wait $pid
 wait $pid

docker-compose.yaml

@@ -365,6 +365,8 @@ services:
       - discovery.type=single-node
       # mmap would need vm.max_map_count raised on the host
       - node.store.allow_mmap=false
+      # No write can create a logs- index before start.sh has set up its template and retention policy; start.sh then allows it
+      - action.auto_create_index=-logs-*,+*
       - DISABLE_SECURITY_PLUGIN=true
       - DISABLE_INSTALL_DEMO_CONFIG=true
       - OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m

docs/operations/opensearch.md

@@ -43,7 +43,9 @@ The container's entrypoint is [`backend/opensearch/start.sh`](https://github.com
 --8<-- "backend/opensearch/logs-retention-policy.json"
 ```
 
-The policy deletes each daily index 30 days after it was created, the same retention as Loki. A stored policy is never overwritten, so a changed policy needs a new id. An index created before the policy existed, on the very first start, gets it attached by the same script.
+The policy deletes each daily index 30 days after it was created, the same retention as Loki. A stored policy is never overwritten, so a changed policy needs a new id.
+
+The collector starts writing as soon as OpenSearch answers, a moment before start.sh has put the template in. So compose starts OpenSearch with `action.auto_create_index=-logs-*,+*`: until then a write gets `index_not_found_exception` and is dropped (Loki has it), instead of creating a `logs-` index with default mappings. Once the template and the policy exist, start.sh allows auto-creation with a persistent cluster setting, so a restart needs no setup before writes go through. The cluster reports yellow even so: the retention plugin's own `.opendistro-ism-config` index asks for a replica that a single node can't place.
 
 ## Grafana