access-management

cicd/examples/README.md

@@ -0,0 +1,8 @@
+# IaC examples
+
+- [Update routing](update-routing/README.md): runnable local create, no-op, update,
+  rejected change and teardown, with a self-contained provider registry.
+- [AWS VPC/subnet](aws-vpc-subnet/README.md): explicit network-resource input for the
+  existing AWS IaC path, with apply and teardown commands.
+
+Generated ledgers and transcripts belong under `cicd/out/`, outside these source examples.

cicd/examples/aws-vpc-subnet/README.md

@@ -0,0 +1,135 @@
+# AWS VPC and subnet
+
+## Change the VPC Name tag declaratively
+
+The VPC in `update-public-ip.json` and `update-private-ip.json` now declares:
+
+```json
+"desired": {
+  "CidrBlock": "10.42.0.0/16",
+  "Name": "omni-demo-vpc"
+}
+```
+
+Change `desired.Name` and reapply the same file, collection and state directory. The
+document's `Name` route calls `CreateTags` for the recorded VPC ID. Reapplying an unchanged
+name reads the live tags and performs no tag mutation. External Name drift is repaired.
+This route sets or overwrites only `Name`, preserving all other tags.
+It does not implement tag deletion: keep a string Name in subsequent update inputs.
+
+Regenerate an existing overlay after updating this example:
+
+```sh
+python3 cicd/examples/aws-vpc-subnet/prepare-update.py \
+  --registry test/corpus/registry --out cicd/out/aws-update-registry --refresh
+```
+
+Then use the apply command below with a fresh run ID. The overlay uses the existing response
+and request template DSLs to project the Name tag and encode the tagging request. The SDK
+contains no special AWS tag-reconciliation logic. Mutable Name intent belongs in `desired.Name`.
+
+## Live update of the existing subnet
+
+After creating the VPC and subnet, apply `update-public-ip.json` to enable automatic
+public IPv4 assignment on future instance launches in that subnet. It changes the existing
+subnet through `ModifySubnetAttribute`; it does not launch an instance or add internet routing.
+The API input is documented in [AWS ModifySubnetAttribute](https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ModifySubnetAttribute.html).
+
+From the repository root, using the existing `demo` collection and ledger:
+
+```sh
+go build -o build/omnicli ./cmd/omnicli
+python3 cicd/examples/aws-vpc-subnet/prepare-update.py \
+  --registry test/corpus/registry --out cicd/out/aws-update-registry --refresh
+
+build/omnicli iac-apply cicd/out/aws-update-registry \
+  @cicd/examples/aws-vpc-subnet/update-public-ip.json \
+  --aws-region ap-southeast-2 --name demo \
+  --state "$PWD/cicd/out/aws-demo/state" --run-id public-ip-001
+```
+
+Run the same apply with `--run-id public-ip-check-002`: the attribute should already match,
+so it must not issue another update. Resource identities must remain the same. Current result
+rows use `status: applied` for successful no-op runs too; that label alone does not prove a mutation.
+
+Verify directly if the AWS CLI is installed, using the subnet ID from your create output:
+
+```sh
+aws ec2 describe-subnets --region ap-southeast-2 \
+  --subnet-ids YOUR_SUBNET_ID --query 'Subnets[0].MapPublicIpOnLaunch' --output text
+```
+
+To reverse the setting, apply `update-private-ip.json` with the same command and a fresh run ID.
+Both versions retain the original VPC/subnet keys and CIDRs. Do not delete the ledger or change
+collection names. These update files require the subnet to exist; they are not initial-create inputs.
+
+The Python preparation command only writes an isolated provider-document overlay. It does not
+call AWS. The overlay adds the update route and SQL-verb mapping missing from the corpus, and
+exposes the read-back boolean under its EC2 Query input name `MapPublicIpOnLaunch.Value` so
+diffing, sending and subsequent read-back agree. The source corpus is left untouched. Prepare
+the overlay once; use a fresh output directory if regenerating it.
+
+
+## Self, input and environment bindings
+
+The files declare ordinary `resource-key` tags. Their values use:
+
+```json
+"TagSpecification.1.Tag.1.Value": {"from":"self", "attribute":"key"}
+```
+
+`write_once` explicitly names the tag key and resource type. No label or collection prefix is
+injected. The provider document supplies ordinary tag create/read/update/delete mappings.
+Removing the owned tag value from `desired` requests tag deletion on reapply.
+
+`bindings.json` also demonstrates top-level caller inputs and an explicit environment reference:
+
+```sh
+AWS_REGION=ap-southeast-2 build/omnicli iac-apply cicd/out/aws-update-registry \
+  @cicd/examples/aws-vpc-subnet/bindings.json \
+  --input '{"vpc_label":"chosen-by-caller"}' \
+  --name demo --state "$PWD/cicd/out/aws-demo/state" --run-id bindings-001
+```
+
+This is a real apply command. Refresh the registry first. Missing inputs or referenced environment
+variables fail before mutation. The [binding contract](../../../docs/developer/resource_bindings.md)
+lists available attributes and API inputs. The old correlation/debug-correlation options are rejected.
+
+## Initial creation and teardown
+
+`resources.json` explicitly declares a VPC and a subnet. The subnet receives the created
+VPC identity through its inbox. Both resources explicitly declare their own tags.
+This is the existing supported network foundation: it does not create public routing,
+an instance, SSH keys, nginx or a load balancer.
+
+From the repository root, supply your registry, region, collection and state directory:
+
+```sh
+go build -o build/omnicli ./cmd/omnicli
+python3 cicd/examples/aws-vpc-subnet/prepare-update.py \
+  --registry test/corpus/registry --out cicd/out/aws-update-registry --refresh
+export REGISTRY="$PWD/cicd/out/aws-update-registry"
+export REGION=YOUR_REGION
+export COLLECTION=YOUR_COLLECTION
+export STATE="$PWD/cicd/out/YOUR_DEPLOYMENT/state"
+
+build/omnicli iac-apply "$REGISTRY" \
+  @cicd/examples/aws-vpc-subnet/resources.json \
+  --aws-region "$REGION" --name "$COLLECTION" --state "$STATE" --run-id create-001
+```
+
+This command creates real resources in the account belonging to your supplied AWS
+credentials. Verify that account before running it. Scope is supplied explicitly;
+credentials use the normal AWS credential configuration. Edit CIDRs before the first apply.
+The provider determines available operations; this example does not promise CIDR replacement.
+Keep the state directory so later runs can address the same deployment.
+
+When ready to remove these two resources, use the same registry, region, collection and state:
+
+```sh
+build/omnicli iac-destroy --registry "$REGISTRY" \
+  --aws-region "$REGION" --name "$COLLECTION" --state "$STATE" --run-id destroy-001
+```
+
+Inspect JSONL results for errors and outstanding resources. Teardown uses the ledger's
+dependency order. No live AWS deployment is performed by creating these example files.

cicd/examples/aws-vpc-subnet/bindings.json

@@ -0,0 +1,65 @@
+{
+  "resources": [
+    {
+      "key": "aws/ec2/vpc",
+      "provider": "aws",
+      "address": "ec2.vpcs",
+      "identity": "line_items.VpcId",
+      "addressed_by": "VpcId",
+      "write_once": {
+        "TagSpecification.1.ResourceType": "vpc",
+        "TagSpecification.1.Tag.1.Key": "resource-key",
+        "region": {
+          "from": "env",
+          "attribute": "AWS_REGION"
+        }
+      },
+      "desired": {
+        "CidrBlock": {
+          "from": "input",
+          "attribute": "vpc_cidr"
+        },
+        "TagSpecification.1.Tag.1.Value": {
+          "from": "input",
+          "attribute": "vpc_label"
+        }
+      }
+    },
+    {
+      "key": "aws/ec2/subnet",
+      "provider": "aws",
+      "address": "ec2.subnets",
+      "identity": "line_items.SubnetId",
+      "addressed_by": "SubnetId",
+      "write_once": {
+        "TagSpecification.1.ResourceType": "subnet",
+        "TagSpecification.1.Tag.1.Key": "resource-key",
+        "region": {
+          "from": "env",
+          "attribute": "AWS_REGION"
+        }
+      },
+      "desired": {
+        "CidrBlock": {
+          "from": "input",
+          "attribute": "subnet_cidr"
+        },
+        "TagSpecification.1.Tag.1.Value": {
+          "from": "self",
+          "attribute": "key"
+        }
+      },
+      "inbound": [
+        {
+          "from": "aws/ec2/vpc",
+          "as": "VpcId"
+        }
+      ]
+    }
+  ],
+  "inputs": {
+    "vpc_cidr": "10.42.0.0/16",
+    "subnet_cidr": "10.42.1.0/24",
+    "vpc_label": "demo-network"
+  }
+}

cicd/examples/aws-vpc-subnet/prepare-update.py

@@ -0,0 +1,169 @@
+#!/usr/bin/env python3
+"""Prepare an isolated provider overlay for the subnet public-IP update example."""
+import argparse
+from pathlib import Path
+import shutil
+import re
+
+
+
+NAME_METHOD = """      updates:
+        - priority: 100
+          requires: [DebugCorrelation]
+          permits: [DebugCorrelation]
+          target: {$ref: '#/components/x-stackQL-resources/vpcs/methods/debug_correlation'}
+        - priority: 100
+          requires: [Name]
+          permits: [Name]
+          target: {$ref: '#/components/x-stackQL-resources/vpcs/methods/set_name'}
+      methods:
+        set_name:
+          operation: {$ref: '#/paths/~1?Action=CreateTags&Version=2016-11-15/post'}
+          request:
+            mediaType: application/x-www-form-urlencoded
+            transform:
+              type: golang_template_text_v0.3.0
+              body: '{{ $d := jsonMapFromString . }}{{ if ne (kindOf (index $d "Name")) "string" }}{{ template "Name must be a string; use a string value to set the tag" . }}{{ end }}ResourceId.1={{ urlquery (index $d "VpcId") }}&Tag.1.Key=Name&Tag.1.Value={{ urlquery (index $d "Name") }}'
+          response: {openAPIDocKey: '200'}
+"""
+
+NAME_OPERATION = """    post:
+      operationId: POST_SetVpcName
+      parameters:
+        - {name: VpcId, in: body, required: true, schema: {type: string}}
+      requestBody:
+        content:
+          application/x-www-form-urlencoded:
+            schema:
+              type: object
+              properties:
+                Name: {type: string}
+      responses:
+        '200': {description: Tag set}
+"""
+
+NAME_READ = '''{{ define "tag" }}{{ if eq (index . "key") "Name" }},"Name":{{ toJson (index . "value") }}{{ else if eq (index . "key") "omnisdk:key" }},"DebugCorrelation":{{ toJson (index . "value") }}{{ else if eq (index . "key") "resource-key" }},"TagSpecification.1.Tag.1.Value":{{ toJson (index . "value") }}{{ end }}{{ end }}
+{{ define "vpc" }}{"VpcId":{{ toJson (index . "vpcId") }},"CidrBlock":{{ toJson (index . "cidrBlock") }},"InstanceTenancy":{{ toJson (index . "instanceTenancy") }}
+{{ $tags := index . "tagSet" "item" }}{{ if eq (kindOf $tags) "slice" }}{{ range $tags }}{{ template "tag" . }}{{ end }}{{ else if eq (kindOf $tags) "map" }}{{ template "tag" $tags }}{{ end }}}
+{{ end }}
+{"line_items":[{{ $rows := index . "DescribeVpcsResponse" "vpcSet" "item" }}{{ if eq (kindOf $rows) "slice" }}{{ $sep := separator "," }}{{ range $rows }}{{ call $sep }}{{ template "vpc" . }}{{ end }}{{ else if eq (kindOf $rows) "map" }}{{ template "vpc" $rows }}{{ end }}]}'''
+
+
+SUBNET_READ = NAME_READ.replace('DescribeVpcsResponse', 'DescribeSubnetsResponse').replace('vpcSet', 'subnetSet').replace('"VpcId"', '"SubnetId"').replace('"vpcId"', '"subnetId"').replace('"InstanceTenancy":{{ toJson (index . "instanceTenancy") }}', '"MapPublicIpOnLaunch.Value":{{ toBool (index . "mapPublicIpOnLaunch") }}')
+DEBUG_OPERATION = """  /?Version=2016-11-15:
+    post:
+      operationId: POST_DebugCorrelation
+      parameters:
+        - {name: VpcId, in: body, required: false, schema: {type: string}}
+        - {name: SubnetId, in: body, required: false, schema: {type: string}}
+      requestBody:
+        content:
+          application/x-www-form-urlencoded:
+            schema:
+              type: object
+              properties:
+                DebugCorrelation: {type: string, nullable: true}
+      responses:
+        '200': {description: Debug label reconciled}
+"""
+
+def add_debug_method(resource, name, identity):
+    program = '{{ $d := jsonMapFromString . }}{{ if index $d "DebugCorrelation" }}Action=CreateTags{{ else }}Action=DeleteTags{{ end }}&ResourceId.1={{ urlquery (index $d "' + identity + '") }}&Tag.1.Key=omnisdk%3Akey{{ if index $d "DebugCorrelation" }}&Tag.1.Value={{ urlquery (index $d "DebugCorrelation") }}{{ end }}'
+    method = "        debug_correlation:\n          operation: {$ref: '#/paths/~1?Version=2016-11-15/post'}\n          request:\n            mediaType: application/x-www-form-urlencoded\n            transform:\n              type: golang_template_text_v0.3.0\n              body: '" + program + "'\n          response: {openAPIDocKey: '200'}\n"
+    resource = resource.replace('      methods:\n', '      methods:\n' + method, 1)
+    return resource.replace('        update:\n', "        update:\n          - $ref: '#/components/x-stackQL-resources/" + name + "/methods/debug_correlation'\n", 1)
+
+
+
+def add_resource_tag_method(resource, name, identity):
+    attribute = 'TagSpecification.1.Tag.1.Value'
+    program = '{{ $d := jsonMapFromString . }}{{ if ne (index $d "' + attribute + '") nil }}Action=CreateTags{{ else }}Action=DeleteTags{{ end }}&ResourceId.1={{ urlquery (index $d "' + identity + '") }}&Tag.1.Key=resource-key{{ if ne (index $d "' + attribute + '") nil }}&Tag.1.Value={{ urlquery (index $d "' + attribute + '") }}{{ end }}'
+    method = "        resource_key_tag:\n          operation: {$ref: '#/paths/~1?Version=2016-11-15/post'}\n          request:\n            mediaType: application/x-www-form-urlencoded\n            transform:\n              type: golang_template_text_v0.3.0\n              body: '" + program + "'\n          response: {openAPIDocKey: '200'}\n"
+    route = "      updates:\n        - priority: 100\n          requires: [TagSpecification.1.Tag.1.Value]\n          permits: [TagSpecification.1.Tag.1.Value]\n          target: {$ref: '#/components/x-stackQL-resources/" + name + "/methods/resource_key_tag'}\n"
+    resource = resource.replace('      updates:\n', route, 1)
+    resource = resource.replace('      methods:\n', '      methods:\n' + method, 1)
+    return resource.replace('        update:\n', "        update:\n          - $ref: '#/components/x-stackQL-resources/" + name + "/methods/resource_key_tag'\n", 1)
+
+
+def prepare(registry, output, refresh=False):
+    source = Path(registry) / 'aws/v00.00.00000'
+    target = Path(output) / 'aws/v00.00.00000'
+    document = (source / 'services/ec2.yaml').read_text()
+    start = document.index('    subnets:\n')
+    end = document.index('\n    transit_gateway_multicast_domains:', start)
+    resource = document[start:end]
+    resource = resource.replace('      methods:\n', '''      updates:
+        - priority: 100
+          requires: [DebugCorrelation]
+          permits: [DebugCorrelation]
+          target: {$ref: '#/components/x-stackQL-resources/subnets/methods/debug_correlation'}
+        - priority: 100
+          requires: [MapPublicIpOnLaunch.Value]
+          permits: [MapPublicIpOnLaunch.Value]
+          target: {$ref: '#/components/x-stackQL-resources/subnets/methods/modify_subnet_attribute'}
+      methods:
+''', 1)
+    anchor = '        update:\n'
+    resource = resource.replace(anchor, anchor + "          - $ref: '#/components/x-stackQL-resources/subnets/methods/modify_subnet_attribute'\n", 1)
+    resource = add_debug_method(resource, 'subnets', 'SubnetId')
+    resource = add_resource_tag_method(resource, 'subnets', 'SubnetId')
+    document = document[:start] + resource + document[end:]
+    # Expose the observed boolean under the same attribute name as the EC2 Query input.
+    start = document.index('    SubnetDisplay:\n')
+    end = start + re.search(r'\n    \S', document[start + 5:]).start() + 5
+    schema = document[start:end]
+    old = '        MapPublicIpOnLaunch:\n'
+    if schema.count(old) != 1:
+        raise ValueError('SubnetDisplay public-IP field differs from the supported corpus')
+    schema = schema.replace(old, '        TagSpecification.1.Tag.1.Value: {type: string}\n        DebugCorrelation: {type: string}\n        MapPublicIpOnLaunch.Value:\n', 1)
+    document = document[:start] + schema + document[end:]
+    start = document.index('    subnets:\n')
+    end = start + re.search(r'\n    \S', document[start + 5:]).start() + 5
+    resource = document[start:end]
+    begin = resource.index('        describe_subnets:\n')
+    finish = begin + re.search(r'\n        \S', resource[begin + 9:]).start() + 9
+    read = resource[begin:finish]
+    read = read.replace('type: schema_driven_xml_v0.1.0',
+        'type: golang_template_mxj_v0.2.0\n              body: |\n' +
+        '\n'.join('                ' + line for line in SUBNET_READ.splitlines()))
+    resource = resource[:begin] + read + resource[finish:]
+    document = document[:start] + resource + document[end:]
+    # Name is intent, not a create-only TagSpecification parameter. Existing template DSLs
+    # project the observed Name tag and shape CreateTags without AWS logic in the SDK.
+    start = document.index('    vpcs:\n')
+    end = start + re.search(r'\n    \S', document[start + 5:]).start() + 5
+    resource = document[start:end]
+    resource = resource.replace('      methods:\n', NAME_METHOD, 1)
+    resource = add_debug_method(resource, 'vpcs', 'VpcId')
+    resource = add_resource_tag_method(resource, 'vpcs', 'VpcId')
+    read_start = resource.index('        describe_vpcs:\n')
+    read_end = read_start + re.search(r'\n        \S', resource[read_start + 9:]).start() + 9
+    read = resource[read_start:read_end]
+    read = read.replace('type: schema_driven_xml_v0.1.0',
+                        'type: golang_template_mxj_v0.2.0\n              body: |\n' +
+                        '\n'.join('                ' + line for line in NAME_READ.splitlines()))
+    resource = resource[:read_start] + read + resource[read_end:]
+    resource = resource.replace('        update:\n',
+        "        update:\n          - $ref: '#/components/x-stackQL-resources/vpcs/methods/set_name'\n", 1)
+    document = document[:start] + resource + document[end:]
+    document = document.replace('    VpcDisplay:\n      type: object\n      properties:\n',
+        '    VpcDisplay:\n      type: object\n      properties:\n        Name: {type: string}\n        TagSpecification.1.Tag.1.Value: {type: string}\n        DebugCorrelation: {type: string}\n', 1)
+    start = document.index('  /?Action=CreateTags&Version=2016-11-15:\n')
+    post = document.index('    post:\n', start)
+    end = document.index('\n  /', post)
+    document = document[:post] + NAME_OPERATION + document[end:]
+    document = document.replace('paths:\n', 'paths:\n' + DEBUG_OPERATION, 1)
+    target.mkdir(parents=True, exist_ok=refresh)
+    (target / 'services').mkdir(exist_ok=refresh)
+    shutil.copyfile(source / 'provider.yaml', target / 'provider.yaml')
+    (target / 'services/ec2.yaml').write_text(document)
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument('--registry', required=True)
+    parser.add_argument('--out', required=True, help='fresh isolated registry directory')
+    parser.add_argument('--refresh', action='store_true', help='regenerate the existing example overlay from its source')
+    options = parser.parse_args()
+    prepare(options.registry, options.out, options.refresh)
+    print(options.out)