From 13d1e19ca5f9ec7230c147490a0a4a52f77335cf Mon Sep 17 00:00:00 2001 From: chen21019 Date: Mon, 2 Nov 2026 00:41:57 +0800 Subject: [PATCH] Emit business-only Catalog request DTOs with explicit empty service links --- .github/workflows/security-release-gate.yml | 6 +- COMPATIBILITY.md | 15 ++ README.md | 8 + cataloglaunch/prepare.go | 41 ++++-- cataloglaunch/wire.go | 109 ++++++++++++++ cataloglaunch/wire_test.go | 155 ++++++++++++++++++++ docs/catalog-launch-owner.md | 4 +- docs/releases/compose-executor-0.14.41.md | 26 ++++ 8 files changed, 346 insertions(+), 18 deletions(-) create mode 100644 cataloglaunch/wire.go create mode 100644 cataloglaunch/wire_test.go create mode 100644 docs/releases/compose-executor-0.14.41.md diff --git a/.github/workflows/security-release-gate.yml b/.github/workflows/security-release-gate.yml index 019ba62e..4ff15e65 100644 --- a/.github/workflows/security-release-gate.yml +++ b/.github/workflows/security-release-gate.yml @@ -21,7 +21,7 @@ jobs: env: DAPPER_IMAGE: pasturestack/compose-cli-dapper:${{ github.sha }} TRIVY_IMAGE: aquasec/trivy:0.74.0@sha256:62b1e65e8869bc4b4c6aa4fa2b21595256c7c2f6018a9d9ad61caf87187c1969 - VERSION_OVERRIDE: 0.14.40 + VERSION_OVERRIDE: 0.14.41 PLATFORM_COMPAT_JAR_URL: https://github.com/PastureStack/orchestration-engine/releases/download/v0.183.281/orchestration-engine-0.183.281.jar PLATFORM_COMPAT_JAR_SHA256: da2a8a51562ed16e296f7e29e99482bb44042ff0834cca679bbe01d951ba1682 @@ -69,7 +69,7 @@ jobs: } run_ci - artifact="dist/artifacts/compose-executor-0.14.40-linux-amd64.gz" + artifact="dist/artifacts/compose-executor-0.14.41-linux-amd64.gz" test -s "$artifact" cp "$artifact" /tmp/compose-executor-first.gz @@ -81,7 +81,7 @@ jobs: mkdir -p evidence/product gzip -cd "$artifact" > evidence/product/compose-executor chmod +x evidence/product/compose-executor - evidence/product/compose-executor --version | grep -F '0.14.40' >/dev/null + evidence/product/compose-executor --version | grep -F '0.14.41' >/dev/null cp "$artifact" evidence/ ( cd evidence diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index 85d398e2..cac98973 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -1,5 +1,20 @@ # Compatibility Contract +## 0.14.41 source candidate: private Catalog request wire + +The private Catalog preview producer emits business-only mutation inputs. +`setServiceLinks` always includes an array, including `[]` for no links, and +retains native target IDs and aliases. Service/Volume creation skeletons and +finalized child request structs exclude the generic SDK Resource envelope, +including nested launch configs. Business maps such as metadata, labels, +environment and log-driver options retain their exact keys and values. + +This changes only the private preview serialization. Format version `1`, +normal parser/DTO owners, ordinary CLI/SDK serialization and Engine closed +validation remain unchanged. Qualification, publication and matching Server +runtime acceptance are pending. See the +[candidate note](docs/releases/compose-executor-0.14.41.md). + The migration preserves Docker Compose parsing, stack event names, generated stack fields, service labels, catalog question data, upgrade and rollback payloads, and compatibility API behavior. The published v0.14.40 adds one server-private mode, diff --git a/README.md b/README.md index 39455efa..83a32820 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,14 @@ PastureStack is an independent community effort to preserve, audit, and moderniz The maintained source uses Ubuntu 26.04, Go 1.27.3, and Python 3.14.9 for integration-source verification. Ubuntu build packages, the base image, and Go archives are fixed by snapshot, exact version, and SHA-256. Go dependencies are managed by `go.mod` and a reproducible module vendor tree. AWS access uses AWS SDK for Go v2; container structures use the maintained Moby API module; the command parser uses urfave/cli v3. The abandoned AWS SDK v1, Rancher generated-client modules, event-subscriber module, `docker/libcompose`, and other GOPATH-era dependencies have been removed. The small Rancher 1.6 protocol boundary retained by this compatibility CLI is maintained and tested locally. The Dapper builder does not embed a Docker CLI or mount a Docker socket; Docker is required only on the host to create and run the isolated builder. The maintained binaries are `pasturestack-compose` and `compose-executor`; `platform-compose.yml` is the preferred companion file. Releases are built and verified from the preserved source history; this repository does not enable automated deployment. +The [0.14.41 source candidate](docs/releases/compose-executor-0.14.41.md) fixes +the private Catalog launch wire format: mutation DTOs contain business fields, +and an empty link action explicitly carries `serviceLinks: []`. Native link +targets and aliases are preserved. Generic API resource links/actions are +excluded from these request structs; ordinary CLI and SDK requests retain +their behavior. Candidate qualification, publication and matching Server +runtime acceptance remain pending. + The current published component is [v0.14.40](https://github.com/PastureStack/compose-cli/releases/tag/v0.14.40), from producer `1835f8db645a8488e327b1f6e750bfdf3d1fb84d`. Its original CI archive diff --git a/cataloglaunch/prepare.go b/cataloglaunch/prepare.go index 17f3f17c..b0bde10e 100644 --- a/cataloglaunch/prepare.go +++ b/cataloglaunch/prepare.go @@ -52,18 +52,18 @@ type Inspect struct { } type Service struct { - Name string `json:"name"` - PrimaryName string `json:"primaryName,omitempty"` - ResourceType string `json:"resourceType,omitempty"` - CreateSkeleton *platformapi.CompositeService `json:"createSkeleton,omitempty"` - Config *config.ServiceConfig `json:"config"` - Inspect Inspect `json:"dockerInspect"` + Name string `json:"name"` + PrimaryName string `json:"primaryName,omitempty"` + ResourceType string `json:"resourceType,omitempty"` + CreateSkeleton map[string]interface{} `json:"createSkeleton,omitempty"` + Config *config.ServiceConfig `json:"config"` + Inspect Inspect `json:"dockerInspect"` } type VolumeRequest struct { Name string `json:"name"` ResourceType string `json:"resourceType"` - CreateSkeleton *client.VolumeTemplate `json:"createSkeleton"` + CreateSkeleton map[string]interface{} `json:"createSkeleton"` } type Output struct { @@ -83,10 +83,10 @@ type Output struct { } type ServiceRequest struct { - Name string `json:"name"` - ResourceType string `json:"resourceType"` - CreateBody interface{} `json:"createBody"` - SetServiceLinks *client.SetServiceLinksInput `json:"setServiceLinks"` + Name string `json:"name"` + ResourceType string `json:"resourceType"` + CreateBody map[string]interface{} `json:"createBody"` + SetServiceLinks *serviceLinksWire `json:"setServiceLinks"` } type catalogEnvironment struct { @@ -221,7 +221,11 @@ func Prepare(input Input) (*Output, error) { if !namePattern.MatchString(name) { return nil, ErrUnsupported } - result.VolumeRequests = append(result.VolumeRequests, VolumeRequest{Name: name, ResourceType: "volumeTemplate", CreateSkeleton: platformapi.CatalogVolumeCreateSkeleton(name, p.VolumeConfigs[name])}) + skeleton, err := catalogRequestWire(platformapi.CatalogVolumeCreateSkeleton(name, p.VolumeConfigs[name])) + if err != nil { + return nil, err + } + result.VolumeRequests = append(result.VolumeRequests, VolumeRequest{Name: name, ResourceType: "volumeTemplate", CreateSkeleton: skeleton}) } for _, name := range names { if !namePattern.MatchString(name) { @@ -275,7 +279,12 @@ func Prepare(input Input) (*Output, error) { h.NetworkMode = "" entry := Service{Name: name, PrimaryName: primaries[name], Config: canonical, Inspect: Inspect{Name: "/" + name, Config: c, HostConfig: h}} if entry.PrimaryName == "" { - entry.ResourceType, entry.CreateSkeleton = platformapi.CatalogServiceCreateSkeleton(name, ©) + kind, skeleton := platformapi.CatalogServiceCreateSkeleton(name, ©) + wire, err := catalogRequestWire(skeleton) + if err != nil { + return nil, err + } + entry.ResourceType, entry.CreateSkeleton = kind, wire } result.Services = append(result.Services, entry) } @@ -326,7 +335,11 @@ func Prepare(input Input) (*Output, error) { if kind == "externalService" { links = nil } - result.ServiceRequests = append(result.ServiceRequests, ServiceRequest{Name: entry.Name, ResourceType: kind, CreateBody: platformapi.CatalogServiceRequestPayload(kind, body), SetServiceLinks: links}) + wire, err := catalogRequestWire(platformapi.CatalogServiceRequestPayload(kind, body)) + if err != nil { + return nil, err + } + result.ServiceRequests = append(result.ServiceRequests, ServiceRequest{Name: entry.Name, ResourceType: kind, CreateBody: wire, SetServiceLinks: catalogLinksWire(links)}) } result.Phase = "finalize" } diff --git a/cataloglaunch/wire.go b/cataloglaunch/wire.go new file mode 100644 index 00000000..7fbdf79d --- /dev/null +++ b/cataloglaunch/wire.go @@ -0,0 +1,109 @@ +package cataloglaunch + +import ( + "bytes" + "encoding/json" + "reflect" + "strings" + + client "github.com/PastureStack/compose-cli/internal/rancherclient/v2" +) + +// These are mutation inputs, not generic API resources. Empty links remain an +// explicit business value, and link aliases/targets retain the native parser's values. +type serviceLinkWire struct { + Name string `json:"name,omitempty"` + ServiceID string `json:"serviceId"` +} + +type serviceLinksWire struct { + ServiceLinks []serviceLinkWire `json:"serviceLinks"` +} + +func catalogLinksWire(input *client.SetServiceLinksInput) *serviceLinksWire { + if input == nil { + return nil + } + result := &serviceLinksWire{ServiceLinks: make([]serviceLinkWire, 0, len(input.ServiceLinks))} + for _, link := range input.ServiceLinks { + result.ServiceLinks = append(result.ServiceLinks, serviceLinkWire{Name: link.Name, ServiceID: link.ServiceId}) + } + return result +} + +var apiResourceType = reflect.TypeOf(client.Resource{}) + +// Only this private preview producer projects SDK request structs. Preserve the +// native JSON encoding, then omit the SDK's typed Resource envelope throughout +// those structs. Maps are business data and are never traversed or key-stripped. +// Ordinary SDK/legacy requests keep their original serialization. +func catalogRequestWire(value interface{}) (map[string]interface{}, error) { + raw, err := json.Marshal(value) + if err != nil { + return nil, ErrUnsupported + } + var result map[string]interface{} + decoder := json.NewDecoder(bytes.NewReader(raw)) + decoder.UseNumber() + if decoder.Decode(&result) != nil || result == nil { + return nil, ErrUnsupported + } + projectRequestStruct(reflect.ValueOf(value), result) + return result, nil +} + +func projectRequestStruct(value reflect.Value, encoded interface{}) { + for value.IsValid() && (value.Kind() == reflect.Ptr || value.Kind() == reflect.Interface) { + if value.IsNil() { + return + } + value = value.Elem() + } + if !value.IsValid() { + return + } + switch value.Kind() { + case reflect.Struct: + object, ok := encoded.(map[string]interface{}) + if !ok { + return + } + if value.Type() == apiResourceType { + // A named Resource property is data, not a promoted response envelope. + return + } + for index := 0; index < value.NumField(); index++ { + field := value.Type().Field(index) + name := strings.Split(field.Tag.Get("json"), ",")[0] + if field.PkgPath != "" || name == "-" { + continue + } + if field.Anonymous && name == "" { + embedded := field.Type + if embedded.Kind() == reflect.Ptr { + embedded = embedded.Elem() + } + if embedded == apiResourceType { + for resourceIndex := 0; resourceIndex < apiResourceType.NumField(); resourceIndex++ { + key := strings.Split(apiResourceType.Field(resourceIndex).Tag.Get("json"), ",")[0] + delete(object, key) + } + } else { + projectRequestStruct(value.Field(index), object) + } + } else { + if name == "" { + name = field.Name + } + projectRequestStruct(value.Field(index), object[name]) + } + } + case reflect.Array, reflect.Slice: + items, ok := encoded.([]interface{}) + if ok { + for index := 0; index < value.Len() && index < len(items); index++ { + projectRequestStruct(value.Index(index), items[index]) + } + } + } +} diff --git a/cataloglaunch/wire_test.go b/cataloglaunch/wire_test.go new file mode 100644 index 00000000..cd338eaf --- /dev/null +++ b/cataloglaunch/wire_test.go @@ -0,0 +1,155 @@ +package cataloglaunch + +import ( + "bytes" + "encoding/json" + "reflect" + "testing" + + client "github.com/PastureStack/compose-cli/internal/rancherclient/v2" + "github.com/PastureStack/compose-cli/platformapi" +) + +func wireObject(t *testing.T, value interface{}) map[string]interface{} { + t.Helper() + raw, err := json.Marshal(value) + if err != nil { + t.Fatal(err) + } + var object map[string]interface{} + if err := json.Unmarshal(raw, &object); err != nil { + t.Fatal(err) + } + return object +} + +func noResourceEnvelope(t *testing.T, object map[string]interface{}) { + t.Helper() + for _, field := range []string{"id", "type", "links", "actions"} { + if _, found := object[field]; found { + t.Fatalf("generic Resource envelope field %s entered a mutation DTO", field) + } + } +} + +func TestCatalogWireEmptyAndNonemptyServiceLinks(t *testing.T) { + for _, raw := range [][]string{nil, {"worker:read", "worker:write"}} { + input, err := platformapi.CatalogServiceLinkSkeleton(raw, map[string]string{"worker": ""}) + if err != nil { + t.Fatal(err) + } + legacyBefore, _ := json.Marshal(input) + object := wireObject(t, catalogLinksWire(input)) + if len(object) != 1 { + t.Fatal("link action must contain only serviceLinks") + } + links, ok := object["serviceLinks"].([]interface{}) + if !ok || len(links) != len(raw) { + t.Fatal("empty links must be [] and selected links must retain their count") + } + for index, value := range links { + link := value.(map[string]interface{}) + noResourceEnvelope(t, link) + if len(link) != 2 || link["serviceId"] != input.ServiceLinks[index].ServiceId || link["name"] != input.ServiceLinks[index].Name { + t.Fatal("native link target or alias changed") + } + } + legacyAfter, _ := json.Marshal(input) + if !bytes.Equal(legacyBefore, legacyAfter) || !bytes.Contains(legacyAfter, []byte(`"links":null`)) || !bytes.Contains(legacyAfter, []byte(`"actions":null`)) { + t.Fatal("ordinary client DTO serialization changed") + } + } +} + +func TestCatalogWirePreservesBusinessMapsAndNestedRequests(t *testing.T) { + metadata := map[string]interface{}{"id": "business-id", "type": "business-type", "links": map[string]interface{}{"target": "keep"}, "actions": []interface{}{"keep"}} + header := client.Resource{Id: "server-id", Type: "server-type", Links: map[string]string{"self": "server-link"}, Actions: map[string]string{"update": "server-action"}} + input := &client.Service{Resource: header, Name: "app", Metadata: metadata, + LaunchConfig: &client.LaunchConfig{Resource: header, ImageUuid: "docker:busybox:1.37.0", Labels: metadata, + Environment: map[string]interface{}{"links": "env-value", "actions": "env-action"}, + LogConfig: &client.LogConfig{Resource: header, Driver: "json-file", Config: metadata}}, + SecondaryLaunchConfigs: []client.SecondaryLaunchConfig{{Resource: header, Name: "side", Labels: metadata}}} + before, _ := json.Marshal(input) + wire, err := catalogRequestWire(input) + if err != nil { + t.Fatal(err) + } + object := wireObject(t, wire) + noResourceEnvelope(t, object) + launch := object["launchConfig"].(map[string]interface{}) + noResourceEnvelope(t, launch) + logConfig := launch["logConfig"].(map[string]interface{}) + noResourceEnvelope(t, logConfig) + noResourceEnvelope(t, object["secondaryLaunchConfigs"].([]interface{})[0].(map[string]interface{})) + if !reflect.DeepEqual(object["metadata"], metadata) || !reflect.DeepEqual(launch["labels"], metadata) || !reflect.DeepEqual(logConfig["config"], metadata) { + t.Fatal("business maps were stripped or changed") + } + environment := launch["environment"].(map[string]interface{}) + if environment["links"] != "env-value" || environment["actions"] != "env-action" { + t.Fatal("environment keys were stripped") + } + after, _ := json.Marshal(input) + if !bytes.Equal(before, after) { + t.Fatal("projection mutated the ordinary SDK object") + } + volume, err := catalogRequestWire(&client.VolumeTemplate{Resource: header, Name: "data", StackId: "$createdStack", DriverOpts: metadata}) + if err != nil { + t.Fatal(err) + } + noResourceEnvelope(t, volume) + if !reflect.DeepEqual(volume["driverOpts"], metadata) { + t.Fatal("volume business options changed") + } + named := struct { + client.Resource + Business client.Resource `json:"business"` + }{Resource: header, Business: header} + namedWire, err := catalogRequestWire(named) + if err != nil { + t.Fatal(err) + } + noResourceEnvelope(t, namedWire) + if !reflect.DeepEqual(namedWire["business"], wireObject(t, header)) { + t.Fatal("a named Resource business property was treated as an embedded envelope") + } +} + +func TestCatalogWireFinalizeMarshaledProtocol(t *testing.T) { + input := exampleInput() + input.Phase = "finalize" + input.DockerCompose = "version: '2'\nservices:\n app:\n image: busybox:1.37.0\n links:\n - worker:read\n volumes:\n - proof:/proof\n worker:\n image: busybox:1.37.0\nvolumes:\n proof: {}\n" + input.TransformedLaunchConfigs = map[string]json.RawMessage{ + "app": json.RawMessage(`{"imageUuid":"docker:busybox:1.37.0","logConfig":{"config":{"links":"business-link","actions":"business-action"}}}`), + "worker": json.RawMessage(`{"imageUuid":"docker:busybox:1.37.0"}`), + } + output, err := Prepare(input) + if err != nil { + t.Fatal(err) + } + wire := wireObject(t, output) + for _, value := range wire["services"].([]interface{}) { + noResourceEnvelope(t, value.(map[string]interface{})["createSkeleton"].(map[string]interface{})) + } + for _, value := range wire["volumeRequests"].([]interface{}) { + noResourceEnvelope(t, value.(map[string]interface{})["createSkeleton"].(map[string]interface{})) + } + for _, value := range wire["serviceRequests"].([]interface{}) { + request := value.(map[string]interface{}) + body := request["createBody"].(map[string]interface{}) + noResourceEnvelope(t, body) + launch := body["launchConfig"].(map[string]interface{}) + noResourceEnvelope(t, launch) + noResourceEnvelope(t, launch["logConfig"].(map[string]interface{})) + links := request["setServiceLinks"].(map[string]interface{}) + if len(links) != 1 { + t.Fatal("link action schema is not closed") + } + items, ok := links["serviceLinks"].([]interface{}) + if !ok { + t.Fatal("empty serviceLinks was omitted or became null") + } + for _, item := range items { + noResourceEnvelope(t, item.(map[string]interface{})) + } + } +} diff --git a/docs/catalog-launch-owner.md b/docs/catalog-launch-owner.md index 16fab3a5..57b5a0ae 100644 --- a/docs/catalog-launch-owner.md +++ b/docs/catalog-launch-owner.md @@ -10,7 +10,9 @@ Engine 固定呼叫 `/usr/bin/compose-executor.real --catalog-launch-preview`， 不在 Auth 重寫 YAML 或建立規則：使用既有 `project.Parse/config.Merge/convert.Convert`。Go template 採 bounded、無 host environment／DNS／random／檔案功能的子集，執行錯誤不吞掉；render 後加 `# notemplating` 固定核准內容。context name/version 取原 owner 的 normalization／Catalog config。finalize 與普通 `NormalFactory` 共用 service/volume skeleton、launch config 補值、secret resolver、sidekick、ports 與 service hash。 -finalize 的 `serviceRequests[].createBody` 是正常 POST 的精確 DTO；`volumeRequests[].createSkeleton` 沿正常 VolumeTemplate owner。`$createdStack`／`$service:` 只容許 Engine 綁本計畫實際建立的 ID，不能接受模型 ID。`setServiceLinks` 沿同一 parser／DTO，只支援此 graph 內部服務；順序視為集合。正常 owner 會跑兩次 Create 以完成循環 links，因此可先送已存在 targets 的 subset，再送完整 intended links；完成判定仍需完整關聯收斂。 +finalize 的 `serviceRequests[].createBody` 沿正常 POST owner 的 business fields；`volumeRequests[].createSkeleton` 沿正常 VolumeTemplate owner。0.14.41 source candidate 的專用 wire projection 只在這個 private preview 入口排除 typed SDK `Resource` envelope，涵蓋 Service／Volume skeleton 與巢狀 launch config。`metadata`／`labels`／`environment`／log-driver options 等 business map 不遍歷、不刪鍵；一般 SDK／legacy API 序列化維持原樣。 + +`$createdStack`／`$service:` 只容許 Engine 綁本計畫實際建立的 ID，不能接受模型 ID。`setServiceLinks` 沿同一 native parser／排序，使用專用 `{serviceLinks:[{name,serviceId}]}` wire DTO；空集合明確送 `[]`，保留 target 與 alias，不夾帶通用 API `links/actions`。此階段只支援 graph 內部服務；順序視為集合。正常 owner 會跑兩次 Create 以完成循環 links，因此可先送已存在 targets 的 subset，再送完整 intended links；完成判定仍需完整關聯收斂。 此 candidate 明確拒絕 KVM、provider/driver、host/dependency、build、外部檔案、ignored extends/secret 形狀、外部或跨 Stack／region links、動態 service selectors、未支援的 network graph。這些是尚待 typed adapter 的功能，不能因已有工具就標成完整 Catalog UI 通過。 diff --git a/docs/releases/compose-executor-0.14.41.md b/docs/releases/compose-executor-0.14.41.md new file mode 100644 index 00000000..84add32c --- /dev/null +++ b/docs/releases/compose-executor-0.14.41.md @@ -0,0 +1,26 @@ +# Compose Executor v0.14.41 source candidate + +Status: unpublished source candidate. The latest published component remains +[v0.14.40](https://github.com/PastureStack/compose-cli/releases/tag/v0.14.40). + +The private Catalog preview producer previously serialized generic SDK Resource +envelopes into mutation inputs. Empty link lists were also omitted. A strict +Engine rejected the resulting link action before native child preflight. + +The producer now uses a dedicated business-only link DTO with explicit +`serviceLinks: []` for no links, retaining native target IDs and aliases. +Service/Volume skeletons and finalized child request structs exclude the typed +Resource envelope, including nested launch configs. Metadata, labels, +environment and log-driver option maps retain their keys and values; this is +not a recursive deletion of similarly named business fields. + +Normal parser, conversion, service hash, secret-reference and sidekick owners +are unchanged. Ordinary SDK/legacy serialization and Engine closed validation +are unchanged. The private protocol retains format version `1`. + +The release workflow version, artifact name and binary-version expectation +are synchronized to 0.14.41. No module, vendor, toolchain or compatibility JAR +pin changes. Official producer CI, race/security/CodeQL gates, original artifact +qualification, publication and matching Server runtime acceptance remain +required. Offline component checks do not establish native launch completion +or full Catalog/role/weak-model acceptance.