fix: fix deployment manifest digest mismatch - #68
Conversation
Signed-off-by: sanjujunnuthula <sanjuvardhan.junnuthula@capgemini.com>
| deviceVendorLogger.ErrorfCtx(pCtx, | ||
| "Digest mismatch for deployment %s: requested=%s, actual=%s", | ||
| deploymentId, requestedDigest, actualDigest) | ||
| "Digest mismatch for deployment %s: requested=%s, actual=%s\nYAML content being hashed:\n%s", |
There was a problem hiding this comment.
@sanjujunnuthula no need to print yaml content
There was a problem hiding this comment.
ok, removing
| deploymentId, actualDigest, len(yamlContent)) | ||
|
|
||
| // Return with proper headers | ||
| return createSuccessResponseWithHeaders(span, |
There was a problem hiding this comment.
can you check any other method we have instead of COAResponse
There was a problem hiding this comment.
ok will check on this, but COA is basically the common request/response used by Symphony components to communicate with each other.
Signed-off-by: sanjujunnuthula <sanjuvardhan.junnuthula@capgemini.com>
| margoStdSbiAPI "github.com/margo/sandbox/standard/generatedCode/wfm/sbi" | ||
| "github.com/valyala/fasthttp" | ||
| "gopkg.in/yaml.v2" | ||
| yaml "sigs.k8s.io/yaml" |
There was a problem hiding this comment.
@sanjujunnuthula can you check what is happening at write time, downloadDeployment is read time.
The digest mismatch happens because the digest in the manifest URL is computed at write time (when the deployment is stored/bundle is built), but downloadDeployment computes it at read time by re-marshaling. Even with sigs.k8s.io/yaml, if the write-time digest was computed with a different library or different struct state, they won't match.
There was a problem hiding this comment.
From the above screenshot i had logged the hash for both libraries.
gopkg.in/yaml.v2 produces 344 bytes with the wrong hash.
sigs.k8s.io/yaml produces 503 bytes with the exact hash expected by the device.
sbi.AppDeploymentManifest (generated from snapshot.spec.yaml) only contains json struct tags (json:"applicationInstanceId,omitempty"), with no yaml tags.
gopkg.in/yaml.v2 ignores json tags and lowercases the raw Go field names, whereas sigs.k8s.io/yaml roundtrips through JSON rules first, preserving correct field casing, nested OpenAPI schemas, and omitempty rules.
At Write-Time (Manifest Snapshot Generation):
When Symphony generates the manifest list (deployments[].digest), it serializes the deployment manifest using JSON-compatible schema rules. The resulting SHA-256 digest is published to the Device Agent.
At Read-Time (downloadDeployment):
When the device sends GET deployments/{deploymentId}/{digest} , the server retrieves the deployment from the database and serializes it to YAML bytes.
When using gopkg.in/yaml.v2, the read-time serialization dropped all JSON tags, producing a completely different byte stream and a mismatched hash.
When using sigs.k8s.io/yaml, read-time serialization follows the exact same JSON schema semantics as write-time, producing the exact matching 503-byte payload with hash sha256:8cdaedf....

There was a problem hiding this comment.
@sanjujunnuthula Can you check if we can add yaml tags in struct and test once
| Area | Status | Issue |
|---|---|---|
| YAML Library | ✅ Fixed | Changed to sigs.k8s.io/yaml for JSON-compatible serialization |
| Response Builder | Direct COAResponse used; alternatives not explored/tested | |
| Struct Tags | ❌ OPEN | No YAML tags added to sbi.AppDeploymentManifest struct |
| Write-Time Consistency | ❌ OPEN | No verification that write-time and read-time serialization match |
| Testing | ❌ MISSING | No test showing digest consistency across write/read cycles |
The PR fixes the immediate read-time problem but leaves the structural guarantee incomplete. The last action item (adding YAML tags and testing) appears to be pending.
updated gopkg.in/yaml.v2 with sigs.k8s.io/yaml , Standard gopkg.in/yaml.v2 ignores json tags and lowercases Go field names, causing structural discrepancies. sigs.k8s.io/yaml serializes through JSON rules first, preserving correct field casing and omitempty directives so the manifest digest remains completely consistent.
Replaced createSuccessResponseWithHeaders with Direct v1alpha2.COAResponse, since ResponseBuilder.WithData(...) is designed for Go Data Objects: WithData expects structured Go objects (e.g., structs, maps). During.Build(), it automatically applies JSON serialization / envelope formattingto whatever is passed into it. Because yaml Content was already a pre-marshaled []byte slice containing 503 raw YAML bytes, passing &yamlContent into WithData(*data) caused ResponseBuilder to treat the raw bytes as an object to be JSON-marshaled/enveloped, turning the payload into 2,791 bytes.