[feature/caching-rework] Centralize build planning

[lbussell]

This PR centralizes build planning and policies across multiple separate commands. The design is not perfect as-is but I would like to work on improving it incrementally.

Design

The design centers around three new types: BuildGraph, BuildPlanner, and IBuildPolicy.

A BuildGraph represents the relationships between the images in the manifest.
The BuildPlanner takes the BuildGraph in combination with an IBuildPolicy and uses it to determine what actions need to be taken.

classDiagram
direction LR

class BuildCommand {
}
class GenerateBuildMatrixCommand {
}
class GetStaleImagesCommand {
}
class BuildTarget {
}
class BuildGraph {
  +Targets
  +Parents
  +Children
  +SharedBuildTargets
}
class BuildPlanner {
  +CreatePlanAsync(graph, imageInfo, policy)
}
class IBuildPolicy {
}
class BuildPlanItem {
  +Target
  +Action
  +Reasons
  +PublishedImage
}
class BuildAction {
  NoAction
  UsePublishedImage
  PublishExistingImage
  BuildImage
}
class BuildReason {
  +Message
  +Cause
}
class PublishedImage {
  +Source
  +Image
  +SharedTags
}

BuildCommand --> BuildGraph : creates
GenerateBuildMatrixCommand --> BuildGraph : creates
GetStaleImagesCommand --> BuildGraph : creates

BuildCommand --> BuildPlanner : executes build plan
GenerateBuildMatrixCommand --> BuildPlanner : uses for trimming
GetStaleImagesCommand --> BuildPlanner : checks for stale images

BuildGraph *-- BuildTarget
BuildPlanner --> BuildGraph
BuildPlanner --> IBuildPolicy : evaluates
BuildPlanner --> BuildPlanItem : produces

BuildPlanItem --> BuildTarget
BuildPlanItem --> BuildAction
BuildPlanItem *-- BuildReason
BuildPlanItem --> PublishedImage
PublishedImage --> BuildTarget : metadata source

Loading

Build policies

There are multiple "policies" that we check images against to determine what actions to take. Previously, all these checks were strewn about the codebase and not managed centrally anywhere.

The CompositeBuildPolicy combines multiple different build policies into one that we can pass to the build planner.

classDiagram
direction LR

class IBuildPolicy {
  +EvaluateAsync(context) BuildPolicyResult
}
class BuildPolicyContext {
  +BuildGraph Graph
  +BuildTarget Target
  +PublishedImages
}
class BuildPolicyResult {
  +BuildAction Action
  +BuildReason[] Reasons
}
class CompositeBuildPolicy {
}
class AlwaysBuildPolicy {
}
class MissingPublishedImagePolicy {
}
class TagSetChangedPolicy {
}
class DockerfileChangedPolicy {
}
class BaseImageChangedPolicy {
}
class ImageDigestCache {
}
class IGitService {
}

IBuildPolicy <|.. CompositeBuildPolicy
IBuildPolicy <|.. AlwaysBuildPolicy
IBuildPolicy <|.. MissingPublishedImagePolicy
IBuildPolicy <|.. TagSetChangedPolicy
IBuildPolicy <|.. DockerfileChangedPolicy
IBuildPolicy <|.. BaseImageChangedPolicy

IBuildPolicy --> BuildPolicyContext : input
IBuildPolicy --> BuildPolicyResult : output
CompositeBuildPolicy o-- IBuildPolicy : combines

BuildPolicyResult --> BuildAction
BuildPolicyResult *-- BuildReason
BuildPolicyContext --> BuildGraph
BuildPolicyContext --> BuildTarget
BuildPolicyContext --> PublishedImage

BaseImageChangedPolicy --> ImageDigestCache
DockerfileChangedPolicy --> IGitService

Loading

[mthalman]

This would break if there are multiple platforms across multiple images that point to the same Dockerfile. This would only happen in a scenario like we once had when we support multi-platform manifest lists across Windows and Linux. In that scenario, we had one manifest image for that multi-platform and another for the multi-arch Debian tag, both tags pointing to the same Debian Dockerfile. That would cause a duplicate key exception to occur here. We don't currently have such a configuration but wanted to point it out.

[mthalman]

Because BuildReason is recursively nested and we're relying on the logger's default implementation of calling ToString, this results in bunch of nested output, not flattened.

Example:

Build plan for runtime (...): BuildImage. BuildReason { Message = "Dependency 'runtime-deps (...)' must build., CausedBy = BuildReason { Message = Base image 'mcr.microsoft.com/dotnet/runtime:8.0' changed from 'sha256:old' to 'sha256:new'., CausedBy = } }