Add ordered instance_types with insufficient-capacity fall-through

[cchristous]

Closes #701

What

Adds an optional, ordered instance_types ([]string) to the shared run
config. When set, it replaces the single instance_type: Packer launches the
first type and, on an insufficient-capacity error, falls through to the next
type in the list, succeeding on the first that launches. If every type is
capacity-starved, the build fails with the last error.

source "amazon-ebs" "mac" {
  # cheapest-first; falls through on Dedicated Host capacity exhaustion
  instance_types = ["mac2.metal", "mac2-m2.metal", "mac-m4.metal"]
  tenancy        = "host"
  # ... all other placement config is preserved across attempts ...
}

Rationale — why this is the right change

A build pinned to one instance_type fails hard with
InsufficientInstanceCapacity / InsufficientHostCapacity when that type is
capacity-starved, even when an equivalent type would launch. The driving case:
macOS arm64_mac AMIs build on mac2.metal (M1) via tenancy=host into a
License Manager host resource group; M1 Dedicated Host capacity is
intermittently exhausted, but the same AMI is cross-compatible with
mac2-m2.metal (M2) and mac-m4.metal (M4). An ordered list lets the build
succeed on the first type with capacity.

Design choices:

• instance_types []string, replacing instance_type — chosen over an
  instance_type_fallbacks supplement because it reads as one ordered priority
  list and matches the existing precedent of spot_instance_types on the Spot
  path. It's mutually exclusive with instance_type and spot_instance_types,
  reusing the existing exclusivity rule; instance_type alone is untouched and
  fully backward compatible.
• Centralized, testable capacity classifier (awserrors.IsCapacityError) —
  fall-through happens only on InsufficientInstanceCapacity,
  InsufficientHostCapacity, InsufficientReservedInstanceCapacity. Every
  other error (config/permission/quota — AuthFailure, InstanceLimitExceeded,
  Unsupported, InvalidParameterValue, …) fails the build immediately, so
  real misconfigurations are never masked by silently trying another type.
• Placement config reused across attempts — the RunInstancesInput is
  built once; only InstanceType changes per attempt, so tenancy,
  host_resource_group_arn, subnet/AZ, license specifications, and block device
  mappings are preserved.
• Burstable (T-family) types rejected in instance_types — the credit
  specification (forced standard for burstable families to avoid AWS's
  unlimited default) is derived from instance_type, which is empty when
  instance_types is used. Rather than silently launch a t3/t4g entry with
  surprise unlimited credits, Prepare rejects burstable entries with a clear
  error; the feature targets non-burstable capacity-constrained families anyway.
  (If burstable support is ever wanted, the correct fix is per-candidate credit
  derivation inside the fallback loop — not relaxing this guard.)
• Both common packages — the repo keeps builder/common (aws-sdk-go v1,
  amazon-instance) and common (aws-sdk-go v2, the EBS builders) in lockstep;
  the field, validation, and fallback loop are added to both so all four
  builders behave consistently and the generated docs (driven off the v1
  partial) stay correct.
• Reconciled with the IAM-propagation dry-run (fix: scope IAM instance profile propagation check to IAM only #700) — fix: scope IAM instance profile propagation check to IAM only #700 (now merged)
  added a DryRun RunInstances IAM-propagation check keyed off
  EffectiveInstanceType(), which handled only instance_type /
  spot_instance_types. This branch extends that helper to return the first
  instance_types candidate (the type a launch tries first; IAM propagation is
  type-independent) so an instance_types-only build still exercises the
  propagation wait instead of dry-running with an empty type. v2 common/ only —
  the helper and dry-run don't exist in v1 builder/common, and
  amazon-instance's IAM step has no dry-run.

Alternatives considered: instance_type_fallbacks as a supplement (rejected —
two knobs for one concept); Spot support (out of scope — spot_instance_types
already exists); a broader capacity-code set (rejected — kept to unambiguous
insufficient-capacity codes to avoid masking config errors).

Risk

• Blast radius: Medium. Touches the shared StepRunSourceInstance.Run
  launch path used by all on-demand builders. Behavior is unchanged unless the
  new instance_types field is set.
• Change criticality: Low–Medium. The launch call is core, but the change
  wraps the existing RunInstances retry in an ordered loop without altering
  the single-type path; the classifier is conservative (capacity codes only).
• Overall: Low–Medium.
• Mitigations & why it's safe to merge: With instance_types unset the
  candidate list is [instance_type], so the loop runs exactly one iteration
  issuing the same RunInstances call with the same per-attempt
  iamInstanceProfile retry as before — byte-for-byte identical behavior for
  existing templates. The only new behavior is gated entirely on the new field.
  The classifier is allow-list based, so only the three insufficient-capacity
  codes trigger fall-through; anything else returns immediately, preserving
  today's fail-fast semantics. Unit tests cover the classifier (capacity vs
  non-capacity codes, wrapped errors, nil) and the fallback loop (first success,
  fall-through, all-exhausted returns the last error, non-capacity aborts
  immediately, single-type path) with a mocked EC2 client; the loop is extracted
  behind the ec2iface.EC2API (v1) / clients.Ec2Client (v2) interface for
  isolated testing. HCL2 spec, struct-markdown partials, and compiled web-docs
  were regenerated via make generate.

Testing

• make test (unit, -race) — green.
• New unit tests: awserrors capacity classifier; runInstanceWithFallback
  ordering/selection; instance_types validation — in both builder/common
  (v1) and common (v2).
• Acceptance tests: not added. Deterministically provoking
  InsufficientInstanceCapacity against real AWS isn't practical; the
  fall-through logic is fully covered by unit tests with a mocked client.

Pre-Submission Review

Two automated review passes were run.

First pass (PR-review + simplify, before opening):

• Reject burstable (T-family) types in instance_types to prevent a surprise
  unlimited-credits launch (credit spec is instance_type-derived).
• runInstanceWithFallback returns a clear error instead of (nil, nil) on an
  empty candidate list (defensive).
• v1 IsCapacityError switched to errors.As for parity with v2 and to
  recognize wrapped capacity errors.
• Added tests: single-type success / capacity-error propagation; distinguishable
  "returns the last error" assertion; wrapped-capacity-error classifier cases;
  error-message content assertions on the validation tests; burstable rejection.
• Simplify: hoisted the burstable regexp to a package-level compiled var
  (matching the existing reShutdownBehavior precedent); aligned the duplicated
  v1/v2 IsCapacityError doc comment.

Second pass (full multi-aspect review — correctness, tests, silent-failure,
comments, simplification). No critical/important correctness issues were found
(v1/v2 confirmed in lockstep, backward-compat byte-for-byte). Findings addressed:

• Fixed the burstable regexp (:? → (?: (non-capturing group); it now serves
  as a proper validation gate rather than matching an optional literal colon.
• Renamed the regexp var to reBurstableInstanceType to match the adjacent
  reShutdownBehavior naming precedent.
• On multi-type exhaustion, runInstanceWithFallback now wraps the last error
  with the list of types tried (via %w) so the persisted failure shows the
  fallback ran and was exhausted; the single-type path returns the raw error
  unchanged (preserving byte-for-byte backward compatibility).
• Doc cleanup: removed hardcoded code-owned enumerations (capacity codes,
  placement fields) and the hardware narrative from the InstanceTypes field
  doc; regenerated partials and web-docs.
• Added tests: empty-list guard; instance_type+spot_instance_types and
  all-three-selector exclusivity; zero-selector error-message assertion; and an
  assertion that the exhaustion error names the types tried.
• Consciously skipped (noted, not defects): a non-empty-Instances guard on the
  success path (pre-existing, no real AWS path), per-entry nitro/credit
  validation for list types (outside the metal/GPU use case), and a full
  Run()-level wiring test (heavy state bag; the selection helper is covered).

Third pass (full multi-aspect review of the rebased branch). Findings
addressed:

• AZ machine-type wiring. StepNetworkInfo.RequestedMachineType was fed the
  raw instance_type, which is empty when instance_types is used, so
  filterAZByMachineType queried DescribeInstanceTypeOfferings with an empty
  instance-type filter and silently skipped AZ selection (the same class of gap
  fix: scope IAM instance profile propagation check to IAM only #700 fixed for the IAM dry-run step, missed on the parallel network step). All
  four builders now wire RequestedMachineType to EffectiveInstanceType(),
  matching the adjacent StepIamInstanceProfile line; EffectiveInstanceType()
  was added to builder/common (v1) to keep the two packages in lockstep. Side
  effect: a spot_instance_types-only build now filters AZs by its first spot
  type instead of skipping filtering — a graceful, strictly-more-correct change
  (it degrades to the old skip-with-warning behavior if that type isn't offered).
• Clearer enable_unlimited_credits error. Combining it with instance_types
  is genuinely incompatible (unlimited credits require a burstable type, which
  instance_types rejects); the check now emits a dedicated message naming the
  conflict instead of interpolating the empty instance_type.
• instance_type required tag flipped to required:"false". One of
  instance_type / instance_types / spot_instance_types satisfies the
  launch-type requirement (enforced in Prepare), so the generated docs now
  correctly list instance_type under Optional for every builder. Regenerated
  hcl2spec, doc partials, and web-docs.
• Added tests: v1 EffectiveInstanceType (mirrors v2) and the
  enable_unlimited_credits + instance_types conflict, in both packages.
• Consciously skipped (noted, not defects): re-selecting subnet/AZ per fallback
  candidate (a larger change; the AZ-wiring fix removes its most common trigger),
  and a distinct exhaustion message for a single-element instance_types list
  (the launch path already normalizes a lone instance_type to a one-element
  list, so both behave identically by construction).

[hashicorp-cla-app]

All committers have signed the CLA.

[hashicorp-cla-app]

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes

_{Have you signed the CLA already but the status is still pending? Recheck it.}