docs(lvm-local-storage): document chunk size + expand tuning section

[abonillabeeche]

Summary

Follow-up to csi-driver-lvm PR #57 — documents the new chunkSize, poolMetadataSize, and zeroBlocks StorageClass parameters and expands the existing "Tuning the dm-thin Pool" section (introduced in #1075) with:

• NEW subsection: Chunk size — the primary content of this PR. Explains why the default is 1M (aligns with the RAID full-stripe width common on Harvester hardware, avoiding partial-stripe read-modify-write and, on parity RAID, write-hole exposure), when to override (match your RAID full stripe; 512K/128K only for sub-stripe or non-RAID VGs, never >2M), the hard "cannot change after pool creation" constraint, and how to verify the effective chunk size of a live pool.
• Chunk zeroing — expanded to mention the zeroBlocks StorageClass parameter (in addition to the existing post-creation lvchange --zero n workflow).
• Pool metadata size — expanded to mention the poolMetadataSize StorageClass parameter (default 16G covers 256 TB pools at 1M chunks) and adds the metadata sizing formula.
• NEW: Full example StorageClass — shows all three parameters set explicitly.

Motivation

A field customer running everything from our runbook ended up with a 16 MiB chunk_size on their thin pool. LVM's default auto-select scales chunk_size with pool size to keep metadata bounded, which for multi-TB pools produces 8-16 MiB chunks — appropriate for metadata sizing but disastrous for random-IOPS workloads (up to 4096:1 allocation write amplification on 4K writes to virgin regions).

Customer's measured DI_RANDOM: 326 IOPS. Our lab measurement on hardware in the same performance class with an explicit, stripe-aligned chunk size + zero=n: 15,000+ QD1 4K IOPS. The gap is entirely explained by chunk size + zero-on-allocate.

Test plan

• Local Docusaurus render (npm start) — page renders, no broken links
• Markdown syntax validated
• Docs CI (markdownlint + link check) — will run on this PR

Base branch note

This PR includes the "Tuning the dm-thin Pool" section from PR #1075 (APPROVED + MERGEABLE) because it builds on that section. Please merge #1075 first — this PR will show a clean diff (just the chunk-size additions) once it rebases against post-#1075 main.

Scope

Only edits docs/advanced/addons/lvm-local-storage.md (current-dev / v1.9 tree). Not backported to versioned_docs/ because the chunkSize StorageClass parameter ships in a CSI driver version targeting Harvester v1.9. The lvchange/lvextend post-creation workflows work on older versions but were already covered by #1075.

References

• csi-driver-lvm PR #57 — driver + chart changes this documents
• LVM upstream performance policy — the 512 KiB general-purpose seed (Harvester defaults to 1M for RAID full-stripe alignment)
• Linux kernel dm-thin admin guide — "512 KiB general; 64 KiB snapshot-heavy"
• Red Hat Gluster admin guide — the 2 MiB cap "to reduce COW problems"

---

🤖 Generated with Claude Code

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request

[abonillabeeche]

@jillian-maroket PING

[jillian-maroket]

@abonillabeeche I'll work on this next week. Too many release deliverables.

[jillian-maroket]

Review done

[jillian-maroket]

Suggested change

	- **Chunk size**: The chunk size determines the smallest unit of physical space that a thin pool allocates in response to a write. A write to a previously-unallocated region always allocates a full chunk, so small random writes to virgin space against a large chunk size cause severe write amplification (a 4 KiB write against a 16 MiB chunk allocates 16 MiB of pool space). When you install the harvester-csi-driver-lvm add-on version 0.4.0 or later, the default StorageClass sets `chunkSize: "1M"`. On the hardware-RAID-backed volume groups typical of Harvester nodes, `1M` matches the full-stripe width of common layouts (for example, four data disks at a 256 KiB strip yields a 1 MiB stripe), so each chunk allocation maps to whole stripes rather than partial ones. `1M` also keeps thin-pool metadata bounded and comfortably supports pools up to 256 TB. See the [Linux kernel dm-thin admin guide](https://docs.kernel.org/admin-guide/device-mapper/thin-provisioning.html) for background on chunk sizing.
	- **Chunk size**: The chunk size determines the smallest unit of physical space that a thin pool allocates in response to a write. Because a write to a previously unallocated region always provisions a full chunk, small random writes against a large chunk size cause severe write amplification (for example, a 4 KiB write against a 16 MiB chunk allocates 16 MiB of pool space).
	When you install the `harvester-csi-driver-lvm` add-on v0.4.0 or later, the default StorageClass uses `chunkSize: "1M"`. On the hardware RAID-backed volume groups typical of Harvester nodes, `1M` matches the full-stripe width of common layouts (for example, four data disks with a 256 KiB stripe size yield a 1 MiB stripe). This ensures each chunk allocation maps to whole stripes rather than partial ones. Additionally, a `1M` chunk size keeps thin-pool metadata bounded and comfortably supports pools up to 256 TB. For more information about chunk sizing, see the [Linux kernel dm-thin documentation](https://docs.kernel.org/admin-guide/device-mapper/thin-provisioning.html).

[jillian-maroket]

Suggested change

Align the chunk size to your RAID full-stripe width — that is, (number of data disks) × (RAID strip size). A chunk smaller than the full stripe (for example, a `512K` chunk on a 1 MiB stripe) forces every first-touch allocation into a partial-stripe **read-modify-write**. On parity RAID (5/6), partial-stripe writes also widen the write-hole window, so an unclean shutdown without a protected controller cache (BBU/FBWC) can leave a stripe with inconsistent parity. Choosing a chunk size equal to — or an exact multiple of — the full stripe avoids both problems; `1M` is the safe default because it aligns with the most common power-of-two RAID geometries.

Align the chunk size to your RAID full-stripe width: (number of data disks) × (RAID strip size). A chunk smaller than the full stripe (for example, a `512K` chunk on a 1 MiB stripe) forces every first-touch allocation into a partial-stripe **read-modify-write**. On parity RAID (5/6), partial-stripe writes also widen the write-hole window. This means that an unclean shutdown without a protected controller cache (BBU/FBWC) can leave a stripe with inconsistent parity. Choosing a chunk size equal to, or an exact multiple of, the full stripe avoids both issues. `1M` is the safe default because it aligns with the most common power-of-two RAID geometries.

[jillian-maroket]

Suggested change

	Override the default with the `chunkSize` parameter on the StorageClass. Common values:
	| Value | When to use |
	|---|---|
	| `1M` | **Default** — matches the full-stripe width of common hardware-RAID layouts; general-purpose VM workloads |
	| `512K` / `128K` | Only when the RAID full stripe is that size, or on non-RAID / single-disk volume groups where stripe alignment does not apply and minimizing snapshot copy-on-write cost is the priority |
	| `2M` | Very large sequential-write pools whose RAID full stripe is 2 MiB |
	Override the default with the `chunkSize` parameter on the StorageClass. Common values include the following:
	| Value | Target Environment |
	| :--- | :--- |
	| `1M` (Default) | Standard hardware RAID (1 MiB full stripe); general-purpose virtual machine workloads |
	| `512K` / `128K` | RAID arrays with matching full stripe size; non-RAID/single-disk volume groups where stripe alignment does not apply and minimizing snapshot copy-on-write overhead is primary |
	| `2M` | Large-stripe RAID arrays (2 MiB full stripe) with high-throughput sequential-write workloads |

[jillian-maroket]

Suggested change

	Never exceed `2M`. Larger chunks trigger disproportionate copy-on-write costs for snapshots, as documented in the [Red Hat Gluster admin guide](https://docs.redhat.com/en/documentation/red_hat_gluster_storage/3.5/html/administration_guide/chap-configuring_red_hat_storage_for_enhancing_performance). If you rely on LVM's built-in auto-selection instead of setting `chunkSize` explicitly, LVM chooses the chunk size based on the pool size to keep metadata bounded, which for multi-TB pools produces 8–16 MiB chunks — appropriate for metadata sizing but often not for random-write performance.
	Do not exceed 2M. Larger chunk sizes trigger disproportionate copy-on-write overhead for snapshots, as documented in the [Red Hat Gluster admin guide](https://docs.redhat.com/en/documentation/red_hat_gluster_storage/3.5/html/administration_guide/chap-configuring_red_hat_storage_for_enhancing_performance).
	In addition, avoid relying on LVM's built-in auto-selection. To keep metadata bounded, LVM determines the chunk size based on the total pool size. This results in 8 to 16 MiB chunks for multi-terabyte pools, which is appropriate for metadata sizing but can severely degrade random-write performance. Always set `chunkSize` explicitly in the StorageClass.

[jillian-maroket]

Suggested change

- **Pool metadata size**: When the thin pool is created, the driver sizes its metadata logical volume based on the `poolMetadataSize` StorageClass parameter (default `16G`, sufficient for pools up to 256 TB at the default 1 MiB chunk size). A larger chunk size needs less metadata to address the same capacity, and a smaller chunk size needs more: metadata bytes ≈ 64 × (pool_size ÷ chunk_size). If you did not set the parameter, or if the pool grew significantly after creation, extend the metadata volume proactively to prevent the pool from running out of space and becoming unresponsive later.

- **Pool metadata size**: When the thin pool is created, the driver sizes its metadata logical volume based on the `poolMetadataSize` StorageClass parameter. The default size is `16G`, which supports pools up to 256 TB at the default 1 MiB chunk size. Larger chunk sizes require less metadata to address the same capacity, while smaller chunk sizes require more (`metadata_bytes ≈ (pool_size ÷ chunk_size) × 64`). If you do not set this parameter, or if the pool grows significantly after creation, proactively extend the metadata volume to prevent the pool from exhausting metadata space and becoming unresponsive later.

[jillian-maroket]

Suggested change

	When the first PersistentVolumeClaim is created against a `dm-thin` StorageClass, the driver creates an LVM thin-pool named `<vgName>-thinpool` at `-l 90%FREE` of the volume group. Two settings are worth knowing about:
	When the first PersistentVolumeClaim (PVC) is created against a `dm-thin` StorageClass, the driver creates an LVM thin pool named `<vgName>-thinpool` using `-l 90%FREE` (allocating 90% of the volume group's remaining free space). Consider tuning the following settings based on your workload demands:

[jillian-maroket]

Suggested change

	- **Chunk zeroing.** By default, the pool writes zeros to each newly-allocated chunk before handing it to the writer. On single-tenant clusters this can be disabled to reduce write amplification on first-touch allocations:
	- **Chunk zeroing**: By default, the thin pool writes zeros to each newly allocated chunk before exposing it to a write operation. On single-tenant clusters, you can disable chunk zeroing to significantly reduce write amplification during initial data allocations.

[jillian-maroket]

Suggested change

	- **Pool metadata size.** LVM auto-sizes the thin-pool metadata LV at pool creation. For pools that will hold many snapshots or many thin volumes over time, extending the metadata proactively avoids running short later:
	- **Pool metadata size**: When the thin pool is created, LVM automatically sizes its metadata logical volume. Consider extending this volume proactively if you expect the pool to store a large number of snapshots or thin volumes over time. Doing so prevents the pool from running out of space and becoming unresponsive later.

[jillian-maroket]

Suggested change

	### Coexistence with Longhorn v2 Block-Mode Disks
	If the same node hosts a Longhorn v2 disk in block mode, the underlying device is held exclusively by the SPDK instance manager. Adding that device to the LVM `global_filter` prevents LVM's device scan from attempting to open it. Example, in `/etc/lvm/lvmlocal.conf`:
	### Coexistence with Longhorn V2 Block Mode Disks
	If the same node hosts a Longhorn V2 disk in block mode, the underlying device is held exclusively by the SPDK Instance Manager. You can add this device to the LVM `global_filter` to exclude it from LVM device scans and prevent resource conflicts.
	Example (`/etc/lvm/lvmlocal.conf`):

[jillian-maroket]

Suggested change

	On Harvester's immutable OS, persist the change through an `/oem/*.yaml` cloud-config file so it survives reboots. For background, see [harvester/harvester#11098](https://github.com/harvester/harvester/issues/11098).
	Because Harvester's operating system is immutable, you must persist this change through an `/oem/*.yaml` cloud-config file to ensure it survives system reboots and upgrades.For more information, see issue [#11098](https://github.com/harvester/harvester/issues/11098).