diff --git a/docs/modules/spark-k8s/pages/getting_started/index.adoc b/docs/modules/spark-k8s/pages/getting_started/index.adoc index ed7a05b7..cb3e29aa 100644 --- a/docs/modules/spark-k8s/pages/getting_started/index.adoc +++ b/docs/modules/spark-k8s/pages/getting_started/index.adoc @@ -1,4 +1,5 @@ = Getting started +:description: Getting started with Apache Spark on Kubernetes, covering the Stackable operator and its dependencies and a first Spark job and its result. This guide gets you started with Spark using the Stackable operator for Apache Spark. It guides you through the installation of the operator and its dependencies, executing your first Spark job and reviewing its result. diff --git a/docs/modules/spark-k8s/pages/reference/commandline-parameters.adoc b/docs/modules/spark-k8s/pages/reference/commandline-parameters.adoc index b52765c7..030d53bf 100644 --- a/docs/modules/spark-k8s/pages/reference/commandline-parameters.adoc +++ b/docs/modules/spark-k8s/pages/reference/commandline-parameters.adoc @@ -1,4 +1,5 @@ = Command line parameters +:description: Command-line parameters the Stackable operator for Apache Spark accepts, with their default values and usage examples. This operator accepts the following command line parameters: diff --git a/docs/modules/spark-k8s/pages/reference/crds.adoc b/docs/modules/spark-k8s/pages/reference/crds.adoc index 32be6df4..7544a169 100644 --- a/docs/modules/spark-k8s/pages/reference/crds.adoc +++ b/docs/modules/spark-k8s/pages/reference/crds.adoc @@ -1,3 +1,4 @@ = CRD Reference +:description: CRD reference for the Stackable operator for Apache Spark on the Stackable Hub, covering every field, default and API version. Find all CRD reference for the Stackable Operator for Apache Spark at: {crd-ref}{crd-ref-release}&operator=spark-k8s-operator. diff --git a/docs/modules/spark-k8s/pages/reference/environment-variables.adoc b/docs/modules/spark-k8s/pages/reference/environment-variables.adoc index d30c6f16..ef697dce 100644 --- a/docs/modules/spark-k8s/pages/reference/environment-variables.adoc +++ b/docs/modules/spark-k8s/pages/reference/environment-variables.adoc @@ -1,4 +1,5 @@ = Environment variables +:description: Environment variables the Stackable operator for Apache Spark accepts, with their default values and what each one controls. This operator accepts the following environment variables: diff --git a/docs/modules/spark-k8s/pages/reference/index.adoc b/docs/modules/spark-k8s/pages/reference/index.adoc index 3cfa0289..5374070a 100644 --- a/docs/modules/spark-k8s/pages/reference/index.adoc +++ b/docs/modules/spark-k8s/pages/reference/index.adoc @@ -1,4 +1,5 @@ = Reference +:description: Reference for the Stackable operator for Apache Spark, covering CRD fields, command-line parameters and environment variables. Consult the reference documentation section to find exhaustive information on: diff --git a/docs/modules/spark-k8s/pages/usage-guide/index.adoc b/docs/modules/spark-k8s/pages/usage-guide/index.adoc index 545c3432..b5e185ac 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/index.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/index.adoc @@ -1,3 +1,4 @@ = Usage guide +:description: Guides for the Stackable operator for Apache Spark beyond the basics, covering job dependencies, S3 connections and examples of usage scenarios. Learn how to load your own xref:usage-guide/job-dependencies.adoc[] or configure an xref:usage-guide/s3.adoc[S3 connection] to access data. Have a look at the xref:usage-guide/examples.adoc[] to learn more about different usage scenarios. diff --git a/docs/modules/spark-k8s/pages/usage-guide/logging.adoc b/docs/modules/spark-k8s/pages/usage-guide/logging.adoc index 07edf9e5..30ba0d63 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/logging.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/logging.adoc @@ -1,4 +1,5 @@ = Logging +:description: Logging for Apache Spark applications and the history server, covering the Vector agent the operator adds and forwarding logs to an aggregator. The Spark operator installs a https://vector.dev/docs/setup/deployment/roles/#agent[vector agent] as a side-car container in every application Pod except the `job` Pod that runs `spark-submit`. It also configures the logging framework to output logs in XML format. diff --git a/docs/modules/spark-k8s/pages/usage-guide/openlineage.adoc b/docs/modules/spark-k8s/pages/usage-guide/openlineage.adoc index d80c3746..342401a4 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/openlineage.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/openlineage.adoc @@ -1,4 +1,5 @@ = OpenLineage +:description: OpenLineage events from Apache Spark applications to a backend such as Marquez, covering authentication and job naming without manual sparkConf wiring. https://openlineage.io/[OpenLineage] is an open standard for data lineage collection. The Spark operator can automatically emit lineage events for a `SparkApplication` to an OpenLineage-compatible backend such as https://marquezproject.github.io/marquez/[Marquez], without any manual `sparkConf` wiring. diff --git a/docs/modules/spark-k8s/pages/usage-guide/operations/applications.adoc b/docs/modules/spark-k8s/pages/usage-guide/operations/applications.adoc index 753b0d99..ead61515 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/operations/applications.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/operations/applications.adoc @@ -1,4 +1,5 @@ = Spark applications +:description: The lifecycle of Apache Spark applications submitted as SparkApplication resources, covering resubmission and their Prometheus metrics. Spark applications are submitted to the Spark Operator as SparkApplication resources. These resources are used to define the configuration of the Spark job, including the image to use, the main application file, and the number of executors to start. diff --git a/docs/modules/spark-k8s/pages/usage-guide/operations/graceful-shutdown.adoc b/docs/modules/spark-k8s/pages/usage-guide/operations/graceful-shutdown.adoc index 980e08aa..e4339d8b 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/operations/graceful-shutdown.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/operations/graceful-shutdown.adoc @@ -1,4 +1,5 @@ = Graceful shutdown +:description: Graceful shutdown for Apache Spark, which the Stackable operator has no settings for, with Spark itself creating and stopping driver and executor Pods. Normally you can configure the graceful shutdown as described in xref:concepts:operations/graceful_shutdown.adoc[]. diff --git a/docs/modules/spark-k8s/pages/usage-guide/operations/index.adoc b/docs/modules/spark-k8s/pages/usage-guide/operations/index.adoc index 21d170f9..f14a2c05 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/operations/index.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/operations/index.adoc @@ -1,4 +1,5 @@ = Operations +:description: Operating Apache Spark with the Stackable operator, covering Spark applications, graceful shutdown, Pod disruptions and Pod placement. This section of the documentation is intended for the operations teams that maintain a Stackable Data Platform installation. diff --git a/docs/modules/spark-k8s/pages/usage-guide/operations/pod-disruptions.adoc b/docs/modules/spark-k8s/pages/usage-guide/operations/pod-disruptions.adoc index 38d6fb85..6b26ae20 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/operations/pod-disruptions.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/operations/pod-disruptions.adoc @@ -1,4 +1,5 @@ = Allowed Pod disruptions +:description: PodDisruptionBudgets for the Apache Spark history server, and why the Stackable operator writes none for driver and executor Pods. == Spark applications diff --git a/docs/modules/spark-k8s/pages/usage-guide/overrides.adoc b/docs/modules/spark-k8s/pages/usage-guide/overrides.adoc index 26c17c33..2259e755 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/overrides.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/overrides.adoc @@ -1,4 +1,5 @@ = Configuration & Environment Overrides +:description: Overrides for Apache Spark, covering configuration, spark-env.sh, security.properties, environment variables, Pod specs and JVM arguments. The cluster definition also supports overriding configuration properties and environment variables, either per role or per role group, where the more specific override (role group) has precedence over the less specific one (role). diff --git a/docs/modules/spark-k8s/pages/usage-guide/resources.adoc b/docs/modules/spark-k8s/pages/usage-guide/resources.adoc index bec9bf7f..b8296f0b 100644 --- a/docs/modules/spark-k8s/pages/usage-guide/resources.adoc +++ b/docs/modules/spark-k8s/pages/usage-guide/resources.adoc @@ -1,6 +1,7 @@ = Resource Requests +:description: CPU and memory for Apache Spark jobs, drivers, executors and the history server, with examples of how to size each of them. -include::home:concepts:stackable_resource_requests.adoc[] +include::concepts:partial$stackable_resource_requests.adoc[] If no resources are configured explicitly, the operator uses the following defaults for `SparkApplication` resources: