Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ The install script allows installation of different flavors of the Agent binarie
|`DD_SYSTEM_PROBE_ENSURE_CONFIG`|Create the system probe configuration file from a template if it does not already exist.|
|`DD_RUNTIME_SECURITY_CONFIG_ENABLED`|If set to `true`, ensure creation of security Agent and system probe configuration file (if they don't already exist), and enable Cloud Workload Security (CWS).|
|`DD_COMPLIANCE_CONFIG_ENABLED`|If set to `true`, ensures the creation of a security Agent configuration file if one doesn't already exist, and enables Cloud Security Posture Management (CSPM).|
|`DD_NO_SECURITY_AGENT_INSTALL`|Set to any value to run Cloud Workload Security (CWS) and Cloud Security Posture Management (CSPM) in the system probe instead of the security Agent. With `DD_RUNTIME_SECURITY_CONFIG_ENABLED=true`, CWS is enabled in the system probe configuration file, which also sends the events directly. With `DD_COMPLIANCE_CONFIG_ENABLED=true`, CSPM is enabled in the Agent configuration file and runs in the system probe. The security Agent configuration file is never created, but if it already exists it is updated to tell the security Agent that CWS and CSPM run in the system probe, so that it stops. These settings are applied even when the configuration files are kept from a previous installation.|
|`DD_DISCOVERY_ENABLED`|If set to `true`, and a system probe configuration file does not already exist, creates a system probe configuration file and enables Service Discovery.
|`DD_PRIVILEGED_LOGS_ENABLED`|If set to `true`, and a system probe configuration file does not already exist, creates a system probe configuration file and enables Privileged Logs.
|`DD_SYSTEM_PROBE_SERVICE_MONITORING_ENABLED`|If set to `true`, and a system probe configuration file does not already exist, creates a system probe configuration file and enables Universal Service Monitoring (USM).
Expand Down
71 changes: 69 additions & 2 deletions install_script.sh.template
Original file line number Diff line number Diff line change
Expand Up @@ -923,6 +923,11 @@ if [ -n "$DD_NO_AGENT_INSTALL" ]; then
no_agent=true
fi

no_security_agent=
if [ -n "$DD_NO_SECURITY_AGENT_INSTALL" ]; then
no_security_agent=true
fi

infrastructure_mode=
if [ -n "$DD_INFRASTRUCTURE_MODE" ]; then
infrastructure_mode=$DD_INFRASTRUCTURE_MODE
Expand Down Expand Up @@ -2275,13 +2280,68 @@ function update_par(){
${par_config}
EOF
}
function set_config_option(){
local sudo_cmd="$1"
local config_file="$2"
local section="$3"
local option="$4"
local value="$5"
if ! $sudo_cmd grep -q "^$section:" "$config_file"; then
printf "\033[34m\n* Setting $section.$option to $value in $config_file\n\033[0m\n"
$sudo_cmd sh -c "cat >> '$config_file'" <<EOF

$section:
$option: $value
EOF
return
fi
# Look for the option in the section only, the same option name can be used in other sections
if $sudo_cmd sed -n "/^$section:/,/^[^[:space:]#]/p" "$config_file" | grep -qE "^[[:space:]]+$option:"; then
printf "\033[34m\n* $section.$option is already set in $config_file, skipping the update.\n\033[0m\n"
return
Comment on lines +2299 to +2301

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Force migration flags to true

When a retained configuration already contains any of these keys with false (for example, compliance_config.enabled: false in datadog.yaml or direct_send_from_system_probe: false in system-probe.yaml), this presence-only check skips the requested update. Consequently, DD_NO_SECURITY_AGENT_INSTALL=true DD_COMPLIANCE_CONFIG_ENABLED=true can leave CSPM disabled or leave CWS sending through the security Agent, despite the migration option explicitly requesting that they run in system-probe. Update the existing value when it conflicts with the required true value rather than treating every pre-existing key as idempotent success.

Useful? React with 👍 / 👎.

fi
printf "\033[34m\n* Setting $section.$option to $value in $config_file\n\033[0m\n"
$sudo_cmd sh -c "sed -i '0,/^$section:/s//$section:\n $option: $value/' $config_file"
}
function run_security_in_system_probe(){
local sudo_cmd="$1"
local core_config_file="$2"
local security_config_file="$3"
local probe_config_file="$4"
local enable_compliance="$5"
# The security Agent service starts as soon as its configuration file exists, so it needs to
# know that both CWS and CSPM run in system-probe to stop right away. The file is only updated
# when it already exists, it must never be created.
if $sudo_cmd test -e "$security_config_file"; then
set_config_option "$sudo_cmd" "$security_config_file" "runtime_security_config" "direct_send_from_system_probe" "true"
set_config_option "$sudo_cmd" "$security_config_file" "compliance_config" "run_in_system_probe" "true"
fi
if $sudo_cmd test -e "$core_config_file"; then
if [ "$enable_compliance" == "true" ]; then
set_config_option "$sudo_cmd" "$core_config_file" "compliance_config" "enabled" "true"
set_config_option "$sudo_cmd" "$core_config_file" "compliance_config" "run_in_system_probe" "true"
elif $sudo_cmd grep -q "^compliance_config:" "$core_config_file"; then
# CSPM enabled by a previous installation has to run in system-probe too
set_config_option "$sudo_cmd" "$core_config_file" "compliance_config" "run_in_system_probe" "true"
fi
fi
# Only set the option when CWS is configured, it would have no effect otherwise
if $sudo_cmd test -e "$probe_config_file" && $sudo_cmd grep -q "^runtime_security_config:" "$probe_config_file"; then
set_config_option "$sudo_cmd" "$probe_config_file" "runtime_security_config" "direct_send_from_system_probe" "true"
fi
}
function manage_security_config(){
local sudo_cmd="$1"
local security_config_file="$2"
local enable_security="$3"
local enable_compliance="$4"
local no_security_agent="$5"
if [ "$enable_security" == "true" ] || [ "$enable_compliance" == "true" ]; then
if ensure_config_file_exists "$sudo_cmd" "$security_config_file" "root"; then
if [ "$no_security_agent" == "true" ]; then
# CWS and CSPM run in system-probe, see run_security_in_system_probe, so the security
# Agent configuration file must not be created
printf "\033[34m\n* Not creating $security_config_file: CWS and CSPM run in system-probe\n\033[0m\n"
elif ensure_config_file_exists "$sudo_cmd" "$security_config_file" "root"; then
update_security_and_or_compliance "$sudo_cmd" "$security_config_file" "$enable_security" "$enable_compliance"
fi
fi
Expand Down Expand Up @@ -2389,7 +2449,7 @@ elif [ ! "$no_agent" ]; then
fi
fi

manage_security_config "$sudo_cmd" "$security_agent_config_file" "$DD_RUNTIME_SECURITY_CONFIG_ENABLED" "$DD_COMPLIANCE_CONFIG_ENABLED"
manage_security_config "$sudo_cmd" "$security_agent_config_file" "$DD_RUNTIME_SECURITY_CONFIG_ENABLED" "$DD_COMPLIANCE_CONFIG_ENABLED" "$no_security_agent"
manage_system_probe_config "$sudo_cmd" "$system_probe_config_file" "$DD_RUNTIME_SECURITY_CONFIG_ENABLED" "$DD_DISCOVERY_ENABLED" "$DD_PRIVILEGED_LOGS_ENABLED" "$usm_enabled"
fi

Expand Down Expand Up @@ -2422,6 +2482,12 @@ if [ -n "$system_probe_ensure_config" ]; then
ensure_config_file_exists "$sudo_cmd" "$system_probe_config_file" "root"
fi

# Make sure the security Agent doesn't run when DD_NO_SECURITY_AGENT_INSTALL is set, even when
# the configuration files were kept from a previous installation
if [ -n "$no_security_agent" ]; then
run_security_in_system_probe "$sudo_cmd" "$config_file" "$security_agent_config_file" "$system_probe_config_file" "$DD_COMPLIANCE_CONFIG_ENABLED"
fi

# Install APM SSI if needed
install_apm_ssi "$sudo_cmd" || true

Expand Down Expand Up @@ -2502,6 +2568,7 @@ end_stage 0 "$(cat <<-END | tr -d '\n'
"apm_enabled": "$([ -n "$DD_APM_INSTRUMENTATION_ENABLED" ] && echo "$DD_APM_INSTRUMENTATION_ENABLED" || echo "false")",
"security_enabled": "$([ -n "$DD_RUNTIME_SECURITY_CONFIG_ENABLED" ] && echo "true" || echo "false")",
"compliance_enabled": "$([ -n "$DD_COMPLIANCE_CONFIG_ENABLED" ] && echo "true" || echo "false")",
"no_security_agent": "$([ -n "$no_security_agent" ] && echo "true" || echo "false")",
"logs_collection_enabled": "$([ "$DD_LOGS_CONFIG_PROCESS_COLLECT_ALL" == "true" ] && echo "true" || echo "false")",
"hostname_override": "$([ -n "$hostname" ] && echo "true" || echo "false")",
"tags_configured": "$([ -n "$host_tags" ] && echo "true" || echo "false")",
Expand Down
94 changes: 94 additions & 0 deletions test/e2e/install_no_security_agent_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
// Unless explicitly stated otherwise all files in this repository are licensed
// under the Apache License Version 2.0.
// This product includes software developed at Datadog (https://www.datadoghq.com/).
// Copyright 2016-present Datadog, Inc.

package e2e

import (
"fmt"
"testing"

"github.com/DataDog/datadog-agent/test/new-e2e/pkg/e2e"
awshost "github.com/DataDog/datadog-agent/test/new-e2e/pkg/provisioners/aws/host"
"github.com/stretchr/testify/assert"
)

type installNoSecurityAgentTestSuite struct {
linuxInstallerTestSuite
}

func TestInstallNoSecurityAgentSuite(t *testing.T) {
if flavor != agentFlavorDatadogAgent {
t.Skip("no security agent test supports only datadog-agent flavor")
}
stackName := fmt.Sprintf("install-no-security-agent-%s-%s-%s", flavor, platform, getenv("CI_PIPELINE_ID", "dev"))
t.Run(stackName, func(t *testing.T) {
t.Logf("We will install %s without the security agent with install script on %s", flavor, platform)
testSuite := &installNoSecurityAgentTestSuite{}
e2e.Run(t,
testSuite,
e2e.WithProvisioner(awshost.ProvisionerNoAgentNoFakeIntake(awshost.WithEC2InstanceOptions(getEC2Options(t)...))),
e2e.WithStackName(stackName),
)
})
}

func (s *installNoSecurityAgentTestSuite) TestInstallNoSecurityAgent() {
s.InstallAgent(7, "DD_NO_SECURITY_AGENT_INSTALL=true DD_RUNTIME_SECURITY_CONFIG_ENABLED=true DD_COMPLIANCE_CONFIG_ENABLED=true DD_SITE=\"datadoghq.com\"", "Install latest Agent 7 without the security agent")

s.assertInstallScript()

s.addExtraIntegration()

s.uninstall()

s.assertUninstall()

s.purge()

s.assertPurge()
}

func (s *installNoSecurityAgentTestSuite) assertInstallScript() {
s.linuxInstallerTestSuite.assertInstallScript(true)

t := s.T()
vm := s.Env().RemoteHost

t.Log("Assert system probe config is created and the security agent config is not")
assertFileExists(t, vm, fmt.Sprintf("/etc/%s/%s", s.baseName, systemProbeConfigFileName))
assertFileNotExists(t, vm, fmt.Sprintf("/etc/%s/%s", s.baseName, securityAgentConfigFileName))

t.Log("Assert runtime security is enabled in the system probe config and sent from system-probe")
systemProbeConfig := unmarshalConfigFile(t, vm, fmt.Sprintf("/etc/%s/%s", s.baseName, systemProbeConfigFileName))
runtimeSecurityConfig := systemProbeConfig["runtime_security_config"].(map[any]any)
assert.Equal(t, true, runtimeSecurityConfig["enabled"])
assert.Equal(t, true, runtimeSecurityConfig["direct_send_from_system_probe"])

t.Log("Assert compliance runs in system-probe")
agentConfig := unmarshalConfigFile(t, vm, fmt.Sprintf("/etc/%s/%s", s.baseName, s.configFile))
complianceConfig := agentConfig["compliance_config"].(map[any]any)
assert.Equal(t, true, complianceConfig["enabled"])
assert.Equal(t, true, complianceConfig["run_in_system_probe"])
}

func (s *installNoSecurityAgentTestSuite) assertUninstall() {
s.linuxInstallerTestSuite.assertUninstall()
t := s.T()
vm := s.Env().RemoteHost
t.Log("Assert system probe config is still there after uninstall")
assertFileExists(t, vm, fmt.Sprintf("/etc/%s/%s", s.baseName, systemProbeConfigFileName))
assertFileNotExists(t, vm, fmt.Sprintf("/etc/%s/%s", s.baseName, securityAgentConfigFileName))
}

func (s *installNoSecurityAgentTestSuite) assertPurge() {
if s.shouldSkipPurge() {
return
}
s.linuxInstallerTestSuite.assertPurge()
t := s.T()
vm := s.Env().RemoteHost
t.Log("Assert system probe config is removed after purge")
assertFileNotExists(t, vm, fmt.Sprintf("/etc/%s/%s", s.baseName, systemProbeConfigFileName))
}
93 changes: 93 additions & 0 deletions unit_tests/test_install_script.sh
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,99 @@ testSystemProbeConfigPrivilegedLogsExplicitlyDisabled(){
assertEquals "$(sudo yq eval '.privileged_logs.enabled' $system_probe_config_file)" "false"
}

### DD_NO_SECURITY_AGENT_INSTALL
testNoSecurityAgentDoesNotCreateSecurityAgentConfig(){
sudo rm $security_agent_config_file 2> /dev/null
manage_security_config "sudo" $security_agent_config_file true true true
sudo test -e $security_agent_config_file
assertEquals 1 $?
}
testNoSecurityAgentFullInstall(){
# What the install script does on a fresh install with CWS and CSPM enabled
sudo rm $security_agent_config_file $system_probe_config_file 2> /dev/null
sudo cp ${config_file}.example $config_file
manage_security_config "sudo" $security_agent_config_file true true true
manage_system_probe_config "sudo" $system_probe_config_file true false "" false
run_security_in_system_probe "sudo" $config_file $security_agent_config_file $system_probe_config_file true
yamllint -c "$yaml_config" --no-warnings $config_file
assertEquals 0 $?
yamllint -c "$yaml_config" --no-warnings $system_probe_config_file
assertEquals 0 $?
assertEquals "$(sudo yq eval '.compliance_config.enabled' $config_file)" "true"
assertEquals "$(sudo yq eval '.compliance_config.run_in_system_probe' $config_file)" "true"
assertEquals "$(sudo yq eval '.runtime_security_config.enabled' $system_probe_config_file)" "true"
assertEquals "$(sudo yq eval '.runtime_security_config.direct_send_from_system_probe' $system_probe_config_file)" "true"
sudo test -e $security_agent_config_file
assertEquals 1 $?
}
testSecurityAgentSystemProbeNoDirectSend(){
sudo rm $system_probe_config_file 2> /dev/null
manage_system_probe_config "sudo" $system_probe_config_file true false "" false
yamllint -c "$yaml_config" --no-warnings $system_probe_config_file
assertEquals 0 $?
assertEquals "$(sudo yq eval '.runtime_security_config.enabled' $system_probe_config_file)" "true"
assertEquals "$(sudo yq eval '.runtime_security_config.direct_send_from_system_probe' $system_probe_config_file)" "null"
}

### Run security in system-probe
testRunSecurityInSystemProbeExistingSecurityAgentConfig(){
# A security Agent configuration file kept from a previous installation must tell the security
# Agent that both CWS and CSPM run in system-probe
sudo cp ${security_agent_config_file}.example $security_agent_config_file
sudo cp ${config_file}.example $config_file
sudo rm $system_probe_config_file 2> /dev/null
run_security_in_system_probe "sudo" $config_file $security_agent_config_file $system_probe_config_file false
yamllint -c "$yaml_config" --no-warnings $security_agent_config_file
assertEquals 0 $?
assertEquals "$(sudo yq eval '.runtime_security_config.direct_send_from_system_probe' $security_agent_config_file)" "true"
assertEquals "$(sudo yq eval '.compliance_config.run_in_system_probe' $security_agent_config_file)" "true"
assertEquals "$(sudo yq eval '.runtime_security_config.enabled' $security_agent_config_file)" "null"
assertEquals "$(sudo yq eval '.compliance_config.enabled' $security_agent_config_file)" "null"
# CSPM is not enabled, the Agent configuration file must not be updated
assertEquals "$(sudo yq eval '.compliance_config' $config_file)" "null"
}
testRunSecurityInSystemProbeKeptConfiguration(){
# CWS and CSPM enabled by a previous installation must now run in system-probe
printf 'apm_config:\n enabled: true\n' | sudo tee $config_file > /dev/null
printf 'runtime_security_config:\n enabled: true\n' | sudo tee $system_probe_config_file > /dev/null
printf 'runtime_security_config:\n enabled: true\ncompliance_config:\n enabled: true\n' | sudo tee $security_agent_config_file > /dev/null
run_security_in_system_probe "sudo" $config_file $security_agent_config_file $system_probe_config_file true
yamllint -c "$yaml_config" --no-warnings $config_file
assertEquals 0 $?
# compliance_config must be added even though the Agent configuration file was kept
assertEquals "$(sudo yq eval '.compliance_config.enabled' $config_file)" "true"
assertEquals "$(sudo yq eval '.compliance_config.run_in_system_probe' $config_file)" "true"
assertEquals "$(sudo yq eval '.apm_config.enabled' $config_file)" "true"
assertEquals "$(sudo yq eval '.runtime_security_config.direct_send_from_system_probe' $system_probe_config_file)" "true"
assertEquals "$(sudo yq eval '.runtime_security_config.enabled' $system_probe_config_file)" "true"
assertEquals "$(sudo yq eval '.runtime_security_config.direct_send_from_system_probe' $security_agent_config_file)" "true"
assertEquals "$(sudo yq eval '.compliance_config.run_in_system_probe' $security_agent_config_file)" "true"
# Running it again must not add the options a second time
run_security_in_system_probe "sudo" $config_file $security_agent_config_file $system_probe_config_file true
assertEquals 1 "$(sudo grep -c "direct_send_from_system_probe" $security_agent_config_file)"
assertEquals 1 "$(sudo grep -c "run_in_system_probe" $security_agent_config_file)"
assertEquals 1 "$(sudo grep -c "run_in_system_probe" $config_file)"
}
testRunSecurityInSystemProbeComplianceAlreadyEnabled(){
# CSPM enabled by a previous installation must run in system-probe, even when
# DD_COMPLIANCE_CONFIG_ENABLED is not set again
printf 'compliance_config:\n enabled: true\n' | sudo tee $config_file > /dev/null
sudo rm $security_agent_config_file $system_probe_config_file 2> /dev/null
run_security_in_system_probe "sudo" $config_file $security_agent_config_file $system_probe_config_file false
assertEquals "$(sudo yq eval '.compliance_config.enabled' $config_file)" "true"
assertEquals "$(sudo yq eval '.compliance_config.run_in_system_probe' $config_file)" "true"
}
testRunSecurityInSystemProbeCreatesNothing(){
sudo rm $security_agent_config_file $system_probe_config_file $config_file 2> /dev/null
run_security_in_system_probe "sudo" $config_file $security_agent_config_file $system_probe_config_file true
sudo test -e $security_agent_config_file
assertEquals 1 $?
sudo test -e $system_probe_config_file
assertEquals 1 $?
sudo test -e $config_file
assertEquals 1 $?
}

### Test logs config process collect all function
testLogsConfigProcessCollectAll() {
sudo rm $config_file 2> /dev/null
Expand Down
Loading