kata-deploy: Document drop-in configuration and add warning to config files

When kata-deploy installs Kata Containers, the base configuration files
should not be modified directly. This change adds documentation explaining
how to use drop-in configuration files for customization, and prepends a
warning comment to all deployed configuration files reminding users to use
drop-in files instead.

The warning is added to both standard shim configurations and custom
runtime configurations. It includes a brief explanation of how drop-in
files work and points users to the documentation for more details.

Signed-off-by: Fabiano Fidêncio <ffidencio@nvidia.com>

tools/packaging/kata-deploy/binary/src/artifacts/install.rs

@@ -216,6 +216,9 @@ fn install_custom_runtime_configs(config: &Config) -> Result<()> {
                 )
             })?;
 
+            // Add warning comment to inform users about drop-in files
+            add_kata_deploy_warning(Path::new(&dest_config))?;
+
             info!(
                 "Copied config for custom runtime {}: {} -> {}",
                 runtime.handler, original_config, dest_config
@@ -342,6 +345,41 @@ fn set_executable_permissions(dir: &str) -> Result<()> {
     Ok(())
 }
 
+/// Warning comment to prepend to configuration files installed by kata-deploy.
+/// This informs users to use drop-in files instead of modifying the base config directly.
+const KATA_DEPLOY_CONFIG_WARNING: &str = r#"# =============================================================================
+# IMPORTANT: This file is managed by kata-deploy. Do not modify it directly!
+#
+# To customize settings, create drop-in configuration files in the config.d/
+# directory alongside this file. Drop-in files are processed in alphabetical
+# order, with later files overriding earlier settings.
+#
+# Example: config.d/50-custom.toml
+#
+# See the kata-deploy documentation for more details:
+#   https://github.com/kata-containers/kata-containers/tree/main/tools/packaging/kata-deploy
+# =============================================================================
+
+"#;
+
+/// Prepends the kata-deploy warning comment to a configuration file.
+/// This informs users to use drop-in files instead of modifying the base config.
+fn add_kata_deploy_warning(config_file: &Path) -> Result<()> {
+    let content = fs::read_to_string(config_file)
+        .with_context(|| format!("Failed to read config file: {:?}", config_file))?;
+
+    // Check if the warning is already present (idempotency)
+    if content.contains("IMPORTANT: This file is managed by kata-deploy") {
+        return Ok(());
+    }
+
+    let new_content = format!("{}{}", KATA_DEPLOY_CONFIG_WARNING, content);
+    fs::write(config_file, new_content)
+        .with_context(|| format!("Failed to write config file: {:?}", config_file))?;
+
+    Ok(())
+}
+
 /// Set up the runtime directory structure for a shim.
 /// Creates: {config_path}/runtimes/{shim}/
 ///          {config_path}/runtimes/{shim}/config.d/
@@ -385,6 +423,9 @@ fn setup_runtime_directory(config: &Config, shim: &str) -> Result<()> {
         // Copy the base config file
         fs::copy(&original_config_file, &dest_config_file)
             .with_context(|| format!("Failed to copy config: {} -> {}", original_config_file, dest_config_file))?;
+
+        // Add warning comment to inform users about drop-in files
+        add_kata_deploy_warning(Path::new(&dest_config_file))?;
         
         info!("  Copied base config: {}", dest_config_file);
     }

tools/packaging/kata-deploy/helm-chart/README.md

@@ -448,3 +448,65 @@ kata-qemu-snp-cicd              kata-qemu-snp-cicd              77s
 kata-qemu-tdx-cicd              kata-qemu-tdx-cicd              77s
 kata-stratovirt-cicd            kata-stratovirt-cicd            77s
 ```
+
+## Customizing Configuration with Drop-in Files
+
+When kata-deploy installs Kata Containers, the base configuration files should not
+be modified directly. Instead, use drop-in configuration files to customize
+settings. This approach ensures your customizations survive kata-deploy upgrades.
+
+### How Drop-in Files Work
+
+The Kata runtime reads the base configuration file and then applies any `.toml`
+files found in the `config.d/` directory alongside it. Files are processed in
+alphabetical order, with later files overriding earlier settings.
+
+### Creating Custom Drop-in Files
+
+To add custom settings, create a `.toml` file in the appropriate `config.d/`
+directory. Use a numeric prefix to control the order of application.
+
+**Reserved prefixes** (used by kata-deploy):
+- `10-*`: Core kata-deploy settings
+- `20-*`: Debug settings
+- `30-*`: Kernel parameters
+
+**Recommended prefixes for custom settings**: `50-89`
+
+### Example: Adding Custom Kernel Parameters
+
+```bash
+# SSH into the node or use kubectl exec
+sudo mkdir -p /opt/kata/share/defaults/kata-containers/runtimes/qemu/config.d/
+sudo cat > /opt/kata/share/defaults/kata-containers/runtimes/qemu/config.d/50-custom.toml << 'EOF'
+[hypervisor.qemu]
+kernel_params = "my_param=value"
+EOF
+```
+
+### Example: Changing Default Memory Size
+
+```bash
+sudo cat > /opt/kata/share/defaults/kata-containers/runtimes/qemu/config.d/50-memory.toml << 'EOF'
+[hypervisor.qemu]
+default_memory = 4096
+EOF
+```
+
+### Custom Runtimes
+
+For more complex customizations, you can define custom runtimes in your Helm
+values. Custom runtimes create isolated configuration directories with their
+own drop-in files:
+
+```yaml
+customRuntimes:
+  enabled: true
+  runtimes:
+    - handler: kata-custom
+      baseConfig: qemu
+      dropInFile: /path/to/your/config.toml
+```
+
+This creates a new Runtime Class `kata-custom` that extends the `qemu`
+configuration with your custom settings.