Skip to content

Add support for Dataproc Flexible Machine Types in compute profiles #16204

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
123-komal wants to merge 1 commit into
base: develop
Choose a base branch
from
Open

Add support for Dataproc Flexible Machine Types in compute profiles #16204

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 38 additions & 16 deletions ...-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@
import com.google.cloud.dataproc.v1.EndpointConfig;
import com.google.cloud.dataproc.v1.GceClusterConfig;
import com.google.cloud.dataproc.v1.GetClusterRequest;
import com.google.cloud.dataproc.v1.InstanceFlexibilityPolicy;
import com.google.cloud.dataproc.v1.InstanceFlexibilityPolicy.InstanceSelection;
import com.google.cloud.dataproc.v1.InstanceGroupConfig;
import com.google.cloud.dataproc.v1.LifecycleConfig;
import com.google.cloud.dataproc.v1.ListClustersRequest;
Expand Down Expand Up @@ -242,6 +244,27 @@ ClusterOperationMetadata createCluster(String name, String imageVersion,
.setPreemptibility(InstanceGroupConfig.Preemptibility.NON_PREEMPTIBLE)
.setDiskConfig(workerDiskConfig);

if (!conf.getWorkerFlexVmMachineTypes().isEmpty()) {
InstanceFlexibilityPolicy workerFlexPolicy =
createInstanceFlexibilityPolicy(conf.getWorkerFlexVmMachineTypes());
primaryWorkerConfig.setInstanceFlexibilityPolicy(workerFlexPolicy);
secondaryWorkerConfig.setInstanceFlexibilityPolicy(workerFlexPolicy);
}

InstanceGroupConfig.Builder masterConfig = InstanceGroupConfig.newBuilder()
.setNumInstances(conf.getMasterNumNodes())
.setMachineTypeUri(conf.getMasterMachineType())
.setDiskConfig(DiskConfig.newBuilder()
.setBootDiskType(conf.getMasterDiskType())
.setBootDiskSizeGb(conf.getMasterDiskGb())
.setNumLocalSsds(0)
.build());
if (!conf.getMasterFlexVmMachineTypes().isEmpty()) {
InstanceFlexibilityPolicy masterFlexPolicy =
createInstanceFlexibilityPolicy(conf.getMasterFlexVmMachineTypes());
masterConfig.setInstanceFlexibilityPolicy(masterFlexPolicy);
}

//Set default concurrency settings for fixed cluster
if (Strings.isNullOrEmpty(conf.getAutoScalingPolicy())) {
//Set spark.default.parallelism according to cluster size.
Expand Down Expand Up @@ -274,22 +297,14 @@ ClusterOperationMetadata createCluster(String name, String imageVersion,
}

ClusterConfig.Builder builder = ClusterConfig.newBuilder()
.setEndpointConfig(EndpointConfig.newBuilder()
.setEnableHttpPortAccess(conf.isComponentGatewayEnabled())
.build())
.setMasterConfig(InstanceGroupConfig.newBuilder()
.setNumInstances(conf.getMasterNumNodes())
.setMachineTypeUri(conf.getMasterMachineType())
.setDiskConfig(DiskConfig.newBuilder()
.setBootDiskType(conf.getMasterDiskType())
.setBootDiskSizeGb(conf.getMasterDiskGb())
.setNumLocalSsds(0)
.build())
.build())
.setWorkerConfig(primaryWorkerConfig.build())
.setSecondaryWorkerConfig(secondaryWorkerConfig.build())
.setGceClusterConfig(clusterConfig.build())
.setSoftwareConfig(softwareConfigBuilder);
.setEndpointConfig(EndpointConfig.newBuilder()
.setEnableHttpPortAccess(conf.isComponentGatewayEnabled())

@vsethi09 vsethi09 Aug 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: fix indentation, if broken.

.build())
.setMasterConfig(masterConfig.build())
.setWorkerConfig(primaryWorkerConfig.build())
.setSecondaryWorkerConfig(secondaryWorkerConfig.build())
.setGceClusterConfig(clusterConfig.build())
.setSoftwareConfig(softwareConfigBuilder);

//Cluster TTL if one should be set
if (conf.getIdleTtlMinutes() > 0) {
Expand Down Expand Up @@ -361,6 +376,13 @@ ClusterOperationMetadata createCluster(String name, String imageVersion,
}
}

private InstanceFlexibilityPolicy createInstanceFlexibilityPolicy(List<String> machineTypes) {
return InstanceFlexibilityPolicy.newBuilder()
.addInstanceSelectionList(
InstanceSelection.newBuilder().addAllMachineTypes(machineTypes).build())
.build();
}

protected void setNetworkConfigs(Compute compute, GceClusterConfig.Builder clusterConfig,
boolean privateInstance) throws RetryableProvisionException, IOException {
String network = conf.getNetwork();
Expand Down
72 changes: 53 additions & 19 deletions ...xt-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocConf.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/*

Check warning on line 1 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocConf.java

View workflow job for this annotation

GitHub Actions / Checkstyle

com.puppycrawl.tools.checkstyle.checks.NewlineAtEndOfFileCheck 

File does not end with a newline.
* Copyright © 2018-2020 Cask Data, Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
Expand All @@ -17,6 +17,7 @@
package io.cdap.cdap.runtime.spi.provisioner.dataproc;

import com.google.auth.oauth2.GoogleCredentials;
import com.google.common.base.Splitter;
import com.google.common.base.Strings;
import io.cdap.cdap.runtime.spi.common.DataprocUtils;
import java.io.ByteArrayInputStream;
Expand All @@ -27,11 +28,14 @@
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.LinkedHashSet;
import java.util.List;
import java.util.Map;
import java.util.Optional;

Check warning on line 37 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocConf.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused import 'java.util.Optional'. 


See more on https://sonarcloud.io/project/issues?id=cdapio_cdap&issues=AaBH_0gFxAVgc9qexrKI&open=AaBH_0gFxAVgc9qexrKI&pullRequest=16204

Check warning on line 37 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocConf.java

View workflow job for this annotation

GitHub Actions / Checkstyle

com.puppycrawl.tools.checkstyle.checks.imports.UnusedImportsCheck 

Unused import - java.util.Optional.
import java.util.Set;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
import javax.annotation.Nullable;
Expand Down Expand Up @@ -72,6 +76,7 @@
static final String TEMP_BUCKET = "tempBucket";

static final Pattern CLUSTER_PROPERTIES_PATTERN = Pattern.compile("^[a-zA-Z0-9\\-]+:");
static final String NETWORK_TAGS = "networkTags";
static final int MAX_NETWORK_TAGS = 64;

static final String SECURE_BOOT_ENABLED = "secureBootEnabled";
Expand Down Expand Up @@ -111,6 +116,12 @@
private static final String COMPUTE_CREDENTIALS_MAX_RETRIES_KEY = "compute.credentials.max.retries";
private static final int COMPUTE_CREDENTIALS_MAX_RETRIES_DEFAULT = 50;

public static final String MASTER_FLEX_VM_MACHINE_TYPES = "masterFlexVmMachineTypes";
public static final String WORKER_FLEX_VM_MACHINE_TYPES = "workerFlexVmMachineTypes";

private static final Splitter COMMA_SPLITTER =
Splitter.on(',').trimResults().omitEmptyStrings();

private final String accountKey;
private final String region;
private final String zone;
Expand All @@ -128,6 +139,7 @@
private final int masterDiskGb;
private final String masterDiskType;
private final String masterMachineType;
private final List<String> masterFlexVmMachineTypes;

private final int workerNumNodes;
private final int secondaryWorkerNumNodes;
Expand All @@ -136,6 +148,7 @@
private final int workerDiskGb;
private final String workerDiskType;
private final String workerMachineType;
private final List<String> workerFlexVmMachineTypes;

private final long pollCreateDelay;
private final long pollCreateJitter;
Expand Down Expand Up @@ -189,8 +202,10 @@
@Nullable String networkHostProjectId, @Nullable String network, @Nullable String subnet,
int masterNumNodes, int masterCpus, int masterMemoryMb,
int masterDiskGb, String masterDiskType, @Nullable String masterMachineType,
List<String> masterFlexVmMachineTypes,
int workerNumNodes, int secondaryWorkerNumNodes, int workerCpus, int workerMemoryMb,
int workerDiskGb, String workerDiskType, @Nullable String workerMachineType,
List<String> workerFlexVmMachineTypes,
long pollCreateDelay, long pollCreateJitter, long pollDeleteDelay, long pollInterval,
@Nullable String encryptionKeyName, @Nullable String gcsBucket,
@Nullable String tempBucket, @Nullable String serviceAccount, boolean preferExternalIp,
Expand Down Expand Up @@ -230,13 +245,15 @@
this.masterDiskGb = masterDiskGb;
this.masterDiskType = masterDiskType;
this.masterMachineType = masterMachineType;
this.masterFlexVmMachineTypes = masterFlexVmMachineTypes;
this.workerNumNodes = workerNumNodes;
this.secondaryWorkerNumNodes = secondaryWorkerNumNodes;
this.workerCpus = workerCpus;
this.workerMemoryMb = workerMemoryMb;
this.workerDiskGb = workerDiskGb;
this.workerDiskType = workerDiskType;
this.workerMachineType = workerMachineType;
this.workerFlexVmMachineTypes = workerFlexVmMachineTypes;
this.pollCreateDelay = pollCreateDelay;
this.pollCreateJitter = pollCreateJitter;
this.pollDeleteDelay = pollDeleteDelay;
Expand Down Expand Up @@ -339,6 +356,14 @@
return getMachineType(workerMachineType, workerCpus, workerMemoryMb);
}

public List<String> getMasterFlexVmMachineTypes() {
return formatFlexMachineTypes(masterFlexVmMachineTypes, masterCpus, masterMemoryMb);
}

public List<String> getWorkerFlexVmMachineTypes() {
return formatFlexMachineTypes(workerFlexVmMachineTypes, workerCpus, workerMemoryMb);
}

int getTotalWorkerCpus() {
if (enablePredefinedAutoScaling) {
return workerCpus
Expand Down Expand Up @@ -667,12 +692,14 @@
if (masterDiskType == null) {
masterDiskType = "pd-standard";
}
final List<String> masterFlexVmMachineTypes = getStringList(properties, MASTER_FLEX_VM_MACHINE_TYPES);
final int workerDiskGb = getInt(properties, "workerDiskGB", 1000);
String workerDiskType = getString(properties, "workerDiskType");
final String workerMachineType = getString(properties, "workerMachineType");
if (workerDiskType == null) {
workerDiskType = "pd-standard";
}
final List<String> workerFlexVmMachineTypes = getStringList(properties, WORKER_FLEX_VM_MACHINE_TYPES);

final long pollCreateDelay = getLong(properties, "pollCreateDelay", 60);
final long pollCreateJitter = getLong(properties, "pollCreateJitter", 20);
Expand Down Expand Up @@ -712,14 +739,7 @@
final Map<String, String> clusterLabels = Collections.unmodifiableMap(
DataprocUtils.parseKeyValueConfig(getString(properties, CLUSTER_LABELS), ";", "\\|"));

final String networkTagsProperty = Optional.ofNullable(getString(properties, "networkTags"))
.orElse("");
final List<String> networkTags = Collections.unmodifiableList(
Arrays.stream(networkTagsProperty.split(","))
.map(String::trim)
.filter(s -> !s.isEmpty())
.collect(Collectors.toList()));

final List<String> networkTags = getStringList(properties, NETWORK_TAGS);
if (networkTags.size() > MAX_NETWORK_TAGS) {
throw new IllegalArgumentException(
"Number of network tags cannot be more than " + MAX_NETWORK_TAGS);
Expand Down Expand Up @@ -781,21 +801,16 @@
Boolean.parseBoolean(properties.getOrDefault(DataprocUtils.LOCAL_CACHE_DISABLED,
"false"));

final String scopesProperty = String.format("%s,%s",
Optional.ofNullable(getString(properties, SCOPES)).orElse(""), CLOUD_PLATFORM_SCOPE);
List<String> scopes = Collections.unmodifiableList(
Arrays.stream(scopesProperty.split(","))
.map(String::trim)
.filter(s -> !s.isEmpty())
.distinct()
.collect(Collectors.toList()));
Set<String> scopesSet = new LinkedHashSet<>(getStringList(properties, SCOPES));
scopesSet.add(CLOUD_PLATFORM_SCOPE);
final List<String> scopes = Collections.unmodifiableList(new ArrayList<>(scopesSet));

return new DataprocConf(accountKey, region, zone, projectId, networkHostProjectId, network,
subnet,
masterNumNodes, masterCpus, masterMemoryMb, masterDiskGb,
masterDiskType, masterMachineType,
masterDiskType, masterMachineType, masterFlexVmMachineTypes,
workerNumNodes, secondaryWorkerNumNodes, workerCpus, workerMemoryMb, workerDiskGb,
workerDiskType, workerMachineType,
workerDiskType, workerMachineType, workerFlexVmMachineTypes,
pollCreateDelay, pollCreateJitter, pollDeleteDelay, pollInterval,
gcpCmekKeyName, gcpCmekBucket, tempBucket, serviceAccount, preferExternalIp,
stackdriverLoggingEnabled, stackdriverMonitoringEnabled,
Expand Down Expand Up @@ -856,4 +871,23 @@
valStr));
}
}
}

private List<String> formatFlexMachineTypes(List<String> flexTypes, int cpus, int memoryMb) {

@vsethi09 vsethi09 Aug 24, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can be simplified as:

private List<String> formatFlexMachineTypes(List<String> flexTypes, int cpus, int memoryMb) {
    return flexTypes.stream()
            .map(type -> getMachineType(type, cpus, memoryMb))
            .collect(Collectors.toUnmodifiableList());
}

@123-komal 123-komal Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

toUnmodifiableList() option was present for the given java version so I tweak it the logic that you provided too the below one

private List<String> formatFlexMachineTypes(List<String> flexTypes, int cpus, int memoryMb) {  
    List<String> result = flexTypes.stream()
      .map(type -> getMachineType(type, cpus, memoryMb))
      .collect(Collectors.toList());

    return Collections.unmodifiableList(result);
  }

List<String> result = flexTypes.stream()
.map(type -> getMachineType(type, cpus, memoryMb))
.collect(Collectors.toList());

return Collections.unmodifiableList(result);
}

/**
* Parses a comma-separated string property into a trimmed list of strings,
* or returns an empty list if null/empty.
*/
private static List<String> getStringList(Map<String, String> properties, String key) {

@sahusanket sahusanket Aug 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we check other fields that are "widget-type": "csv"
Example : networkTags ,scopes etc..

How are they parsed ?

And if this function can be made generic to be used for other such fields?

@123-komal 123-komal Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the other "widget-type": "csv" fields (networkTags, scopes, and initActions).

networkTags is a standard CSV, and we can use the same method getStringList for it. However, scopes has domain-specific logic (appending CLOUD_PLATFORM_SCOPE and deduplicating), and initActions has its own constructor and getter lifecycle. I keep their update logic same and it's logic is different from the getStringList.

String val = getString(properties, key);
return Strings.isNullOrEmpty(val)
? Collections.emptyList()
: COMMA_SPLITTER.splitToList(val);
}
}
23 changes: 23 additions & 0 deletions ...proc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocProvisioner.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
package io.cdap.cdap.runtime.spi.provisioner.dataproc;

import com.google.api.gax.grpc.GrpcStatusCode;
import com.google.api.gax.rpc.ApiException;

Check warning on line 20 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocProvisioner.java

View workflow job for this annotation

GitHub Actions / Checkstyle

com.puppycrawl.tools.checkstyle.checks.imports.UnusedImportsCheck 

Unused import - com.google.api.gax.rpc.ApiException.
import com.google.api.gax.rpc.StatusCode;
import com.google.cloud.dataproc.v1.ClusterOperationMetadata;
import com.google.cloud.dataproc.v1.ClusterStatus.State;
Expand All @@ -42,9 +42,11 @@
import io.cdap.cdap.runtime.spi.ssh.SSHContext;
import io.cdap.cdap.runtime.spi.ssh.SSHKeyPair;
import io.cdap.cdap.runtime.spi.ssh.SSHPublicKey;
import java.util.ArrayList;

Check warning on line 45 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocProvisioner.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Remove this unused import 'java.util.ArrayList'. 


See more on https://sonarcloud.io/project/issues?id=cdapio_cdap&issues=AaBH_0iDxAVgc9qexrKJ&open=AaBH_0iDxAVgc9qexrKJ&pullRequest=16204

Check warning on line 45 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocProvisioner.java

View workflow job for this annotation

GitHub Actions / Checkstyle

com.puppycrawl.tools.checkstyle.checks.imports.UnusedImportsCheck 

Unused import - java.util.ArrayList.
import java.util.Collections;
import java.util.EnumSet;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Optional;
Expand All @@ -57,7 +59,7 @@
import java.util.regex.Pattern;
import javax.annotation.Nullable;

import io.grpc.Status;

Check warning on line 62 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocProvisioner.java

View workflow job for this annotation

GitHub Actions / Checkstyle

com.puppycrawl.tools.checkstyle.checks.imports.CustomImportOrderCheck 

Extra separation in import group before 'io.grpc.Status'

Check warning on line 62 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocProvisioner.java

View workflow job for this annotation

GitHub Actions / Checkstyle

com.puppycrawl.tools.checkstyle.checks.imports.CustomImportOrderCheck 

Wrong lexicographical order for 'io.grpc.Status' import. Should be before 'javax.annotation.Nullable'.
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand Down Expand Up @@ -85,6 +87,9 @@
private static final Set<ClusterStatus> TERMINAL_STATES =
EnumSet.of(ClusterStatus.RUNNING, ClusterStatus.FAILED, ClusterStatus.NOT_EXISTS);

private static final Pattern MACHINE_TYPE_PATTERN =
Pattern.compile("^[a-z\\d]+(-[a-z\\d]+)+$");

Check warning on line 91 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocProvisioner.java

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Refactor this repetition that can lead to a stack overflow for large inputs. 


See more on https://sonarcloud.io/project/issues?id=cdapio_cdap&issues=AaAi9DGMb7Yt1os_-aUx&open=AaAi9DGMb7Yt1os_-aUx&pullRequest=16204

private final DataprocClientFactory clientFactory;

@SuppressWarnings("WeakerAccess")
Expand All @@ -101,8 +106,26 @@
@Override
public void validateProperties(Map<String, String> properties) {
DataprocConf conf = DataprocConf.create(properties);
boolean privateInstance = Boolean.parseBoolean(

Check warning on line 109 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocProvisioner.java

View workflow job for this annotation

GitHub Actions / Checkstyle

com.puppycrawl.tools.checkstyle.checks.coding.VariableDeclarationUsageDistanceCheck 

Distance between variable 'privateInstance' declaration and its first usage is 4, but allowed 3.  Consider making that variable final if you still need to store its value in advance (before method calls that might have side effects on the original value).
getSystemContext().getProperties().get(PRIVATE_INSTANCE));
Set<String> allFlexTypes = new HashSet<>();
allFlexTypes.addAll(conf.getMasterFlexVmMachineTypes());
allFlexTypes.addAll(conf.getWorkerFlexVmMachineTypes());

for (String machineType : allFlexTypes) {
if (!MACHINE_TYPE_PATTERN.matcher(machineType).matches()) {
String errorMessage = String.format(
"Invalid flexible VM machine type '%s'. "
+ "Machine types should follow standard GCP format.",
machineType);
throw new DataprocRuntimeException.Builder()
.withErrorCategory(DataprocRuntimeException.ERROR_CATEGORY_PROVISIONING_CONFIGURATION)
.withErrorReason(errorMessage)
.withErrorMessage(errorMessage)
.withErrorType(ErrorType.USER)
.build();
}
}

if (privateInstance && conf.isPreferExternalIp()) {
// When prefer external IP is set to true it means only Dataproc external ip can be used for
Expand Down Expand Up @@ -429,7 +452,7 @@
* @param clusterName cluster name
* @param runLabels map with labels to look for the cluster
* @throws Exception if wait failed, interrupted or cluster can't be found even after operation
* finished.

Check warning on line 455 in cdap-runtime-ext-dataproc/src/main/java/io/cdap/cdap/runtime/spi/provisioner/dataproc/DataprocProvisioner.java

View workflow job for this annotation

GitHub Actions / Checkstyle

com.puppycrawl.tools.checkstyle.checks.javadoc.JavadocTagContinuationIndentationCheck 

Line continuation have incorrect indentation level, expected level should be 4.
*/
private static void waitForLabelsUpdateToApply(DataprocClient client, DataprocConf conf,
Future<?> updateLabelsFuture, String clusterName, Map<String, String> runLabels) throws Exception {
Expand Down
28 changes: 28 additions & 0 deletions cdap-runtime-ext-dataproc/src/main/resources/gcp-dataproc.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,6 +170,20 @@
"size": "medium"
}
},
{
"widget-type": "multi-select",
Comment thread
123-komal marked this conversation as resolved.
"label": "Master Flexible Machine Types",
"name": "masterFlexVmMachineTypes",
"description": "Optional comma-separated list of fallback machine types in priority order if the primary master machine type is unavailable.",
"widget-attributes": {
"options": [
"n1",
"n2",
"n2d",
"e2"

@vsethi09 vsethi09 Aug 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should n4 be present in the list?

@vsethi09 vsethi09 Aug 24, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Below as well.

@123-komal 123-komal Aug 28, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have kept the supported option same as the one with we listed in masterMachineType. But we can still n4 during runtime, which I also tested and it worked.

]
}
},
{
"widget-type": "select",
"label": "Master Cores",
Expand Down Expand Up @@ -254,6 +268,20 @@
"size": "medium"
}
},
{
"widget-type": "multi-select",
"label": "Worker Flexible Machine Types",
"name": "workerFlexVmMachineTypes",
"description": "Machine types in priority order if the primary worker machine type is unavailable.",
"widget-attributes": {
"options": [
"n1",
"n2",
"n2d",
"e2"
]
}
},
{
"widget-type": "select",
"label": "Worker Cores",
Expand Down
Loading
Loading