If I enable SASL/IAM, SASL/SCRAM, and TLS mutual auth all at once, which one does a client actually use?

Each client connection authenticates via exactly one of the enabled mechanisms, chosen by how that specific client is configured to connect, not a fallback chain or combined check. Enabling multiple mechanisms simultaneously supports a heterogeneous population of clients using different auth methods (useful during a migration from SCRAM to IAM, for instance), not multi-factor authentication on a single connection.

Does increasing ProvisionedThroughput require increasing VolumeSize too?

No, they're independent settings, similar to gp3 EBS volumes generally, you can provision more throughput/IOPS without necessarily needing more raw storage capacity, and vice versa. This matters for right-sizing cost, a workload needing high throughput but modest total storage doesn't need to over-provision capacity just to unlock more throughput headroom.

What's the cost implication of PER_TOPIC_PER_PARTITION enhanced monitoring versus the default?

It publishes a meaningfully larger number of CloudWatch metrics (broken out per topic and per partition rather than aggregated at the broker or cluster level), which directly increases CloudWatch metrics cost proportional to your topic/partition count. It's valuable for actively diagnosing a specific partition-level issue but isn't usually left on as a permanent default for a cluster with many topics and partitions, given the cost scales with that count.

AWS MSK Cluster Config Builder

MonitoringAWS

Build MSK (Kafka) cluster configurations with broker sizing, authentication, encryption, monitoring, and logging.

Amazon MSK Configuration

Build MSK (Kafka) cluster configurations with broker sizing, authentication, encryption, monitoring, and logging.

Required Fields

ClusterNameKafkaVersionNumberOfBrokerNodesBrokerNodeGroupInfo

{
  "ClusterName": "prod-event-streaming",
  "KafkaVersion": "3.6.0",
  "NumberOfBrokerNodes": 6,
  "BrokerNodeGroupInfo": {
    "InstanceType": "kafka.m5.2xlarge",
    "ClientSubnets": [
      "subnet-0abc123def456789a",
      "subnet-0abc123def456789b",
      "subnet-0abc123def456789c"
    ],
    "SecurityGroups": ["sg-0abc123def456789a"],
    "StorageInfo": {
      "EbsStorageInfo": {
        "VolumeSize": 500,
        "ProvisionedThroughput": {
          "Enabled": true,
          "VolumeThroughput": 250
        }
      }
    },
    "ConnectivityInfo": {
      "PublicAccess": {
        "Type": "DISABLED"
      },
      "VpcConnectivity": {
        "ClientAuthentication": {
          "Sasl": {
            "Iam": {"Enabled": true},
            "Scram": {"Enabled": true}
          },
          "Tls": {"Enabled": true}
        }
      }
    }
  },
  "ClientAuthentication": {
    "Sasl": {
      "Iam": {"Enabled": true},
      "Scram": {"Enabled": true}
    },
    "Tls": {
      "CertificateAuthorityArnList": [
        "arn:aws:acm-pca:us-east-1:123456789012:certificate-authority/abcd1234"
      ],
      "Enabled": true
    }
  },
  "EncryptionInfo": {
    "EncryptionAtRest": {
      "DataVolumeKMSKeyId": "arn:aws:kms:us-east-1:123456789012:key/abcd1234"
    },
    "EncryptionInTransit": {
      "ClientBroker": "TLS",
      "InCluster": true
    }
  },
  "ConfigurationInfo": {
    "Arn": "arn:aws:kafka:us-east-1:123456789012:configuration/prod-config/abcd1234",
    "Revision": 3
  },
  "LoggingInfo": {
    "BrokerLogs": {
      "CloudWatchLogs": {
        "Enabled": true,
        "LogGroup": "/aws/msk/prod-event-streaming"
      },
      "S3": {
        "Enabled": true,
        "Bucket": "msk-broker-logs",
        "Prefix": "logs/prod/"
      }
    }
  },
  "OpenMonitoring": {
    "Prometheus": {
      "JmxExporter": {"EnabledInBroker": true},
      "NodeExporter": {"EnabledInBroker": true}
    }
  },
  "EnhancedMonitoring": "PER_TOPIC_PER_PARTITION",
  "Tags": {
    "Environment": "production",
    "Service": "event-streaming"
  }
}

Generated Output

Output will appear here...

What This Tool Does

Build an Amazon MSK (managed Kafka) cluster with per-broker EBS storage and optional ProvisionedThroughput (independent of volume size, since gp3-style EBS lets you provision IOPS/throughput separately from capacity), multiple concurrent ClientAuthentication mechanisms (SASL/IAM, SASL/SCRAM, mutual TLS all enabled simultaneously), and EnhancedMonitoring at PER_TOPIC_PER_PARTITION granularity for detailed per-partition metrics at a real CloudWatch cost. Enabling multiple authentication mechanisms simultaneously (as the example does) doesn't mean clients get to pick loosely, each client connection must fully authenticate via exactly one configured mechanism, and having all three enabled is about supporting a heterogeneous client population (some using IAM, some using SCRAM, some using mTLS) during a migration, not about layering multiple auth checks on one connection.

Technical Details

The builder validates that ClusterName, KafkaVersion, NumberOfBrokerNodes, and BrokerNodeGroupInfo all resolve before accepting the JSON as a valid CreateCluster request, the fields MSK needs to know the cluster's identity, Kafka version, broker count, and network/storage/instance configuration; it can't verify the referenced subnet/security group/KMS key/certificate authority ARNs actually exist, those checks happen only against the live account.

Common Use Cases

1Enabling ProvisionedThroughput on the EBS storage independently of VolumeSize, so a workload needing high sustained throughput per broker doesn't have to over-provision raw storage capacity just to get more IOPS/throughput headroom.
2Enabling both SASL/IAM and SASL/SCRAM authentication simultaneously during a migration where some existing clients use SCRAM credentials and newer clients are being migrated to IAM-based auth, letting both client populations connect during the transition rather than requiring a hard cutover.
3Setting EnhancedMonitoring to PER_TOPIC_PER_PARTITION when diagnosing a specific hot-partition or consumer-lag issue that requires visibility below the broker level, accepting the higher CloudWatch metrics cost that comes with that granularity.
4Disabling PublicAccess and requiring VPC-based ClientAuthentication exclusively, so the cluster is only reachable from within the VPC (or peered/connected networks), never exposed to the public internet regardless of client authentication method.

Common Questions

If I enable SASL/IAM, SASL/SCRAM, and TLS mutual auth all at once, which one does a client actually use?: Each client connection authenticates via exactly one of the enabled mechanisms, chosen by how that specific client is configured to connect, not a fallback chain or combined check. Enabling multiple mechanisms simultaneously supports a heterogeneous population of clients using different auth methods (useful during a migration from SCRAM to IAM, for instance), not multi-factor authentication on a single connection.
Does increasing ProvisionedThroughput require increasing VolumeSize too?: No, they're independent settings, similar to gp3 EBS volumes generally, you can provision more throughput/IOPS without necessarily needing more raw storage capacity, and vice versa. This matters for right-sizing cost, a workload needing high throughput but modest total storage doesn't need to over-provision capacity just to unlock more throughput headroom.
What's the cost implication of PER_TOPIC_PER_PARTITION enhanced monitoring versus the default?: It publishes a meaningfully larger number of CloudWatch metrics (broken out per topic and per partition rather than aggregated at the broker or cluster level), which directly increases CloudWatch metrics cost proportional to your topic/partition count. It's valuable for actively diagnosing a specific partition-level issue but isn't usually left on as a permanent default for a cluster with many topics and partitions, given the cost scales with that count.

Expert Tips

TIP

Supporting multiple ClientAuthentication mechanisms simultaneously is a legitimate and common migration pattern, but plan to eventually consolidate to one primary mechanism, permanently running with three enabled auth paths widens the security surface area unnecessarily once the migration that justified it is complete.

TIP

ProvisionedThroughput is a real lever for cost-efficient sizing, don't default to over-provisioning EBS volume size just to get more throughput, tune throughput independently based on actual observed broker I/O patterns.

TIP

PER_TOPIC_PER_PARTITION enhanced monitoring is a diagnostic tool, not a permanent default, for a cluster with a large number of topics/partitions, leaving it on indefinitely can become a meaningful, easily-overlooked line item in the CloudWatch bill.

Related Learning Guides

DynamoDB Data Modeling28 min read
Cost Optimization Strategies26 min read

Was this tool helpful?

Disclaimer: This tool runs entirely in your browser. No data is sent to our servers. Always verify outputs before using them in production. AWS, Azure, and GCP are trademarks of their respective owners.

AWS MSK Cluster Config Builder

MonitoringAWS

Build MSK (Kafka) cluster configurations with broker sizing, authentication, encryption, monitoring, and logging.

Amazon MSK Configuration

Build MSK (Kafka) cluster configurations with broker sizing, authentication, encryption, monitoring, and logging.

Required Fields

ClusterNameKafkaVersionNumberOfBrokerNodesBrokerNodeGroupInfo

Generated Output

Output will appear here...

What This Tool Does

Technical Details

Common Use Cases

1Enabling ProvisionedThroughput on the EBS storage independently of VolumeSize, so a workload needing high sustained throughput per broker doesn't have to over-provision raw storage capacity just to get more IOPS/throughput headroom.
2Enabling both SASL/IAM and SASL/SCRAM authentication simultaneously during a migration where some existing clients use SCRAM credentials and newer clients are being migrated to IAM-based auth, letting both client populations connect during the transition rather than requiring a hard cutover.
3Setting EnhancedMonitoring to PER_TOPIC_PER_PARTITION when diagnosing a specific hot-partition or consumer-lag issue that requires visibility below the broker level, accepting the higher CloudWatch metrics cost that comes with that granularity.
4Disabling PublicAccess and requiring VPC-based ClientAuthentication exclusively, so the cluster is only reachable from within the VPC (or peered/connected networks), never exposed to the public internet regardless of client authentication method.

Common Questions

If I enable SASL/IAM, SASL/SCRAM, and TLS mutual auth all at once, which one does a client actually use?: Each client connection authenticates via exactly one of the enabled mechanisms, chosen by how that specific client is configured to connect, not a fallback chain or combined check. Enabling multiple mechanisms simultaneously supports a heterogeneous population of clients using different auth methods (useful during a migration from SCRAM to IAM, for instance), not multi-factor authentication on a single connection.
Does increasing ProvisionedThroughput require increasing VolumeSize too?: No, they're independent settings, similar to gp3 EBS volumes generally, you can provision more throughput/IOPS without necessarily needing more raw storage capacity, and vice versa. This matters for right-sizing cost, a workload needing high throughput but modest total storage doesn't need to over-provision capacity just to unlock more throughput headroom.
What's the cost implication of PER_TOPIC_PER_PARTITION enhanced monitoring versus the default?: It publishes a meaningfully larger number of CloudWatch metrics (broken out per topic and per partition rather than aggregated at the broker or cluster level), which directly increases CloudWatch metrics cost proportional to your topic/partition count. It's valuable for actively diagnosing a specific partition-level issue but isn't usually left on as a permanent default for a cluster with many topics and partitions, given the cost scales with that count.

Expert Tips

TIP

Related Learning Guides

DynamoDB Data Modeling28 min read
Cost Optimization Strategies26 min read

Was this tool helpful?