Build OCI Streaming (Kafka-compatible) stream pool and stream configurations.
Last verified: May 2026
Build OCI Streaming (Kafka-compatible) stream pool, stream, and connector configurations.
Required Fields
compartmentIdstreamPoolstreamsOutput will appear here...OCI Streaming is a managed Apache Kafka-compatible event streaming platform for ingesting and processing high-volume, real-time data streams. Streams organize messages into partitions for parallel consumption, with configurable retention periods and consumer group semantics. This builder helps you configure streams with partition counts, retention settings, consumer group definitions, and integration patterns for connecting to OCI Functions, Service Connector Hub, and custom consumers.
OCI Streaming is Kafka-compatible at the API level, so existing Kafka clients (producers and consumers) work with minimal configuration changes. The key differences are: Streaming is fully managed (no brokers to patch or scale), automatically handles replication and failover, charges per PUT request and data retention rather than per-broker-hour, and integrates natively with OCI services via Service Connector Hub. Self-managed Kafka offers more tuning control, custom plugins, and Kafka Connect ecosystem but requires significant operational overhead.
Partitions determine parallelism — each partition can be consumed by one consumer in a consumer group. Choose partitions based on your target throughput and consumer count. A single partition supports approximately 1 MB/s write and 2 MB/s read. If you need 10 MB/s write throughput, provision at least 10 partitions. Also consider the number of consumers — you need at least as many partitions as consumers in a group. Partitions can be increased but never decreased, so start conservatively.
Your team is replacing a self-managed Apache Kafka cluster (3 brokers, ~$600/month + significant ops time) with OCI Streaming. The builder generates: a stream pool for resource sharing, 8-partition stream with 24-hour retention, Kafka client config snippets for existing Java/Python producers and consumers (minimal changes — just URL and SASL config), Service Connector Hub forwarding events to Object Storage for archival. Cost drops to ~$200/month for equivalent throughput, and the operational burden of running brokers disappears.
The builder constructs OCI Streaming configurations: stream resource (compartment, name, partitions, retention hours, stream pool reference for shared resources), consumer group definitions, and Service Connector Hub bindings for automatic event routing to Object Storage / Functions / Notifications / other streams. Output is generated as oci streaming commands and Terraform oci_streaming_stream + oci_streaming_stream_pool + oci_sch_service_connector resources.
OCI Streaming is Kafka API-compatible (KafkaProducer/KafkaConsumer Java clients work with minor config changes), but it's NOT 100% Kafka. Some advanced features (transactional producers, Kafka Connect, KSQL) aren't supported. Test compatibility before committing migrations from existing Kafka stacks.
Partitions can be increased but NEVER decreased. Start conservatively — typically 4-8 partitions for moderate throughput, scale up as needed. Over-provisioning partitions on day 1 limits future flexibility and adds per-partition operational overhead.
Service Connector Hub eliminates most consumer code. Need to forward stream events to Object Storage, Functions, or another stream? Configure a Service Connector instead of writing+running consumer apps. Free tier covers a lot of integration patterns.
Was this tool helpful?
Disclaimer: This tool runs entirely in your browser. No data is sent to our servers. Always verify outputs before using them in production. AWS, Azure, and GCP are trademarks of their respective owners.