Skip to content

Commit

Permalink
fix after preview
Browse files Browse the repository at this point in the history
  • Loading branch information
aimurphy committed Oct 16, 2024
1 parent 046bbc1 commit 7b5e82f
Show file tree
Hide file tree
Showing 15 changed files with 105 additions and 100 deletions.
7 changes: 4 additions & 3 deletions modules/ROOT/pages/astream-faq.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,12 @@ See xref:operations:astream-pricing.adoc[].

== Why is {product} based on Apache Pulsar?

See our https://www.datastax.com/blog/four-reasons-why-apache-pulsar-essential-modern-data-stack[blog post] that explains why we are excited about Apache Pulsar and why we decided it was the best technology to base {product} on.
For information about the decision to use Apache Pulsar, see https://www.datastax.com/blog/four-reasons-why-apache-pulsar-essential-modern-data-stack[Four Reasons Why Apache Pulsar is Essential to the Modern Data Stack].

== What will happen to Kesque?
== What happened to Kesque?

{product} is based heavily on technology originally created as part of Kesque. With the launch of {product} we will begin the process of shutting down the Kesque service and migrating customers to the new {product} platform.
{product} is based heavily on technology originally created as part of Kesque.
With the launch of {product}, {company} began shutting down the Kesque service and migrated customers to {product}.

== Who should use {product}?

Expand Down
2 changes: 1 addition & 1 deletion modules/ROOT/pages/astream-subscriptions-exclusive.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -85,4 +85,4 @@ Caused by: org.apache.pulsar.client.api.PulsarClientException$ConsumerBusyExcept
* xref:astream-subscriptions.adoc[Subscriptions in Pulsar]
* xref:astream-subscriptions-shared.adoc[Shared subscriptions]
* xref:astream-subscriptions-failover.adoc[Failover subscriptions]
* xref:astream-subscriptions-keyshared.adoc[Key_shared subscriptions]
* xref:astream-subscriptions-keyshared.adoc[Key shared subscriptions]
16 changes: 9 additions & 7 deletions modules/ROOT/pages/astream-subscriptions-failover.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,14 @@ If the primary consumer disconnects, the standby consumers begin consuming the s

This page explains how to use Pulsar's failover subscription model to manage your topic consumption.

.Failover subscription video
[%collapsible]
====
This video from the *Five Minutes About Pulsar* series demonstrates failover subscriptions:

video::ckB87OLs5eM[youtube, list=PL2g2h-wyI4SqeKH16czlcQ5x4Q_z-X7_m, height=445px,width=100%]
====

include::ROOT:partial$subscription-prereq.adoc[]

[#example]
Expand Down Expand Up @@ -84,15 +92,9 @@ In the second `SimplePulsarConsumer` terminal, the backup consumer begins consum
You can configure as many backup consumers as you like.
To test them, you can progressively end each `SimplePulsarConsumer` process, and then check that the next backup consumer has begun receiving messages.

=== Failover subscription video

Follow along with this video from our *Five Minutes About Pulsar* series to see failover subscriptions in action:

video::ckB87OLs5eM[youtube, list=PL2g2h-wyI4SqeKH16czlcQ5x4Q_z-X7_m, height=445px,width=100%]

== See also

* xref:astream-subscriptions.adoc[Subscriptions in Pulsar]
* xref:astream-subscriptions-exclusive.adoc[Exclusive subscriptions]
* xref:astream-subscriptions-shared.adoc[Shared subscriptions]
* xref:astream-subscriptions-keyshared.adoc[Key_shared subscriptions]
* xref:astream-subscriptions-keyshared.adoc[Key shared subscriptions]
40 changes: 19 additions & 21 deletions modules/ROOT/pages/astream-subscriptions-keyshared.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
= Key shared subscriptions in Pulsar
:navtitle: Key_Shared
:navtitle: Key shared
:page-tag: pulsar-subscriptions,quickstart,admin,dev,pulsar

_Subscriptions_ in Pulsar describe which consumers are consuming data from a topic and how they want to consume that data.
Expand All @@ -15,13 +15,21 @@ Keys are generated with hashing that converts arbitrary values like `topic-name`
This page explains how to use Pulsar's key shared subscription model to manage your topic consumption.

.Key shared subscription video
[%collapsible]
====
This video from the *Five Minutes About Pulsar* series demonstrates key shared subscriptions:

video::_49wlA53L_8[youtube, list=PL2g2h-wyI4SqeKH16czlcQ5x4Q_z-X7_m, height=445px,width=100%]
====

include::ROOT:partial$subscription-prereq.adoc[]

[#example]
== Key shared subscription example

To try out a Pulsar key shared subscription, add `.subscriptionType(SubscriptionType.Key_Shared)` to the `pulsarConsumer` in `SimplePulsarConsumer.java`:

. To try out a Pulsar key shared subscription, add `.subscriptionType(SubscriptionType.Key_Shared)` to the `pulsarConsumer` in `SimplePulsarConsumer.java`:
+
.SimplePulsarConsumer.java
[source,java]
----
Expand All @@ -37,19 +45,15 @@ pulsarConsumer = pulsarClient.newConsumer(Schema.JSON(DemoBean.class))
.keySharedPolicy(KeySharedPolicy.autoSplitHashRange())
.subscribe();
----

=== keySharedPolicy

+
The `keySharedPolicy` defines the how hashed values are assigned to subscribed consumers.
+
The above example uses `autoSplitHashRange`, which is an auto-hashing policy.
Running multiple consumers with auto-hashing balances the messaging load across all available consumers, like a xref:astream-subscriptions-shared.adoc[shared subscription]
+
If you want to manually set a hash range, use `KeySharedPolicy.stickyHashRange()`, as demonstrated in the following steps.

The above example used `autoSplitHashRange`, which is an auto-hashing policy.
Running multiple consumers with auto-hashing balances the messaging load across all available consumers.

If you want to manually set a hash range, use `KeySharedPolicy.stickyHashRange()`.

==== Test sticky hashed key shared subscriptions

. To test out sticky hashed key shared subscriptions, import the following additional classes:
. To use a sticky hashed key shared subscription, import the following classes to `SimplePulsarConsumer.java`:
+
.SimplePulsarConsumer.java
[source,java]
Expand Down Expand Up @@ -143,15 +147,9 @@ Caused by: org.apache.pulsar.client.api.PulsarClientException$ConsumerAssignExce
at com.datastax.pulsar.SimplePulsarConsumer.main(SimplePulsarConsumer.java:47)
----

. To run multiple consumers with sticky hashing, modify the `SimplePulsarConsumer.java` configuration to split the hash range between consumers.
. To run multiple consumers with sticky hashing, modify the `SimplePulsarConsumer.java` configuration to split the hash range between consumers or use auto-hashing.
Then, you can launch multiple instances of `SimplePulsarConsumer.java` to consume messages from different hash ranges.

== Key shared subscription video

Follow along with this video from our *Five Minutes About Pulsar* series to see key shared subscriptions in action:

video::_49wlA53L_8[youtube, list=PL2g2h-wyI4SqeKH16czlcQ5x4Q_z-X7_m, height=445px,width=100%]

== See also

* xref:astream-subscriptions.adoc[Subscriptions in Pulsar]
Expand Down
16 changes: 9 additions & 7 deletions modules/ROOT/pages/astream-subscriptions-shared.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,14 @@ However, there is a risk of losing message ordering guarantees and acknowledgeme

This page explains how you can use Pulsar's shared subscription model to manage your topic consumption.

.Shared subscription video
[%collapsible]
====
This video from the *Five Minutes About Pulsar* series demonstrates shared subscriptions:

video::mmukXqGsauA[youtube, list=PL2g2h-wyI4SqeKH16czlcQ5x4Q_z-X7_m, height=445px,width=100%]
====

include::ROOT:partial$subscription-prereq.adoc[]

[#example]
Expand Down Expand Up @@ -83,15 +91,9 @@ If you run this test with xref:astream-subscriptions-exclusive.adoc[exclusive su
To continue testing the shared subscription configuration, you can continue running new instances of `SimplePulsarConsumer.java` in new temrinal windows.
All the consumers subscribe to the topic and consume messages in a round-robin fashion.

== Shared subscription video

Follow along with this video from our *Five Minutes About Pulsar* series to see shared subscriptions in action:

video::mmukXqGsauA[youtube, list=PL2g2h-wyI4SqeKH16czlcQ5x4Q_z-X7_m, height=445px,width=100%]

== See also

* xref:astream-subscriptions.adoc[Subscriptions in Pulsar]
* xref:astream-subscriptions-exclusive.adoc[Exclusive subscriptions]
* xref:astream-subscriptions-failover.adoc[Failover subscriptions]
* xref:astream-subscriptions-keyshared.adoc[Key_shared subscriptions]
* xref:astream-subscriptions-keyshared.adoc[Key shared subscriptions]
2 changes: 1 addition & 1 deletion modules/ROOT/pages/astream-subscriptions.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ pulsarConsumer = pulsarClient.newConsumer(Schema.BYTES)
* xref:astream-subscriptions-exclusive.adoc[Exclusive subscriptions]
* xref:astream-subscriptions-shared.adoc[Shared subscriptions]
* xref:astream-subscriptions-failover.adoc[Failover subscriptions]
* xref:astream-subscriptions-keyshared.adoc[Key_shared subscriptions]
* xref:astream-subscriptions-keyshared.adoc[Key shared subscriptions]

== See also

Expand Down
2 changes: 1 addition & 1 deletion modules/developing/nav.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -23,4 +23,4 @@
* xref:gpt-schema-translator.adoc[]
* xref:astra-cli.adoc[]
* xref:astream-cdc.adoc[]
* xref:streaming-learning:pulsar-io:connectors/index.adoc[IO Connectors]
* xref:streaming-learning:pulsar-io:connectors/index.adoc[IO connectors]
17 changes: 9 additions & 8 deletions modules/developing/pages/astream-functions.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,7 @@ Must have 4 steps to maintain numbering.
////
======

[start=5]
. If you haven't done so already, xref:configure-pulsar-env.adoc[set up your environment for the Pulsar binaries].

. Create a deployment configuration YAML file that defines the function metadata and associated topics:
Expand Down Expand Up @@ -262,7 +263,7 @@ Organizations on the *Free* plan can use xref:streaming-learning:functions:index
[TIP]
====
If your Python function contains only a single script and no dependencies, you can deploy the `.py` file directly, without packaging it into a `.zip` file or creating a configuration file:
+
[source,bash,subs="+quotes"]
----
$ ./pulsar-admin functions create \
Expand Down Expand Up @@ -306,16 +307,16 @@ See <<controlling-your-function,Controlling your function>> for more information
Upload your own code::
+
--
. Select *Upload my own code*.
.. Select *Upload my own code*.
. Select your function file:
.. Select your function file:
+
* `.py`: A single, independent Python script
* `.zip`: A Python script with dependencies
* `.jar`: A Java function
* `.go`: A Go function
. Based on the uploaded file, select the specific class (function) to deploy.
.. Based on the uploaded file, select the specific class (function) to deploy.
+
{astra_db} generates a list of acceptable classes detected in the code.
A file can contain multiple classes, but only one is used per deployment.
Expand All @@ -332,7 +333,7 @@ image::astream-exclamation-function.png[Exclamation Function]
Use {company} transform function::
+
--
. Select *Use {company} transform function*.
.. Select *Use {company} transform function*.
This is the only function option available on the {product} *Free* plan.
Expand Down Expand Up @@ -395,7 +396,7 @@ The trigger sends the message string to the function.
Your function should output the result of processing the message.

[#controlling-your-function]
=== Control functions
=== Stop and start functions

In the {astra_ui}, on your tenant's *Functions* tab, you can use *Function Controls* to start, stop, and restart functions.

Expand All @@ -408,7 +409,7 @@ image::astream-function-log.png[Function Log]

If you specified a log topic when deploying your function, function logs also output to that topic.

== Edit functions
=== Edit functions

. In the {astra_ui}, on your tenant's *Functions* tab, click *Update Function*.

Expand All @@ -422,7 +423,7 @@ If you specified a log topic when deploying your function, function logs also ou

If you need to change any other function settings, you must delete and redeploy the function with the desired settings.

== Delete functions
=== Delete functions

[IMPORTANT]
====
Expand Down
14 changes: 8 additions & 6 deletions modules/developing/pages/astream-kafka.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,14 @@ By integrating two popular event streaming ecosystems, {kafka_for_astra} unlocks

This document will help you get started producing and consuming Kafka messages on a Pulsar cluster.

.Starlight for Kafka video
[%collapsible]
====
This video from the *Five Minutes About Pulsar* series explains how to migrate from Kafka to Pulsar:

video::Qy2ZlelLjXg[youtube, list=PL2g2h-wyI4SqeKH16czlcQ5x4Q_z-X7_m, height=445px,width=100%]
====

== {kafka_for_astra} Quickstart

:page-tag: starlight-kafka,quickstart,install,admin,dev,pulsar,kafka
Expand Down Expand Up @@ -98,12 +106,6 @@ Your Kafka messages are being produced and consumed in a Pulsar cluster:
+
image::astream-kafka-monitor.png[Monitor Kafka Activity]

== Starlight for Kafka video

Follow along with this video from our *Five Minutes About Pulsar* series to migrate from Kafka to Pulsar:

video::Qy2ZlelLjXg[youtube, list=PL2g2h-wyI4SqeKH16czlcQ5x4Q_z-X7_m, height=445px,width=100%]

== See also

* https://github.com/datastax/starlight-for-kafka[{company} Starlight for Kafka project]
Expand Down
14 changes: 7 additions & 7 deletions modules/developing/pages/clients/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@
To connect to your service, use the open-source client APIs provided by the Apache Pulsar project.
{product} is running Pulsar version {pulsar_version}. You should use this API version or higher.

Popular Pulsar clients include the following:
For more information and examples, see the following:

* xref:clients/csharp-produce-consume.adoc[image:csharp-icon.png[] C#]
* xref:clients/golang-produce-consume.adoc[image:golang-icon.png[] Golang]
* xref:clients/java-produce-consume.adoc[image:java-icon.png[] Java]
* xref:clients/nodejs-produce-consume.adoc[image:node-icon.png[] Node.js]
* xref:clients/python-produce-consume.adoc[image:python-icon.png[] Python]
* xref:clients/spring-produce-consume.adoc[image:spring-boot-icon.png[] Spring Boot]
* xref:clients/csharp-produce-consume.adoc[C#]
* xref:clients/golang-produce-consume.adoc[Golang]
* xref:clients/java-produce-consume.adoc[Java]
* xref:clients/nodejs-produce-consume.adoc[Node.js]
* xref:clients/python-produce-consume.adoc[Python]
* xref:clients/spring-produce-consume.adoc[Spring Boot]
19 changes: 8 additions & 11 deletions modules/developing/pages/clients/spring-produce-consume.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,23 +5,20 @@

You can produce and consume messages with the Java Pulsar client, {product}, and Spring.

== Prerequisites
Go to the https://github.com/datastax/astra-streaming-examples[examples repo] for the complete source of this example.

You will need the following prerequisites in place to complete this guide:
== Prerequisites

* JRE 8 installed (https://sdkman.io/[install now,title=Install java])
* (https://maven.apache.org/install.html[Maven,title=Maven]) or (https://gradle.org/install/[Gradle,title=Gradle]) installed
* A working Pulsar topic (get started xref:getting-started:index.adoc[here] if you don't have a topic)
* A basic text editor or IDE
For this example, you need the following:

[TIP]
====
Visit our https://github.com/datastax/astra-streaming-examples[examples repo] to see the complete source of this example.
====
* JRE 8
* https://maven.apache.org/install.html[Maven]
* A Pulsar topic in {product}
* A text editor or IDE

== Create a Maven project in Spring Initializr

Spring Initializr is a great tool to quickly create a project with the dependencies you need. Let's use it to create a project with the Pulsar client dependency.
You can use Spring Initializr to quickly create a Java project with the required dependencies, including the Pulsar client dependency.

. Go to https://start.spring.io/[Spring Initializr] to initialize a new project.

Expand Down
10 changes: 5 additions & 5 deletions modules/operations/pages/astream-limits.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Guardrails are provisioned in the default settings for {product}.
You can't directly change these guardrails.
For dedicated clusters, some guardrails can be changed by contacting {support_url}[{company} Support].

[cols="1,1"]
[cols="1,1,1"]
|===
|Guardrail |Limit |Comments

Expand Down Expand Up @@ -329,10 +329,10 @@ The following configurations can't be changed:
* Data persistency (`En`, `Qw`, `Qa`)
* `Managedledger` policy/deletion
* Namespace bundle configurations:
* Bundle split
* Bundle level clear backlog
* Bundle level unload
* Bundle level subscribe and unsubscribe
** Bundle split
** Bundle level clear backlog
** Bundle level unload
** Bundle level subscribe and unsubscribe
* Replication
* Delayed delivery
* Offload policy
Expand Down
Loading

0 comments on commit 7b5e82f

Please sign in to comment.