A Series of Catalyst Questions

Catalyst
1

Good evening Diagridians, I have a number of questions that I’m hoping to get some help on. Please see below.

  1. How does catalyst support dapr version upgrades & dapr version selection by clients?
  2. Is there a way to ci/cd the component definitions, subscriptions, resiliency policies, etc? (maybe via the management apis when exposed)?
  3. Does pubsub subscription support routing rules?
  4. Do I understand correctly that secret store is not a supported dapr component type currently?
  5. Can we change the catalyst log level to get debug logs in dev, warn in higher?
  6. If a component is updated, does the catalyst server-side automatically hot-load the updated definition? Does this cause any downtime of the Catalyst server-side from a client’s POV? (This could effectively be a form of dynamic reload for both subscriptions & components that we currently don’t have with plain Dapr)
  7. Is there an API we could call to create new pubsub components & subscriptions on the fly from our apps? (I think the answer here is not yet, but this may be possible with the management apis at a later stage)
2
  1. How does catalyst support dapr version upgrades & dapr version selection by clients?

We completely abstract dapr management, therefore the serverless sidecars we host are going to be upgrade to newer versions of dapr transparently. We maintain a dapr fork which gives us the ability to ensure compatibility. We will likely have to consider deprecations and breaking changes in upstream and how we would support those as a managed service so I can get back to you in that regard. Dapr itself has to maintain compatibility due to its user base so that also would play a factor in being able to ensure compat. If you are live serving a Dapr upgrade should not affect uptime

  1. Is there a way to ci/cd the component definitions, subscriptions, resiliency policies, etc? (maybe via the management apis when exposed)?

Yes - we currently have declarative manifests which could be deployed via scripting + CLI but we also are going to expose our management API and long term want to provide support for some type of templating solution like terraform for deployments

  1. Does pubsub subscription support routing rules?
    Yes!

  2. Do I understand correctly that secret store is not a supported dapr component type currently?

Correct- we transparently store and encrypt secret values that are used to create components. Long term, we also are working to support passwordless component connectivity to allow you to use a trusted identity to authenticate to the cloud-hosted backing services. We will consider support for using a BYO secret store based on demand

  1. Can we change the catalyst log level to get debug logs in dev, warn in higher?

Not today- we dont stream the dapr logs as is because they contain low level details that you as a Catalyst consumer would not have control over. Things like component loading errors or init errors are propogated as normalized errors vs just a log stream. Our goal is to enhance the log output to ensure you can always properly troubleshoot any user error i.e. connection misconfigured, connection auth invalid, etc.

  1. If a component is updated, does the catalyst server-side automatically hot-load the updated definition? Does this cause any downtime of the Catalyst server-side from a client’s POV? (This could effectively be a form of dynamic reload for both subscriptions & components that we currently don’t have with plain Dapr)

Hot reload was contributed in upstream primarily by one of our Dapr OSS engineers, the goal will be to allow customers to opt in to hot reloading / leverage this in Catalyst. Currently to avoid downtime, we update the component and perform a rolling restart, which means we bring up a new sidecar, make sure it’s running, divert all traffic to it, and only then kill the previous one

  1. Is there an API we could call to create new pubsub components & subscriptions on the fly from our apps? (I think the answer here is not yet, but this may be possible with the management apis at a later stage)

We are working on an SDK poc which would allow you to create app id resources / connect to them / subscribe to them via code :slight_smile:

2 Likes