= Overview

== What is Kroxylicious Proxy?

Kroxylicious Proxy provides a pluggable, protocol-aware ("Layer 7") proxy for https://kafka.apache.org[Apache Kafka(R)] brokers and clusters, together with an API for conveniently implementing custom logic within such a proxy.

== Why?

Proxies are a powerful and flexible architectural pattern.
For Kafka, they can be used to add functionality to Kafka clusters which is not available out-of-the-box with Apache Kafka.
In an ideal world, such functionality would be implemented directly in Apache Kafka.
But there are numerous practical reasons that can prevent this, for example:

* Organizations having very niche requirements which are unsuitable for implementation directly in Apache Kafka.
* Functionality which requires changes to Kafka's public API and which the Apache Kafka project is unwilling to implement.
  This is the case for https://lists.apache.org/thread/x1p119hkpoy01vq9ck3d0ql67jtvm875[broker interceptors], for example.
* Experimental functionality which might end up being implemented in Apache Kafka eventually.
For example using Kroxylicious proxy it's easier to experiment with alternative transport protocols, such as Quic, or operating system APIs, such as io_uring, because there is already support for this in Netty, the networking framework on which Kroxylicious is built.

== How it works

First let's define the concepts in the landscape surrounding Kroxylicious.

. _Kafka Client_, or _Client_ refers to any client application using a Kafka Client library to talk to a *Kafka Cluster*.
. _Kafka Cluster_ or _Cluster_ refers to a cluster comprising one or more Kafka Brokers.
. _Downstream_ refers to the area between Kafka Client and Kroxylicious.
. _Upstream_ refers to the area between Kroxylicious and a Kafka Cluster.

.Kroxylicious landscape
[a2s, format="svg"]
....
.---------------------------------------------------------------------------.
|                                  :                                        |
|                                  :              +===============+         |
|                                  :              :[Cluster]      :         |
|                         .-----------------.     :  .----------. :         |
|     .----------.        |[Kroxylicious]   |     :  |[Broker1] | :         |
|     |[Client]  |        |                 +------->|          | :         |
|     |          +------->|                 |     :  |          | :         |
|     |          |        |                 |     :  .----------. :         |
|     |          |        |                 |     :               :         |
|     .----------.        |                 |     :  .----------. :         |
|                         |                 |     :  |[Broker2] | :         |
|                         |                 +------->|          | :         |
|                         |                 |     :  |          | :         |
|                         |                 |     :  .----------. :         |
|     .----------.        |                 |     :               :         |
|     |[Client]  |        |                 |     :  .----------. :         |
|     |          |        |                 |     :  |[Broker3] | :         |
|     |          +------->|                 +------->|          | :         |
|     |          |        |                 |     :  |          | :         |
|     .----------.        |                 |     :  .----------. :         |
|                         .-----------------.     .===============+         |
|                                  :                                        |
|                                  :                                        |
|     [downstream]                 :                  [upstream]            |
.---------------------------------------------------------------------------.
[0,0]: {"fill":"#99d","a2s:delref":1}
....

Now let's define some concepts used within Kroxylicious itself.

=== Virtual Cluster

The _Virtual Cluster_ is the downstream representation of a Kafka Cluster.  At the conceptual level, a Kafka Client
connects to a Virtual Cluster.  Kroxylicious proxies all communications made to the Virtual Cluster through to a
(physical) Kafka Cluster, passing it through the _Filter Chain_.

So far, this explanation has elided the detail of Kafka Brokers.  Let's talk about that now.

The Virtual Cluster automatically exposes a bootstrap endpoint for the Virtual Cluster.  This is what the Kafka Client
must specify as the https://kafka.apache.org/documentation/#producerconfigs_bootstrap.servers[bootstrap.servers] property
in the client configuration.

In addition to the bootstrap endpoint, Kroxylicious automatically exposes broker endpoints.  There is one broker endpoint
for each broker of the physical cluster.  When the Client connects to a broker endpoint, Kroxylicious proxies all
communications to the corresponding broker of the (physical) Kafka Cluster.

Kroxylicious automatically intercepts all the Kafka RPC responses that contain a broker address.  It rewrites the address
so that it refers to the corresponding broker endpoint of the Virtual Cluster.  This means when the Kafka Client
goes to connect to, say broker 0, it does so through the Virtual Cluster.

=== Target Cluster

The _Target Cluster_ is the definition of physical Kafka Cluster within the Kroxylicious itself.

A _Virtual Cluster_ has exactly one _Target Cluster_.

There can be a _one-to-one_ relationship between Virtual Clusters and Target Clusters.
The other possibility is _many-to-one_, where many Virtual Clusters point to the same Target Cluster.  The
many-to-one pattern is exploited by filters such as the xref:available-filters.adoc#_multi_tenancy[Multi-tenancy]
filter.

.One-to-One relationship between Virtual Cluster and Target Cluster
[a2s, format="svg"]
....
.-------------------------------------------------------------------------------------.
|                                                                                     |
|                 '-------------------------------------'    '===================='   |
| '---------'     |[Kroxylicious]                       |    :[Cluster]           :   |
| |[Client] |=+   |                                     |    :                    :   |
| '---------' :   |    '----------------------------'   |    :      '---------'   :   |
|             :   |    |[Virtual Cluster]           |===+====|=====>|[Broker] |   :   |
|             +===|===>|                            |   |    :      |         |   :   |
|             :   |    |                            |   |    :      '---------'   :   |
|             +===|===>|                            |   |    :                    :   |
|                 |    |                            |   |    :      '---------'   :   |
|                 |    |                            |===+====:==+==>|[Broker] |   :   |
|                 |    '----------------------------'   |    :      |         |   :   |
|                 |                                     |    :      '---------'   :   |
|                 '-------------------------------------'    '===================='   |
.-------------------------------------------------------------------------------------.
[0,0]: {"fill":"#99d","a2s:delref":1}
....

.Many-to-one between Virtual Cluster and Target Cluster
[a2s, format="svg"]
....
.-------------------------------------------------------------------------------------.
|                                                                                     |
|                 '-------------------------------------'    '===================='   |
| '---------'     |[Kroxylicious]                       |    :[Cluster]           :   |
| |[Client] |=+   |                                     |    :                    :   |
| '---------' :   |    '----------------------------'   |    :      '---------'   :   |
|             :   |    |[Virtual Cluster A]         |===+=+==|==+==>|[Broker] |   :   |
|             +===|===>|                            |   | :  :      |         |   :   |
|             :   |    |                            |   | :  :      '---------'   :   |
|             +===|===>|                            |   | :  :                    :   |
|                 |    |                            |   | :  :      '---------'   :   |
|                 |    |                            |===+====:==+==>|[Broker] |   :   |
|                 |    '----------------------------'   | :  :  :   |         |   :   |
| '---------'     |                                     | :  :  :   '---------'   :   |
| |[Client] |=+   |    '----------------------------'   | :  :  :                 :   |
| '---------' :   |    |[Virtual Cluster B]         |===+=+  '===================='   |
|             +===|===>|                            |   |       :                     |
|             :   |    |                            |   |       :                     |
|             +===|===>|                            |   |       :                     |
|                 |    |                            |   |       :                     |
|                 |    |                            |===+=======+                     |
|                 |    '----------------------------'   |                             |
|                 '-------------------------------------'                             |
.-------------------------------------------------------------------------------------.
[0,0]: {"fill":"#99d","a2s:delref":1}
....

A one-to-many pattern, where one Virtual Cluster points to many Target Clusters (providing amalgamation),
is not a supported use-case.



=== Filter Chain

A _Filter Chain_ consists of an *ordered list* of pluggable _protocol filters_.

A  _protocol filter_ implements some logic for intercepting, inspecting and/or manipulating Kafka protocol messages.
Kafka protocol requests (such as `Produce` requests) pass sequentially through each of the protocol filters in the
chain, beginning with the 1st filter in the chain and then following with the subsequent filters, before being
forwarded to the broker.

When the broker returns a response (such as a `Produce` response) the protocol filters in the chain are invoked in the
reverse order (that is, beginning with the nth filter in the chain, then the n-1th and so on) with each having the
opportunity to inspect and/or manipulating the response. Eventually a response is returned to the client.

The description above describes only the basic capabilities of the protocol filter. Richer features of filters
are described later.

// TODO document additional filter behaviours https://github.com/kroxylicious/kroxylicious/issues/420

.Illustration of a request and response being manipulated by filters in a chain
[a2s, format="svg"]
....
.----------------------------------------------------------------------------------------------------------------------.
|                                                                                                                      |
|                       '---------------------------------------------------------------'                              |
|                       |[Kroxylicious]                                                 |                              |
|                       |                                                               |                              |
|                       |   '----------------------------------------------------'      |      '--------------------'  |
|                       |   |[Virtual Cluster]                                   |      |      |[Cluster]           |  |
|  '-------------'      |   |   '----------'     '----------'     '----------'   |      |      |    '------------'  |  |
|  |[Client]     |      |   |   |[Filter1] |     |[Filter2] |     |[Filter3] |   |      |      |    |[Broker]    |  |  |
|  |             |======|===|==>|          |====>|          |====>|          |===|======|======|===>|            |  |  |
|  |             |  A   |   |   | F(A)-->B |  B  | F(B)-->C |  C  | F(C)-->D |   |      |      | D  |            |  |  |
|  |             |      |   |   |          |     |          |     |          |   |      |      |    |            |  |  |
|  |             |<=====|===|===|          |<====|          |<====|          |<==|======|======|====|            |  |  |
|  |             |  W   |   |   | f(X)-->W |  X  | f(Y)-->X |  Y  | f(Z)-->Y |   |      |      | Z  |            |  |  |
|  '-------------'      |   |   '----------'     '----------'     '----------'   |      |      |    '------------'  |  |
|                       |   |                                                    |      |      '--------------------'  |
|                       |   '----------------------------------------------------'      |                              |
|                       |                                                               |                              |
|                       '---------------------------------------------------------------'                              |
|                                                                                                                      |
.----------------------------------------------------------------------------------------------------------------------.
[0,0]: {"fill":"#99d","a2s:delref":1}
....

As mentioned above, Kroxylicious takes the responsibility to rewrite the Kafka RPC responses that carry broker address
information so that they reflect the broker addresses exposed by the Virtual Cluster. These are the
https://kafka.apache.org/protocol.html#The_Messages_Metadata[`Metadata`],
https://kafka.apache.org/protocol.html#The_Messages_DescribeCluster[`DescribeCluster`] and
https://kafka.apache.org/protocol.html#The_Messages_FindCoordinator[`FindCoordinator`] responses. This processing is
entirely transparent to the work of the protocol filters.  _Filter authors_ are free to write their own filters that
intercept these responses too.

=== Filter composition

An important principal for the protocol filter API is that filters should _compose_ nicely.
That means that filters generally don't know what other filters might be present in the chain, and what they might be doing to messages.
When a filter forwards a request or response it doesn't know whether the message is being sent to the next filter in the chain, or straight back to the client.

Such composition is important because it means a _proxy user_ can configure multiple filters (possibly written by several _filter authors_) and expect to get the combined effect of all of them.

It's never quite that simple, of course.
In practice they will often need to understand what each filter does in some detail in order to be able to operate their proxy properly, for example by understanding whatever metrics each filter is emitting.

== Implementation

The proxy is written in Java, on top of https://netty.io[Netty].
The usual https://netty.io/4.1/api/io/netty/channel/ChannelHandler.html[`ChannelHandlers`] provided by the Netty project are used where appropriate (e.g. SSL support uses https://netty.io/4.1/api/io/netty/handler/ssl/SslHandler.html[`SslHandler`]), and Kroxylicious provides Kafka-specific handlers of its own.

The Kafka-aware parts use the Apache Kafka project's own classes for serialization and deserialization.

Protocol filters get executed using a handler-per-filter model.

== Deployment topologies

The proxy supports a range of possible deployment topologies.
Which style is used depends on what the proxy is meant to _achieve_, architecturally speaking.
Broadly speaking a proxy instance can be deployed:

As a forward proxy::
Proxying the access of one or more clients to a particular cluster/broker that might also accessible (to other clients) directly.
+
// TODO include a diagram
+
Topic-level encryption provides one example use case for a forward proxy-style deployment.
This might be applicable when using clients that don't support interceptors, or if an organisation wants to apply the same encryption policy in a single place, securing access to the keys within their network.

As a reverse proxy::
Proxying access for all clients trying to reach a particular cluster/broker.
+
// TODO include a diagram
+
Transparent multi-tenancy provides an example use case for a reverse proxy.
While Apache Kafka itself has some features that enable multi-tenancy, they rely on topic name prefixing as the primary mechanism for ensuring namespace isolation.
Tenants have to adhere to the naming policy and know they're a tenant of a larger shared cluster.
+
_Transparent_ multi-tenancy means each tenant has the illusion of having their own cluster, with almost complete freedom over topic and group naming, while still actually sharing a cluster.

// TODO we probably don't need the level of detail below, just summarize
// and provide the detail in the deploying section

We can further classify deployment topologies in how many proxy instances are used. For example:

Single proxy instance::

Proxy pool::
//-

== More about filters
