// Module included in the following assemblies:
//  assembly-using-kafka-client-serdes

[id='registry-serdes-concepts-serde_{context}']

= Kafka client applications and {registry}

[role="_abstract"]
{registry} decouples schema management from client application configuration. You can enable a Java client application to use a schema from {registry} by specifying its URL in your client code.

You can store the schemas in {registry} to serialize and deserialize messages, which are referenced from your client applications to ensure that the messages that they send and receive are compatible with those schemas. Kafka client applications can push or pull their schemas from {registry} at runtime.

Schemas can evolve, so you can define rules in {registry}, for example, to ensure that schema changes are valid and do not break previous versions used by applications. {registry} checks for compatibility by comparing a modified schema with previous schema versions.

[discrete]
== {registry} schema technologies
{registry} provides schema registry support for schema technologies such as:

* Avro
* Protobuf
* JSON Schema

These schema technologies can be used by client applications through the Kafka client serializer/deserializer (SerDes) services provided by {registry}.  The maturity and usage of the SerDes classes provided by {registry} might vary. The sections that follow provide more details about each schema type.

[discrete]
== Producer schema configuration

A producer client application uses a serializer to put the messages that it sends to a specific broker topic into the correct data format. 

To enable a producer to use {registry} for serialization:

* xref:registry-serdes-register_{context}[Define and register your schema with {registry}] (if it does not already exist).
* xref:registry-serdes-config-producer_{context}[Configure your producer client code] with the following:

** URL of {registry}
** {registry} serializer to use with messages
** Strategy to map the Kafka message to a schema artifact in {registry}
** Strategy to look up or register the schema used for serialization in {registry}

After registering your schema, when you start Kafka and {registry}, you can access the schema to format messages sent to the Kafka broker topic by the producer. Alternatively, depending on configuration, the producer can automatically register the schema on first use.

If a schema already exists, you can create a new version using the registry REST API based on compatibility rules defined in {registry}. Versions are used for compatibility checking as a schema evolves. A group ID, artifact ID, and version represents a unique tuple that identifies a schema.

[discrete]
== Consumer schema configuration
A consumer client application uses a deserializer to get the messages that it consumes from a specific broker topic into the correct data format.

To enable a consumer to use {registry} for deserialization:

* xref:registry-serdes-register_{context}[Define and register your schema with {registry}] (if it does not already exist)
* xref:registry-serdes-config-consumer_{context}[Configure the consumer client code]  with the following:
** URL of {registry}
** {registry} deserializer to use with the messages
** Input data stream for deserialization

.Retrieve schemas using a content ID
By default, the schema is retrieved from {registry} by the deserializer using a content ID (which is an ID unique to the *content* of an artifact version, but not unique to the version itself), which is specified in the message being consumed. The schema content ID can be located in the message headers or in the message payload, depending on the configuration of the producer application.  By default, the content ID will be located in the message body.

When locating the content ID in the message payload, the format of the data begins with a magic byte, used as a signal to consumers, followed by the content ID, and the message data as normal. For example:

[source,shell,subs="+quotes,attributes"]
----
# ...
[MAGIC_BYTE]
[CONTENT_ID]
[MESSAGE DATA]
----

Then when you start Kafka and {registry}, you can access the schema to format messages received from the Kafka broker topic.

.Retrieve schemas using a global ID
Alternatively, you can configure to retrieve schemas from {registry} based on the global ID, which is the unique ID of the artifact version.  The same options are available to you when using global ID instead of contentID.  You can either send the global ID in the message headers or the message body (default).

When locating the global ID in the message payload, the format of the data begins with a magic byte, used as a signal to consumers, followed by the global ID, and the message data as normal. For example:

[source,shell,subs="+quotes,attributes"]
----
# ...
[MAGIC_BYTE]
[GLOBAL_ID]
[MESSAGE DATA]
----

