include::{mod-loc}shared/all-attributes.adoc[]

[id="intro-to-the-registry_{context}"]
= Introduction to {registry}
//If the assembly covers a task, start the title with a verb in the gerund form, such as Creating or Configuring.

[role="_abstract"]
This chapter introduces {registry} concepts and features and provides details on the supported artifact types that are stored in the registry:

* xref:registry-overview_{context}[]
* xref:registry-artifacts_{context}[]
* xref:registry-web-console_{context}[]
* xref:registry-rest-api_{context}[]
* xref:registry-storage_{context}[]
* xref:client-serde_{context}[]
* xref:kafka-connect_{context}[]
* xref:registry-demo_{context}[]
* xref:registry-distros_{context}[]

//INCLUDES


// Metadata created by nebel

[id="registry-overview_{context}"]
== What is {registry}?

[role="_abstract"]
{registry} is a datastore for sharing standard event schemas and API designs across event-driven and API architectures. You can use {registry} to decouple the structure of your data from your client applications, and to share and manage your data types and API descriptions at runtime using a REST interface.

Client applications can dynamically push or pull the latest schema updates to or from {registry} at runtime without needing to redeploy. Developer teams can query {registry} for existing schemas required for services already deployed in production, and can register new schemas required for new services in development.

You can enable client applications to use schemas and API designs stored in {registry} by specifying the {registry} URL in your client application code. {registry} can store schemas used to serialize and deserialize messages, which are referenced from your client applications to ensure that the messages that they send and receive are compatible with those schemas.

Using {registry} to decouple your data structure from your applications reduces costs by decreasing overall message size, and creates efficiencies by increasing consistent reuse of schemas and API designs across your organization. {registry} provides a web console to make it easy for developers and administrators to manage registry content.

You can configure optional rules to govern the evolution of your {registry} content. These include rules to ensure that uploaded content is valid, or is compatible with other versions. Any configured rules must pass before new versions can be uploaded to {registry}, which ensures that time is not wasted on invalid or incompatible schemas or API designs.

ifndef::apicurio-registry[]
{registry-name-full} is based on the Apicurio Registry open source community project. For details, see https://github.com/apicurio/apicurio-registry. 
endif::[]


[discrete]
=== {registry} capabilities

* Multiple payload formats for standard event schema and API specifications such as Apache Avro, JSON Schema, Google Protobuf, AsyncAPI, OpenAPI, and more.

* Pluggable {registry} storage options for storing content in Apache Kafka or PostgreSQL database.

* Rules for content validation, compatibility, and integrity to govern how {registry} content evolves over time.

* {registry} content management using web console, REST API, command line, Maven plug-in, or language SDKs.

* Full Apache Kafka schema registry support, including integration with Kafka Connect for external systems.

* Kafka client serializers/deserializers (SerDes) to validate message types at runtime.

* Compatibility with existing Confluent schema registry client applications.

* Cloud-native Quarkus Java runtime for low memory footprint and fast deployment times.

* Operator-based installation of {registry} on OpenShift/Kubernetes.

* OpenID Connect (OIDC) authentication using {keycloak}.



// Metadata created by nebel

[id="registry-artifacts_{context}"]
== Schema and API artifacts in {registry}

[role="_abstract"]
The items stored in {registry}, such as event schemas and API designs, are known as registry _artifacts_. The following shows an example of an Apache Avro schema artifact in JSON format for a simple share price application:

.Example Avro schema
[source,json]
----
{
   "type": "record",
   "name": "price",
   "namespace": "com.example",
   "fields": [
       {
           "name": "symbol",
           "type": "string"
       },
       {
           "name": "price",
           "type": "string"
       }
   ]
}
----

When a schema or API design is added as an artifact in {registry}, client applications can then use that schema or API design to validate that the client messages conform to the correct data structure at runtime.

Artifacts have metadata, both generated and editable.  Standard metadata for an artifact includes (but may not be limited to):

==== Generated or immutable properties

- groupId
- artifactId
- artifactType
- createdOn
- modifiedBy
- modifiedOn

==== Editable properties
- name
- description
- labels
- owner

[discrete]
=== Artifact Versions
Every artifact is composed of zero or more _artifact version_s.  Only artifact versions have actual
content (as well as metadata).  These versions represent the evolution of the content of an artifact,
and are immutable.  You can think of an Artifact as an ordered sequence of Versions, typically with
the most recent version representing the "current" schema or API design content.


Artifact Versions have metadata, both generated and editable.  Standard metadata for an artifact version includes (but may not be limited to):

==== Generated or immutable properties

- groupId
- artifactId
- version
- globalId
- contentId
- owner
- createdOn
- modifiedBy
- modifiedOn

==== Editable properties
- name
- description
- labels
- state


[discrete]
=== Groups of schemas and APIs

An _artifact group_ is an optional named collection of schema or API artifacts. Each group contains a logically related set of schemas or API designs, typically managed by a single entity, belonging to a particular application or organization.

You can create optional artifact groups when adding your schemas and API designs to organize them in {registry}. For example, you could create groups to match your `development` and `production` application environments, or your `sales` and `engineering` organizations.

Schema and API groups can contain multiple artifact types. For example, you could have Protobuf, Avro, JSON Schema, OpenAPI, or AsyncAPI artifacts all in the same group.

Groups have metadata, both generated and editable.  Standard metadata for a group includes (but may not be limited to):

==== Generated or immutable properties

- groupId
- owner
- createdOn
- modifiedBy
- modifiedOn

==== Editable properties
- description
- labels

You can create schema and API artifacts and groups using the {registry} web console, REST API, command line, Maven plug-in, or Java client application.

NOTE: Specifying a group is optional when using the {registry} web console, and the `default` group is used automatically. When using the REST API or Maven plug-in, specify the `default` group in the API path if you do not want to create a unique group.

[role="_additional-resources"]
.Additional resources

* For information on supported artifact types, see {registry-artifact-reference}.
* For information on the Core Registry API, see the {registry-rest-api}.


[discrete]
=== References to other schemas and APIs

Some {registry} artifact types can include _artifact references_ from one artifact file to another. You can create efficiencies by defining reusable schema or API components, and then referencing them from multiple locations. For example, you can specify a reference in JSON Schema or OpenAPI using a `$ref` statement, or in Google Protobuf using an `import` statement, or in Apache Avro using a nested namespace.

The following example shows a simple Avro schema named `TradeKey` that includes a reference to another schema named `Exchange` using a nested namespace:

.Tradekey schema with nested Exchange schema
[source,json]
----
{
 "namespace": "com.kubetrade.schema.trade",
 "type": "record",
 "name": "TradeKey",
 "fields": [
   {
     "name": "exchange",
     "type": "com.kubetrade.schema.common.Exchange"
   },
   {
     "name": "key",
     "type": "string"
   }
 ]
}
----

.Exchange schema
[source,json]
----
{
 "namespace": "com.kubetrade.schema.common",
 "type": "enum",
 "name": "Exchange",
 "symbols" : ["GEMINI"]
}
----

An artifact reference is stored in {registry} as a collection of artifact metadata that maps from an artifact type-specific reference to an internal {registry} reference. Each artifact reference in {registry} is composed of the following:

* Group ID
* Artifact ID
* Artifact version
* Artifact reference name

You can manage artifact references using the {registry} core REST API, Maven plug-in, and Java serializers/deserializers (SerDes). {registry} stores the artifact references along with the artifact content. {registry} also maintains a collection of all artifact references so you can search them or list all references for a specific artifact.

[discrete]
==== Supported artifact types
{registry} currently supports artifact references for the following artifact types only:

* Avro
* Protobuf
* JSON Schema
* OpenAPI
* AsyncAPI

[role="_additional-resources"]
.Additional resources

* For details on managing artifact references, see:
** {managing-registry-artifacts-api}.
** {managing-registry-artifacts-maven}.
* For a Java example, see the https://github.com/Apicurio/apicurio-registry/tree/main/examples/serdes-with-references[Apicurio Registry SerDes with references demonstration].



// Metadata created by nebel

[id="registry-web-console_{context}"]
== Manage content using the {registry} web console

[role="_abstract"]
You can use the {registry} web console to browse and search the schema and API artifacts and optional groups stored in the registry, and to add new schema and API artifacts, groups, and versions. You can search for artifacts by label, name, group, and description. You can view an artifact’s content or its available versions, or download an artifact file locally.

You can also configure optional rules for registry content, globally, for groups, and for each schema and API artifact. These optional rules for content validation and compatibility are applied when new schema and API artifacts or versions are uploaded to the registry.

For more details, see {registry-rule-reference}.

.{registry} web console
image::images/getting-started/registry-web-console.png[{registry} web console]

The {registry} web console is available from `{registry-url}`.

[role="_additional-resources"]
.Additional resources
* {managing-registry-artifacts-ui}



// Metadata created by nebel

[id="registry-rest-api_{context}"]

[role="_abstract"]
== {registry} REST API for clients
Client applications can use the Core Registry API v3 to manage the schema and API artifacts in {registry}. This API provides operations for the following features:

Admin::
Export or import {registry} data in a `.zip` file, and manage logging levels for the {registry} at runtime.
Groups::
Manage groups of artifacts in {registry}.  You can create groups to better organize your artifacts.
Group Rules::
Configure rules to govern the content evolution of schemas or API artifacts within a specific group to prevent invalid or incompatible content from being added to {registry}. Group rules override any global rules configured.
Artifacts::
Manage schema and API artifacts stored in {registry}.
Artifact metadata::
Manage details about a schema or API artifact. You can edit details such as artifact name, description, or labels. Details such as artifact group, and when the artifact was created or modified are read-only.
Artifact rules::
Configure rules to govern the content evolution of a specific schema or API artifact to prevent invalid or incompatible content from being added to {registry}. Artifact rules override any group and/or global rules configured.
Artifact versions::
Manage the sequence of versions that make up the content of a schema or API artifact. You can also manage the lifecycle state of an artifact version: enabled, disabled, or deprecated.
Global rules::
Configure rules to govern the content evolution of all schema and API artifacts to prevent invalid or incompatible content from being added to {registry}. Global rules are applied only if an artifact and its group do not have rules configured.
Search::
Browse or search for schema and API artifacts and versions, for example, by name, group, description, or label.
System::
Get the {registry} version and the limits on resources for the {registry} server.
Users::
Get the current {registry} user.

[discrete]
=== Compatibility with other schema registry REST APIs
{registry} also provides compatibility with the following schema registries by including implementations of their respective REST APIs:

* {registry} Core Registry API v2
* Confluent Schema Registry API v7

Applications using Confluent client libraries can use {registry} as a drop-in replacement.
ifdef::rh-service-registry[]
For more details, see link:https://developers.redhat.com/blog/2019/12/17/replacing-confluent-schema-registry-with-red-hat-integration-service-registry/[Replacing Confluent Schema Registry].
endif::[]

[role="_additional-resources"]
.Additional resources
* For more information on the Core Registry API v3, see the {registry-rest-api}.
* For API documentation on the Core Registry API v3 and all compatible APIs, browse to the `/apis` endpoint of your {registry}, for example, `\http://MY-REGISTRY-URL/apis`.



// Metadata created by nebel

[id="registry-storage_{context}"]
== {registry} storage options

[role="_abstract"]
{registry} provides the following options for the underlying storage of registry data:

.{registry} data storage options
[%header,cols="1,3"]
|===
|Storage option
|Description

ifdef::apicurio-registry[]
|In-memory
|The in-memory storage option is suitable for a development environment only. All data is lost when restarting {registry} with this storage. The PostgreSQL or Kafka storage option is recommended for a production environment.
endif::[]

|PostgreSQL database
|PostgreSQL is the recommended data storage option for performance, stability, and data management (backup/restore, and so on) in a production environment.

|Apache Kafka
|Kafka storage is provided for production environments where database management expertise is not available, or where storage in Kafka is a specific requirement.
|===


[role="_additional-resources"]
.Additional resources
ifdef::apicurio-registry[]
* {installing-the-registry-docker}
* {installing-the-registry-openshift}
* {installing-the-registry-storage-openshift}
endif::[]
ifdef::rh-service-registry[]
* For more details on storage options, see link:{LinkServiceRegistryInstall}[{NameServiceRegistryInstall}].
endif::[]


// Metadata created by nebel


[id="client-serde_{context}"]
==  Validate Kafka messages using schemas and Java client serializers/deserializers

[role="_abstract"]
Kafka producer applications can use serializers to encode messages that conform to a specific event schema. Kafka consumer applications can then use deserializers to validate that messages have been serialized using the correct schema, based on a specific schema ID.

.{registry} and Kafka client SerDes architecture
image::images/getting-started/registry-serdes-architecture.png[Kafka client SerDes architecture]

{registry} provides Kafka client serializers/deserializers (SerDes) to validate the following message types at runtime:

* Apache Avro
* Google Protobuf
* JSON Schema

The {registry} Maven repository and source code distributions include the Kafka SerDes implementations for these message types, which Kafka client application developers can use to integrate with {registry}.

These implementations include custom Java classes for each supported message type, for example, `io.apicurio.registry.serde.avro`, which client applications can use to pull schemas from {registry} at runtime for validation.

[role="_additional-resources"]
.Additional resources
* {kafka-client-serdes}



// Metadata created by nebel

[id="kafka-connect_{context}"]
== Stream data to external systems with Kafka Connect converters

[role="_abstract"]
You can use {registry} with Apache Kafka Connect to stream data between Kafka and external systems. Using Kafka Connect, you can define connectors for different systems to move large volumes of data into and out of Kafka-based systems.

.{registry} and Kafka Connect architecture
image::images/getting-started/registry-connect-architecture.png[Registry and Kafka Connect architecture]

{registry} provides the following features for Kafka Connect:

* Storage for Kafka Connect schemas
* Kafka Connect converters for Apache Avro and JSON Schema
* Core Registry API to manage schemas

You can use the Avro and JSON Schema converters to map Kafka Connect schemas into Avro or JSON schemas. These schemas can then serialize message keys and values into the compact Avro binary format or human-readable JSON format. The converted JSON is less verbose because the messages do not contain the schema information, only the schema ID.

{registry} can manage and track the Avro and JSON schemas used in the Kafka topics. Because the schemas are stored in {registry} and decoupled from the message content, each message must only include a tiny schema identifier. For an I/O bound system like Kafka, this means more total throughput for producers and consumers.

The Avro and JSON Schema serializers and deserializers (SerDes) provided by {registry} are used by Kafka producers and consumers in this use case. Kafka consumer applications that you write to consume change events can use the Avro or JSON SerDes to deserialize these events. You can install the {registry} SerDes in any Kafka-based system and use them along with Kafka Connect, or with a Kafka Connect-based system such as Debezium.

[role="_additional-resources"]
.Additional resources

ifdef::rh-service-registry[]
* link:https://docs.redhat.com/en/documentation/red_hat_build_of_debezium/3.0.8/html/debezium_user_guide/configuring-debezium-connectors-for-your-application#configuring-debezium-connectors-to-use-avro-serialization[Configuring Debezium to use Avro serialization and {registry}]
endif::[]
ifdef::apicurio-registry[]
* link:https://debezium.io/documentation/reference/stable/configuration/avro.html[Configuring Debezium to use Avro serialization and {registry}]
endif::[]
* link:https://github.com/Apicurio/apicurio-registry/tree/main/examples/event-driven-architecture[Example of using Debezium to monitor the PostgreSQL database used by Apicurio Registry]
* link:https://kafka.apache.org/documentation/#connect[Apache Kafka Connect documentation]



// Metadata created by nebel

[id="registry-demo_{context}"]
== {registry} demonstration examples

[role="_abstract"]
{registry} provides open source example applications that demonstrate how to use {registry} in different use case scenarios. For example, these include storing schemas used by Kafka serializer and deserializer (SerDes) Java classes. These classes fetch the schema from {registry} for use when producing or consuming operations to serialize, deserialize, or validate the Kafka message payload.

These applications demonstrate use cases such as the following examples:

* Apache Avro Kafka SerDes
* Apache Avro Maven plug-in
* Apache Camel Quarkus and Kafka
* CloudEvents
* Confluent Kafka SerDes
* Custom ID strategy
* Event-driven architecture with Debezium
* Google Protobuf Kafka SerDes
* JSON Schema Kafka SerDes
* REST clients

[role="_additional-resources"]
.Additional resources
* For more details, see link:https://github.com/Apicurio/apicurio-registry/tree/main/examples[]



// Metadata created by nebel

[id="registry-distros_{context}"]
== {registry} available distributions

[role="_abstract"]
{registry} provides the following components as part of its distribution.

ifdef::apicurio-registry[]

.{registry} images
[%header,cols="2,4"]
|===
|Component
|Container Image
|Back-end
|https://hub.docker.com/r/apicurio/apicurio-registry
|User Interface
|https://hub.docker.com/r/apicurio/apicurio-registry-ui
|{registry} Operator
|https://hub.docker.com/r/apicurio/apicurio-registry-operator
|===

.Additional resources
* For details on building from source code and running Docker images, see link:https://github.com/Apicurio/apicurio-registry[]
* For details on using the Operator to deploy, see link:https://github.com/Apicurio/apicurio-registry/tree/main/operator[]

endif::[]

ifdef::rh-service-registry[]

.{registry} Operator and images
[%header,cols="3,3,2"]
|===
|Distribution
|Location
|Release category
|{registry} Operator
|OpenShift web console under *Operators* → *OperatorHub*
|General Availability
|Container image for {registry} Operator
|link:{download-url-registry-container-catalog}[{org-name} Ecosystem Catalog]
|General Availability
|Container image for Apicurio Registry (Back End)
|link:{download-url-registry-container-catalog}[{org-name} Ecosystem Catalog]
|General Availability
|Container image for Apicurio Registry (User Interface)
|link:{download-url-registry-container-catalog}[{org-name} Ecosystem Catalog]
|General Availability
|===


.{registry} zip downloads
[%header,cols="3,3,2"]
|===
|Distribution
|Location
|Release category
|Example custom resource definitions for installation
|link:{download-url-registry-distribution}[{org-name} Software Downloads]
|General Availability
|Maven repository
|link:{download-url-registry-distribution}[{org-name} Software Downloads]
|General Availability
|Source code
|link:{download-url-registry-distribution}[{org-name} Software Downloads]
|General Availability
|Kafka Connect converters
|link:{download-url-registry-distribution}[{org-name} Software Downloads]
|General Availability
|===

NOTE: You must have a subscription for {prodnamefull} and be logged into the {org-name} Customer Portal to access the available {registry} distributions.
endif::[]

:leveloffset!:
