[[integration]]
= Integration
:doctype: book
:sectnums:
:icons: font


[[integrationOverview]]
== Overview

OptaPlanner's input and output data (the planning problem and the best solution) are plain old JavaBeans (POJOs), so integration with other Java technologies is straightforward.
For example:

* To read a planning problem from the database (and store the best solution in it), annotate the domain POJOs with JPA annotations.
* To read a planning problem from an XML file (and store the best solution in it), annotate the domain POJOs with XStream or JAXB annotations.
* To expose the Solver as a REST Service that reads the planning problem and responds with the best solution, annotate the domain POJOs with XStream, JAXB or Jackson annotations and hook the `Solver` in Camel or RESTEasy.

image::integration/integrationOverview.png[align="center"]


[[integrationWithPersistentStorage]]
== Persistent storage


[[integrationWithJpaAndHibernate]]
=== Database: JPA and Hibernate

Enrich domain POJOs (solution, entities and problem facts) with JPA annotations
to store them in a database by calling `EntityManager.persist()`.

[NOTE]
====
Do not confuse JPA's `@Entity` annotation with OptaPlanner's `@PlanningEntity` annotation.
They can appear both on the same class:

[source,java,options="nowrap"]
----
@PlanningEntity // OptaPlanner annotation
@Entity // JPA annotation
public class Talk {...}
----
====

Add a dependency to the `optaplanner-persistence-jpa` jar to take advantage of these extra integration features:


[[jpaAndHibernatePersistingAScore]]
==== JPA and Hibernate: persisting a `Score`

When a `Score` is persisted into a relational database, JPA and Hibernate will default to Java serializing it to a `BLOB` column.
This has several disadvantages:

* The Java serialization format of `Score` classes is currently not backwards compatible. Upgrading to a newer OptaPlanner version can break reading an existing database.
* The score is not easily readable for a query executed in the database console. This is annoying during development.
* The score cannot be used in a SQL or JPA-QL query to efficiently filter the results: for example to query all infeasible schedules.

To avoid these issues, configure it to instead use INTEGER (or other) columns, by using the appropriate `*ScoreHibernateType` for your `Score` type, for example for a ``HardSoftScore``:

[source,java,options="nowrap"]
----
@PlanningSolution
@Entity
@TypeDef(defaultForType = HardSoftScore.class, typeClass = HardSoftScoreHibernateType.class)
public class CloudBalance {

    @PlanningScore
    @Columns(columns = {@Column(name = "initScore"), @Column(name = "hardScore"), @Column(name = "softScore")})
    protected HardSoftScore score;

    ...
}
----

[NOTE]
====
Configure the same number of `@Column` annotations as the number of score levels in the score plus one (for the ``initScore``), otherwise Hibernate will fail fast because a property mapping has the wrong number of columns.
====

In this case, the DDL will look like this:

[source,sql]
----
CREATE TABLE CloudBalance(
    ...
    initScore INTEGER,
    hardScore INTEGER,
    softScore INTEGER
);
----

When using a `BigDecimal` based ``Score``, specify the precision and scale of the columns to avoid silent rounding:

[source,java,options="nowrap"]
----
@PlanningSolution
@Entity
@TypeDef(defaultForType = HardSoftBigDecimalScore.class, typeClass = HardSoftBigDecimalScoreHibernateType.class)
public class CloudBalance{

    @PlanningScore
    @Columns(columns = {
            @Column(name = "initScore")
            @Column(name = "hardScore", precision = 10, scale = 5),
            @Column(name = "softScore", precision = 10, scale = 5)})
    protected HardSoftBigDecimalScore score;

    ...
}
----

In this case, the DDL will look like this:

[source,sql]
----
CREATE TABLE CloudBalance(
    ...
    initScore INTEGER,
    hardScore DECIMAL(10, 5),
    softScore DECIMAL(10, 5)
);
----

When using any type of bendable ``Score``, specify the hard and soft level sizes as parameters:

[source,java,options="nowrap"]
----
@PlanningSolution
@Entity
@TypeDef(defaultForType = BendableScore.class, typeClass = BendableScoreHibernateType.class, parameters = {
        @Parameter(name = "hardLevelsSize", value = "3"),
        @Parameter(name = "softLevelsSize", value = "2")})
public class Schedule {

    @PlanningScore
    @Columns(columns = {
            @Column(name = "initScore")
            @Column(name = "hard0Score"),
            @Column(name = "hard1Score"),
            @Column(name = "hard2Score"),
            @Column(name = "soft0Score"),
            @Column(name = "soft1Score")})
    protected BendableScore score;

    ...
}
----

All this support is Hibernate specific because currently JPA 2.1's converters do not support converting to multiple columns.


[[jpaAndHibernatePlanningCloning]]
==== JPA and Hibernate: planning cloning

In JPA and Hibernate, there is usually a `@ManyToOne` relationship from most problem fact classes to the planning solution class.
Therefore, the problem fact classes reference the planning solution class, which implies that when the solution is xref:planner-configuration/planner-configuration.adoc#cloningASolution[planning cloned], they need to be cloned too.
Use an `@DeepPlanningClone` on each such problem fact class to enforce that:

[source,java,options="nowrap"]
----
@PlanningSolution // OptaPlanner annotation
@Entity // JPA annotation
public class Conference {

    @OneToMany(mappedBy="conference")
    private List<Room> roomList;

    ...
}
----

[source,java,options="nowrap"]
----
@DeepPlanningClone // OptaPlanner annotation: Force the default planning cloner to planning clone this class too
@Entity // JPA annotation
public class Room {

    @ManyToOne
    private Conference conference; // Because of this reference, this problem fact needs to be planning cloned too

}
----

Neglecting to do this can lead to persisting duplicate solutions, JPA exceptions or other side effects.


[[integrationWithXStream]]
=== XML or JSON: XStream

Enrich domain POJOs (solution, entities and problem facts) with XStream annotations to serialize them to/from XML or JSON.

Add a dependency to the `optaplanner-persistence-xstream` jar to take advantage of these extra integration features:


[[xStreamMarshallingAScore]]
==== XStream: marshalling a `Score`

When a `Score` is marshalled to XML or JSON by the default XStream configuration, it's verbose and ugly.
To fix that, configure the appropriate ``ScoreXStreamConverter``:

[source,java,options="nowrap"]
----
@PlanningSolution
@XStreamAlias("CloudBalance")
public class CloudBalance {

    @PlanningScore
    @XStreamConverter(HardSoftScoreXStreamConverter.class)
    private HardSoftScore score;

    ...
}
----

For example, this generates pretty XML:

[source,xml,options="nowrap"]
----
<CloudBalance>
   ...
   <score>0hard/-200soft</score>
</CloudBalance>
----

The same applies for a bendable score:

[source,java,options="nowrap"]
----
@PlanningSolution
@XStreamAlias("Schedule")
public class Schedule {

    @PlanningScore
    @XStreamConverter(BendableScoreXStreamConverter.class)
    private BendableScore score;

    ...
}
----

For example, this generates:

[source,xml,options="nowrap"]
----
<Schedule>
   ...
   <score>[0/0]hard/[-100/-20/-3]soft</score>
</Schedule>
----

When reading a bendable score from an XML element, the implied `hardLevelsSize` and `softLevelsSize`
must always be in sync with those in the solver.


[[integrationWithJaxb]]
=== XML or JSON: JAXB

Enrich domain POJOs (solution, entities and problem facts) with JAXB annotations to serialize them to/from XML or JSON.

Add a dependency to the `optaplanner-persistence-jaxb` jar to take advantage of these extra integration features:


[[jaxbMarshallingAScore]]
==== JAXB: marshalling a `Score`

When a `Score` is marshalled to XML or JSON by the default JAXB configuration, it's corrupted.
To fix that, configure the appropriate ``ScoreJaxbAdapter``:

[source,java,options="nowrap"]
----
@PlanningSolution
@XmlRootElement @XmlAccessorType(XmlAccessType.FIELD)
public class CloudBalance {

    @PlanningScore
    @XmlJavaTypeAdapter(HardSoftScoreJaxbAdapter.class)
    private HardSoftScore score;

    ...
}
----

For example, this generates pretty XML:

[source,xml,options="nowrap"]
----
<cloudBalance>
   ...
   <score>0hard/-200soft</score>
</cloudBalance>
----

The same applies for a bendable score:

[source,java,options="nowrap"]
----
@PlanningSolution
@XmlRootElement @XmlAccessorType(XmlAccessType.FIELD)
public class Schedule {

    @PlanningScore
    @XmlJavaTypeAdapter(BendableScoreJaxbAdapter.class)
    private BendableScore score;

    ...
}
----

For example, with a `hardLevelsSize` of `2` and a `softLevelsSize` of `3`, that will generate:

[source,xml,options="nowrap"]
----
<schedule>
   ...
   <score>[0/0]hard/[-100/-20/-3]soft</score>
</schedule>
----

The `hardLevelsSize` and `softLevelsSize` implied, when reading a bendable score from an XML element, must always be in sync with those in the solver.


[[integrationWithJackson]]
=== JSON: Jackson

Enrich domain POJOs (solution, entities and problem facts) with Jackson annotations to serialize them to/from JSON.

Add a dependency to the `optaplanner-persistence-jackson` jar and register `OptaPlannerJacksonModule`:

[source,java,options="nowrap"]
----
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.registerModule(OptaPlannerJacksonModule.createModule());
----


[[jacksonMarshallingAScore]]
==== Jackson: marshalling a `Score`

When a `Score` is marshalled to/from JSON by the default Jackson configuration, it fails.
The `OptaPlannerJacksonModule` fixes that, by using `HardSoftScoreJacksonSerializer`,
`HardSoftScoreJacksonDeserializer`, etc.

[source,java,options="nowrap"]
----
@PlanningSolution
public class CloudBalance {

    @PlanningScore
    private HardSoftScore score;

    ...
}
----

For example, this generates:

[source,json]
----
{
   "score":"0hard/-200soft"
   ...
}
----

[NOTE]
====
When reading a `BendableScore`, the `hardLevelsSize` and `softLevelsSize` implied in the JSON element,
must always be in sync with those defined in the `@PlanningScore` annotation in the solution class.For example:

[source,json]
----
{
   "score":"[0/0]hard/[-100/-20/-3]soft"
   ...
}
----

This JSON implies the `hardLevelsSize` is 2 and the `softLevelsSize` is 3,
which must be in sync with the `@PlanningScore` annotation:

[source,java,options="nowrap"]
----
@PlanningSolution
public class Schedule {

    @PlanningScore(bendableHardLevelsSize = 2, bendableSoftLevelsSize = 3)
    private BendableScore score;

    ...
}
----
====

When a field is the `Score` supertype (instead of a specific type such as `HardSoftScore`), it uses `PolymorphicScoreJacksonSerializer` and `PolymorphicScoreJacksonDeserializer`
to record the score type in JSON too, otherwise it would be impossible to deserialize it:

[source,java,options="nowrap"]
----
@PlanningSolution
public class CloudBalance {

    @PlanningScore
    private Score score;

    ...
}
----

For example, this generates:

[source,json]
----
{
   "score":{"HardSoftScore":"0hard/-200soft"}
   ...
}
----


[[integrationWithJsonb]]
=== JSON: JSON-B

Enrich domain POJOs (solution, entities and problem facts) with JSON-B annotations to serialize them to/from JSON.

Add a dependency to the `optaplanner-persistence-jsonb` jar and use `OptaPlannerJsonbConfig` to create a `Jsonb` instance:

[source,java,options="nowrap"]
----
JsonbConfig config = OptaPlannerJsonbConfig.createConfig();
Jsonb jsonb = JsonbBuilder.create(config);
----


[[jsonbMarshallingAScore]]
==== JSON-B: marshalling a `Score`

When a `Score` is marshalled to/from JSON by the default JSON-B configuration, it fails.
The `OptaPlannerJsonbConfig` fixes that, by using adapters including `BendableScoreJsonbAdapter`, `HardSoftScoreJsonbAdapter`, etc.

[source,java,options="nowrap"]
----
@PlanningSolution
public class CloudBalance {

    @PlanningScore
    private HardSoftScore score;

    ...
}
----

For example, this generates:

[source,json]
----
{"hardSoftScore":"0hard/-200soft"}
----

The same applies for a bendable score:

[source,java,options="nowrap"]
----
@PlanningSolution
public class CloudBalance {

    @PlanningScore
    private BendableScore score;

    ...
}
----

This generates:

[source,json]
----
{"bendableScore":"[0/0]hard/[-200/-20/0]soft"}
----

[[integrationWithQuarkus]]
== Quarkus

To use OptaPlanner with Quarkus, read the xref:quickstart/quarkus/quarkus-quickstart.adoc#quarkusJavaQuickStart[Quarkus Java quick start].
If you are starting a new project, visit the https://code.quarkus.io/[code.quarkus.io] and select
the _OptaPlanner AI constraint solver_ extension before generating your application.

[WARNING]
====
xref:drools-score-calculation/drools-score-calculation.adoc#droolsScoreCalculation[Drools score calculation] is incompatible with the `quarkus:dev` mode and deprecated.
====

Following properties are supported in the Quarkus `application.properties`:

:property_prefix: quarkus.
include::config-properties.adoc[]

[[integrationWithSpringBoot]]
== Spring Boot

To use OptaPlanner on Spring Boot, add the `optaplanner-spring-boot-starter` dependency
and read the xref:quickstart/spring-boot/spring-boot-quickstart.adoc#springBootJavaQuickStart[Spring Boot Java quick start].

[WARNING]
====
xref:drools-score-calculation/drools-score-calculation.adoc#droolsScoreCalculation[Drools score calculation] is currently incompatible with the dependency `spring-boot-devtools`:
none of the DRL rules will fire, due to ClassLoader issues.
Also, Drools score calculation is deprecated.
====

These properties are supported in Spring's `application.properties`:

:property_prefix:
include::config-properties.adoc[]


[[integrationWithSoaAndEsb]]
== SOA and ESB


[[integrationWithCamel]]
=== Camel and Karaf

http://camel.apache.org/[Camel] is an enterprise integration framework which includes support for OptaPlanner (starting from Camel 2.13). It can expose a use case as a REST service, a SOAP service, a JMS service, ...

http://camel.apache.org/optaplanner.html[Read the documentation for the camel-optaplanner component.]
That component works in Karaf too.


[[integrationWithOtherEnvironments]]
== Other environments


[[integrationWithJPMS]]
=== Java platform module system (Jigsaw)

When using OptaPlanner from code on the modulepath (Java 9 and higher),
_open_ your packages that contain your domain objects, constraints and solver configuration
_to all modules_ in your `module-info.java` file:

[source,java,options="nowrap"]
----
module org.optaplanner.cloudbalancing {
    requires org.optaplanner.core;
    ...

    opens org.optaplanner.examples.cloudbalancing; // Solver configuration
    opens org.optaplanner.examples.cloudbalancing.domain; // Domain classes
    opens org.optaplanner.examples.cloudbalancing.score; // Constraints
    ...
}
----

Otherwise OptaPlanner can't reach those classes or files, even if they are exported.

[[integrationWithOSGi]]
=== OSGi

Integration with OSGi is not supported.

[[integrationWithAndroid]]
=== Android

Android is not a complete JVM (because some JDK libraries are missing),
but OptaPlanner works on Android with xref:score-calculation/score-calculation.adoc#easyJavaScoreCalculation[easy Java] or xref:score-calculation/score-calculation.adoc#incrementalJavaScoreCalculation[incremental Java] score calculation.
The Drools rule engine does not work on Android yet,
so xref:constraint-streams/constraint-streams.adoc#constraintStreams[Constraint Streams] and xref:drools-score-calculation/drools-score-calculation.adoc#droolsScoreCalculation[Drools score calculation (Deprecated)] doesn't work on Android and its dependencies need to be excluded.

*Workaround to use OptaPlanner on Android:*

. Add a dependency to the `build.gradle` file in your Android project to exclude OptaPlanner's `optaplanner-constraint-streams-drools` and `optaplanner-constraint-drl` modules:
+
[source,gradle]
----
dependencies {
    ...
    implementation('org.optaplanner:optaplanner-core:...') {
        exclude group: 'org.optaplanner', module: 'optaplanner-constraint-streams-drools'
        exclude group: 'org.optaplanner', module: 'optaplanner-constraint-drl'
    }
    ...
}
----


[[integrationWithHumanPlanners]]
== Integration with human planners (politics)

A good OptaPlanner implementation beats any good human planner for non-trivial datasets.
Many human planners fail to accept this, often because they feel threatened by an automated system.

But despite that, both can benefit if the human planner becomes the supervisor of OptaPlanner:

* *The human planner defines, validates and tweaks the score function.*
** The human planner tweaks the constraint weights of the xref:score-calculation/score-calculation.adoc#constraintConfiguration[constraint configuration] in a UI, as the business priorities change over time.
** When the business changes, the score function often needs to change too.
The human planner can notify the developers to add, change or remove score constraints.
* *The human planner is always in control of OptaPlanner.*
** As shown in the course scheduling example, the human planner can pin down one or more planning variables to a specific planning value.
Because they are xref:repeated-planning/repeated-planning.adoc#pinnedPlanningEntities[pinned], OptaPlanner does not change them: it optimizes the planning around the enforcements made by the human.
If the human planner pins down all planning variables, he/she sidelines OptaPlanner completely.
** In a prototype implementation, the human planner occasionally uses pinning to intervene, but as the implementation matures, this should become obsolete.
The feature should be kept available as a reassurance for the humans, and in the event that the business changes dramatically before the score constraints are adjusted accordingly.

For this reason, it is recommended that the human planner is actively involved in your project.

image::integration/keepTheUserInControl.png[align="center"]

[[sizingHardwareAndSoftware]]
== Sizing hardware and software

Before sizing a OptaPlanner service, first understand the typical behaviour of a `Solver.solve()` call:

image::integration/sizingHardware.png[align="center"]

Understand these guidelines to decide the hardware for a OptaPlanner service:

* **RAM memory**: Provision plenty, but no need to provide more.
** The problem dataset, loaded before OptaPlanner is called, often consumes the most memory. It depends on the problem scale.
*** For example, in the Machine Reassignment example some datasets use over 1GB in memory. But in most examples, they use just a few MB.
*** If this is a problem, review the domain class structure: remove classes or fields that OptaPlanner doesn't need during solving.
*** OptaPlanner usually has up to three solution instances: the internal working solution, the best solution and the old best solution (when it's being replaced). However, these are all a xref:planner-configuration/planner-configuration.adoc#cloningASolution[planning clone] of each other, so many problem fact instances are shared between those solution instances.
** During solving, the memory is very volatile, because solving creates many short-lived objects. The Garbage Collector deletes these in bulk and therefore needs some heap space as a buffer.
** The maximum size of the JVM heap space can be in three states:
*** **Insufficient**: An `OutOfMemoryException` is thrown (often because the Garbage Collector is using more than 98% of the CPU time).
*** **Narrow**: The heap buffer for those short-lived instances is too small, therefore the Garbage Collector needs to run more than it would like to, which causes a performance loss.
**** Profiling shows that in the heap chart, the used heap space frequently touches the max heap space during solving. It also shows that the Garbage Collector has a significant CPU usage impact.
**** Adding more heap space increases the xref:score-calculation/score-calculation.adoc#scoreCalculationSpeed[score calculation speed].
*** **Plenty**: There is enough heap space. The Garbage Collector is active, but its CPU usage is low.
**** Adding more heap space does _not_ increase performance.
**** Usually, this is around 300 to 500MB above the dataset size, _regardless of the problem scale_ (except with xref:move-and-neighborhood-selection/move-and-neighborhood-selection.adoc#nearbySelection[nearby selection] and caching move selector, neither are used by default).
* **CPU power**: More is better.
** Improving CPU speed directly increases the xref:score-calculation/score-calculation.adoc#scoreCalculationSpeed[score calculation speed].
*** If the CPU power is twice as fast, it takes half the time to find the same result. However, this does not guarantee that it finds a better result in the same time, nor that it finds a similar result for a problem twice as big in the same time.
*** Increasing CPU power usually does not resolve scaling issues, because planning problems scale exponentially. Power tweaking the solver configuration has far better results for scaling issues than throwing hardware at it.
** During the `solve()` method, the CPU power will max out until it returns (except in xref:repeated-planning/repeated-planning.adoc#daemon[daemon mode] or if your xref:optimization-algorithms/optimization-algorithms.adoc#SolverEventListener[SolverEventListener] writes the best solution to disk or the network).
* **Number of CPU cores**: one CPU core per active Solver, plus at least one one for the operating system.
** So in a multitenant application, which has one Solver per tenant, this means one CPU core per tenant, unless the number of solver threads is limited, as that limits the number of tenants being solved in parallel.
** With Partitioned Search, presume one CPU core per partition (per active tenant), unless the number of partition threads is limited.
*** To reduce the number of used cores, it can be better to reduce the partition threads (so solve some partitions sequentially) than to reduce the number of partitions.
** In use cases with many tenants (such as scheduling Software as a Service) or many partitions, it might not be affordable to provision that many CPUs.
*** Reduce the number of active Solvers at a time. For example: give each tenant only one minute of machine time and use a `ExecutorService` with a fixed thread pool to queue requests.
*** Distribute the Solver runs across the day (or night). This is especially an opportunity in SaaS that's used across the globe, due to timezones: UK and India can use the same CPU core when scheduling at night.
** The SolverManager will take care of the orchestration, especially in those underfunded environments in which solvers (and partitions) are forced to share CPU cores or wait in line.
* **I/O (network, disk, ...)**: Not used during solving.
** OptaPlanner is not a web server: a solver thread does not block (unlike a servlet thread), each one fully drains a CPU.
*** A web server can handle 24 active servlets threads with eight cores without performance loss, because most servlets threads are blocking on I/O.
*** However, 24 active solver threads with eight cores will cause each solver's xref:score-calculation/score-calculation.adoc#scoreCalculationSpeed[score calculation speed] to be three times slower, causing a big performance loss.
** Note that calling any I/O during solving, for example a remote service in your score calculation, causes a huge performance loss because it's called thousands of times per second, so it should complete in microseconds. So no good implementation does that.

Keep these guidelines in mind when selecting and configuring the software.
See https://www.optaplanner.org/blog/archive.html[our blog archive] for the details of our experiments, which use our diverse set of examples.
Your mileage may vary.

* Operating System
** No experimentally proven advice yet (but prefer Linux anyway).
* JDK
** Version: Our benchmarks have consistently shown improvements in performance when comparing new JDK releases with their predecessors. It is therefore recommended using the latest available JDK. If you're interested in the performance comparisons of OptaPlanner running of different JDK releases, you can find them in the form of blog posts in https://www.optaplanner.org/blog/archive.html[our blog archive].
** Garbage Collector: ParallelGC can be potentially between 5% and 35% faster than G1GC (the default). Unlike web servers, OptaPlanner needs a GC focused on throughput, not latency. Use `-XX:+UseParallelGC` to turn on ParallelGC.
* Logging can have a severe impact on performance.
** Debug logging `org.drools` can reduce performance by a factor of 7.
** Debug logging `org.optaplanner` can be between 0% and 15% slower than info logging. Trace logging can be between 5% and 70% slower than info logging.
** Synchronous logging to a file has an additional significant impact for debug and trace logging (but not for info logging).
* Avoid a cloud environment in which you share your CPU core(s) with other virtual machines or containers. Performance (and therefore solution quality) can be unreliable when the available CPU power varies greatly.

Keep in mind that the perfect hardware/software environment will probably _not_ solve scaling issues (even Moore's law is too slow).
There is no need to follow these guidelines to the letter.
