Create and Deploy a Streaming Application with the Oracle Cloud Infrastructure Streaming Service

This guide describes how to create and deploy a streaming application that uses Micronaut Streaming and the Oracle Cloud Infrastructure Streaming service. The application consists of two Micronaut microservices deployed to Oracle Cloud Infrastructure that use the Oracle Cloud Infrastructure Streaming service to communicate with each other in an asynchronous and decoupled way.

Oracle Cloud Infrastructure Streaming service is a real-time, serverless, streaming platform compatible with Apache Kafka.

This guide assumes that you are familiar with the Oracle Cloud Infrastructure Command Line Interface (CLI).

Prerequisites #

• JDK 17 or higher. See Setting up Your Desktop.
• An Oracle Cloud Infrastructure account. See Setting up Your Cloud Accounts.
• The Oracle Cloud Infrastructure CLI installed with local access configured.
• An Oracle Cloud Infrastructure compartment where you will create a stream. Ensure appropriate permissions are granted to your user account to manage the Streaming service.
• A Virtual Cloud Network in Oracle Cloud Infrastructure with a single regional public subnet, which will be used by a Compute instance.
• A Docker-API compatible container runtime such as Rancher Desktop or Docker for running tests.
• The GCN CLI. See Setting up Your Desktop. (Optional.)

Follow the steps below to create the application from scratch. However, you can also download the completed example:

A note regarding your development environment

Consider using Visual Studio Code that provides native support for developing applications with the Graal Cloud Native extension.

If you use IntelliJ IDEA, enable annotation processing.

Windows platform: The GCN guides are compatible with Gradle only. Maven support is coming soon.

1. Create the Microservices #

The two microservices are:

• Books returns a list of books. It uses a domain consisting of a book name and an International Standard Book Number (ISBN). It also publishes a message to the streaming service every time a book is accessed.
• Analytics connects to the streaming service to update the analytics for every book (a counter). It also exposes an endpoint to retrieve the counter.

1.1. Create the Books Microservice #

1. Open the GCN Launcher in advanced mode.

2. Create a new project using the following selections.
   • Project Type: Application (Default)
   • Project Name: books
   • Base Package: com.example.publisher
   • Clouds: OCI
   • Language: Java (Default)
   • Build Tool: Gradle (Groovy) or Maven
   • Test Framework: JUnit (Default)
   • Java Version: 17 (Default)
   • Micronaut Version: (Default)
   • Cloud Services: Streaming
   • Features: Awaitility Framework, GraalVM Native Image, Micronaut Serialization Jackson Core, and Reactor
   • Sample Code: No
3. Click Generate Project, then click Download Zip. The GCN Launcher creates an application with the default package com.example.publisher in a directory named books. The application ZIP file will be downloaded in your default downloads directory. Unzip it, open in your code editor, and proceed to the next steps.

Alternatively, use the GCN CLI as follows:

gcn create-app com.example.publisher.books \
    --clouds=oci \
    --services=streaming \
    --features=awaitility,graalvm,reactor,serialization-jackson \
    --build=gradle \
    --jdk=17 \
    --lang=java \
    --example-code=false

gcn create-app com.example.publisher.books \
    --clouds=oci \
    --services=streaming \
    --features=awaitility,graalvm,reactor,serialization-jackson \
    --build=maven \
    --jdk=17 \
    --lang=java \
    --example-code=false

1.1.1. Book Domain Class

The GCN Launcher created a Book domain class in a file named lib/src/main/java/com/example/publisher/Book.java, as follows:

package com.example.publisher;

import io.micronaut.core.annotation.Creator;
import io.micronaut.serde.annotation.Serdeable;

import java.util.Objects;

@Serdeable
public class Book {

    private final String isbn;
    private final String name;

    @Creator
    public Book(String isbn, String name) {
        this.isbn = isbn;
        this.name = name;
    }

    public String getIsbn() {
        return isbn;
    }

    public String getName() {
        return name;
    }

    @Override
    public String toString() {
        return "Book{" +
                "isbn='" + isbn + '\'' +
                ", name='" + name + '\'' +
                '}';
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        Book other = (Book) o;
        return Objects.equals(isbn, other.isbn) &&
                Objects.equals(name, other.name);
    }

    @Override
    public int hashCode() {
        return Objects.hash(isbn, name);
    }
}

1.1.2. BookService

To keep this guide simple there is no database persistence: the Books microservice keeps the list of books in memory. The GCN Launcher created a class named BookService in lib/src/main/java/com/example/publisher/BookService.java with the following contents:

package com.example.publisher;

import jakarta.annotation.PostConstruct;
import jakarta.inject.Singleton;
import java.util.ArrayList;
import java.util.List;
import java.util.Optional;

@Singleton
public class BookService {

    private final List<Book> bookStore = new ArrayList<>();

    @PostConstruct
    void init() {
        bookStore.add(new Book("1491950358", "Building Microservices"));
        bookStore.add(new Book("1680502395", "Release It!"));
        bookStore.add(new Book("0321601912", "Continuous Delivery"));
    }

    public List<Book> listAll() {
        return bookStore;
    }

    public Optional<Book> findByIsbn(String isbn) {
        return bookStore.stream()
                .filter(b -> b.getIsbn().equals(isbn))
                .findFirst();
    }
}

1.1.3. BookController

The GCN Launcher created a controller to access Book instances in a file named lib/src/main/java/com/example/publisher/BookController.java with the following contents:

package com.example.publisher;

import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Get;

import java.util.List;
import java.util.Optional;

@Controller("/books") // <1>
class BookController {

    private final BookService bookService;

    BookController(BookService bookService) { // <2>
        this.bookService = bookService;
    }

    @Get // <3>
    List<Book> listAll() {
        return bookService.listAll();
    }

    @Get("/{isbn}") // <4>
    Optional<Book> findBook(String isbn) {
        return bookService.findByIsbn(isbn);
    }
}

1 The @Controller annotation defines the class as a controller mapped to the root URI /books.

2 Use constructor injection to inject a bean of type BookService.

3 The @Get annotation maps the listAll method to an HTTP GET request on /books.

4 The @Get annotation maps the findBook method to an HTTP GET request on /books/{isbn}.

1.1.4. BookControllerTest

The GCN Launcher created a test for BookController to verify the interaction with the Analytics microservice in a file named oci/src/test/java/com/example/publisher/BookControllerTest.java with the following contents:

package com.example.publisher;

import io.micronaut.configuration.kafka.annotation.KafkaListener;
import io.micronaut.configuration.kafka.annotation.Topic;
import io.micronaut.core.type.Argument;
import io.micronaut.http.HttpRequest;
import io.micronaut.http.client.HttpClient;
import io.micronaut.http.client.annotation.Client;
import io.micronaut.http.client.exceptions.HttpClientResponseException;
import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestInstance;

import jakarta.inject.Inject;
import java.util.Collection;
import java.util.Optional;
import java.util.concurrent.ConcurrentLinkedDeque;

import static io.micronaut.configuration.kafka.annotation.OffsetReset.EARLIEST;
import static java.util.concurrent.TimeUnit.SECONDS;
import static org.awaitility.Awaitility.await;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNotNull;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.TestInstance.Lifecycle.PER_CLASS;

@MicronautTest
@TestInstance(PER_CLASS) // <1>
class BookControllerTest {

    private static final Collection<Book> received = new ConcurrentLinkedDeque<>();

    @Inject
    AnalyticsListener analyticsListener; // <2>

    @Inject
    @Client("/")
    HttpClient client; // <3>

    @Test
    void testMessageIsPublishedToKafkaWhenBookFound() {
        String isbn = "1491950358";

        Optional<Book> result = retrieveGet("/books/" + isbn); // <4>
        assertNotNull(result);
        assertTrue(result.isPresent());
        assertEquals(isbn, result.get().getIsbn());

        await().atMost(5, SECONDS).until(() -> !received.isEmpty()); // <5>

        assertEquals(1, received.size()); // <6>
        Book bookFromKafka = received.iterator().next();
        assertNotNull(bookFromKafka);
        assertEquals(isbn, bookFromKafka.getIsbn());
    }

    @Test
    void testMessageIsNotPublishedToKafkaWhenBookNotFound() throws Exception {
        assertThrows(HttpClientResponseException.class, () -> {
            retrieveGet("/books/INVALID");
        });

        Thread.sleep(5_000); // <7>
        assertEquals(0, received.size());
    }

    @AfterEach
    void cleanup() {
        received.clear();
    }

    @KafkaListener(offsetReset = EARLIEST)
    static class AnalyticsListener {

        @Topic("analytics")
        void updateAnalytics(Book book) {
            received.add(book);
        }
    }

    private Optional<Book> retrieveGet(String url) {
        return client.toBlocking().retrieve(HttpRequest.GET(url), Argument.of(Optional.class, Book.class));
    }
}

1 Classes that implement TestPropertyProvider must use this annotation to create a single class instance for all tests.

2 Dependency injection for the AnalyticsListener class declared later in the file. This is a listener class that replicates the functionality of the class of the same name in the Analytics microservice.

3 Dependency injection for an HTTP client that the Micronaut framework will implement at compile to make calls to BookController.

4 Use the HttpClient to retrieve the details of a Book, which will trigger sending a message.

5 Wait a few seconds for the message to arrive; it should happen very quickly, but the message will be sent on a separate thread.

6 Verify that the message was received and that it has the correct data.

7 Wait a few seconds to ensure no message is received.

1.1.5. AnalyticsClient

The GCN Launcher created a client interface to send messages to the streaming service in a file named lib/src/main/java/com/example/publisher/AnalyticsClient.java with the contents shown below. (Micronaut generates an implementation for the client interface at compilation time.)

package com.example.publisher;

import io.micronaut.configuration.kafka.annotation.KafkaClient;
import io.micronaut.configuration.kafka.annotation.Topic;
import reactor.core.publisher.Mono;

@KafkaClient
public interface AnalyticsClient {

    @Topic("analytics") // <1>
    Mono<Book> updateAnalytics(Book book); // <2>
}

1 Set the name of the topic. This matches the name of the topic used by AnalyticsListener in the Analytics microservice.

Note: This must match the name of the stream that you will create later in Oracle Cloud Infrastructure.

2 Send the Book POJO. Micronaut will automatically convert it to JSON before sending it.

1.1.6. AnalyticsFilter

Sending a message to the streaming service is as simple as injecting AnalyticsClient and calling its updateAnalytics method. The goal is to send a message every time the details of a book are returned from the Books microservice or, in other words, every time there is a call to http://localhost:8080/books/{isbn}. To achieve this, the GCN Launcher created an Http Server Filter in a file named lib/src/main/java/com/example/publisher/AnalyticsFilter.java as follows:

package com.example.publisher;

import io.micronaut.http.HttpRequest;
import io.micronaut.http.MutableHttpResponse;
import io.micronaut.http.annotation.Filter;
import io.micronaut.http.filter.HttpServerFilter;
import io.micronaut.http.filter.ServerFilterChain;
import reactor.core.publisher.Flux;
import org.reactivestreams.Publisher;

@Filter("/books/?*") // <1>
class AnalyticsFilter implements HttpServerFilter { // <2>

    private final AnalyticsClient analyticsClient;

    AnalyticsFilter(AnalyticsClient analyticsClient) { // <3>
        this.analyticsClient = analyticsClient;
    }

    @Override
    public Publisher<MutableHttpResponse<?>> doFilter(HttpRequest<?> request,
                                                      ServerFilterChain chain) { // <4>
        return Flux
                .from(chain.proceed(request)) // <5>
                .flatMap(response -> {
                    Book book = response.getBody(Book.class).orElse(null); // <6>
                    if (book == null) {
                        return Flux.just(response);
                    }
                    return Flux.from(analyticsClient.updateAnalytics(book)).map(b -> response); // <7>
                });
    }
}

1 Annotate the class with @Filter and define the Ant-style matcher pattern to intercept all calls to the desired URIs.

2 The class must implement HttpServerFilter.

3 Dependency injection for AnalyticsClient.

4 Implement the doFilter method.

5 Call the request; this will invoke the controller action.

6 Get the response from the controller and return the body as an instance of the Book class.

7 If the book is retrieved, use the client to send a message.

1.1.7. Test the Microservice

Use the following command to test the Books microservice:

./gradlew :oci:test

./mvnw install -pl lib -am
./mvnw package -pl oci -DskipTests
./mvnw test

1.2. Create the Analytics Microservice #

1. Open the GCN Launcher in advanced mode.

2. Create a new project using the following selections.
   • Project Type: Application (Default)
   • Project Name: analytics
   • Base Package: com.example.consumer
   • Clouds: OCI
   • Language: Java (Default)
   • Build Tool: Gradle (Groovy) or Maven
   • Test Framework: JUnit (Default)
   • Java Version: 17 (Default)
   • Micronaut Version: (Default)
   • Cloud Services: Streaming
   • Features: Awaitility Framework, GraalVM Native Image and Micronaut Serialization Jackson Core
   • Sample Code: No
3. Click Generate Project, then click Download Zip. The GCN Launcher creates an application with the default package com.example.consumer in a directory named analytics. The application ZIP file will be downloaded in your default downloads directory. Unzip it, open in your code editor, and proceed to the next steps.

Alternatively, use the GCN CLI as follows:

gcn create-app com.example.consumer.analytics \
    --clouds=oci \
    --services=streaming \
    --features=awaitility,graalvm,serialization-jackson \
    --build=gradle \
    --jdk=17 \
    --lang=java \
    --example-code=false

gcn create-app com.example.consumer.analytics \
    --clouds=oci \
    --services=streaming \
    --features=awaitility,graalvm,serialization-jackson \
    --build=maven \
    --jdk=17 \
    --lang=java \
    --example-code=false

1.2.1. Domain Classes

The GCN Launcher created a Book domain class in a file named lib/src/main/java/com/example/consumer/Book.java, as shown below. (This Book POJO is the same as the one in the Books microservice. In a real application this would be in a shared library.)

package com.example.consumer;

import io.micronaut.core.annotation.Creator;
import io.micronaut.serde.annotation.Serdeable;

import java.util.Objects;

@Serdeable
public class Book {

    private final String isbn;
    private final String name;

    @Creator
    public Book(String isbn, String name) {
        this.isbn = isbn;
        this.name = name;
    }

    public String getIsbn() {
        return isbn;
    }

    public String getName() {
        return name;
    }

    @Override
    public String toString() {
        return "Book{" +
                "isbn='" + isbn + '\'' +
                ", name='" + name + '\'' +
                '}';
    }

    @Override
    public boolean equals(Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;
        Book other = (Book) o;
        return Objects.equals(isbn, other.isbn) &&
                Objects.equals(name, other.name);
    }

    @Override
    public int hashCode() {
        return Objects.hash(isbn, name);
    }
}

The GCN Launcher also created a BookAnalytics domain class in a file named lib/src/main/java/com/example/consumer/BookAnalytics.java, as follows:

package com.example.consumer;

import io.micronaut.core.annotation.Creator;
import io.micronaut.serde.annotation.Serdeable;

@Serdeable
public class BookAnalytics {

    private final String bookIsbn;
    private final long count;

    @Creator
    public BookAnalytics(String bookIsbn, long count) {
        this.bookIsbn = bookIsbn;
        this.count = count;
    }

    public String getBookIsbn() {
        return bookIsbn;
    }

    public long getCount() {
        return count;
    }
}

1.2.2. AnalyticsService

To keep this guide simple there is no database persistence: the Analytics microservice keeps book analytics in memory. The GCN Launcher created a class named AnalyticsService in lib/src/main/java/com/example/consumer/AnalyticsService.java with the following contents:

package com.example.consumer;

import jakarta.inject.Singleton;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.stream.Collectors;

@Singleton
public class AnalyticsService {

    private final Map<Book, Long> bookAnalytics = new ConcurrentHashMap<>(); // <1>

    public void updateBookAnalytics(Book book) { // <2>
        bookAnalytics.compute(book, (k, v) -> {
            return v == null ? 1L : v + 1;
        });
    }

    public List<BookAnalytics> listAnalytics() { // <3>
        return bookAnalytics
                .entrySet()
                .stream()
                .map(e -> new BookAnalytics(e.getKey().getIsbn(), e.getValue()))
                .collect(Collectors.toList());
    }
}

1 Keep the book analytics in memory.

2 Initialize and update the analytics for the book passed as parameter.

3 Return all the analytics.

1.2.3. AnalyticsServiceTest

The GCN Launcher created a test for the AnalyticsService class, in a file named oci/src/test/java/com/example/consumer/AnalyticsServiceTest.java, as follows:

package com.example.consumer;

import static org.junit.jupiter.api.Assertions.assertEquals;

import io.micronaut.test.extensions.junit5.annotation.MicronautTest;
import org.junit.jupiter.api.Test;

import jakarta.inject.Inject;
import java.util.List;

@MicronautTest
class AnalyticsServiceTest {

    @Inject
    AnalyticsService analyticsService;

    @Test
    void testUpdateBookAnalyticsAndGetAnalytics() {
        Book b1 = new Book("1491950358", "Building Microservices");
        Book b2 = new Book("1680502395", "Release It!");

        analyticsService.updateBookAnalytics(b1);
        analyticsService.updateBookAnalytics(b1);
        analyticsService.updateBookAnalytics(b1);
        analyticsService.updateBookAnalytics(b2);

        List<BookAnalytics> analytics = analyticsService.listAnalytics();
        assertEquals(2, analytics.size());

        assertEquals(3, findBookAnalytics(b1, analytics).getCount());
        assertEquals(1, findBookAnalytics(b2, analytics).getCount());
    }

    private BookAnalytics findBookAnalytics(Book b, List<BookAnalytics> analytics) {
        return analytics
                .stream()
                .filter(bookAnalytics -> bookAnalytics.getBookIsbn().equals(b.getIsbn()))
                .findFirst()
                .orElseThrow(() -> new RuntimeException("Book not found"));
    }
}

1.2.4. AnalyticsController

The GCN Launcher created a Controller to create an endpoint for the Analytics microservice in a file named lib/src/main/java/com/example/consumer/AnalyticsController.java, as follows:

package com.example.consumer;

import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Get;

import java.util.List;

@Controller("/analytics") // <1>
class AnalyticsController {

    private final AnalyticsService analyticsService;

    AnalyticsController(AnalyticsService analyticsService) {
        this.analyticsService = analyticsService;
    }

    @Get // <2>
    List<BookAnalytics> listAnalytics() {
        return analyticsService.listAnalytics();
    }
}

1 The @Controller annotation defines the class as a controller mapped to the root URI /analytics.

2 The @Get annotation maps the listAnalytics method to an HTTP GET request on /analytics.

The application doesn’t expose the method updateBookAnalytics created in AnalyticsService. This method will be invoked when reading messages from Kafka.

1.2.5. AnalyticsListener

The GCN Launcher created a class to act as a consumer of the messages sent to the streaming service by the Books microservice. The Micronaut framework implements logic to invoke the consumer at compile time. The AnalyticsListener class is in a file named lib/src/main/java/com/example/consumer/AnalyticsListener.java, as follows:

package com.example.consumer;

import io.micronaut.configuration.kafka.annotation.KafkaListener;
import io.micronaut.configuration.kafka.annotation.Topic;
import io.micronaut.context.annotation.Requires;
import io.micronaut.context.env.Environment;

@Requires(notEnv = Environment.TEST) // <1>
@KafkaListener // <2>
class AnalyticsListener {

    private final AnalyticsService analyticsService; // <3>

    AnalyticsListener(AnalyticsService analyticsService) { // <3>
        this.analyticsService = analyticsService;
    }

    @Topic("analytics") // <4>
    void updateAnalytics(Book book) {
        analyticsService.updateBookAnalytics(book); // <5>
    }
}

1 Do not load this bean in the test environment: you can run tests without access to a streaming service.

2 Annotate the class with @KafkaListener to indicate that this bean consumes messages from Kafka.

3 Constructor injection for AnalyticsService.

4 Annotate the method with @Topic and specify the topic name. This matches the name of the topic used by AnalyticsClient in the Books microservice.

Note: This must match the name of the stream that you will create later in Oracle Cloud Infrastructure. (See section 2.4.)

5 Call AnalyticsService to update the analytics for the book.

1.2.6. Test the Microservice

Use the following command to test the Analytics microservice:

./gradlew :oci:test

./mvnw install -pl lib -am
./mvnw package -pl oci -DskipTests
./mvnw test

1.2.7. Change the port for the Analytics Microservice

The Books and Analytics microservices are both run on the same Oracle Cloud Infrastructure Compute instance, so they must run on different ports. Change the port that Analytics runs on by editing its oci/src/main/resources/application.properties to include the following configuration:

micronaut.server.port=8081

2. Run the Microservices #

2.1. Start the Books Microservice #

To run the Books microservice, use the following command, which starts the application on port 8080.

./gradlew :oci:run

./mvnw install -pl lib -am
./mvnw -pl oci mn:run

2.2. Start the Analytics Microservice #

To run the Analytics microservice, use the following command, which starts the application on port 8081.

./gradlew :oci:run

./mvnw install -pl lib -am
./mvnw -pl oci mn:run

2.3. Test the Microservices #

Use curl to test the microservices, as follows.

1. Retrieve the list of books:

    curl http://localhost:8080/books
   

    [{"isbn":"1491950358","name":"Building Microservices"},{"isbn":"1680502395","name":"Release It!"},{"isbn":"0321601912","name":"Continuous Delivery"}]
   

2. Retrieve the details of a specified book:

    curl http://localhost:8080/books/1491950358
   

    {"isbn":"1491950358","name":"Building Microservices"}
   

3. Retrieve the analytics:

    curl http://localhost:8081/analytics
   

    [{"bookIsbn":"1491950358","count":1}]
   

Update the curl command to the Books microservice to retrieve other books and repeat the invocations, then re-run the curl command to the Analytics microservice to see that the counts increase.

3. Set up Oracle Cloud Infrastructure Resources #

In this section, you find the OCID of the compartment and configure a stream.

3.1. Find the OCID of the Compartment #

Find the OCID of the compartment where you will deploy the microservices. Run this command to list all the compartments in your root compartment:

oci iam compartment list

Find the compartment by the name or description in the JSON output. It should look like this:

{
    "compartment-id": "ocid1.tenancy.oc1..aaaaaaaaud4g4e5ovjaw...",
    "defined-tags": {},
    "description": "GCN Guides",
    "freeform-tags": {},
    "id": "ocid1.compartment.oc1..aaaaaaaarkh3s2wcxbbm...",
    "inactive-status": null,
    "is-accessible": null,
    "lifecycle-state": "ACTIVE",
    "name": "gcn-guides",
    "time-created": "2021-05-02T23:54:28.392000+00:00"
}

In this case, there is a compartment named “gcn-guides”.

Use the OCID from the id property; the compartment-id property is the parent compartment.

For convenience, save the compartment OCID as an environment variable. For Linux or macOS, run the following command:

export C=ocid1.compartment.oc1..aaaaaaaarkh3s2wcxbbm...

or for Windows, if using cmd run this command:

set C=ocid1.compartment.oc1..aaaaaaaarkh3s2wcxbbm...

and if using PowerShell run

$C = "ocid1.compartment.oc1..aaaaaaaarkh3s2wcxbbm..."

The examples below use Linux/macOS syntax for environment variables, for example -c $C. If you use Windows cmd, change those to -c %C% (but no change is needed if you use PowerShell).

3.2. Create a Stream Pool and Streams #

Use the Oracle Cloud Infrastructure CLI to create a stream pool:

oci streaming admin stream-pool create --compartment-id $C --name "gcn-guide-pool"

Copy value in the id row and export it for later. Also save the fully-qualified domain name (FQDN) URL.

export STREAM_POOL_ID=<replace_with_the_copied_value>

Create a stream within the pool you created with the topic name used in the microservices (“analytics”):

oci streaming admin stream create --stream-pool-id $STREAM_POOL_ID --partitions 1 --name "analytics"

3.3. Create a User #

Use the following command to create a user:

oci iam user create --description "gcn-guide-streaming-user" --name "gcn-guide-streaming-user"

Copy the value in the id row and export it for later.

export USER_OCID=<replace_with_the_copied_value>

3.4. Create a Dynamic Group #

Instance Principal authentication enables Oracle Cloud Infrastructure applications to retrieve secrets from Oracle Cloud Infrastructure Vault. (For more information about using Oracle Cloud Infrastructure Vault, see Securely Store Application Secrets in Oracle Cloud Infrastructure Vault.) To use this, create a dynamic group and add a policy statement granting permission.

Choose a group name, such as “gcn-guide-dg”, and a matching rule (to determine group membership). For this guide, use a fairly broad rule:

ALL {instance.compartment.id = <compartment_ocid>}

as follows:

oci iam dynamic-group create --name gcn-guide-dg \
       --description gcn-guide-dg \
       --matching-rule "ALL {instance.compartment.id = '<compartment_ocid>'}" \

Replace <compartment_ocid> with the OCID of the compartment where you create your Compute instances.

3.4.1. Create Auth Token

You need an auth token to use as the password in the configuration. Create one as follows:

oci iam auth-token create --description "gcn-guide-token" --user-id $USER_OCID

3.4.2. Write a Policy

Create a policy to grant the stream access to the user and group you created. Use the compartment OCID where you created the streams.

oci iam policy create --compartment-id $C --description “gcn-guide-streaming-policy” --name "gcn-guide-streaming-policy" \
  --statements '["Allow dynamic-group gcn-guide-dg to manage streams in compartment id <compartment_ocid>"]'

3.5. Configure an Oracle Cloud Infrastructure Compute Instance #

To run the applications in the oraclecloud environment, add the following configuration in both microservices books/oci/src/main/resources/application-oraclecloud.properties and analytics/oci/src/main/resources/application-oraclecloud.properties:

oci.config.instance-principal.enabled=true

3.6. Launch an Oracle Cloud Infrastructure Compute Instance #

Use the Create and Connect a Micronaut Application to an Oracle Cloud Infrastructure MySQL Database guide to create a compute instance. Follow the steps in the “Create a Compute Instance” .

Note: You will use this Compute instance to deploy the Books and Analytics microservices.

4. Generate a Native Executable Using GraalVM #

GCN supports compiling a Java application ahead-of-time into a native executable using GraalVM Native Image. You can use the Gradle plugin for GraalVM Native Image building/Maven plugin for GraalVM Native Image building. Packaged as a native executable, it significantly reduces application startup time and memory footprint.

To generate a native executable, run the following command for each microservice:

./gradlew :oci:nativeCompile

MICRONAUT_ENVIRONMENTS=oraclecloud oci/build/native/nativeCompile/oci

gradlew :oci:nativeCompile

cmd /C "set MICRONAUT_ENVIRONMENTS=oraclecloud && oci\build\native\nativeCompile\oci"

./mvnw install -pl lib -am
./mvnw package -pl oci -Dpackaging=native-image

MICRONAUT_ENVIRONMENTS=oraclecloud oci/target/oci

mvnw install -pl lib -am
mvnw package -pl oci -Dpackaging=native-image

cmd /C "set MICRONAUT_ENVIRONMENTS=oraclecloud && oci\target\oci"

Start the native executables for the two microservices and run the same curl requests as before to check that everything works as expected.

You can see that the microservices behave identically as if you run them from JAR files, but with reduced startup time and smaller memory footprint.

5. Deploy the Books Microservice to Oracle Cloud Infrastructure #

1. Ensure that the private key you downloaded has the correct permissions, as follows:

    chmod 400 /path/to/ssh-key-*.key
   

2. Create a JAR file containing all the microservice’s dependencies, as follows:

   • Gradle
   • Maven

   ./gradlew :oci:shadowJar

   ./mvnw install -pl lib -am
   ./mvnw package -pl oci -DskipTests

3. Copy the JAR file to your Compute instance, as follows:

   • Gradle
   • Maven

   scp -i /path/to/ssh-key-*.key oci/build/libs/oci-0.1-SNAPSHOT-all.jar opc@[COMPUTE_INSTANCE_PUBLIC_IP]:~/booking_application.jar

   scp -i /path/to/ssh-key-*.key oci/target/oci-0.1-SNAPSHOT-all.jar opc@[COMPUTE_INSTANCE_PUBLIC_IP]:~/booking_application.jar

4. Connect to the Compute instance:

    ssh -i /path/to/ssh-key-*.key opc@[COMPUTE_INSTANCE_PUBLIC_IP]
   

5. Once connected, install GraalVM Native Image and JDK for Java 17. See Install GraalVM JDK with Native Image.

6. Open up the firewall to port 8080 using the following commands:

    sudo firewall-cmd --permanent --zone=public --add-port=8080/tcp
    sudo firewall-cmd --reload
   

7. Set environment variables, as follows:

    export OCI_STREAM_POOL_FQDN=<the FQDN URL you saved earlier, for example cell-1.streaming.us-ashburn-1.oci.oraclecloud.com>
    export OCI_TENANCY_NAME=<the name of your Oracle Cloud Infrastructure tenancy>
    export OCI_USERNAME=<the username you created above>
    export OCI_STREAM_POOL_OCID=<the stream pool OCID you saved earlier>
    export OCI_AUTH_TOKEN=<the auth token value you saved earlier>
   

8. Start the Books microservice, as follows:

    MICRONAUT_ENVIRONMENTS=oraclecloud java -jar booking_application.jar
   

9. Verify that the application is running by invoking the controller at http://[COMPUTE_INSTANCE_PUBLIC_IP]:8080/books using curl:

    curl -i http://[COMPUTE_INSTANCE_PUBLIC_IP]:8080/books
   

10. Invoke the controller endpoint to trigger a message to be published to the Streaming service. You can test other ISBNs as well.

    curl -i http://[COMPUTE_INSTANCE_PUBLIC_IP]:8080/books/1491950358
    

6. Deploy the Analytics Microservice to Oracle Cloud Infrastructure #

1. Create a JAR file containing all the microservice’s dependencies, as follows:

   • Gradle
   • Maven

   ./gradlew :oci:shadowJar

   ./mvnw install -pl lib -am
   ./mvnw package -pl oci -DskipTests

2. Copy the JAR file to your Compute instance, as follows:

   • Gradle
   • Maven

   scp -i /path/to/ssh-key-*.key oci/build/libs/oci-0.1-all.jar opc@[COMPUTE_INSTANCE_PUBLIC_IP]:~/analytics_application.jar

   scp -i /path/to/ssh-key-*.key oci/target/oci-0.1-all.jar opc@[COMPUTE_INSTANCE_PUBLIC_IP]:~/analytics_application.jar

3. Connect to the Compute instance:

    ssh -i /path/to/ssh-key-*.key opc@[COMPUTE_INSTANCE_PUBLIC_IP]
   

4. Open up the firewall to port 8081 using the following commands:

    sudo firewall-cmd --permanent --zone=public --add-port=8081/tcp
    sudo firewall-cmd --reload
   

5. Set environment variables, as follows:

    export OCI_STREAM_POOL_FQDN=<the FQDN URL you saved earlier, for example cell-1.streaming.us-ashburn-1.oci.oraclecloud.com>
    export OCI_TENANCY_NAME=<the name of your Oracle Cloud Infrastructure tenancy>
    export OCI_USERNAME=<the username you created above>
    export OCI_STREAM_POOL_OCID=<the stream pool OCID you saved earlier>
    export OCI_AUTH_TOKEN=<the auth token value you saved earlier>
   

6. Start the Analytics microservice, as follows:

    MICRONAUT_ENVIRONMENTS=oraclecloud java -jar analytics_application.jar
   

7. Verify that the application is running by invoking the controller at http://[COMPUTE_INSTANCE_PUBLIC_IP]:8081/analytics using curl:

    curl -i http://[COMPUTE_INSTANCE_PUBLIC_IP]:8081/analytics
   

7. Deploy Native Executables to Oracle Cloud Infrastructure #

7.1. Run Native Executables on the Compute Instance #

1. Download the example ZIP file and copy it to the Compute instance:

   scp -i /path/to/ssh-key-*.key /path/to/downloaded/streaming_sample.zip opc@[COMPUTE_INSTANCE_PUBLIC_IP]:~/streaming_sample.zip
   

2. Once copied, connect to the OCI instance (if it has disconnected by now):

    ssh -i /path/to/ssh-key-*.key opc@[COMPUTE_INSTANCE_PUBLIC_IP]
   

3. Unpack the compressed file:

   sudo yum install unzip
   unzip streaming_sample.zip
   

4. Install the packages that GraalVM needs to work properly on OCI:

   sudo yum -y install libstdc++
   sudo ln -s /usr/lib64/libstdc++.so.6 /usr/lib/libstdc++.so
   

5. Set the value of the JAVA_HOME environment variable to the location of GraalVM, for example:

   export JAVA_HOME=/usr/lib64/graalvm/graalvm22-ee-java17
   

6. You must apply the settings from section 4.5 (Configure an OCI Compute Instance).

7. To generate a native executable, run the following command for each microservice:

   • Gradle
   • Maven

   ./gradlew :oci:nativeCompile

   The native executable is created in the oci/build/native/nativeCompile/ directory and can be run with the following command.

   oci/build/native/nativeCompile/oci

   ./mvnw install -pl lib -am
   ./mvnw package -pl oci -Dpackaging=native-image

   The native executable is created in the oci/target/ directory and can be run with the following command:

   oci/target/oci

Start the native executables for the two microservices and run the same curl requests as before to check that everything works as expected.

You can see that the microservices behave identically as if you run them as Java applications, but with reduced startup time and smaller memory footprint.

Summary #

In this guide you created and deployed a streaming application with the Micronaut framework, Kafka, and Oracle Cloud Infrastructure Streaming service. The communication between two microservices acting as a producer and consumer ran asynchronously. Then you packaged these microservices into native executables with GraalVM Native Image for their faster startup and reduced memory footprint, and deployed to Oracle Cloud Infrastructure.

Related Documentation #

• Oracle Cloud Infrastructure Streaming service
• Securely Store Application Secrets in Oracle Cloud Infrastructure Vault
• Kafka support in Micronaut
• Graal Cloud Native
• GraalVM Native Image Quick Start