Publish Application Logs to Amazon CloudWatch Logs

This guide describes how to create an application that publishes logs to Amazon Cloudwatch Logs using Micronaut AWS integration. Typically, application logs are written to stdout and to files, but the Micronaut Logging module supports multiple logging frameworks such as Logback, and logging to various other appenders, to email, a database, or other destinations.

Micronaut AWS provides integration between Micronaut and Amazon Web Services (AWS), including CloudWatch Logs.

CloudWatch Logs is a logging service from Amazon to monitor, store, and access your log files from Amazon Elastic Compute Cloud (Amazon EC2) instances, AWS CloudTrail, Route 53, and other sources.

Prerequisites #

• JDK 17 or higher. See Setting up Your Desktop.
• An Amazon Web Services (AWS) account. See Setting up Your Cloud Accounts.
• The GCN CLI. See Setting up Your Desktop. (Optional.)

Follow the steps below to create the application from scratch. However, you can also download the completed example in Java:

A note regarding your development environment

Consider using Visual Studio Code that provides native support for developing applications with the Graal Cloud Native extension.

If you use IntelliJ IDEA, enable annotation processing.

Windows platform: The GCN guides are compatible with Gradle only. Maven support is coming soon.

1. Create the Application #

Create an application using the GCN Launcher.

1. Open the GCN Launcher in advanced mode.

2. Create a new project using the following selections.
   • Project Type: Application (Default)
   • Project Name: aws-logging-demo
   • Base Package: com.example (Default)
   • Clouds: AWS
   • Language: Java (Default)
   • Build Tool: Gradle (Groovy) or Maven
   • Test Framework: JUnit (Default)
   • Java Version: 17 (Default)
   • Micronaut Version: (Default)
   • Cloud Services: Logging
   • Features: GraalVM Native Image (Default)
   • Sample Code: Yes (Default)
3. Click Generate Project, then click Download Zip. The GCN Launcher creates an application with the default package com.example in a directory named aws-logging-demo. The application ZIP file will be downloaded in your default downloads directory. Unzip it, open in your code editor, and proceed to the next steps.

Alternatively, use the GCN CLI as follows:

gcn create-app com.example.aws-logging-demo \
    --clouds=aws \
    --services=logging \
    --features=graalvm \
    --build=gradle \
    --jdk=17 \
    --lang=java

gcn create-app com.example.aws-logging-demo \
    --clouds=aws \
    --services=logging \
    --features=graalvm \
    --build=maven \
    --jdk=17 \
    --lang=java

The GCN Launcher creates a multimodule project with two subprojects: aws for Amazon Web Services, and lib. You develop the application logic in the aws subproject. If your application is to be deployed to multiple cloud providers, use the lib subproject to create classes that can be shared between the providers. This enables you to separate the code that is different between cloud providers, while keeping most of the implementation in the common lib subproject.

The Logging service that you selected at the project generation step bundles Logback, Jackson Databind, Oracle Cloud Infrastructure Logging, and other necessary dependencies. The Logback appender publishes logs to Amazon CloudWatch Logs.

1.1. Controller Class #

The example code includes a controller in a file named aws/src/main/java/com/example/LogController.java, which enables you to send POST requests to publish a message to a log:

package com.example;

import io.micronaut.http.annotation.Body;
import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Post;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Controller
class LogController {

    private static final Logger LOG = LoggerFactory.getLogger(LogController.class);

    @Post("/log")
    void log(@Body String message) {
        LOG.info(message);
    }
}

2. Get Started with CloudWatch Logs #

To use CloudWatch Logs, you need to authenticate to AWS, and then create a Log group and a Log stream.

To authenticate to AWS, use any of these methods.

2.1. Create a Log Group and a Log Stream #

You can create a Log group and a Log stream using either the AWS SDK v2, or the AWS console. Use the console to complete this guide.

1. Sign in to the AWS console and navigate to CloudWatch:

   

2. In the left sidebar, to go Logs, then Log groups.

3. Click Create log group. In the dialog box, enter a log group name and set the retention setting to “Never expire”. Click Create to confirm the action:

   

4. Once your log group is created, open it and click Create log stream. Enter a custom name. Click Create to confirm the action.

   

3. Configure Appender #

The GCN Launcher provided a file named aws/src/main/resources/logback.xml containing the configuration for an appender that publishes log statements to CloudWatch Logs.

1. Open the aws/src/main/resources/logback.xml file, and add these lines within the <appender> element named ‘CloudWatch’:

    <groupName>LogGroupName</groupName>
    <streamName>LogStreamName</streamName>
   

   Your logback.xml file should look like this:

    <configuration debug='false'>
   
        <!--
        You can un-comment the STDOUT appender and <appender-ref ref='STDOUT'/> in
        the cloud appender to log to STDOUT as the 'emergency' appender.
        -->
   
        <!--
        <appender name='STDOUT' class='ch.qos.logback.core.ConsoleAppender'>
            <encoder>
                <pattern>%cyan(%d{HH:mm:ss.SSS}) %gray([%thread]) %highlight(%-5level) %magenta(%logger{36}) - %msg%n</pattern>
            </encoder>
        </appender>
        -->
   
        <appender name='CLOUDWATCH' class='io.micronaut.aws.cloudwatch.logging.CloudWatchLoggingAppender'>
            <!-- <appender-ref ref='STDOUT'/> -->
            <encoder class='ch.qos.logback.core.encoder.LayoutWrappingEncoder'>
                <layout class='ch.qos.logback.contrib.json.classic.JsonLayout'>
                    <jsonFormatter class='io.micronaut.aws.cloudwatch.logging.CloudWatchJsonFormatter'/>
                </layout>
            </encoder>
            <groupName>LogGroupName</groupName>
            <streamName>LogStreamName</streamName>
        </appender>
   
        <root level='INFO'>
            <appender-ref ref='CLOUDWATCH'/>
        </root>
   
    </configuration>
   

2. Replace LogGroupName with the name of the Log group inside the <groupName> element.

3. Replace LogStreamName with the name of the Log stream inside the <streamName> element.

Note: You can un-comment the STDOUT appender as the ‘emergency’ appender. See the file for details.

Having configured the appender, you can proceed to run the application, publishing the logs.

4. Run the Application, Publish and View Logs #

1. Run the application using the following command (it will start the application on port 8080):

   • Gradle
   • Maven

   ./gradlew :aws:run

   ./mvnw install -pl lib -am
   ./mvnw mn:run -pl aws

2. Send some curl requests to test logging:

   curl -id '{"message":"your message here"}' \
      -H "Content-Type: application/json" \
      -X POST http://localhost:8080/log
   

3. From the AWS CloudWatch console, navigate to Logs, open your Log group and a Log group stream to view the logs published by your application.

5. Generate a Native Executable Using GraalVM #

GCN supports compiling a Java application ahead-of-time into a native executable using GraalVM Native Image. You can use the Gradle plugin for GraalVM Native Image building/Maven plugin for GraalVM Native Image building. Packaged as a native executable, it significantly reduces application startup time and memory footprint.

To generate a native executable, run the following command:

./gradlew :aws:nativeCompile

aws/build/native/nativeCompile/aws

./mvnw install -pl lib -am
./mvnw package -pl aws -Dpackaging=native-image

aws/target/aws

6. Run and Test the Native Executable #

Run the native executable, and then perform the same request as in step 4.

Summary #

This guide demonstrated how to use GCN to create an application that writes and publishes logs to CloudWatch Logs. Then, using the AWS console, you viewed the logs produced by the application. Finally, you saw how to build a native executable for this application with GraalVM Native Image, and ran it to test publishing logs to AWS CloudWatch Logs.

Related Documentation #

• Micronaut CloudWatch Logging
• CloudWatch Logs
• GraalVM Native Image