Sending Spring Boot Metrics to Datadog

Hello and welcome to the next article in which we will briefly discuss Datadog and Micrometer, but what’s more important- we will learn how to send Spring Boot metrics to Datadog in practice.

Moreover, we will take a quick look into the Spring Boot Actuator and how it can help us, so, to let’s not waste our time and let’s get to work!

Video Tutorial

If you prefer video content, then check out my video:

If you find this content useful, please leave a subscription

Datadog

If you came here directly, then I bet you already know what Datadog is and what purpose it will serve in your project.

However, for those who hear about it for the first time, let me give you a really short intro.

Long story short- Datadog is an integrated platform for monitoring & security. It provides tools for monitoring applications, databases, distributed tracing, logs management, and many many more. In other words, makes all the things that are not so cool, but necessary in production-ready environments, a bit less painful

However, what is the most important for us today- it provides tools to collect metrics from our applications so that later we can make use of them.

But, what are they?

What Are Metrics?

According to the Datadog documentation:

Metrics are numerical values that can track anything about your environment over time.

And I must admit- I love this definition for its completeness in its simplicity.

Because basically, metric is everything in our system that we can assign a number to and track. From Garbage Collector stats, JVM memory, CPU, and request latency, to the average cart size. Literally everything.

And in this tutorial, we will see how to send metrics collected automatically by our Spring Boot app to DataDog.

Create New Project

As the first step, let’s navigate to the Spring Initializr page, select the following and import the project to our IDE:

Apart from versions and metadata, we can see that we added two dependencies: Spring Web and Spring Boot Actuator.

The first one is quite obvious, but in terms of metrics Spring Boot Actuator provides dependency management and auto-configuration for Micrometer – a vendor-neutral application observability facade.

With that, Spring Boot takes care of configuring MeterRegistry and the only thing we need to do is to add the micrometer-registry-{system} dependency to the build.gradle.kts :

implementation(«io.micrometer:micrometer-registry-datadog»)

Note: we could achieve exactly the same using the Spring Initializr page and searching for the Datadog dependency, but I wanted to emphasize the fact, that other vendors, like Elastic, or OpenTelemetry can be added in the same manner.

Verify and Expose Metrics Endpoint

After we import our project, let’s run it, and let’s take a look at the endpoints exposed by default by Spring Boot Actuator:

curl —location —request GET ‘http://localhost:8080/actuator’

As a result, we should see the following:

And the first lesson is that the metrics endpoint is not exposed by default.

So to change that, let’s navigate to the application.yaml and add the following:

management:
endpoints:
web:
exposure:
include: metrics, health

After that, let’s rerun the application.

As a result, this time, we should see that two new endpoints are present:

{
«_links»: {
«self»: {
«href»: «http://localhost:8080/actuator»,
«templated»: false
},
«health»: {
«href»: «http://localhost:8080/actuator/health»,
«templated»: false
},
«health-path»: {
«href»: «http://localhost:8080/actuator/health/{*path}»,
«templated»: true
},
«metrics-requiredMetricName»: {
«href»: «http://localhost:8080/actuator/metrics/{requiredMetricName}»,
«templated»: true
},
«metrics»: {
«href»: «http://localhost:8080/actuator/metrics»,
«templated»: false
}
}
}

The last one- /actuator/metrics– simply displays a list of available meter names, and when we invoke it, we should see the following JSON response:

{
«names»: [
«application.ready.time»,
«application.started.time»,
«disk.free»,
«disk.total»,
«executor.active»,
«executor.completed»,
«executor.pool.core»,
«executor.pool.max»,
«executor.pool.size»,
«executor.queue.remaining»,
«executor.queued»,
«http.server.requests»,
«http.server.requests.active»,
«jvm.buffer.count»,
«jvm.buffer.memory.used»,
«jvm.buffer.total.capacity»,
«jvm.classes.loaded»,
«jvm.classes.unloaded»,
«jvm.compilation.time»,
«jvm.gc.concurrent.phase.time»,
«jvm.gc.live.data.size»,
«jvm.gc.max.data.size»,
«jvm.gc.memory.allocated»,
«jvm.gc.memory.promoted»,
«jvm.gc.overhead»,
«jvm.gc.pause»,
«jvm.info»,
«jvm.memory.committed»,
«jvm.memory.max»,
«jvm.memory.usage.after.gc»,
«jvm.memory.used»,
«jvm.threads.daemon»,
«jvm.threads.live»,
«jvm.threads.peak»,
«jvm.threads.started»,
«jvm.threads.states»,
«logback.events»,
«process.cpu.time»,
«process.cpu.usage»,
«process.start.time»,
«process.uptime»,
«system.cpu.count»,
«system.cpu.usage»,
«tomcat.sessions.active.current»,
«tomcat.sessions.active.max»,
«tomcat.sessions.alive.max»,
«tomcat.sessions.created»,
«tomcat.sessions.expired»,
«tomcat.sessions.rejected»
]
}

In simple words, this is the list of all metrics collected at this point by our Spring Boot application. And yes- we will be able to export them to Datadog.

Nevertheless, let’s verify what the second endpoint- /actuator/metrics/{requiredMetricName}– does.

To do that, let’s make a GET request and specify any metric name:

curl —location —request GET ‘http://localhost:8080/actuator/metrics/process.cpu.usage’

This time, the endpoint will return metric details along with the current value:

{
«name»: «process.cpu.usage»,
«description»: «The «recent cpu usage» for the Java Virtual Machine process»,
«measurements»: [
{
«statistic»: «VALUE»,
«value»: 0.038209510879134205
}
],
«availableTags»: []
}

Create Datadog Account

Excellent! At this point, we are sure that our Spring Boot app collects metrics, so it’s time to proceed with the Datadog part.

If you haven’t done it yet, then let’s create a trial account together.

Let’s navigate to the https://www.datadoghq.com/ and click the “Free trial” button:

Following, let’s fill out the signup form:

And with that done, when we confirm our e-mail address, we will land on this page:

As we can see, the above page contains a bunch of guides on how to set up the agent.

And although we are not interested in any of those, this page has one, valuable information for us- the API key.

Note: I think that readers of this blog are well aware of that, but for my own peace of mind- API keys are vulnerable data, so you must be cautious with them!

At this point, please copy this value and we will use it in the proceeding steps.

Configure App To Send Metrics

When we have our Datadog account ready, we can finally configure our app.

API Key

Firstly, let’s get back to our Spring Boot project and update the application.yaml :

management:
endpoints:
web:
exposure:
include: metrics, health
datadog:
metrics:
export:
api-key: YOUR_KEY_VALUE

Of course, we must replace the YOUR_KEY_VALUE with the actual API key.

Update Datadog URI

If you are using Datadog US (the option you could select when signing up), then that’s pretty much it and you can restart the application.

By default, metrics are sent to the Datadog US site (api.datadoghq.com). If your Datadog project is hosted on one of the other sites, or you need to send metrics through a proxy, configure the URI accordingly.

So given the above, if we use another region, then we need to add one more line:

management:
endpoints:
web:
exposure:
include: metrics, health
datadog:
metrics:
export:
api-key: YOUR_KEY_VALUE
uri: https://api.datadoghq.eu

Update Metrics Upload Interval

Following, let’s see how easily we can update the interval, in which data are sent to Datadog:

management:
endpoints:
web:
exposure:
include: metrics, health
datadog:
metrics:
export:
api-key: YOUR_KEY_VALUE
uri: https://api.datadoghq.eu
step: 10s

The default value is 30 seconds. And to change that, the only thing we need to do is to add the step value.

Set Spring Logging Level to DEBUG

As the last thing, which technically is not directly related to sending metrics, I would like to add the following to the root of our application.yaml file:

logging:
level:
root: DEBUG

This way we set the logging level in our Spring Boot application to DEBUG.

Why? It will be useful in a moment when we will be testing our implementation.

Testing

With all of that done, we can finally verify that everything is working properly.

Verify Spring Boot Sends Data to Datadog

Firstly, let’s rerun our application and take a look at the logs.

If we see the following:

i.m.datadog.DatadogMeterRegistry : successfully sent 78 metrics to datadog

Then data are successfully sent to the Datadog API.

Let’s leave it running for some time (like 2 minutes, or something), so that we have actual data later to work with

Review Metrics in Datadog

After that time, the only thing left is to take a look at metrics in Datadog.

If we haven’t closed the Datadog “Agent Setup” page, then it should automatically close and redirect us to the dashboard.

However, if that didn’t happen, then no worries, we can simply navigate to the metrics summary page:

On this page, we can see all the metrics that have been gathered so far. Long story short, this page contains meta-information about metrics themselves, like tags, etc.

I strongly recommend you spend here some time and figure out different options.

And if we would like to take a look at the actual values, then let’s navigate to the metrics explorer:

I know, the image is quite small (the “Open Image in New Tab” option will help here ). However, the only thing we need to do here is to specify the metric name in the left upper part and the period we would like to view in the right upper part.

After that, we will see the above chart.

And basically, that’s how metrics work with Spring Boot and Datadog, but please don’t close this article yet, I wanted to show you two, interesting things that may be useful to you in real life.

Sending Custom Tags

Tags are nothing else, than additional metadata you can send with metrics.

Why are they useful? Well, thanks to them we can later differentiate different metrics. For example, we can send a tag with the application name, so that later we will group metrics by microservices, or environment value, like dev, staging, or prod.

In Spring Boot, we have several ways to achieve that. And probably the easiest way is to (again) update the application.yaml:

management:
endpoints:
web:
exposure:
include: metrics, health
datadog:
metrics:
export:
api-key: YOUR_KEY_VALUE
uri: https://api.datadoghq.eu
step: 10s
metrics:
tags:
application: «my-app-one»

As we can see, we can add a custom tag by using metrics.tags and specifying that as a key-value pair. In our case- the application tag has the my-app-one value.

This approach is not only straightforward but also quite flexible. We could always put a placeholder for the environment variable instead of the hardcoded value.

Anyway, when we rerun our application and let Spring send some metrics to the Datadog, we will see that from now on we can filter values using the “from” field:

Again, the “Open Image in New Tab” may be helpful here

Turn Off Metrics Export

As the last thing, let’s take a look at how we can instruct our Spring app to stop sending metrics to Datadog.

Again, let’s navigate to the application.yaml file and set the enabled flag to false:

management:
endpoints:
web:
exposure:
include: metrics, health
datadog:
metrics:
export:
enabled: false
api-key: YOURE_KEY_VALUE
uri: https://api.datadoghq.eu
step: 10s

Of course, we can use an environment variable placeholder (like ${DATADOG_ENABLED}) and turn on/off export in different environments.

Summary

That’s all for this tutorial on how to send Spring Boot metrics to Datadog.

As always, you can find the source code in this GitHub repository.

Have a great day and don’t forget to leave a comment!

The post Sending Spring Boot Metrics to Datadog appeared first on Codersee | Kotlin, Ktor, Spring.

coding genius