LIGHT

  • News
  • Docs
  • Community
  • Reddit
  • GitHub
Star

How to use Prometheus for metrics

Introduction

Light-4j provide Prometheus metrics handler as MiddlewareHandler to collects the API runtime information and save it to Prometheus Metric data module. And then use pre-defined HttpHandler (PrometheusGetHandler) to expose the prometheus Metric data module; Prometheus server will pull the metric from metrics_path which is configurable on

Prometheus config yml file (for exampe: /v1/prometheus) ; The pulling interval is second base on the config file (scrape_interval: 30s).

Configuration

Configuration file (prometheus.yml) for the MiddlewareHandler (com.networknt.metrics.prometheus.PrometheusHandler)

# Metrics handler configuration

# If prometheus metrics handler is enabled or not
enabled: false

Change the enabled value to true to enable the Prometheus metrics handler;

Configuration for services

If user want to use Prometheus metrics handler as metrics MiddlewareHandler handler for the service, simply set the config “prometheusMetrics” as true

Config to generate new service by light-codegen

Here is an example of config.json for openapi generator.

{
  "name": "petstore",
  "version": "1.0.1",
  "groupId": "com.networknt",
  "artifactId": "petstore",
  "rootPackage": "com.networknt.petstore",
  "handlerPackage":"com.networknt.petstore.handler",
  "modelPackage":"com.networknt.petstore.model",
  "overwriteHandler": true,
  "overwriteHandlerTest": true,
  "overwriteModel": true,
  "httpPort": 8080,
  "enableHttp": true,
  "httpsPort": 8443,
  "enableHttps": false,
  "enableRegistry": false,
  "supportDb": true,
  "dbInfo": {
    "name": "mysql",
    "driverClassName": "com.mysql.jdbc.Driver",
    "jdbcUrl": "jdbc:mysql://mysqldb:3306/oauth2?useSSL=false",
    "username": "root",
    "password": "my-secret-pw"
  },
  "supportH2ForTest": false,
  "supportClient": false,
  "prometheusMetrics": true
}

Then following the step to generate service API.

The generated service will use Prometheus metrics handler as metrics MiddlewareHandler handler and the the API runtime information will be save to prometheus Metric data module.

Change existing service to use light-4j Prometheus MiddlewareHandler handler

If user want to change existing service which created before based on light-4j palteform (default use [InfluxDB metrics handler]: /concern/metrics/) to use Prometheus metrics handler, please change the service by following steps:

  1. Change the service POM to add light-4j Prometheus MiddlewareHandler handler dependency:
        <dependency>
            <groupId>com.networknt</groupId>
            <artifactId>prometheus</artifactId>
            <version>${version.light-4j}</version>
        </dependency>

  1. Change the service.yml file to use Prometheus MiddlewareHandler handler:
- com.networknt.handler.MiddlewareHandler:
  # Exception Global exception handler that needs to be called first to wrap all middleware handlers and business handlers
  - com.networknt.exception.ExceptionHandler
  # Metrics handler to calculate response time accurately, this needs to be the second handler in the chain.
  - com.networknt.metrics.prometheus.PrometheusHandler
  ...

  1. Change the api spec (openapi.json or swagger.json) by adding Prometheus metrics_path:
"/prometheus":{"get":{"responses":{"200":{"description":"successful operation"}},"parameters":[]}}
  1. Change the
import com.networknt.metrics.prometheus.PrometheusGetHandler;

public class PathHandlerProvider implements HandlerProvider {
    @Override
    public HttpHandler getHandler() {
        return Handlers.routing()
                .add(Methods.GET, "/v1/prometheus", new PrometheusGetHandler());
    }
}

The metrics_path is configurable. In light-codegen, we set it as /prometheus, but it can be change to any url user want. Then user just to to set the same “metrics_path” in the Prometheus config file.

Configuration for Prometheus (config with Consul )

Prometheus is configured via a configuration file. The file is written in YAML format, defined by the Prometheus scheme. By default, the targets ( nodes to monitor ) are provided statically in the prometheus config file. Static targets are simple when we know in advance our infrastructure. But for the microservice architecture, service API will have dynamic target on the cloud. So our solution is use consul to provide targets for prometheus to monitor dynamically

In prometheus, we need to configure it use consul_sd_targets. Prometheus will then query consul http interface for the catalog of targets to monitor.

Here is the sample config file. It can be found from link:

  - job_name: 'services'
    scrape_interval: 15s
    consul_sd_configs:
      - server: 'consul:8500'
        services:
          - 'multimodule_tram-todolist-command-service'
          - 'multimodule_tram-todolist-view-service'

    relabel_configs:
      - source_labels: ['__meta_consul_service']
        regex:         '(.*)'
        target_label:  'service'
        replacement:   '$1'
      - source_labels: ['__meta_consul_tags']
        target_label: 'slave_name'
        regex: '.*slave_name=([^,]*).*'
        replacement: '${1}'
      - source_labels: ['__meta_consul_tags']
        target_label: 'public_hostname'
        regex: '.*public_hostname=([^,]*).*'
        replacement: '${1}'

    metrics_path: /v1/prometheus
    # metrics_path defaults to '/metrics'
    # scheme defaults to 'http'.


Verification:

  1. The latest version of light-tram-4j example todo-list has been change to use prometheus metrics handler; Please follow the step in the tutrial to start the sample service api:

light-tram todo-list service API

  1. Starts consul service discovery by docker-compose:

 cd ~/networknt/light-docker
 docker-compose -f docker-compose-consul.yml up

  1. Starts registrator service by docker-compose:

 cd ~/networknt/light-docker
 docker-compose -f docker-compose-registrator.yml up

  1. Starts Prometheus server by docker-compose (the Prometheus server will start based on the config on ~/networknt/light-docker/prometheus/prometheus.yml. User can change it for different api):

 cd ~/networknt/light-docker
 docker-compose -f docker-compose-prometheus.yml up

  1. Open a browser to verify Consul:
http://127.0.0.1:8500/

  1. Open a browser to verify Prometheus:
http://localhost:9090/

Using Prometheus in Grafana:

Light framwork provided docker-compose file (docker-compose-prometheus.yml) includes the Grafana image: image: grafana/grafana:4.6.3

So user can open browser to view the Prometheus from Grafana

http://127.0.0.1:3000/

user: admin
passwork: admin

Please follow the following step to set the Prometheus in Grafana:

Adding the data source to Grafana:

  1. Open the side menu by clicking the Grafana icon in the top header.

  2. In the side menu under the Dashboards link you should find a link named Data Sources.

  3. Click the + Add data source button in the top header.

  4. Select Prometheus from the Type dropdown.

You can get the detail from: http://docs.grafana.org/features/datasources/prometheus/

  • About Light Platform
    • Overview
    • Testimonials
    • What is Light
    • Features
    • Principles
    • Benefits
    • Roadmap
    • Community
    • Articles
    • Videos
    • License
  • Getting Started
    • Get Started Overview
    • Environment
    • Light Codegen Tool
    • Light Rest 4j
    • Light Tram 4j
    • Light Graphql 4j
    • Light Hybrid 4j
    • Light Eventuate 4j
    • Light Oauth2
    • Light Portal Service
    • Light Proxy Server
    • Light Router Server
    • Light Config Server
    • Light Saga 4j
    • Light Session 4j
    • Webserver
    • Websocket
    • Spring Boot Servlet
  • Architecture
    • Architecture Overview
    • API Category
    • API Gateway
    • Architecture Patterns
    • CQRS
    • Eco System
    • Event Sourcing
    • Fail Fast vs Fail Slow
    • Integration Patterns
    • JavaEE declining
    • Key Distribution
    • Microservices Architecture
    • Microservices Monitoring
    • Microservices Security
    • Microservices Traceability
    • Modular Monolith
    • Platform Ecosystem
    • Plugin Architecture
    • SOA
    • Scalability and Performance
    • Serverless
    • Service Collaboration
    • Service Mesh
    • Spring is bloated
    • Stages of API Adoption
    • Transaction Management
    • Microservices Cross-cutting Concerns Options
    • Service Mesh Plus
    • Service Discovery
  • Design
    • Design Overview
    • Design First vs Code First
    • Desgin Pattern
    • Service Evolution
    • Consumer Contract and Consumer Driven Contract
    • Handling Partial Failure
    • Idempotency
    • Environment Segregation
    • Multi-Tenancy
    • Why check token expiration
    • WebServices to Microservices
  • Cross-Cutting Concerns
    • Concerns Overview
  • API Styles
    • Light-4j for absolute performance
    • Style Overview
    • Distributed session on IMDG
    • Eventuate - Event Sourcing and CQRS
    • Hybrid - Modularized Monolithic
    • REST - Representational state transfer
    • Saga - Distributed Transactions
    • Tram - Transactional Messaging
    • Web Server with Light Platform
    • Websocket with light platform
    • Spring Boot Integration
    • Single Page Application
    • GraphQL - A query language for your API
  • Infrastructure Services
    • Service Overview
    • Light Proxy
    • Light Router
    • Introduction
    • Architecture
    • Light Portal
    • Messaging Infrastructure
    • Centralized Logging
    • Light OAuth2
    • Services
    • Metrics and Alerts
    • Reference
    • Config Server
    • Tokenization
  • Tool Chain
    • Tool Chain Overview
  • Utility Library
  • Service Consumer
    • Service Consumer
  • Development
    • Development Overview
    • Best Practices
    • Development Flow
    • Platform Developer
    • Develop Build
    • Application
    • Service Provider Developer
    • Service Consumer Developer
  • Deployment
    • Deployment Overview
    • Frontend Backend
    • Linux Service
    • Windows Service
    • Install Eventuate on Windows
    • Secure API
    • Client vs light-router
    • Memory Limit
    • Deploy to Kubernetes
  • Benchmark
    • Benchmark Overview
  • Tutorial
    • Tutorial Overview
  • Troubleshooting
    • Troubleshoot
  • FAQ
    • FAQ Overview
  • Milestones
  • Contribute
    • Contribute to Light
    • Development
    • Documentation
    • Example
    • Tutorial
“How to use Prometheus for metrics” was last updated: April 2, 2019: fixes #62 add Chinese language for the document site (5c820aa)
Improve this page
  • News
  • Docs
  • Community
  • Reddit
  • GitHub
  • About Light Platform
  • Getting Started
  • Architecture
  • Design
  • Cross-Cutting Concerns
  • API Styles
  • Infrastructure Services
  • Tool Chain
  • Utility Library
  • Service Consumer
  • Development
  • Deployment
  • Benchmark
  • Tutorial
  • Troubleshooting
  • FAQ
  • Milestones
  • Contribute