Build MongoDB Exporter for Prometheus

There is a number of exporters for Prometheus, that are contributed and maintained by a community. And here we talk about MongoDB Exporter. Welcome!

Table of Contents

Installing MongoDB on your own server

apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 9DA31620334BD75D9DCB49F368818C72E52529D4

echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/4.0 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.0.list

sudo apt-get update

apt-get install -y mongodb-org

Start and MongoDB is working

systemctl start mongod

systemctl enable mongod

Check if MongoDB listens on default port 27017 in netstat

netstat -tulpna

If something goes wrong check status

systemctl status mongod

and check logs

tail -f /var/log/mongodb/mongod.log

Open ports for mongoDB and Exporter in iptables

// Accept all traffic on loopback interfaces
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

// Allow Established and Related Incoming Connections
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

//And the same  for Outgoing
iptables -A OUTPUT -m conntrack --ctstate ESTABLISHED -j ACCEPT

//Drop invalid packets
iptables -A INPUT -m conntrack --ctstate INVALID -j DROP

//Open ports 22(ssh), 9216 (mongoDB exporter), 27017 (mongoDB)
for i in 22 9216 27017; do iptables -A INPUT -p tcp --dport $i -m conntrack --ctstate NEW,ESTABLISHED -j ACCEPT && iptables -A OUTPUT -p tcp --sport $i -m conntrack --ctstate ESTABLISHED -j ACCEPT; done

More information about iptables configuration you can find here on Digital Ocean

# mongo -u admin -p --authenticationDatabase admin

MongoDB shell version: 3.2.22
Enter password: 
connecting to: test
Server has startup warnings: 
2019-06-25T19:29:01.063+0200 I CONTROL  [initandlisten] 
2019-06-25T19:29:01.063+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2019-06-25T19:29:01.063+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2019-06-25T19:29:01.063+0200 I CONTROL  [initandlisten]

Download MongoDB Exporter source code and compile using GOLang

Download / Clone from github

https://github.com/percona/mongodb_exporter (current version 0.7.1)

Install Go on server, get the link to the latest version from official website

wget https://dl.google.com/go/go1.12.6.linux-amd64.tar.gz

Extract it and put in /usr/local

tar -xvf go1.12.6.linux-amd64.tar.gz
mv go /usr/local

Setup GoLang environment: GOROOT, GOPATH, PATH

Dont forget to create folder MyProjects in yours user Home Directory and folder bin inside it

export GOROOT=/usr/local/go
export GOPATH=$HOME/MyProjects
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH

Now verify it’s working, type:

go version

And check environment set properly

go env

Move recenlty downloaded source code of Exporter to MyProjects folder get inside of it and…

Compile MongoDB Exporter

run

make
make build

There could be several warnings you may ignore, but check if its asking you to create additional bin folder somewhere. Just do it and run again.

Compiled binary file mongodb_exporter may be found in subfolder named bin inside your project.

In case something goes wrong I’ve uploaded it to my github account

Configuring Prometheus MongoDB_Exporter to connect with MongoDB

By default mongoDB accepts connections only from localhost, but you can change it in file:

/etc/mongod.conf

Change bindIp from 127.0.0.1 to 0.0.0.0

Save, restart mongod service

service mongod restart

Don’t forget to check service status if it’s online.

Start MongoDB Exporter

./mongodb_exporter --mongodb.uri=mongodb://admin:password@192.0.2.30:27017/admin

May be you’ll need to create special user for monitoring purposes with limited access privileges

db.getSiblingDB("admin").createUser({
    user: "user",
    pwd: "password",
    roles: [
        { role: "clusterMonitor", db: "admin" },
        { role: "read", db: "local" }
    ]
})

That one should work.

Errors and test

Possible errors:

ERRO[0590] Can't create mongo session to [mongodb://admin:password@192.0.2.30:27017]  source="mongodb_collector.go:203"

Check your connection parameters and iptables configuration.

ERRO[0030] Could not get MongoDB BuildInfo: server selection error: server selection timeout
current topology: Type: Single
Servers:
Addr: 192.0.2.30:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: dial tcp 192.0.2.30:27017: connect: connection refused
!  source="connection.go:191"
ERRO[0030] Problem gathering the mongo server version: server selection error: server selection timeout
current topology: Type: Single
Servers:
Addr: 192.0.2.30:27017, Type: Unknown, State: Connected, Average RTT: 0, Last error: dial tcp 192.0.2.30:27017: connect: connection refused  source="mongodb_collector.go:211"

Same. Check your connection parameters and iptables configuration. Don’t forget to specify DB name.

Check traffic is passing using tcpdump (monitor all traffic on port 27017):

tcpdump -i eth1 'port 27017'

Also check connections are passing from proper IP:

tcpdump -i ens3 host 192.0.2.30

Check mongod service

systemctl status mongod

And that it’s assigned on specified port

nestat -tulpna

Integrating with Prometheus

Now add new job and scrape config to /etc/prometheus/prometheus.conf

- job_name: 'mongodb_exporter'
  scrape_interval: 5s
  static_configs:
    - targets:
      - '192.0.2.30:9216'

And reload Prometheus configuration:

systemctl status prometheus.service // to check PID
Main PID: 29055 (prometheus)
kill -SIGHUP 29055 // to reload configuration

And viola! 🙂

Then you can download Dashboard for Grafana (ID: 2583)

And configure service for systemd like that one:

[Unit]
Description=mongodb_exporter
Wants=network-online.target
After=network-online.target

[Service]
User=monitoring
Group=monitoring
Restart=on-failure
Type=simple
ExecStart=/usr/local/bin/mongodb_exporter --mongodb.uri=mongodb://password@192.0.2.30:27017/admin

[Install]
WantedBy=multi-user.target

Thank you for reading, hope it’s help!