Running Node Exporter on Pfsense

Using node_exporter to monitor pfsense

Why what where

I’m using Pfsense as my edge device in a small homelab-colocation running in Montreal. It’s running under ESXI as a VM but I still required some level of visibility.

Because $dayjob, I selected Prometheus as my central monitoring platform. The Prometheus stack is based around active scraping of “exporters” that are gathering data from local or remote source. By default, Pfsense does not provide an API or an endpoint that can be scraped and there is no such thing as a prometheus_exporter.

That said, Prometheus does offer a generic Linux exporter called node_exporter which was ported to FreeBSD.

It’s not available as a Pfsense bundle package so the installation process is a bit different. Do it at your own risk ;)

How to install node_exporter under Pfsense.

We can see that the FreeBSD port does exist here :

You will need to sudo package installed + ssh enabled to log into Pfsense directly.

It is available for FreeBSD 11 (which is what Pfsense is based on)

[2.4.5-RELEASE][[email protected]]/home/laurentdumont: sudo FreeBSD-version

We can then install the package from our mirror.

sudo pkg add
Fetching node_exporter-1.0.1.txz: 100%    3 MiB   3.3MB/s    00:01    
Installing node_exporter-1.0.1...
Newer FreeBSD version for package node_exporter:
To ignore this error set IGNORE_OSVERSION=yes
- package: 1104001
- running kernel: 1103507
Ignore the mismatch and continue? [y/N]: y
Extracting node_exporter-1.0.1: 100%
Message from node_exporter-1.0.1:

If upgrading from a version of node_exporter <0.15.0 you'll need to update any
custom command line flags that you may have set as it now requires a
double-dash (--flag) instead of a single dash (-flag).
The collector flags in 0.15.0 have now been replaced with individual boolean
flags and the -collector.procfs` and -collector.sysfs` flags have been renamed
to --path.procfs and --path.sysfs respectively.

You can see a warning for what seems to be a FreeBSD mismatch but I was able to accept the warning and go on my merry way.

Things have been running for a few days without any fires so it seems rather safe. I’m not sure how Pfsense itself will deal with the extra package if I do an upgrade of the OS so that remains to be seen…

[2.4.5-RELEASE][[email protected]]/home/laurentdumont: node_exporter --help
usage: node_exporter [<flags>]

  -h, --help                    Show context-sensitive help (also try --help-long and --help-man).
                                Regexp of mount points to ignore for filesystem collector.
                                Regexp of filesystem types to ignore for filesystem collector.
                                Regexp of net devices to blacklist (mutually exclusive to device-whitelist).
                                Regexp of net devices to whitelist (mutually exclusive to device-blacklist).
                                NTP server to use for ntp collector
                                NTP protocol version
                                Certify that collector.ntp.server address is not a public ntp server
      --collector.ntp.ip-ttl=1  IP TTL to use while sending NTP query
                                Max accumulated distance to the root
                                Offset between local clock and local ntpd time to tolerate
      --path.procfs="/proc"     procfs mountpoint.
      --path.sysfs="/sys"       sysfs mountpoint.
      --path.rootfs="/"         rootfs mountpoint.
                                Path to runit service directory.
                                XML RPC endpoint.""  
                                Directory to read text files with metrics from.
      --collector.boottime      Enable the boottime collector (default: enabled).
      --collector.buddyinfo     Enable the buddyinfo collector (default: disabled).
      --collector.cpu           Enable the cpu collector (default: enabled).
      --collector.exec          Enable the exec collector (default: enabled).
      --collector.filesystem    Enable the filesystem collector (default: enabled).
      --collector.loadavg       Enable the loadavg collector (default: enabled).
      --collector.meminfo       Enable the meminfo collector (default: enabled).
      --collector.netdev        Enable the netdev collector (default: enabled).
      --collector.ntp           Enable the ntp collector (default: disabled).
      --collector.runit         Enable the runit collector (default: disabled).
      --collector.supervisord   Enable the supervisord collector (default: disabled).
      --collector.textfile      Enable the textfile collector (default: enabled).
      --collector.time          Enable the time collector (default: enabled).
      --collector.uname         Enable the uname collector (default: enabled).
      --collector.zfs           Enable the zfs collector (default: enabled).
      --collector.devstat       Enable the devstat collector (default: disabled).
                                Address on which to expose metrics and web interface.
                                Path under which to expose metrics.
                                Exclude metrics about the exporter itself (promhttp_*, process_*, go_*).
      --web.max-requests=40     Maximum number of parallel scrape requests. Use 0 to disable.
                                Set all collectors to disabled by default.
      --web.config=""           [EXPERIMENTAL] Path to config yaml file that can enable TLS or authentication.
      --log.level=info          Only log messages with the given severity or above. One of: [debug, info, warn, error]
      --log.format=logfmt       Output format of log messages. One of: [logfmt, json]
      --version                 Show application version.

Once the package has been installed, you can start it with.

sudo service node_exporter onestart

This starts the node_exporter service but will not auto-start if Pfsense does reboot.

[2.4.5-RELEASE][[email protected]]/home/laurentdumont: level=info ts=2020-11-03T17:08:58.360Z caller=node_exporter.go:177 msg="Starting node_exporter" version="(version=1.0.1, branch=release-1.0, revision=0)"
level=info ts=2020-11-03T17:08:58.360Z caller=node_exporter.go:178 msg="Build context" build_context="(go=go1.15.2, user=root, date=)"
level=info ts=2020-11-03T17:08:58.360Z caller=node_exporter.go:105 msg="Enabled collectors"
level=info ts=2020-11-03T17:08:58.360Z caller=node_exporter.go:112 collector=boottime
level=info ts=2020-11-03T17:08:58.360Z caller=node_exporter.go:112 collector=cpu
level=info ts=2020-11-03T17:08:58.360Z caller=node_exporter.go:112 collector=exec
level=info ts=2020-11-03T17:08:58.360Z caller=node_exporter.go:112 collector=filesystem
level=info ts=2020-11-03T17:08:58.360Z caller=node_exporter.go:112 collector=loadavg
level=info ts=2020-11-03T17:08:58.360Z caller=node_exporter.go:112 collector=meminfo
level=info ts=2020-11-03T17:08:58.360Z caller=node_exporter.go:112 collector=netdev
level=info ts=2020-11-03T17:08:58.361Z caller=node_exporter.go:112 collector=textfile
level=info ts=2020-11-03T17:08:58.361Z caller=node_exporter.go:112 collector=time
level=info ts=2020-11-03T17:08:58.361Z caller=node_exporter.go:112 collector=uname
level=info ts=2020-11-03T17:08:58.361Z caller=node_exporter.go:112 collector=zfs
level=info ts=2020-11-03T17:08:58.361Z caller=node_exporter.go:191 msg="Listening on" address=:9100
level=info ts=2020-11-03T17:08:58.361Z caller=tls_config.go:170 msg="TLS is disabled and it cannot be enabled on the fly." http2=false
level=error ts=2020-11-03T17:08:58.840Z caller=collector.go:161 msg="collector failed" name=zfs duration_seconds=1.7671e-05 err="couldn't get sysctl: no such file or directory"

[2.4.5-RELEASE][[email protected]]/home/laurentdumont: 
[2.4.5-RELEASE][[email protected]]/home/laurentdumont: 
[2.4.5-RELEASE][[email protected]]/home/laurentdumont: sudo ps aux | grep node_exporter
laurentdumont 60965   0.0  0.8 722916 17296  1  S    17:08       0:00.01 node_exporter
laurentdumont 61329   0.0  0.1   6560  2332  1  S+   17:09       0:00.00 grep node_exporter

You can test it by accessing : http://IP_OF_PFSENSE:9100/metrics

And it works!

# HELP node_network_receive_bytes_total Network device statistic receive_bytes.
# TYPE node_network_receive_bytes_total counter
node_network_receive_bytes_total{device="em0"} 2.24300166503e+12
node_network_receive_bytes_total{device="em1"} 3.401224344e+09
node_network_receive_bytes_total{device="em2"} 1.558897176e+09
node_network_receive_bytes_total{device="enc0"} 0
node_network_receive_bytes_total{device="lo0"} 105356
node_network_receive_bytes_total{device="ovpns1"} 2.28780693e+08
node_network_receive_bytes_total{device="pflog0"} 0
node_network_receive_bytes_total{device="pfsync0"} 0
# HELP node_network_receive_drop_total Network device statistic receive_drop.
# TYPE node_network_receive_drop_total counter
node_network_receive_drop_total{device="em0"} 0
node_network_receive_drop_total{device="em1"} 0
node_network_receive_drop_total{device="em2"} 0
node_network_receive_drop_total{device="enc0"} 0

The FreeBSD ports of node_exporter seems rather limited and is missing some stuff I’d like to export (like total sessions, UDP/TCP stats). Looking at the Github node_exporter page, these are just not supported for FreeBSD :(

The next step would be to try the NetData Prometheus integration and see if it provides more information.

We can then graph the data in Grafana!


Laurent Dumont
I route packets and juggle bytes