wiki:GrafanaMonitoring

Version 5 (modified by Ben, 3 years ago) (diff)

--

Overview and Installation

Grafana is a web-based application used for monitoring and analyzing data Grafana Docs. Grafana can be installed here.

Graphite is a database tool for storing time-series metrics. Graphite Docs. Graphite can be installed here (make sure all steps are followed; for any problems installing graphite check here).

Graphite consists of 2 important software components which we use:

  • Carbon: A service that listens for time-series data, which we can send via the command line, using pickle via python, or using our C++ Sensors class.
  • Whisper: A simple database library used for storing the data.


Feeding Data to Grafana

First, start Grafana and Carbon:

  • To start Grafana, run the following command:
    sudo service grafana-server start
    
  • To start Carbon, run the following 4 commands
    sudo systemctl start carbon-cache
    sudo systemctl enable httpd
    sudo systemctl start httpd
    /opt/graphite/bin/carbon-cache.py start
    

Following are 2 methods of sending data to .wsp files, or Whisper database files, which Graphite uses to store the data that is ultimately sent to Grafana.

  1. Using netcat directly from the terminal:
  • The command
    echo “data.test 4 `date +%s`” | nc 127.0.0.1 2003
    

This command:

  • Creates a whisper datafile PATH_TO_WSPFILES/data/test.wsp if it doesn't already exist. On the machine which I first tested this on, this path was /var/lib/carbon/whisper/data/test.wsp
  • Stores the value 4 in this file.
  • Uses the port 2003 to store data related to netcat (Graphite uses port 2003 for netcat and port 2004 for pickle).
  1. Using pickle via python:

On Siqi's machine I wrote a python script /home/bkilian/work/grafanaPlotting/apollo_read.py which effectively does the same thing, writing a value to a whisper file every 60 seconds, incrementing the value each time.