Hello! We are MTR Design and site making is our speciality. We work with UK based startups, established businesses, media companies and creative individuals and turn their ideas into reality. This is a thrill and we totally love it.

Server monitoring with S2Mon - Part 2

Author: Emil Filipov

In part 1 I covered the reasons why it is in your best interest to monitor your servers, and how can S2Mon help with that task. Well, we know that monitoring can be all cool and shiny, but how hard is it to set up? After all, the (real or perceived) effort required for the initial configuration is the single biggest reason why people avoid monitoring. In this part I'll explore the configuration process with S2Mon.

1. Overview

The S2 system relies on an agent installed on the server side to send information to the central brain over an encrypted SSL connection. The agent we are using is, of course, open-source script (written in Bash), so anyone can look inside it, and see what it is doing exactly. Don't know about you but I would feel very uneasy if I had to install some "proprietary" monitoring binary on my machines - it could be a remotely controlled trojan horse for all I know. So, keeping the agent open is key to us.

The open-source agent confirms another key point - it only *sends* information out to the central S2Mon servers, it does not *receive* any commands/configurations back. The communication here is one way - from the agent to the S2Mon center. The S2Mon center cannot modify the agent behavior in any way whatsoever.

2. Requirements

Since the agent is mainly written in Bash, it, obviously, requires Bash to be available on the monitored system. Fortunately, Bash is available on any Linux system released during the last 15 years or so. The other requirements are:

  • Perl
  • curl
  • bzip
  • netstat
  • Linux OS

The required tools are all present in most of the contemporary Linux distributions, but in case you have any doubts, you can check out the prerequisites page for distro-specific tips.

The Linux OS requirement is the major one here - S2Mon currently runs on Linux only. We have plans to make it available to Mac OS, *BSD and Windows users in the future, but for the time being, these platforms are not supported.

3. Registering an account

You will obviously need an S2Mon account, so, in case you do not have one, head to https://www.s2mon.com/registration/. Once you submit your desired account name and your email address, you will be taken straight to your Dashboard, and your password will be sent over email to you (so make sure you get that email field right).

The Dashboard is very restricted at this point - you need to verify your email address to unlock the full functionality of the system. To complete the verification, simply click on the activation link you got in your mailbox. That's it, your S2Mon account is now fully functional!

4. Adding a server entry to the S2Mon site

OK, this is where the fun starts. Before the S2Mon system starts accepting any data from your server, you need to create a server record in the S2Mon system. Go to https://www.s2mon.com/host/add/ (or Servers -> Add host , if you would prefer) and fill in the following form:

Add Host

Hostname should be a unique identifier of your server - it does not need to be a Fully Qualified Domain Name (FQDN) - though it is a good idea to use that. For an Address, you should enter the external IP address of the host. This is the IP address is where Ping probes will be sent to, should you choose to enable them from the drop-down menu. It is a good idea to enable these probes if the server is ping-able; in this way you will get an alert if the ping dies - this is considered an indication that something is wrong. The Label field is optional free-text that you may use to describe your server. If you fill it in, it will be the server identifier used throughout the S2 site; otherwise the Hostname would be used.

After you submit the form, you will be presented with the basic steps that you will need to follow to get the probe running. Since you are reading this blog post, you can just copy the Pushdata Agent URL and ignore everything else :). The Pushdata Agent URL is the address where the agent would send all of the monitoring information, so it is the most important piece of data on that page. In case you forgot it, accidentally closed the page,or the dog ate your computer, don't worry, you can always get back to this page via Servers -> Edit button -> Probe setup tab.

5. Activating the services you want to monitor

Now go to https://www.s2mon.com/servers/. You will see the list of your servers there, but there is also a convenient panel where you can enable or disable certain services. Go on and activate the ones you are interested in (or, if you are like me - all of them):

Service Activation

6. Running the probe on the server

This is the trickiest part of it all, as there is a lot of different ways to do it, depending on the server controls you have at hand. I'll assume that you have SSH access to the server, so you can run commands directly. If you do not have this kind of access though, you may still be able to run the S2 probe if you can:

  • Download the probe archive, extract it, and put the extracted files onto your server;
  • Run a periodical task (cron job) every minute, which would fire the executable agent script at the specified URL.

The S2Mon agent does NOT require a root account - you can run it from an unprivileged account. Even though I trust the agent completely, I run it from an unprivileged account on all my servers - it's a good approach security-wise, and it is more tidy. In some cases, however, unprivileged accounts may not have access to all built-in metrics, so you might want to run the cron job with root privileges - it's up to you and your specific setup

So, regardless of which account you decide to run the agent under, you can log in with that account and do the following:

s2mon ~$ wget https://dc1.s2-monitoring.com/active-node/a/s2-pushdata.tar.gz # download
s2mon ~$ tar xzf s2-pushdata.tar.gz # extract
s2mon ~$ ls -la s2-pushdata/post.sh # verify that the post.sh script has executable permissions
s2mon ~$ cd s2-pushdata/
s2mon ~/s2-pushdata$ DEBUG_ENABLED=1 ./post.sh "https://dc1.s2-monitoring.com/rblmon/collector-vahzeegh/index.php/my-hostname.com" # Use your specific Pushdata Agent URL here, enclosed in single or double quotes!

You will get some debug output out of the last command; it will abort if there is anything missing (for example, if curl is not installed on the system). If everything is OK, the last line would indicate successful data submission, e.g.:

DEBUG: POST(https://dc1.s2-monitoring.com/rblmon/collector-vahzeegh/index.php/my-hostname.com): 21 keys (8879 -> 2539 bytes).

The only thing that's left to do is to set the agent to be executed every minute. Again, there are a few different ways to do this, but the most common one is to run 'crontab -e', which will open up your user's crontab for editing. Then you only need to append the line:

* * * * * cd /path/to/s2-pushdata/ && ./post.sh "https://dc1.s2-monitoring.com/rblmon/collector-vahzeegh/index.php/my-hostname.com" &>/dev/null

Please make sure that you substitute /path/to/s2-pushdata/ with the actual path to the s2-pushdata directory on your system, and to change the URL to the value that you got after adding your host record in the S2Mon website (note: changing just the hostname part at the end will NOT work).

7. Profit!

OK, if you were able to complete steps 5,6 and 7, then you should see the nifty monitoring widget on your S2Mon Dashboard turn all green. Congrats, your server is now monitored and you are recording the history of its most intimate parameters!

Widget - OK Status

8. (Optional) MySQL monitoring configuration

The MySQL service requires some extra configuration for the S2Mon agent to be able to look inside it, so you will need to take some extra steps if you want to monitor any of the MySQL services. The easiest way to do this is to:

  • Create a MySQL user for S2Mon to use, by using the following query (ran as MySQL root or equivalent)
GRANT USAGE ON *.* TO 's2-monitor'@'localhost' IDENTIFIED BY '*******';

Make sure to replace '*******' with a completely random password. Don't worry, you will not need to remember it for long!

  • Create the files /etc/s2-pushdata/mysql-username and /etc/s2-pushdata/mysql-password on your system, and put the username (s2-monitor in this case) and password in the respective file (on a single line).
  • Change the ownership of those so that only the user that you are running S2Mon under can read them (for example set them to 0400).

After this is all done, you will see the MySQL data charts slowly starting to fill in with data in the next few minutes.

9. Post-setup

Now that you have a host successfully added to the interface, the next logical step would be to setup some kind of notification that would poke you when the some parameter goes too high or too low. Additionally, you might want to enable other people to view or modify the server data in your account. Both tasks are easy with S2Mon and I will show you how to do it in the next part.