Beat security woes with better system intelligence: Elastic preps Linux audit tool alternative

Security is always top of mind for IT and business people, and the frequency of attacks leaves many wondering if the Internet will ever be safe. But better system, and hence operational, intelligence can help eliminate many common problems as engineers from Elastic outlines at this year’s TechSummit in Berlin

Many organizations, particularly younger startups, are too preoccupied with running their business to focus on security. But if you ignore security for long enough someone will eventually be running bitcoin miners on your servers.

That’s the sentiment two Elastic engineers conveyed at the recent TechSummit conference in Berlin. Team Lead Monica Sarbu and Software Engineer Tudor Golubenco, spoke about the need to apply more system intelligence to combat unauthorized access to computer systems, both on-premises and in the cloud.

Elastic is the company behind Elasticsearch and Sarbu and Golubenco work for the Beats product line. Beats are lightweight “shippers” that collect and ship various types of Linux system operational data to Elasticsearch. Beats is an open source project part of the Elastic stack and is written in the Go language.

“We use the plural 'Beats' because it’s not a single product, but a family of projects,“ Sarbu said. “We have a project for each data type that we collect. For example, the Filebeat collects logs from servers and the Metricbeat interrogates external services and fetches metrics from them.”

When the data from Beats is in Elasticsearch it can be analysed and visualized. Use of the Elastic stack is well known for logging and monitoring, which is useful for security purposes.

“Even if you follow good security practices, sometimes you can get breached due to something not in your control or due to a silly mistake,” Sarbu said. “My favourite mistake is when developers commit server credentials in a public GitHub account. This is something that happens more often than you think.”

Computer security forensics gets Elastic

How do you find out you have been breached? The knowledge varies from never finding out, which is the worst case, to finding out yourself and proving no harm was done.

To discover security breaches logging and monitoring technologies can be employed either in real-time or forensically after an attack.

According to Sarbu, logging tools collect data from all different sources relevant to security like authorization logs, which can tell you the password (or key) and IP address of the user. Another thing you might follow is the number of failed “sudo” attempts. This is when a regular user (or attacker) attempts to elevate their privileges on a Linux system.

“Tools like auditd [audit daemon] can monitor every system call, which can result in quite a lot of messages so you have to be specific on which data you look at, including who accesses a file, new network connections and when a new process is started,” Sarbu said.

“Auditd logs are in a difficult to parse format because they use the multiline log format. But Filebeat has a specific module for auditd that takes care of the regular expression to parse the logs.”

With Linux’s auditd tool quite complex to use and “get right”, Elastic engineers are now working on a solution that is an alternative to auditd, which will also get all the system information directly from the kernel.

No release date was given for the auditd alternative, but like many open source projects it will be out when it’s ready.

Sarbu’s colleague, Elastic Software Engineer, Tudor Golubenco, auditd can take as much as 20 per cent of the system’s CPU capacity and this high overhead is one of the reasons the team wants to move away from using it.

“If we can get this data directly from the kernel we can implement it in a more efficient manner,” Golubenco said. “You can start with SSH logs as all the time there are attempts to brute force attack them to gain system access.”

A recent Beats development is the concept of modules. A module is designed to simplify the collecting, parsing and even the visualization of data. There are modules for common log formats like the Nginx web server. The monitoring will identify a successful login and whether it used a password. This can be alerted on to distinguish between an attack and a misguided user.

Elastic has the Metricbeat shipper for system and process logs, including what is consuming CPU and memory.

“I can see the processes the user has started and the ones one I would expect,” Golubenco said. “If there are four processes and three don’t make sense there is probably a security breach.”

This type of security analysis can generate a lot of logs, but they can be indexed in Elasticsearch.

“Filtering these logs to reduce the number is something we are working on. And we don’t want to rely on auditd to generate log files, and then need to parse them – we want to talk directly to the kernel,” Golubenco said.

Sarbu said running process are always interesting to monitor as you can retrieve the commands that are executed on the server and identify the suspicious ones when they happen.

Beats provide developers with a platform that allows them to be extended built from scratch. There are more than 30 community developed Beats now available.