{"id":296,"date":"2022-12-19T23:31:00","date_gmt":"2022-12-19T22:31:00","guid":{"rendered":"https:\/\/securityeye.wordifysites.com\/?p=296"},"modified":"2024-03-07T20:40:43","modified_gmt":"2024-03-07T19:40:43","slug":"log-management","status":"publish","type":"post","link":"https:\/\/swissmakers.ch\/en\/log-management\/","title":{"rendered":"Centralised log management of a modern company with the ELK stack"},"content":{"rendered":"

User activity within your own company in the form of logs is incredibly valuable data.<\/strong> System failures or performance statuses also provide rich information about the quality<\/strong> of a network or a product. However, the manual collection of all such actions, both from people and machines, is in most cases a very complex task.<\/p>\n\n\n\n

ELK<\/strong> is a technology stack that Elasticsearch<\/strong>, Logstash<\/strong> and Kibana<\/strong> combined to provide a comprehensive approach to consolidating, managing and Analysing logs and protocols<\/strong> from your entire company. It allows Real-time insights<\/strong> into network processes, firewall activities, domain controller log data and also carries out automated and continuous security analyses.<\/p>\n\n\n\n

\n

Centrally collect, search, automatically analyse and evaluate all events and log data and trigger alarms as soon as security problems or unauthorised processes are detected.<\/strong><\/p>\n<\/blockquote>\n\n\n\n

The centrepiece - Elasticsearch<\/h2>\n\n\n\n

As Main component of an ELK stack<\/strong> comes Elasticsearch<\/strong> as the centrepiece and in the productive environment in the form of a cluster. It serves as Database<\/strong>the Storage location<\/strong> of all data stored in separate log indexes, Indices<\/code> can be subdivided. This is also where the monitoring cluster comes to life. All storage and searches, whether automated or manual, are carried out here via API requests and bilateral communication between the cluster member nodes.<\/p>\n\n\n\n

Since Elasticsearch is very extensive, we publish under: Elasticsearch - The High Performance Search Engine<\/em> a separate post about this.<\/p>\n\n\n\n

The normalisation of logs by Logstash<\/h2>\n\n\n\n

The integration of custom systems, as is often the case with larger industrial companies such as gas, water or electricity companies, often requires the logs to be normalised. This means that the log data produced by the different devices must first be converted into a standardised format that can then be imported into the Elasticsearch cluster. This applies in particular to technologies that are not already supported by the official data shippers<\/strong> like Filebeat<\/a><\/strong>, Winlogbeat<\/a><\/strong>, Auditbeat<\/a><\/strong>, Metricbeat<\/a><\/strong> or Packetbeat<\/a><\/strong> are covered.<\/p>\n\n\n\n