With the launch of Safety Engine 1.0.x, we enabled the Safety Engine to perform as an HTTP REST API, permitting it to assemble alerts from different Safety Engines.
I’ll information you thru the steps to arrange the CrowdSec Safety Engine throughout a number of servers, the place one server will function the mother or father and two further machines will ahead alerts to it.
Advantages
Sharing cybersecurity incidents throughout machines utilizing the CrowdSec Safety Engine is a extremely efficient technique to reinforce collective safety defenses. By leveraging CrowdSec’s functionality to distribute remediations amongst linked machines, every machine advantages from real-time updates about new threats detected elsewhere within the community.
Structure
Within the diagram above, the mother or father Safety Engine, designated as server-1, can be arrange because the HTTP REST API, generally generally known as the LAPI (Native API). This engine can be in control of storing and distributing the gathered alerts. Remediation is managed via the Remediation Elements, which depend upon the LAPI provided by server-1. It is essential to grasp that mitigation can happen independently from detection.
Server-2 and server-3 are designated as internet-facing machines that can host providers accessible to the general public and can be generally known as the kid Log Processors. On these servers, we’ll set up CrowdSec Safety Engine and Remediation Elements, which can work together with the server-1 LAPI.
Be aware: The phrase little one Log Processors refers to a CrowdSec Safety Engine that operates with its LAPI turned off. For extra data on this, seek the advice of our Taxonomy Replace Article.
We strongly encourage you to discover the CrowdSec Hub to be taught concerning the in depth vary of providers the Safety Engine can defend. This platform showcases the varied capabilities of the Engine in securing all the pieces from net purposes to databases in opposition to cyber threats.
Structure Choices
I selected a postgresql backend for the server-1 LAPI to attain higher stability in database learn and write operations. Nonetheless, relying in your operational scale, you may uncover that the default SQLite with WAL (Write-Forward Logging) enabled meets your wants, if that’s the case you may skip part 1b.
Stipulations
To comply with this tutorial, you’ll need the next:
Two internet-facing Ubuntu 22.04 machines internet hosting providers.
One Ubuntu 22.04 machine.
An area community connection between the Mother or father and Little one machines.
Step 1: Setup and Configure Mother or father LAPI server-1
Step 1a: Set up CrowdSec Safety Engine
Let’s set up the Safety Engine, following the set up information.
bash
curl -s https:/packagecloud.io/set up/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash
sudo apt set up crowdsec
Step 1b (Optionally available): Utilizing postgresql on Mother or father server-1
Set up the PostgreSQL package deal utilizing the apt package deal supervisor.
bash
sudo apt set up postgresql
Subsequent, transition to the ‘postgres’ Linux person after which join by executing the psql command.
bash
sudo -i -u postgres
psql
You possibly can arrange the database and create a licensed person utilizing the instructions beneath. Substitute <PASSWORD> with a password you choose, you MUST hold it inside the single quotes.
bash
postgres=# CREATE DATABASE crowdsec;
CREATE DATABASE
postgres=# CREATE USER crowdsec WITH PASSWORD ‘<PASSWORD>‘; CREATE ROLE
postgres=# GRANT ALL PRIVILEGES ON DATABASE crowdsec TO crowdsec;
GRANT
Now, we’ll arrange the Safety Engine to make the most of this newly created database as its backend. This requires updating the db_config part within the /and so forth/crowdsec/config.yaml file.
yaml
db_config:
log_level: information
sort: postgres
person: crowdsec
password: “”
db_name: crowdsec
host: 127.0.0.1
port: 5432
Throughout the set up of the Safety Engine, the native machine was configured to make use of the SQLite database. To change to the newly arrange postgres database, you’ll need to regenerate the credentials after which proceed to restart the Safety Engine.
bash
sudo cscli machines add -a –drive
sudo systemctl restart crowdsec
Step 1c: Expose LAPI port
To allow communication between the LAPI and the kid Log Processors/Remediation Elements, it’s crucial to regulate the LAPI’s settings to simply accept connections from exterior sources, since its default configuration binds it to the machine’s loopback deal with (127.0.0.1). This adjustment might be made by modifying the /and so forth/crowdsec/config.yaml configuration file and altering the desired settings.
yaml
api:
server:
listen_uri: 10.0.0.1:8080
Within the talked about setup, we modify the settings to pay attention on the ten.0.0.1 interface on port 8080. Do you have to want to pay attention on a number of interfaces, you may change this to 0.0.0.0 and implement firewall guidelines to allow particular connections.
Step 2: Setup and Configure Little one Log Processors
Step 2a: Set up CrowdSec Safety Engine
Let’s set up the Safety Engine, following the set up information.
bash
curl -s https:/packagecloud.io/set up/repositories/crowdsec/crowdsec/script.deb.sh | sudo bash
sudo apt set up crowdsec
Step 2b: Configure to make use of LAPI server
First, lets register the Log Processor to the LAPI server utilizing the next command
bash
sudo cscli lapi register -u http://10.0.0.1:8080
Make sure you modify the -u flag to fit your community. Make the most of the IP deal with if it is static, or go for the hostname in case your community permits it.
Subsequent, we’ll flip off the native API on the Safety Engine, turning it right into a Log Processor. This motion is taken as a result of the API will not be utilized, which can preserve system sources and keep away from occupying a TCP port unnecessarily.
To realize this, we will disable the API within the configuration with:
yaml
api:
server:
allow: false
Step 2c: Validate the registration request on LAPI
Since we used the cscli lapi register on the kid Log Processor we should validate the request on server-1 through the next instructions:
bash
sudo cscli machines listing
NAME VERSION
IP ADDRESS
LAST UPDATE
STATUS
dc6f34b3a4994700a2e333df43728701D0iARTSQ6dxiwyMR
10.0.0.1
2021-04-13T12:16:11Z
✔️
v1.0.9-4-debian-pragmatic-a8b16a66b110ebe03bb330cda2600226a3a862d7
9f3602d1c9244f02b0d6fd2e92933e75zLVg8zSRkyANxHbC
10.0.0.3
2021-04-13T12:24:12Z
🚫
From this output, it is evident there is a new machine that hasn’t been validated but by the 🚫 inside the standing column. We have to manually validate this machine to make sure the LAPI acknowledges which machines are licensed to transmit alerts.
Be aware: If you happen to do not see a brand new machine marked with a 🚫 within the standing column, be sure to are executing the command on the LAPI server.
bash
sudo cscli machines validate 9f3602d1c9244f02b0d6fd2e92933e75zLVg8zSRkyANxHbC
Be sure that to vary the argument following “validate” to correspond with the brand new machine identify displayed within the listing output.
Step second: Restart the kid Log Processor service
On the kid Log Processor machine you may run the next command to restart the service:
bash
sudo systemctl restart crowdsec
Then, for every machine you want to join, repeat step 2. In our case, we’ll carry out this motion twice, as soon as for every Ubuntu machine.
Step 3: Establishing Remediation
Now, it is necessary to configure remediation measures to your internet-facing servers since merely operating the Log Processor doesn’t implement enforcement actions. On this article, we’ll concentrate on establishing the Linux firewall Remediation Part. For extra remediation choices, remember to discover the in depth listing accessible within the CrowdSec Documentation.
Step 3a: Producing API key on LAPI
First, we’ll create API token on the LAPI server by executing the next command:
bash
sudo cscli bouncers add server-2-firewall
Api key for ‘server-2-firewall’:
02954e85c72cf442a4dee357f0ca5a7c
Please hold this key because you will be unable to retrieve it!
I used server-2-firewall because the identify for the important thing, however you may select any identify you like. It is essential to pick out a descriptive identify for the important thing to facilitate future administration, particularly if it’s worthwhile to revoke a key resulting from a token compromise.
Step 3b: Set up the Remediation Part
IPtables firewall is among the many mostly used on Linux, so we’ll proceed to put in the Part that interacts with it, utilizing the apt package deal supervisor.
bash
sudo apt set up cs-firewall-bouncer-iptables
As soon as the Part is put in, we’ll edit the configuration underneath /and so forth/crowdsec/bouncers/crowdsec-firewall-bouncer.yaml to level in direction of the LAPI
yaml
api_url: http://10.0.0.1:8080/
api_key: 02954e85c72cf442a4dee357f0ca5a7c
Make sure you modify the api_url to align together with your LAPI deal with and replace the api_key to the one generated by the earlier command. Keep in mind you should utilize both the IP deal with or the hostname. After you have altered the configuration let’s restart the firewall Remediation Part.
bash
sudo systemctl restart crowdsec-firewall-bouncer
Then, for every Remediation Part you want to join, repeat step 3. In our case, we’ll carry out this motion twice, as soon as for every firewall on the Ubuntu machines. Be sure that to change the naming scheme of the api key.
A number of closing ideas
This information illustrated the method for establishing a multi-server Safety Engine setup. Whereas this instance utilized three servers, the structure permits for straightforward growth. The useful resource consumption on server-2 and server-3 stays minimal because the majority of operations are directed in direction of server-1, facilitating simple scalability of the system:
Register and validate further Safety Engines on the LAPI server
Add any further Remediation Elements
As beforehand acknowledged, there is no requirement for the Remediation Elements and Safety Engines to be put in on the identical server. This suggests that the Safety Engine needs to be put in on the location the place logs are produced, whereas the Remediation Part might be deployed at any desired location.
It is necessary to notice that this configuration comes with sure limitations:
The communication between Safety Engines happens through unencrypted HTTP, which is appropriate for an area community however not safe for web use. Nevertheless, the CrowdSec Safety Engine helps using HTTPS for these interactions.
This text doesn’t delve into monitoring or alerting. Nonetheless, the Safety Engine helps complete monitoring capabilities through Prometheus, and you could find extra detailed details about it on this article.
Having each the CrowdSec LAPI and PostgreSQL on server-1 creates a single level of failure, doubtlessly resulting in delays in menace response ought to any points come up with the server.
Now it’s possible you’ll be questioning — how do I construct a extremely accessible multi-server CrowdSec setup? We may have a devoted article on that within the coming weeks, so keep tuned!
We’re at all times very happy to obtain your suggestions! Don’t hesitate to succeed in out to us on our neighborhood platforms on Discord and Discourse.