Zabbix: Secure LDAP (LDAPS)18th Feb 2021
This article describes how to Secure LDAP (LDAP over TLS) on Zabbix LDAP authentication.
A bit of a theory
Despite the topic is quite simple I am seeing a lot of questions about this so let's get started. I assume that you know what LDAP is if you are planning to use it so what is secure LDP (LDAPS)? It's still the same old LDAP but encapsulated into TLS. I am not going to give any links here about what TLS is, anybody can find plenty of information on the Internet but in two words: before any byte of data sent between Client and Server secure negotiation happens between them to establish 'trust'. If this succeeds then all the data sent-received further is encrypted and cannot be (easily) read by any third-party. TLS certificate is one of the pillars of this secure communication. The certificate is the first thing sent by the server to a client and the client must decide whether it trusts this certificate (server) and obviously proceed only if it does. Note: there might be a case when a server also needs a certificate from client to validate client (mutual authentication) but this is rare case and not covered in this article. Thus we came to a question: how do we tell client to trust this server's certificate?
LDAPS in Zabbix
I always prefer to play with something that can be configured and tested as opposed to just theory so in this article we use two containers:
docker run -p 3389:389 -p 6636:636 --name openldap-server --detach bgmot42/openldap-server:0.1.1 docker run -p 8080:80 --name zabbix-appliance -t --link openldap-server:ldap-host -d bgmot42/zabbix-appliance-ubuntu:6.0.0-bg
By doing the above you will start two docker containers:
- openldap-server - the image created by me specifically for testing/demo, based on https://github.com/osixia/docker-openldap . We use bgmot42's image because as opposed to osixia's it does have a certificate signed by a Certificate Authority (CA)
- zabbix-appliance - fully functional Zabbix server (baked off https://github.com/BGmot/zabbix repository)
If you don't want (of for some reason uncomfortable) to play with dockers Authentication using LDAP can be set up in any Zabbix server following instructions at this link.
First of all make sure LDAP ("non-secure") works properly - login into Zabbix with default credentials Admin/zabbix and configure LDAP authentication (all passwords are plain "password"):
Click Test and you should get:
Now switch to LDAS by putting ldaps://ldap-host in LDAP host field. If ldap:// or ldaps:// is used in LDAP host field is used then the value in Port field is ignored but for consistency put 636 in Port field (default port for LDAPS). Now click Test again and it should miserably fail:
So why is if failing? Most probably LDAP client does not trust LDAP server's certificate. Fortunately it is very easy to see what certificate the server is using. Login into Zabbix server (over SSH) and just execute:
root@614f72b420c6:/var/lib/zabbix# openssl s_client -port 636 -connect ldap-host -showcerts CONNECTED(00000003) <snip> --- Certificate chain 0 s:C = CA, ST = Ontario, L = Washago, O = BGmot, OU = Root operations, CN = ldap-host i:CN = BGmot CA -----BEGIN CERTIFICATE----- MIIDIjCCAqegAwIBAgIQQ6LPau8xELSC+ZTOvePKvDAKBggqhkjOPQQDBDATMREw DwYDVQQDDAhCR21vdCBDQTAeFw0yMDA5MDcxMzU0NDdaFw0yMzA4MjMxMzU0NDda MG8xCzAJBgNVBAYTAkNBMRAwDgYDVQQIDAdPbnRhcmlvMRAwDgYDVQQHDAdXYXNo YWdvMQ4wDAYDVQQKDAVCR21vdDEYMBYGA1UECwwPUm9vdCBvcGVyYXRpb25zMRIw EAYDVQQDDAlsZGFwLWhvc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIB AQDmpvPWJNcTn5UtrTIooyEP5METmWRZKCsvP3BOEA3N1xSmemRZQXrvv+Fl4eGG wzEbFImmLG1kziaCF3rhX4cb4ShloGZApyBKmSXGA0c1U2gsmBTAbwcBUtF+BglU aexgofpxM9PNtqW6qqnQke3WToFLVIlLFbilZN/5tqXnbzPt2ny2GEi5F9TIFFJG LDzMNMfCJCt2Gl26y3t4U2JszgX7wAbukIoBzs2fKnxs5Yu2GNCZ8Y7KmrFvwS4S GFAy0xYvDv62jrtZrhka+iIjbBAvGc/W+KfUXKbLfzeEisD8g3J0KCTScS+lZlcT JMWVIhneDZSG/zNxTPNZze6vAgMBAAGjgbUwgbIwCQYDVR0TBAIwADAdBgNVHQ4E FgQUVqJy8sN1xh9OIB4MEfifhFkvpk0wTgYDVR0jBEcwRYAU2e/W9f0Son17mt6k BmLnNd6wiEqhF6QVMBMxETAPBgNVBAMMCEJHbW90IENBghRIORoN0Zz4MSWcORs2 gfQy6dpVmDATBgNVHSUEDDAKBggrBgEFBQcDATALBgNVHQ8EBAMCBaAwFAYDVR0R BA0wC4IJbGRhcC1ob3N0MAoGCCqGSM49BAMEA2kAMGYCMQDvNbLnbJCl4lklWPbx BEV+vM+b37Q32nANyA/MIGNE+QAPRb731ZH6P4zp5ma6ursCMQCBMtxxguTc7Qto 0QKJsghvP8nMOUVfRBWxvMB7saeoeVFxgvGTp8PvDtDCKYrNzoY= -----END CERTIFICATE----- 1 s:CN = BGmot CA i:CN = BGmot CA -----BEGIN CERTIFICATE----- MIIB8jCCAXmgAwIBAgIUSDkaDdGc+DElnDkbNoH0MunaVZgwCgYIKoZIzj0EAwQw EzERMA8GA1UEAwwIQkdtb3QgQ0EwHhcNMjAwOTA1MTQ0MzA4WhcNMzAwOTAzMTQ0 MzA4WjATMREwDwYDVQQDDAhCR21vdCBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IA BPBHSsn53IIQ/aQsnpeWvbkPVbqPJcMbvJm36yXjO0LWMw8h4dj8TDsxjDNQ1RMB LSIJdPVlUr45fIygmPkB9P1ZciaZZzJt09YoO4DmyJG3YxevnuaDw1RXNPgwrk1g 5KOBjTCBijAdBgNVHQ4EFgQU2e/W9f0Son17mt6kBmLnNd6wiEowTgYDVR0jBEcw RYAU2e/W9f0Son17mt6kBmLnNd6wiEqhF6QVMBMxETAPBgNVBAMMCEJHbW90IENB ghRIORoN0Zz4MSWcORs2gfQy6dpVmDAMBgNVHRMEBTADAQH/MAsGA1UdDwQEAwIB BjAKBggqhkjOPQQDBANnADBkAjBbVR18WFy3sfTjeGPhqKwdfRNtVxMyUW2VrB+7 kceIjAnC5zughFE51lR8NsG6xQwCMA3wJpJoQBI1XibqanWfz0trA/F8bAbRN+gz opxZAqtr9EfDY5LLdCUwgK1xlxrVUg== -----END CERTIFICATE----- --- Server certificate subject=C = CA, ST = Ontario, L = Washago, O = BGmot, OU = Root operations, CN = ldap-host issuer=CN = BGmot CA --- <snip>
Find "Certificate chain" where we can see two certificates ("s:" stands for "subject" and "i:" stands for "Issuer") numbered as 0 and 1. Certificate 0 is this server's certificate (look at CN = ldap-host in its subject) and this certificate is signed by certificate 1 - BGmot CA - it's a "Certificate Authority" certificate as it is signed by itself.
Now you need to decide whether we want to trust this particular certificate (certificate 0) or any certificate signed by "BGmot CA". I highly advise the second variant as individual certificates tend to expire and if it happens then your LDAPS will stop working. All we need to do is
1) store this certificate in a file. In this example I put it in /etc/ldap/certs/ca.crt (create /etc/ldap/certs folder). Other Linux distributives might have /etc/openldap instead of /etc/ldap. So the file should have:
-----BEGIN CERTIFICATE----- MIIB8jCCAXmgAwIBAgIUSDkaDdGc+DElnDkbNoH0MunaVZgwCgYIKoZIzj0EAwQw EzERMA8GA1UEAwwIQkdtb3QgQ0EwHhcNMjAwOTA1MTQ0MzA4WhcNMzAwOTAzMTQ0 MzA4WjATMREwDwYDVQQDDAhCR21vdCBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IA BPBHSsn53IIQ/aQsnpeWvbkPVbqPJcMbvJm36yXjO0LWMw8h4dj8TDsxjDNQ1RMB LSIJdPVlUr45fIygmPkB9P1ZciaZZzJt09YoO4DmyJG3YxevnuaDw1RXNPgwrk1g 5KOBjTCBijAdBgNVHQ4EFgQU2e/W9f0Son17mt6kBmLnNd6wiEowTgYDVR0jBEcw RYAU2e/W9f0Son17mt6kBmLnNd6wiEqhF6QVMBMxETAPBgNVBAMMCEJHbW90IENB ghRIORoN0Zz4MSWcORs2gfQy6dpVmDAMBgNVHRMEBTADAQH/MAsGA1UdDwQEAwIB BjAKBggqhkjOPQQDBANnADBkAjBbVR18WFy3sfTjeGPhqKwdfRNtVxMyUW2VrB+7 kceIjAnC5zughFE51lR8NsG6xQwCMA3wJpJoQBI1XibqanWfz0trA/F8bAbRN+gz opxZAqtr9EfDY5LLdCUwgK1xlxrVUg== -----END CERTIFICATE-----
Make sure the user Zabbix server is running under has access to this file.
Now tell OpenLDAP client to trust any certificate signed by this CA by adding just one line to /etc/ldap/ldap.conf:
Let's test again in Zabbix WebUI:
You are all set!
If your LDAP server's certificate is signed by a cert and which in turn signed by a CA cert then you need to put the whole certificate chain not including just the server's certificate. In this case the file should have .pem extension (ca.pem).
WARNING: do not user IP addresses instead of hostnames/FQDNs, the value in LDAP host field must match what the server's certificate was issued for!
Conversation the Zabbix server is trying to make with LDAP server can be simulated from command line just install ldap-utils package (for Ubuntu, in your distro the name might be a bit different) on Zabbix server.
First of all make sure you can perform the simplest operation with LDAP and LDAPs:
$ ldapwhoami -x -H ldap://ldap-host anonymous $ ldapwhoami -x -H ldaps://ldap-host anonymous
If you get an error, add debugging (sometimes very helpful!):
$ ldapwhoami -x -H ldaps://172.17.0.2 -d 4 TLS: hostname (172.17.0.2) does not match common name in certificate (ldap-host). TLS: can't connect: (unknown error code). ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
Finally you can perform the whole search:
$ ldapsearch -D "cn=ldap_search,dc=example,dc=org" -w password -b "ou=Users,dc=example,dc=org" -H ldaps://ldap-host uid # extended LDIF # # LDAPv3 # base <ou=Users,dc=example,dc=org> with scope subtree # filter: (objectclass=*) # requesting: uid # # Users, example.org dn: ou=Users,dc=example,dc=org # user1, Users, example.org dn: uid=user1,ou=Users,dc=example,dc=org uid: user1 # user2, Users, example.org dn: uid=user2,ou=Users,dc=example,dc=org uid: user2 # user3, Users, example.org dn: uid=user3,ou=Users,dc=example,dc=org uid: user3 # search result search: 2 result: 0 Success # numResponses: 5 # numEntries: 4