SSN Linux

A blog for my Linux stuff. You can find my main blog at tuxconcept.tumblr.com. Thanks for stopping by!

[ssnlinux]

Things I’ve Learned: FreeIPA

I've been a user of FreeIPA for quite a long time, mostly in home and lab environments that didn't really have any bearing on enterprise environments. I always saw it as a sort of Active Directory (it's not) for Linux and UNIX systems and used it to test various things. One example of using it to test "things" was practicing for my RHCE in 2016 and also studying my Red Hat Security Hardening. Now I've used it since version 3.0, and it definitely has come a long way since then. I never really took a lot of the time I thought was necessary to learn specific things about it, in terms of having it work with an Active Directory and even 2FA/OTP among other things. Many years ago, there were functions such as winsync (which is now deprecated) that helped with this, while even the sssd components began to slowly integrate Active Directory support directly.

To give some backstory to the title of this post: In my current job, we are a mixed environment shop. We have a handful of Windows, Linux, and UNIX systems. Because of this, we have two separate directory servers. We have an LDAP and an Active Directory. A few years back, I respawned conversations that were shut down by previous management: We need to consolidate directory servers into at least Active Directory. Practically all of our workstations run Windows or Mac. Our usernames between both directories are different (LDAP uses underscores between first and last names, AD had first initial, last name up to 7 characters and now it's first.last). You can imagine how confusing this is and how annoying this actually is.

You can imagine the challenges this brings. In fact, here's a list of challenges this poses for the organization.

New users get confused and think they login with user.name@domain, domain\user.name, user.name on Linux/UNIX systems

Users at times do not know what username is used for what application or system (is it user_name or user.name for $x?)

Introduction of OIM (oracle identity manager) and OAM (oracle access manager), which was setup incorrectly and managed poorly by incompetent team called "information security"

Now you might wonder why does this matter to my FreeIPA topic. Simple, it causes issues for the UNIX engineering team (which I now co-lead).

Managers and above as well as new users expect all their access to be already there for all Linux/UNIX systems they need to work on

Again, new users get confused and use AD usernames to login to Linux/UNIX systems

Access is defined on their LDAP objects as "host: hostname"

SUDO is hybrid: LDAP and /etc/sudoers.d files

Identity management tools, which have been in the environment for almost 4 years, were never designed to handle the above

Management is reluctant to go to "member" groups to define access to systems (to emulate "role based" access)

This is where I stepped in and suggested that we need to use a different product, even though I initially suggested we go to AD entirely. The problem with going to AD entirely is that it takes away the UNIX team's management, and plus we would probably need to buy other products or software just to get group policies to work. This did not sit well with majority of my team and I didn't like the idea either, especially because we then begin to lose a sense of control on our own systems. This is when I suggested FreeIPA, as FreeIPA at the time (version 4.4) supported IPA-AD trusts. This meant that it gave control back to the UNIX team to our own infrastructure. Logins would be from our AD, we wouldn't need to create accounts in FreeIPA. This reduces a lot of overhead on us, leaving LDAP to start to (hopefully) die off. My team as a whole agreed to the idea so I took the project and started it up.

This is what I've learned and achieved.

The Setup

The setup was easy for me. The network team delegated me a subdomain of one of our internal domains to make my life easier. The Windows team was also extremely helpful and helped me get the AD trust up. I didn't expect it to be a pain, as my tests at home succeeded. The trust was setup, enabling compat and posix attributes (uidNumber, gidNumber, unixHomedirectory, loginShell) to try and make the migration from LDAP to IPA easier. The Windows engineer also setup my team as well as two other teams with the attributes so I had a biggger test bed.

After doing this, I also set the domain resolution order (FreeIPA 4.5) to be the AD domain first. That way, logging in with just user.name without the @domain works without much an issue. And then, clients using sssd had full_name_format set to %1$s to make it less confusing for users on their prompts. I didn't like the idea of having a prompt like [email protected]@servername - that's extremely annoying to look at. When this is set, you are able to do id for users in both AD or IPA without the domain on the name and it succeeds on RHEL 7 and higher. Setting domain resolution order back to blank or not setting it at all will require the @domain to be used for logins, including in the compat tree. There's really no other clear way around this as far as I know.

SID Numbers

The issue with trying to do a cleanish migration is that you can't change gid numbers out from under people. FreeIPA creates a range for your domain and also creates a range for AD, even if you don't use it/you're using posix attributes. When you create groups outside of IPA's defaulted range, they will not receive SID numbers. In this case, I had to create another range and then rerun the proper SID generating scripts.

I created a range with this data:

Base ID: 10

Range size: 50000

Primary RID base: 200000000

Secondary RID base: 200050000

I then did an update across the board. This took a little bit.

# cat > /root/task/80-sidgen-task-conf.uipdate <<EOF dn: cn=$TIME-$FQDN,cn=ipa-sidgen-task,cn=tasks,cn=config add:objectClass: top add:objectClass: extensibleObject add:cn:$TIME-$FQDN add:nsslapd-basedn: $SUFFIX add:delay: 0 EOF # ipa-ldap-updater /root/task/80-sidgen-task-conf.update

SUDO

SUDO for clients older than RHEL 7 was very interesting when in a IPA-AD trust.

Legacy Clients

Our environment is ridiculously mixed. We have Solaris 8, 10, 11, RHEL 5, 6, and 7. We also have Fedora workstations that my team uses because we dislike Windows, but that's a different story. We have still been trying to get rid of Solaris 8 forever now, but without any movement. We decided to not attach Solaris 8 to FreeIPA to hopefully let the systems rot.

RHEL 5

"RHEL 5???" you might be thinking. Yes. It's been out of support since March 31, 2017. This company has no concept of anything "new" or wanting to get rid of technology debt. So since we were probably going to be stuck with RHEL 5 for another year or so, I had to figure out how to get it to connect to FreeIPA - not only as a client but also be able to see Active Directory users. The first thing I learned here: Always use the ipa-advise command - Seriously. This will help you out so much.

In my case, I used nss pam ldap to help here. What this provided at the very least is the ability to get sudo to actually work 100% of the time - sudo didn't work all the time when sssd was the id provider. Here's the issue:

The domain resolution order configuration causes multiple uid attributes to appear on objects in the compat tree - this is normal.

Group membership, even in the compat tree, is defined by [email protected], rather than just user. As far as I know, there's no clear way around this. Running groups or id will not show proper membership.

sudo probably only worked here because of the id output, even though I was logging in with first.last, my id shows up as [email protected] and associated that properly with my group membership

When using sssd, while the user would appear as first.last, group membership would never be applied. Here's what I mean.

dn: uid=first.last,cn=users,cn=compat,dc=ipa,dc=example,dc=com cn: First Last objectClass: posixAccount objectClass: top gidNumber: 1006800013 gecos: First Last uidNumber: 10000 loginShell: /bin/bash homeDirectory: /home/first.last uid: [email protected] uid: first.last dn: cn=unixadm,cn=groups,cn=compat,dc=ipa,dc=example,dc=com gidNumber: 5900 objectClass: posixGroup objectClass: ipaOverrideTarget objectClass: ipaexternalgroup objectClass: top ipaAnchorUUID:: removed cn: unixadm memberUid: [email protected]

I'm sure you see the problem. This causes other issues: If you're trying to use pam_hbac, it seems to want to work using pam_sss in the stack, not pam_ldap. With pam_ldap, it doesn't even try. With pam_sss, it tries, but it ultimately fails because it doesn't seem to be properly evaluating the rules and groups - again, this goes back to what I said: group membership is not seen with sssd fully while nss pam ldap can associate it to an extent.

As of this writing, I was never able to properly figure this out. What I noticed is in sssd, the group memberships were always intermittent. In fact, it always acted weird - if I did getent group unixadm, it'd show me. As soon as I ran id against my user, I'd disappear from the group. And I found this to be very troublesome. Sometimes it worked, other times it didn't. I could never really figure out why this was happening. One thing I did try was updating SSSD to 1.9.6 from a COPR repo. That didn't help.

Long story short: get rid of your RHEL 5 systems. At least RHEL 6 worked. Sort of.

Solaris 10

Alright, so this is where it gets interesting. Getting it to work with FreeIPA was a chore, a serious chore. The worst part about all of this is the communication from the system to FreeIPA is unencrypted. Yes, you heard this right. Unencrypted. I couldn't even get it to be OK with Kerberos login to at least have a secure channel. The saving grace here is that if you have a keytab on your system and login, it should just work. The other problem I had is I needed to use pam_ldap in my pam configuration.

First thing I needed is a profile that the ldapclient could use. I also created a solaris system account. Here's an example ldif below you can add into IPA.

dn: cn=solaris_authpam,ou=profile,dc=ipa,dc=example,dc=com serviceAuthenticationMethod: pam_ldap:simple authenticationMethod: simple objectClass: top objectClass: DUAConfigProfile bindTimeLimit: 5 cn: default cn: solaris_authpam defaultSearchBase: dc=ipa,dc=example,dc=com defaultServerList: pentl01.ipa.example.com pentl02.ipa.example.com followReferrals: TRUE objectclassMap: shadow:shadowAccount=posixAccount objectclassMap: passwd:posixAccount=posixaccount objectclassMap: group:posixGroup=posixgroup profileTTL: 6000 searchTimeLimit: 15 serviceSearchDescriptor: group:cn=groups,cn=compat,dc=ipa,dc=example,dc=com serviceSearchDescriptor: passwd:cn=users,cn=compat,dc=ipa,dc=example,dc=com serviceSearchDescriptor: netgroup:cn=ng,cn=compat,dc=ipa,dc=example,dc=com serviceSearchDescriptor: ethers:cn=computers,cn=accounts,dc=ipa,dc=example,dc=com serviceSearchDescriptor: sudoers:ou=sudoers,dc=ipa,dc=example,dc=com dn: uid=solaris,cn=sysaccounts,cn=etc,dc=ipa,dc=example,dc=com objectClass: account objectClass: simpleSecurityObject objectClass: top uid: solaris userPassword: secret123

I needed to create a keytab for the host.

# ipa host-add hostname.example.com # ipa-getkeytab -s pentl01.ipa.example.com -p host/[email protected] -k /tmp/hostname.keytab

I then transferred the keytab to the system and put it at /etc/krb5/krb5.keytab. I configured /etc/krb5/krb5.conf.

[libdefaults] default_realm = IPA.EXAMPLE.COM dns_lookup_kdc = true verify_ap_req_nofail = false [realms] IPA.EXAMPLE.COM = { } DOMAIN.TLD = { } [domain_realm] ipa.example.com = IPA.EXAMPLE.COM .ipa.example.com = IPA.EXAMPLE.COM domain.tld = DOMAIN.TLD .domain.tld = DOMAIN.TLD [logging] default = FILE:/var/krb5/kdc.log kdc = FILE:/var/krb5/kdc.log kdc_rotate = { period = 1d version = 10 } [appdefaults] kinit = { renewable = true forwardable= true }

Set the permissions of the keytab to 600, owned by root:sys. Once it was configured, I tested kinit [email protected]. If you get a message about "kt warn", it can be ignored. After this, set /etc/defaultdomain to your ipa domain (ipa.example.com). Configure /etc/ldap.conf.

base dc=ipa,dc=example,dc=com scope sub TLS_CACERTDIR /var/ldap TLS_CERT /var/ldap/cert8.db TLS_CACERT /var/ldap/ipa.pem tls_checkpeer no ssl off bind_timelimit 120 timelimit 120 uri ldap://pentl01.ipa.example.com sudoers_base ou=sudoers,dc=ipa,dc=example,dc=com pam_lookup_policy yes

Create /var/ldap if needed and create an NSS certificate database. You should also have your CA certificate as /var/ldap/ipa.pem.

# cd /var/ldap # /usr/sfw/bin/certutil -A -n "ca-cert" -i /var/ldap/ipa.pem -a -t CT -d .

At this point, I do the ldapclient init.

# ldapclient init -a profileName=solaris_authpam \ -a domainName=ipa.example.com \ -a proxyDN="uid=solaris,cn=sysaccounts,cn=etc,dc=ipa,dc=example,dc=com" \ -a proxyPassword="secret123" \ -D uid=solaris,cn=sysaccounts,cn=etc,dc=ipa,dc=example,dc=com \ -w qdbKnW1et pentl01.ipa.example.com

Now configure /etc/nsswitch.conf.

passwd: files ldap [NOTFOUND=return] group: files ldap [NOTFOUND=return] sudoers: files ldap netgroup: ldap

Now configure /etc/pam.conf

# Console login auth requisite pam_authtok_get.so.1 login auth sufficient pam_krb5.so.1 login auth required pam_dhkeys.so.1 login auth required pam_unix_cred.so.1 login auth required pam_dial_auth.so.1 login auth required pam_unix_auth.so.1 use_first_pass login auth required pam_ldap.so.1 rlogin auth sufficient pam_rhosts_auth.so.1 rlogin auth requisite pam_authtok_get.so.1 rlogin auth sufficient pam_krb5.so.1 rlogin auth required pam_dhkeys.so.1 rlogin auth required pam_unix_cred.so.1 rlogin auth required pam_unix_auth.so.1 rlogin auth required pam_ldap.so.1 # Needed for krb krlogin auth required pam_unix_cred.so.1 krlogin auth sufficient pam_krb5.so.1 # Remote Shell rsh auth sufficient pam_rhosts_auth.so.1 rsh auth required pam_unix_cred.so.1 rsh auth binding pam_unix_auth.so.1 server_policy rsh auth required pam_ldap.so.1 # Needed for krb krsh auth required pam_unix_cred.so.1 krsh auth required pam_krb5.so.1 # ? ppp auth requisite pam_authtok_get.so.1 ppp auth required pam_dhkeys.so.1 ppp auth required pam_dial_auth.so.1 ppp auth binding pam_unix_auth.so.1 server_policy ppp auth required pam_ldap.so.1 # Other, used by sshd and "others" as a fallback other auth requisite pam_authtok_get.so.1 other auth sufficient pam_krb5.so.1 other auth required pam_dhkeys.so.1 other auth required pam_unix_cred.so.1 other auth binding pam_unix_auth.so.1 server_policy other auth required pam_ldap.so.1 other account requisite pam_roles.so.1 other account required pam_projects.so.1 other account binding pam_unix_account.so.1 server_policy other account required pam_ldap.so.1 other session required pam_unix_session.so.1 other password required pam_dhkeys.so.1 other password requisite pam_authtok_get.so.1 other password requisite pam_authtok_check.so.1 other password required pam_authtok_store.so.1 server_policy # passwd and cron passwd auth binding pam_passwd_auth.so.1 server_policy passwd auth required pam_ldap.so.1 cron account required pam_unix_account.so.1 # SSH Pubkey - Needed for openldap and still probably needed for ipa # without this, ssh keys stopped working. sshd-pubkey account required pam_unix_account.so.1

I installed pkgutil from OpenCSW. Seriously, this will help you out tremendously.

# /opt/csw/bin/pkgutil -y -i sudo sudo_ldap # vi /etc/opt/csw/sudo.conf Plugin sudoers_policy sudoers-ldap.so Plugin sudoers_io sudoers-ldap.so # ln -s /etc/ldap.conf /etc/opt/csw/ldap.conf

After that, I went ahead and tested id, logins, etc.

bash-3.2# id louis.abel uid=25439([email protected]) gid=1006800013(aocusers)

The current problem is that id won't show group membership, yet if you run the groups command, it will. Just because it shows the groups, doesn't mean you can really work with the files that are group owned as such. This only changes if you login via [email protected].

Solaris 11

Solaris 11 was a lot more forgiving. I was able to get SSL to work and the sudo that they provide to work also. Most of the things above can be used. I'll point out which ones you can pull from above.

Create another profile on the IPA server.

dn: cn=solaris_authssl,ou=profile,dc=ipa,dc=example,dc=com authenticationMethod: tls:simple objectClass: top objectClass: DUAConfigProfile bindTimeLimit: 5 cn: default cn: solaris_authssl defaultSearchBase: dc=ipa,dc=example,dc=com defaultServerList: pentl01.ipa.example.com pentl02.ipa.example.com followReferrals: TRUE objectclassMap: shadow:shadowAccount=posixAccount objectclassMap: passwd:posixAccount=posixaccount objectclassMap: group:posixGroup=posixgroup searchTimeLimit: 15 serviceSearchDescriptor: group:cn=groups,cn=compat,dc=ipa,dc=example,dc=com serviceSearchDescriptor: passwd:cn=users,cn=compat,dc=ipa,dc=example,dc=com serviceSearchDescriptor: netgroup:cn=ng,cn=compat,dc=ipa,dc=example,dc=com serviceSearchDescriptor: ethers:cn=computers,cn=accounts,dc=ipa,dc=example,dc=com serviceSearchDescriptor: sudoers:ou=sudoers,dc=ipa,dc=example,dc=com profileTTL: 6000

Generate your keytabs and move them to the server.

Configure the krb5.conf file as necessary.

Set /etc/defaultdomain to your IPA domain.

Place IPA cacert into /var/ldap/ipa.pem

Your /etc/ldap.conf will need to be a tad different on Solaris 11. You will need to remove the TLS_CERT line. Solaris 11 no longer uses NSS.

base dc=ipa,dc=example,dc=com scope sub TLS_CACERTDIR /var/ldap TLS_CACERT /var/ldap/ipa.pem tls_checkpeer no ssl off bind_timelimit 120 timelimit 120 uri ldap://pentl01.ipa.example.com sudoers_base ou=sudoers,dc=ipa,dc=example,dc=com pam_lookup_policy yes

To configure nsswitch, you have to run commands (thanks oracle).

/usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/default = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/sudoer = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/password = astring: "files ldap [NOTFOUND=return]"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/group = astring: "files ldap [NOTFOUND=return]"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/host = astring: "files dns"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/network = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/protocol = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/rpc = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/ether = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/netmask = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/bootparam = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/publickey = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/netgroup = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/automount = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/alias = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/printer = astring: "user files"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/service = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/project = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/auth_attr = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/prof_attr = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/tnrhtp = astring: "files ldap"' /usr/sbin/svccfg -s svc:/system/name-service/switch 'setprop config/tnrhdb = astring: "files ldap"' /usr/sbin/svcadm refresh svc:/system/name-service/switch ; /usr/sbin/svcadm restart svc:/system/name-service/switch ; /usr/sbin/svcadm restart ldap/client

Now you need to setup all of your pam files. Here are the ones I configured and it worked.

# /etc/pam.d/krlogin auth required pam_unix_cred.so.1 auth required pam_krb5.so.1 # /etc/pam.d/krsh auth required pam_unix_cred.so.1 auth required pam_krb5.so.1 # /etc/pam.d/login auth requisite pam_authtok_get.so.1 auth sufficient pam_krb5.so.1 auth required pam_dhkeys.so.1 auth required pam_unix_cred.so.1 auth required pam_dial_auth.so.1 auth required pam_unix_auth.so.1 server_policy auth required pam_ldap.so.1 # /etc/pam.d/other auth definitive pam_user_policy.so.1 auth requisite pam_authtok_get.so.1 auth sufficient pam_krb5.so.1 auth required pam_dhkeys.so.1 auth required pam_unix_cred.so.1 auth required pam_unix_auth.so.1 server_policy auth required pam_ldap.so.1 account requisite pam_roles.so.1 account definitive pam_user_policy.so.1 account required pam_unix_account.so.1 server_policy account required pam_krb5.so.1 account required pam_tsol_account.so.1 account required pam_ldap.so.1 session definitive pam_user_policy.so.1 session required pam_unix_session.so.1 password definitive pam_user_policy.so.1 password include pam_authtok_common password sufficient pam_krb5.so.1 password required pam_authtok_store.so.1 server_policy # /etc/pam.d/passwd auth binding pam_passwd_auth.so.1 server_policy auth required pam_ldap.so.1 account requisite pam_roles.so.1 account definitive pam_user_policy.so.1 account required pam_unix_account.so.1 # /etc/pam.d/ppp auth requisite pam_authtok_get.so.1 auth required pam_dhkeys.so.1 auth required pam_unix_cred.so.1 auth required pam_dial_auth.so.1 auth required pam_unix_auth.so.1 server_policy auth required pam_ldap.so.1 # /etc/pam.d/rlogin auth definitive pam_user_policy.so.1 auth sufficient pam_rhosts_auth.so.1 auth requisite pam_authtok_get.so.1 auth sufficient pam_krb5.so.1 auth required pam_dhkeys.so.1 auth required pam_unix_cred.so.1 auth required pam_unix_auth.so.1 auth required pam_ldap.so.1 # /etc/pam.d/rsh auth definitive pam_user_policy.so.1 auth sufficient pam_rhosts_auth.so.1 auth required pam_unix_cred.so.1 auth required pam_ldap.so.1 # /etc/pam.d/sshd-pubkey account required pam_unix_account.so.1

The only issue is if you are using domain resolution order, trying to login without the realm name for AD users will cause an assertion error. I asked Oracle to fix this because Solaris 10 was fine. I don't know when they'll fix it. As of this writing, they claim October 20, 2017.

The actual current problem is that id won't show group membership, yet if you run the groups command, it will. Just because it shows the groups, doesn't mean you can really work with the files that are group owned as such. This only changes if you login via [email protected] or remove the domain resolution order entirely - this would cause you to have to login with username@domain.

HBAC

For legacy clients to authenticate AD users properly, you need an HBAC rule against the IPA servers that perform the task of talking to AD. I'd recommend creating an ipaservers group (if one doesn't exist already) of all of your IPA servers, and then creating the hbac rules.

# ipa hbacsvc-add system-auth # ipa hbacrule-add system-auth-legacy # ipa hbacrule-add-host --hostgroup=ipaservers # ipa hbacrule-mod --usercat=all system-auth-legacy

For legacy clients, you need the pam_hbac module. For RHEL 5, there is a copr repo that you can pull the RPM's for. For Solaris, you are required to compile them, which isn't fun, I understand. Below I'll show you how to compile the modules and how to utilize the module.

Solaris 10 ++++++++++

You need to install OpenCSW's pkgutil. I also brought the sources of pam_hbac from github over as a git clone in a tar file.

root# pkgutil -y -i libglib2_dev gmake openssl binutils gcc4g++ glib2 ar libnet root# cd /tmp root# tar xf pam_hbac.tar root# cd /tmp/pam_hbac root# PATH=$PATH:/opt/csw/bin root# autoconf -o configure ; autoreconf -i root# ./configure AR=/opt/csw/bin/gar --with-pammoddir=/usr/lib/security --sysconfdir=/etc/ --disable-ssl root# vi Makefile.am ## Comment out the man pages #if HAVE_MANPAGES #SUBDIRS += doc #endif root# make root# make install root# /etc/pam_hbac.conf URI = ldap://pentl01.ipa.example.com BASE = dc=ipa,dc=example,dc=com BIND_DN = uid=hbac,cn=sysaccounts,cn=etc,dc=ipa,dc=example,dc=com BIND_PW = secret123 SSL_PATH = /var/ldap HOST_NAME = hostname.example.com

After this, you can place the hbac lines in your pam file. I usually put it under 'other' and 'login' as "account".

login auth requisite pam_authtok_get.so.1 login auth sufficient pam_krb5.so.1 login auth required pam_dhkeys.so.1 login auth required pam_unix_cred.so.1 login auth required pam_dial_auth.so.1 login auth required pam_unix_auth.so.1 use_first_pass login auth required pam_ldap.so.1 login account required pam_hbac.so ignore_unknown_user ignore_authinfo_unavail debug # other account requisite pam_roles.so.1 other account required pam_projects.so.1 other account binding pam_unix_account.so.1 server_policy other account required pam_ldap.so.1 other account required pam_hbac.so ignore_unknown_user ignore_authinfo_unavail debug

To answer your question, yes. I had to disable SSL. This is because IPA has a lot of ciphers turned off. Solaris 10 doesn't support much higher, and because these are dynamically compiled (not static), they pick up both SSL libraries from OpenCSW and the system. But the system is picked up first and used.

Solaris 11 ++++++++++

Compiling on Solaris 11 is easier (thankfully).

root# pkg install autoconf libtool pkg-config automake gcc asciidoc docbook (some of these won't install, that's fine) root# cd /tmp root# tar xf pam_hbac.tar root# cd /tmp/pam_hbac root# autoreconf -if root# ./configure --with-pammoddir=/usr/lib/security --mandir=/usr/share/man --sysconfdir=/etc/ root# make root# make install root# /etc/pam_hbac.conf URI = ldap://pentl01.ipa.example.com BASE = dc=ipa,dc=example,dc=com BIND_DN = uid=hbac,cn=sysaccounts,cn=etc,dc=ipa,dc=example,dc=com BIND_PW = secret123 SSL_PATH = /var/ldap HOST_NAME = hostname.example.com

Pam is a little different here. I configured login and other just like on Solaris 10.

# goes at the VERY end of the account block in /etc/pam.d/other and /etc/pam.d/login account required pam_hbac.so ignore_unknown_user ignore_authinfo_unavail debug

General +++++++

You also need to create an hbac service in IPA.

# ipa hbacsvc-add sshd-gssapi # ipa hbacsvc-add sshd-kbdint

You would need to associate those in particular to your HBAC rule sets as needed for your legacy hosts.

Summary

What I've learned overall is that trying to get legacy clients to work with FreeIPA is actually simple if you were only using IPA. It begins to get tricky when you add in AD. The other thing I've learned is that there are companies that will not let go of their dying systems and we just have to live with it. Because this is the case and I know it will not change any time soon, I had to really put a lot of (unnecessary) effort into making this all work. RHEL 6 and RHEL 7? Easy. RHEL 5, Solaris 10/11? Not so easy.

This also gave me a more of a sour taste in my mouth about the political landscape of an enterprise. It's just flat out unfortunate. But, I am glad I got this experience so I can show others what I had to deal with and hopefully others can conquer it in the same way or even better than I have.

[ssnlinux]

SaltStack on Solaris 10

Unfortunately, I work in a mixed environment. While we have majority of Red Hat based Linux servers, we also have over a couple hundred Solaris UNIX servers. We manage our Linux servers using SaltStack, which is great for us, but not for Solaris. Our Solaris servers are NOT consistent in configuration and on top of that, they're all Solaris 10 and we've never had configuration management for them.

Fortunately, there's OpenCSW that assists with providing necessary packages for usability on Solaris. Combining OpenCSW packages and pip, we can install salt and start managing them. This is what I did to get salt working without using the outdated salt package in OpenCSW.

Note: There are some caveats with this method. The main one that you need to know about is ZMQ. The ZMQ in OpenCSW is ridiculously old. We're talking version 2 kind of old. This means that the Solaris minions will consistently lose contact with the master and the service will require a restart constantly to stay up. Until it's updated, you will need to deal with this fact.

1. Install pkgutil from OpenCSW

# pkgadd -d http://get.opencsw.org/now

2. Update the catalog

# pkgutil -U -u

3. Install python related packages

# pkgutil -y -i py_pip py_pyzmq py_m2crypto py_crypto py_jinja2 py_msgpack_python py_yaml py_markupsafe py_requests

4. Use pip to install salt

# /opt/csw/bin/pip install salt

My environment is stuck at 2015.5.10 currently. So we needed to use that version to keep everything in line and avoid potential master and minion conflicts.

# /opt/csw/bin/pip install salt==2015.5.10

5. Modify /opt/csw/lib/python2.7/site-packages/salt/_syspaths.py to look like this

ROOT_DIR = '/' CONFIG_DIR = '/etc/salt' CACHE_DIR = '/var/cache/salt' SOCK_DIR = '/var/run/salt' SRV_ROOT_DIR = '/srv' BASE_FILE_ROOTS_DIR = '/srv/salt' BASE_PILLAR_ROOTS_DIR = '/srv/pillar' BASE_MASTER_ROOTS_DIR = '/srv/salt-master' LOGS_DIR = '/var/log/salt' PIDFILE_DIR = '/var/run'

6. Optionally, create /etc/salt ahead of time with a minion configuration (with master set and other settings you need)

7. Create /var/svc/manifest/system/salt-minion.xml with the following content:

<?xml version='1.0'?> <!DOCTYPE service_bundle SYSTEM '/usr/share/lib/xml/dtd/service_bundle.dtd.1'>

<service_bundle type='manifest' name='salt-minion:salt-minion'>

<service  name='network/salt-minion'  type='service'  version='1'>  <create_default_instance enabled='false' />  <single_instance />

 <dependency name='fs'    grouping='require_all'    restart_on='none'    type='service'>    <service_fmri value='svc:/system/filesystem/local' />  </dependency>

 <dependency name='net'    grouping='require_all'    restart_on='none'    type='service'>    <service_fmri value='svc:/network/loopback' />  </dependency>

 <exec_method    type='method'    name='start'    exec='/lib/svc/method/svc-salt-minion start'    timeout_seconds='120'>  </exec_method>

 <exec_method    type='method'    name='stop'    exec='/lib/svc/method/svc-salt-minion stop'    timeout_seconds='60'>  </exec_method>

 <exec_method    type='method'    name='restart'    exec='/lib/svc/method/svc-salt-minion restart'    timeout_seconds='180'>  </exec_method>

</service>

</service_bundle>

8. Create /lib/svc/method/svc-salt-minion with the following content:

#!/bin/sh # #AUTOENABLE no #

PATH=/opt/csw/bin:$PATH export PATH CONF_DIR=/etc/salt PIDFILE=/var/run/salt-minion.pid SALTMINION=/opt/csw/bin/salt-minion

[ ! -d ${CONF_DIR} ] &&  exit $CONF_DIR

start_service() {        /bin/rm -f ${PIDFILE}        $SALTMINION -d -c ${CONF_DIR} 2>&1 }

stop_service() {        if [ -f "$PIDFILE" ]; then                /usr/bin/kill -TERM `/usr/bin/cat $PIDFILE`        fi }

case "$1" in start)        start_service        ;; stop)        stop_service        ;; restart)        stop_service        sleep 1        start_service        ;; *)        echo "Usage: $0 {start|stop|restart}"        exit 1        ;; esac

9. Set /lib/svc/method/svc-salt-minion to 755 using chmod.

At this point, we can create a package. Let's go ahead and do that now.

1. Create a list of files/folders in /opt/csw that have the salt name.

# cd /tmp # find /opt/csw | grep salt >> /tmp/saltlist

Add these to the /tmp/saltlist file...

/etc/salt /var/svc/manifest/system/salt-minion.xml /lib/svc/method/svc-salt-minion

2. Pipe the list through pkgproto.

# cat /tmp/saltlist | pkgproto > /tmp/salt.prototype

3. Open the /tmp/salt.prototype file and add this at the top.

i pkginfo

4. Create the pkginfo file like this (changing the versions as needed):

PKG=salt NAME="Salt 2015.5.10" VERSION="2015.5.10" ARCH="sparc" CATEGORY="application" DESC="Distributed remote execution system" BASEDIR="/"

5. Build the package.

# pkgmk -o -r / -d /tmp -f salt.prototype

6. Move the package into a package stream.

# pkgtrans -s /tmp salt-2015.5.10.pkg salt

Now you can install the package elsewhere using these steps (assuming you are using pkgutil everywhere else):

# pkgutil -y -i py_pip py_pyzmq py_m2crypto py_crypto py_jinja2 py_msgpack_python py_yaml py_markupsafe py_requests # pkgadd -d salt-2015.5.10.pkg salt

Now, to run Salt, you'll need to configure /etc/salt/minion and then enable the service.

1. /etc/salt/minion

. . . master: cm.example.com . . .

2. Enable the service

# svccfg import /var/svc/manifest/system/salt-minion.xml # svcadm enable salt-minion

Check... # svcs -xv grep salt-minion

3. Check /var/log/salt/minion for content. You'll more than likely see the ZMQ warning.

2016-08-02 10:35:25,158 [salt.minion                              ][WARNING ][9233] You have a version of ZMQ less than ZMQ 3.2! There are known connection keep-alive issues with ZMQ < 3.2 which ma y result in loss of contact with minions. Please upgrade your ZMQ!

[ssnlinux]

RHEL/CentOS and Informix Raw Storage

My team is a combination of UNIX, Linux, and Database Administrators. There is usually a day or two where I have to interact closely with the database administrators for either setting up servers or assisting in providing more storage for them on already built servers.

We have mostly virtual machines running our informix databases. This is generally fine because we used cooked file systems and those are straight forward to setup. However, it’s a whole other ball game on new physical servers where we are not used cooked file systems. Basically, they want everything in “raw chunks”, similar to how we do our Oracle database servers.

Basically, I took the time to use multipath to manage the LUN’s presented to the server and name them accordingly. Example.

multipaths { multipath { wwid 360000970000195700675533030383046 alias ifxdwdba } ... (much more here, spanning to ifxdwdbz) }

So I figured this was fine and then I spent the time to script out a way to carve each of these LUN's up they wanted (long story short, these luns are 1TB each, and they wanted 20GB partitions each). Before I handed it off, I needed to come up with a way to give them full ownership of the devices as their user.

# cd /dev/mapper # for x in $(ls | egrep -v 'mpatha|control'); do printf "%s %s\n" "$x" \ "$(udevadm info --query=all --name=/dev/mapper/$x | grep -i dm_uuid| \ cut -d'=' -f 2)" >> /tmp/info; done # for x in $(cat /tmp/info) ; do echo "KERNEL==\"dm-*\",ENV{DM_UUID}==\"$x\",OWNER=\"informix\",GROUP=\"informix\",MODE=\"0660\"" >> /etc/udev/rules.d/99-informix.rules ; done # udevadm control --reload-rules

This seemed to work and they were in business. Or so I thought.

Turns out that Informix is picky on the storage you give it. It expects character devices, not block devices. My initial thought was that all disks should just be block devices and that character devices were for things like printers, sound cards, etc, not for disks. Informix doesn't care; it wants its storage to be "raw", and the "raw" way to do it is to be a character device.

An easy way around this is to do something like raw /dev/raw/raw1 /dev/dm-7. However, that's tiring, especially with lots of partitions. And plus, you can't dictate the name of the "raw" devices we're making. So I decided there needed to be a better way, for me and my database admins.

At the bottom of my udev rules, I put similar rules to this (note, you will need to run the command yourself and find the "result" first).

ACTION=="add",KERNEL=="dm-*",PROGRAM=="/usr/lib/udev/scsi_id -g -u -d %N",RESULT=="360000970000195700675533030383046",RUN+="/usr/local/sbin/udevlinker %n %N" ... (others) ACTION=="add",KERNEL=="raw*",OWNER="informix",GROUP="informix",MODE="0660"

Basically what the %n means is "minor" number. Each device on a system gets a major and a minor number. Major being the device type, minor being the number of the device. The %N variable is the full path of the block device. If you actually look, the minor numbers are represented on the dm-* devices in /dev.

Normally the RUN that others use is the raw command directly. Instead, I used a script of my own. On top of that, I ensured any "raw" devices that get added to the system, the informix user will own them.

#!/bin/bash # /usr/local/sbin/udevlinker MINOR=$1 BLPATH=$2 DMNAME=$(ls -l /dev/mapper | awk "/dm-$MINOR$/ { print \$9 }") /bin/mkdir -p /dev/ifx /bin/raw /dev/raw/raw$MINOR $BLPATH /bin/ln -sf /dev/raw/raw$MINOR /dev/ifx/$DMNAME

The script above is the one in question. I ensured that it would take in the minor number and the block device path. It would also take the time to look at the minor number and compare it to the list of multipath symlinks in /dev/mapper and find the appropriate name. That way, I can make a symbolic link in /dev/ifx that leads to the correct /dev/raw/* device.

Once I had put everything together, I ran udevadm control --reload-rules and then udevadm trigger --action=add. This was a success. The raw devices got created and the symbolic links did too.

# ls -l /dev/ifx | head total 0 lrwxrwxrwx. 1 root root 13 May 12 00:26 ifxdwdba -> /dev/raw/raw8 lrwxrwxrwx. 1 root root 14 May 12 00:26 ifxdwdba1 -> /dev/raw/raw54 lrwxrwxrwx. 1 root root 15 May 12 00:26 ifxdwdba10 -> /dev/raw/raw111 lrwxrwxrwx. 1 root root 15 May 12 00:26 ifxdwdba11 -> /dev/raw/raw114 lrwxrwxrwx. 1 root root 15 May 12 00:26 ifxdwdba12 -> /dev/raw/raw125 lrwxrwxrwx. 1 root root 15 May 12 00:26 ifxdwdba13 -> /dev/raw/raw130 lrwxrwxrwx. 1 root root 15 May 12 00:26 ifxdwdba14 -> /dev/raw/raw137 lrwxrwxrwx. 1 root root 15 May 12 00:26 ifxdwdba15 -> /dev/raw/raw151 lrwxrwxrwx. 1 root root 15 May 12 00:26 ifxdwdba16 -> /dev/raw/raw152 # ls -l /dev/raw | head total 0 crw-rw----. 1 informix informix 162, 100 May 12 00:26 raw100 crw-rw----. 1 informix informix 162, 101 May 12 00:26 raw101 crw-rw----. 1 informix informix 162, 102 May 12 00:26 raw102 crw-rw----. 1 informix informix 162, 103 May 12 00:26 raw103 crw-rw----. 1 informix informix 162, 104 May 12 00:26 raw104 crw-rw----. 1 informix informix 162, 105 May 12 00:26 raw105 crw-rw----. 1 informix informix 162, 106 May 12 00:26 raw106 crw-rw----. 1 informix informix 162, 107 May 12 00:26 raw107 crw-rw----. 1 informix informix 162, 108 May 12 00:26 raw108

However, there was one small problem. When the system is booting up, there is no proper multipath names in mapper by the time udev is doing its job. This means the symbolic link that gets created is literally named the same as the raw device we're linking to.

There's two options. I could enable rc.local (since this is on RHEL 7) or make a one-shot. Either way, we can get the same accomplished. I made a one-shot like this.

# cat /etc/systemd/system/udevlinker.service [Unit] Description=udev linker After=network.target remote-fs.target nss-lookup.target [Service] Type=oneshot User=root ExecStart=/bin/sh -c 'sleep 3 ; /bin/rm /dev/ifx/* ; /usr/sbin/udevadm trigger --action=add' [Install] WantedBy=multi-user.target

This ensured that after all of these targets were reached, I would go and re-run the udev trigger after deleting all of the "bad" links. This definitely helps on reboots.

For my RHEL 6 guys or people who want to use rc.local, you can do similar.

sleep 3 /bin/rm /dev/ifx/* /usr/sbin/udevadm trigger --action=add

I'm sure I could do much better than this. But I couldn't think of a better way at the time. Hopefully this helps anyone else facing a similar issue or finding a better way to tackle this problem.

[ssnlinux]

Cisco IP Communicator 8.6

We have desk phones and we have softphones. These softphones are great, except for when we don’t have a client on Linux.

Since winehq decided to reject my submission, I’m going to post what I did here to get IPC to work properly under Fedora 23, Wine 1.9.0 staging.

This installation was done via PlayOnLinux, choosing a "non-listed" program. Keep in mind that PlayOnLinux is separate from wine development. But, this should NOT stop any user from being able to do it in their own manner. However, I used the System Wine, which is 1.9.0 staging in Fedora 23's current base repo (as of Jan 13, 2016).

*******************************************INSTALL******************************************* More than likely, you can emulate this setup on multiple wine versions without issues, or on different distributions. These are the steps that I took to ensure installation:

1. Ensure crypt32 has an override (native,builtin)  a. This is because the certificates and other things won't install properly, and the installation will crash 2. Emulate as Windows XP. I could not get this to work under 7 or higher. 3. Run the setup and install the program. At this point, it may crash while trying to open the communicator. This is fine. 4. Open communicator and go through the Audio Tuning Wizard. It may crackle, etc. But you should be able to go right through it without any issues. 5. Do a test call.

If you have issues, consider doing the following (regardless of POL or not): 1. If you have issues, install corefonts and crypt32 to your wineprefix. 2. If you are using POL, select the additional software for: POL_Function_OverrideDLL, POL_Install_corefonts, POL_Install_crypt32

*********************************************************************************************

Special Notes:

-> If you are using this over a VPN, you it is highly recommended, to prevent any headaches, to set the Communicator to use a specific IP. This is how: 1. Click the menu icon in the corner -> Preferences 2. Click Audio -> Network (button) 3. Select "Use this address" and place the IP your VPN interface has 4. Press OK. It will reload.

-> Fedora has pulseaudio by default (most modern distros should). If you have crackling issues, fix it like so:

# sed -i.bak '/load-module module-udev-detect/ s/$/ tsched=0/' /etc/pulse/default.pa (Verify...) # grep tsched /etc/pulse/default.pa

To do it manually, if you don't trust sed...

# vi /etc/pulse/default.pa (Scroll down to load-module module-udev-detect) (Add tsched=0 to the end)

.ifexists module-udev-detect.so load-module module-udev-detect tsched=0 .else

[ssnlinux]

vSphere Web Client on Linux

Before I get into showing you how to get the vSphere web client to work, I need to give a quick back story. If you’re not interested in this, jump down a few paragraphs to “The Problem”. 

I work in a place that uses a heavy amount of VMware. Because of this, I have to be able to login to a client and manage the virtual machines that my team owns. This is fine, assuming you are running Windows or OS X, but not so fine when under Linux. I don’t use Windows at work (or try not to) and just a few months ago, all machines at my desk are now Fedora Linux.

So I knew ahead of time of the web client stopped working because Google did the right thing and disabled NPAPI for good on Linux. This basically made the vSphere Client Integration Plugin useless to me. I ran into this before and was aware that Flash is no longer supported on the Linux platform. Sure it gets security updates once in a while, but it’s stuck on an extremely low version. So low in fact, that my company’s web client asks for at least 11.5 of flash. 

So, I did manage to get vSphere Web to work in Firefox, combination of the plugin, pepper flash and fresh player plugin. However, I ran into one issue. 

This error is annoying. Windows and OS X have this binary packaged with the plugin and yet, here’s the Linux version, clearly missing these binaries. Why? VMware support, who I would think would try to test my method internally, decided to tell me ‘no’ without trying to give any hints on where I could obtain the binaries for Linux. Basically they dodged my question. You don’t tell or do that to a Linux admin. Ever. 

Also, shout out to the VMware support rep and engineers for caring about their customer base. It’s clear you guys really do care about your customers and the communities.

The Problem:

Google disabled NPAPI fully on Chrome for Linux (this is a good thing, regardless)

VMware decided to stay in the dark ages for a while and their 6.0 client is using PPAPI to an extent. So they finally caught up. (Still needs flash)

Adobe no longer updates Flash aside from security updates (this is a good thing because Flash needs to die). 

VMware support decided to not give me any hints or ideas where to obtain “vmware-authd”

Prerequisites: 

Fedora 22 or higher, 64 bit (tested with F23 also)

Firefox browser (chrome or chromium not supported)

vCenter/vSphere 5.5 or above (6.0 also applies)

Third-party RPM’s (links will follow)

Understanding that this is sort of a hack

1. First thing is first, you need a couple of RPM’s. You need freshplayerplugin and chromium-pepper-flash (flash that comes with Chrome). Links below: Chromium Pepper Flash: http://koji.russianfedora.pro/koji/packageinfo?packageID=110  Fresh Player Plugin: http://koji.russianfedora.pro/koji/packageinfo?packageID=134 Make sure to install them... # dnf install *.rpm I’m unsure of any repo that handles these. I have my own and update them occasionally. 2. Ensure that firefox see’s the plugin in the ‘add ons’ (click your menu icon on the top right and click the add ons button). Set it to Ask to Activate.

3. Go to a vSphere Web Console in your environment. 

4. The bottom will usually state to download the Client plugin. Click the link and download the “bundle” file.

5. Install the bundle file. You’ll need to elevate to root to do so. You can also use sudo. 

[nazu@diurne 05:46 PM ~]$ su - [root@diurne 05:47 PM ~]# cd /home/nazu/Downloads/ [root@diurne 05:47 PM ~]# chmod +x *.bundle [root@diurne 05:47 PM Downloads]# ./VMware-ClientIntegrationPlugin-5.5.0.x86_64.bundle

6. Optional: This is only needed if you face an error “invalid username/password” -- Create a file called /etc/vmware/config and place the following in it: 

vmauthd.server.alwaysProxy = "TRUE"

Note: Steps 7-9 is needed for versions lower than 6.0 -- 6.0 should not require this. If you see the vmware-authd error, then proceed with these steps.

7. Download vmware player for linux here

8. Instead of installing it, extract it instead. This will take a few moments for the prompt to come back.

# chmod +x VMware-Player-12.0.0-2985596.x86_64.bundle # ./VMware-Player-12.0.0-2985596.x86_64.bundle -x /root/vm Extracting VMware Installer...done.

9. Copy the binaries and set their permissions. Note: We use SUID because otherwise, the console will still fail to load with one of a few cryptic errors.

# cp ~/vm/vmware-vmx/sbin/vmware-authd* /usr/sbin/ # chmod ugo+x /usr/sbin/vmware-authd* # chmod u+s /usr/sbin/vmware-authd

10. Next, tell SELinux to stop whining (This usually sufficies, you can use audit2why < /var/log/audit/audit.log later if you still have issues/errors)

# setsebool mozilla_plugin_can_network_connect 1

11. Close firefox completely (if it’s not already) and reopen it. Check your plugins.

12. Go back to vSphere and attempt to login. If you see pop-ups to allow the plugins, make sure to set them to allow and refresh the page.

So far so good.

The bad part (yes, there is a bad part):  Pepper flash is not “sandboxed” like most other plugins you would install and use in firefox (due to the wrapper), so it’s sort of a security concern there. Recently, firefox has done the right thing and disabled flash by default. However, because firefox doesn’t know (or recognize) pepperflash actually is flash, my recommendation is to set flash to “Ask to activate” (as noted in Step 2). Otherwise it will just keep activating itself. I’m not a fan of that obviously. I’d prefer to say “never activate”, but unfortunately there are some sites I still need to use flash.

I also had a chance to try this with a 6.0 vCenter/vSphere. Works as intended without issues (and the remote console looks and feels much better). Also, it’s using PPAPI. However, it still needs flash because VMware is working on a VMRC client for Linux™ circa April 17, 2015. I foresee an actual client sometime in 2020.

I hope everyone has found this informative and helpful. 

[ssnlinux]

Ethernet Interface Hangs

As most know, I run a mini-atx CentOS router, because of some of the testing and things I like to do. However, I noticed that I started getting hangs and other internet issues lately. These were in my logs.

Mar 17 00:25:24 palaceredirect kernel: e1000e 0000:00:19.0 eno1: Detected Hardware Unit Hang:  TDH                  <1>  TDT                  <16>  next_to_use          <16>  next_to_clean        <1> buffer_info[next_to_clean]:  time_stamp           <134564978>  next_to_watch        <1>  jiffies              <134566ed8>  next_to_watch.status <0> MAC Status             <80083> PHY Status             <796d> PHY 1000BASE-T Status  <3c00> PHY Extended Status    <3000> PCI Status             <10> Mar 17 00:25:24 palaceredirect kernel: e1000e 0000:00:19.0 eno1: Reset adapter unexpectedly Mar 17 00:25:28 palaceredirect kernel: e1000e: eno1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: Rx/Tx

You may or may not have seen this before. It’s pretty cryptic, but it spams /var/log/messages. Enough for me to be concerned and a little annoyed. I also had this issue on an actual racked dell server (I forget the model). This is what I did to fix the issue:

ethtool -K eno1 tso off gso off

That was about it. I’m not a fan of /etc/rc.local, but you could always put it in there to make sure they are called when the system is down coming up.

echo “ethtool -K eno1 tso off gso off” >> /etc/rc.d/rc.local chmod +x /etc/rc.d/rc.local

If you’re feeling particularly fun, you could always make a systemd unit to take care of it. Just ensure your command is “/usr/sbin/ethtool -K %i tso off gso off” and it comes after network.target.

[ssnlinux]

SUDO Lesson: /etc/hosts

For those who use LDAP/AD with SUDO containers for elevated access, this will be an important lesson for you.

If someone asks to add an entry in /etc/hosts that's like the below for an application or something else...

127.0.0.1 hostname.example.com

Don't do it. Or if it's already there, remove it immediately. 

Let's say you have this container. This controls sudo access to those hosts for those users.

dn: cn=tomcat_users,ou=SUDOers,dc=bromosapien,dc=net objectClass: sudoRole cn: tomcat_users description: Administration Role sudoCommand: ALL sudoHost: web1-tc1.bromosapien.net sudoHost: app1-tc1.bromosapien.net sudoUser: chris sudoUser: zera

Here's zera.

# sudo -l -U zera . . . User zera may run the following commands on this host:   (ALL : ALL) ALL

Alright, cool. So it looks like Zera can still perform her commands. Let's add our special /etc/hosts entry, using the example from earlier, and see what happens.

# sudo -l -U zera Matching Defaults entries for zera on this host: . . . User zera may run the following commands on this host:

Looks like her commands disappeared. 

$ sudo -i [sudo] password for zera: zera is not allowed to run sudo on web1-tc1.  This incident will be reported.

So you may be wondering why that happens. It comes down to host resolution and the order in which names are prioritized. By default, /etc/hosts already has a localhost entry for 127.0.0.1. When sudo tries to do a look up in LDAP, it more than likely attempts a look up of its name, finds out it's 127.0.0.1, thinks its name is localhost instead, and looks it up that way. (That's how it appears to happen, anyway). As a result, sudo LDAP queries fail. 

Note: If you use "sudoHost: ALL", this will not apply. The sudo query looks for "ALL" on top of the system's hostname. 

Conclusion: Do NOT put an /etc/hosts entry for 127.0.0.1 that is the hostname of the system. You will create unnecessary and unneeded problems. And there should be no reason to be doing that in the first place.

[ssnlinux]

SSSD, Local Accounts, LDAP Groups, and You

So recently, I ran into a little problem where I work. Most applications run under "service" accounts that we create for our customers because running things as root is not best practice. However, since the introduction to SSSD, this has posed some problems for those local service accounts that are part of groups in LDAP. Check the below example.

dn: cn=radusers,ou=Group,dc=bromosapien,dc=net objectClass: posixGroup objectClass: top cn: radusers description: Group for radian gidNumber: 20000 memberUid: zera memberUid: chris memberUid: ana

We have an account called "zera" in particular. Thing is, this account isn't in LDAP. And, on this machine, zera's groups are not showing up.

[root@library ~]# ldapsearch -x -LLL uid=zera [root@library ~]# id zera uid=11000(zera) gid=11000(zera) groups=11000(zera)

To fix this, you have to make a change to your sssd.conf. Remember: This is under the assumption you are using rfc2307 schema. Not bis. 

[domain/default] . . .  ldap_rfc2307_fallback_to_local_users = True

Once that's in, restart sssd. You may need to clean the cache out in /var/lib/sss/db, though.

[root@library ~]# id zera uid=11000(zera) gid=11000(zera) groups=11000(zera),20000(radusers)

[ssnlinux]

CentOS/RHEL 7 Active Directory with SUDO

A challenge I had to face recently was trying to get RHEL 6 and RHEL 7 integrated with Active Directory authentication along with SUDO. While this isn't extremely difficult (since there's an adcli and realm command on our Linux machines to add to the domain), it becomes a whole different story when you need to do true ID mapping and sudo. 

First thing that has to be done, is we need to get the SUDO schema added. Our environment is 2012 R2. I pulled a sudo schema from /usr/share/doc/sudo-1.8.6p7/schema.ActiveDirectory. I replaced DC=X to my domain using sed.

% sed -i 's/DC=X/DC=moo,DC=cloud/g' schema.ActiveDirectory % unix2dos schema.ActiveDirectory

Afterwards, I transfered the file over to my Active Directory server, opened a powershell prompt as administrator, and added in the schema.

PS C:\Users\label\Desktop> ldifde -i -f schema.ActiveDirectory

That should be fine and should import without issue. You will need to make an OU and a sudoRole next. Example below.

dn: OU=SUDOers,DC=moo,DC=cloud changetype: add objectClass: top objectClass: organizationalUnit ou: SUDOers dn: CN=ADMINS,OU=SUDOers,DC=moo,DC=cloud changetype: add objectClass: top objectClass: sudoRole sudoCommand: ALL sudoHost: ALL sudoHost: * sudoUser: label

The same command above can be used to add that in. Simple enough, so now on our example RHEL 7 machine, I did the following to get it added in.

# realm join mootare.cloud -U label

It'll ask for a password, and when a prompt returns, you should be ready. If you get errors about required packages, install them and try again. For RHEL 6, try using adcli join.

I made these changes to my /etc/sssd/sssd.conf

[sssd] services = nss, pam, pac, sudo [domain/moo.cloud] ldap_id_mapping = False

Afterwards, I stopped sssd, destroyed the cache in /var/lib/sss/db/* and started sssd back up. My ID shows up fine.

[[email protected]@moosbl-03 ~]$ id uid=10001([email protected]) gid=10001([email protected]) groups=10001([email protected]),10(wheel) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

Remember, this is under the assumption that you set your user(s) with an ID and a group with a gid in it! Examples below.

[ssnlinux]

CNTLM - Self Service Proxy

At work today before I left, the Lead Operator noted to me a handy package called cntlm, which is apparently a way to create a local proxy on the box, according to an internal proxy inside of your enterprise network.

This is handy, because you can route all of your browsers, services, etc, through this proxy instead of having to use http_proxy or leaving it in a yum.conf or other places. This has made my

Check it out.

% yum install cntlm

What I did is I set it up like so...

## I set my user and domain, with password. ## It is NOT recommended for it be in clear text, ## the conf tells you what to do in that case. Username zeranalika Domain clockcorp Password this_is_my_pass ## I set my proxies by IP. You can use hostname. proxy 10.100.0.1:8080 proxy 10.100.0.2.8080 ## I left this default. Listen 3128 ## This is for what domains or IP's that the proxy should be ignored NoProxy localhost, 127.0.0.*, 10.*, *.angelsofclockwork.net, *.bromosapien.net

After that, I saved the file, started it up and tried out yum.

[root@syan ~]# vi /etc/yum.conf . . . proxy=http://localhost:3128 [root@syan ~]# systemctl start cntlm

[root@syan ~]# yum update -y Loaded plugins: langpacks fedora/21/x86_64/metalink | 14 kB 00:00:00 fedora | 3.8 kB 00:00:00 updates/21/x86_64/metalink | 16 kB 00:00:00 updates | 4.9 kB 00:00:00 (1/4): fedora/21/x86_64/group_gz | 232 kB 00:00:00 (2/4): updates/21/x86_64/group_gz | 405 kB 00:00:00 (3/4): updates/21/x86_64/primary_db | 2.6 MB 00:00:00 (4/4): fedora/21/x86_64/primary_db | 17 MB 00:00:05 (1/2): updates/21/x86_64/updateinfo | 273 kB 00:00:00 (2/2): updates/21/x86_64/pkgtags | 1.3 MB 00:00:00

Success!

[ssnlinux]

FirewallD, NAT, and DHCP

I'm the kind of person who enjoys having a Linux box for my router at home. It was always of use. I started doing this when CentOS 5 was the latest out there for enterprise systems. 

But now, we're in CentOS 7 and Fedora 20/21 with SystemD, FirewallD, and what not. I decided that instead of falling back to regular iptables, I'd give my hand at FirewallD, a frontend to iptables. This is what I did to make it all work.

I set up DHCP as I did here. 

Now to setup FirewallD. Keep in mind, your modem interface would be your external, and your LAN interface would be internal.

% firewall-cmd --change-interface=enp2s0 --zone=external --permanent % firewall-cmd --change-interface=eno1 --zone=internal --permanent % firewall-cmd --set-default-zone=internal % firewall-cmd --complete-reload

That's about it. And then you can use firewall-cmd --list-all --zone=internal or external to see them.   

I do suggest if ssh shows up in external to remove it.

% firewall-cmd --remove-service=ssh --zone=external --permanent

You can always change the port to something really high if you want ssh open. I don't recommend it (even though I do it). I would setup SSH keys and disable root login. That's talk for another time though. :)

If you want to go and forward some ports, you can do something like this. I'll use StarCraft Brood War/WarCraft III ports as an example.

% firewall-cmd --zone=external --add-forward-port=port=6112:proto=udp:toport=6112:toaddr=10.100.1.101 --permanent % firewall-cmd --zone=external --add-forward-port=port=6112:proto=tcp:toport=6112:toaddr=10.100.1.101 --permanent % firewall-cmd --complete-reload

[ssnlinux]

SystemD Things

Since I have been getting a little bit more time to spend with SystemD (since work hasn't had a chance to start rolling out RHEL 7), I've discovered a couple of settings while just poking around. 

[Login] #NAutoVTs=6 #ReserveVT=6

Above is a snippet of /etc/systemd/logind.conf. This is read/understood by the getty service, which provides the consoles on the F keys to show up. F1 shows up by default, and the rest are "socket activated", meaning a login will spawn when you go to it. This number can be increased to 12. Though, if you have logging on F12, it's probably not a good idea to have it that high. (I'm not sure why you would have that many gettys to be honest)

In /etc/systemd/journald.conf, you can forward console messages (that show up in journalctl) to F12 like this:

[Journal] . . . ForwardToConsole=yes TTYPath=/dev/tty12 % systemctl restart systemd-journald

Kind of nifty actually! But I'm the kind of person who still likes my logs in /var/log, so I install the rsyslog package anyway. But I could see why it would be useful on a server.

[ssnlinux]

Useful SSN Links

For those who are interested, here are some useful links.

YouTube - Has most of my tutorials in video form

Wiki - My personal wiki of Linux tutorials between Fedora and CentOS

Glossary - It's an aging, but still useful document of some Linux things. I don't think it takes SystemD into consideration due to its age.

[ssnlinux]

Fedora 20 to 21 Upgrade

Hello all.

So today at work I decided that instead of doing a clean install of the new Fedora release (like I normally do for every release), I would give my hand at FedUp, which is provided in the base repositories. With Fedora's new product lineup of Workstation, Cloud, and Server, it made it a little confusing on what to really choose on the process. For sanity reasons when using something that isn't GNOME, choose "nonproduct" instead.

# yum update -y ; init 6 # yum install fedup -y # fedup --product=nonproduct --network 21 # init 6 # Upgrade begins, it will reboot by itself. # rpm --rebuilddb # yum distro-sync --setopt=deltarpm=0 # package-cleanup --orphans ## Review the list and clean up

For those who chose "workstation" as the product and you were using KDE or perhaps another desktop environment, here is what ended up happening to me. It installed a bunch of gnome packages that I did not want, so I did close to the below to get everything back to the way it was.

# yum update -y ; init 6 # yum install fedup -y # fedup --product=workstation --network 21 # init 6 ## During the upgrade process, I noticed gnome packages. I was upset. # rpm --rebuilddb # yum distro-sync --setopt=deltarpm=0 ## This is when I needed to get rid of the gnome stuff. # rpm -qa | grep gnome >> /tmp/list # yum remove $(cat /tmp/list) ; yum groupinstall "KDE Plasma Workspaces" # rpm -qa | grep fc20 >> /tmp/list2 # package-cleanup --orphans

Hopefully this helps others out.