- Try to trace/follow the intruder back to his origin via looking at
- who
- w
- last
- lastcomm
- netstat
- snmpnetstat
- router information.
- /var/adm/messages (many crackers send e-mail to their "home" accounts)
- syslog (sends logs to other hosts as well)
- wrapper logs
- do a 'finger' to all local users(and check where they last logged in
from)
- history files from shells, such as .history, .rchist, and similiar
files.
Footnote: 'who', 'w', 'last', and 'lastcomm' are commands that rely on
/var/adm/pacct, /usr/adm/wtmp, and /etc/utmp to report the information to you.
Most backdoors will keep the intruder from being shown in these logs. Even if
the intruder has not installed any backdoors yet, it is trivial to remove any
detection in these logs. But they may just forget about one or two of them.
Especially if you have some additional, non-standard ones.
Suggestion: Install xinetd or tcp_wrapper that will log all connections to
your machine to see if someone is knocking on its doors. Forward syslogs to
another machine so intruder will not easily detect the logs and modify. Other
possibilities: netlog from net.tamu.edu:/pub/security.
It might be wise to monitor the intruder via some ethernet network packet
capture program to see how he is exploiting his systems before taking
corrective measures.
- Close the machine from outside access. Remove from network to stop further
access via intruder. If the intruder finds out that the administrator is unto
him, he may try to hide his tracks by rm -rf /.
- Check the binaries with the originals. Especially check the following
binaries because they are commonly replaced backdoors for regaining access:
- /bin/login
- all the /usr/etc/in.* files (ie. in.telnetd)
- and /lib/libc.so.* (on Suns).
- anything called from inetd
Other commonly replaced backdoor binaries:
- netstat - allows hiding connections
- ps - allows hiding processes (ie Crack)
- ls - allows hiding directories
- ifconfig - hides the fact that promiscuity mode is on the ethernet
- sum - fools the checksum for binaries, not necessarily replaced anymore
because its possible to change the checksum of the binaries to the correct
value without modifying sum. *EMPHASIZE* Do NOT Rely on sum.
Use 'ls -lac' to find the real modification time of files. Check /etc/wtmp
(if you still have one) for any system time adjustments. Check the files with
the distribution media (CD or tape) or calculate MD5 checksums and compare
them with the originals kept offline (you did calculate them sometime ago,
didn't you?) Suggestion: cmp the files with copies that are known to be good.
Another popular backdoor is suid'ing a common command (ie. /bin/time) to
allow root access with regular accounts.
To find all suid programs you may use:
find / -type f -perm -4000 -ls
To be thorough you may
need to re-load the entire OS to make sure there are no backdoors. Tripwire
helps prevent modifying binaries and system files (ie. inetd.conf) on the
system, without the administrator knowing.
- Implement some password scheme for your users to verify that they change
their passwords often. Install anlpasswd, npasswd, or passwd+ in place of
passwd (or yppasswd) so that your users are forced to set reasonable
passwords. Then run Crack, which is available on ftp://ftp.cert.org/pub/tools/crackto
make sure that your users aren't bypassing the password check. Crack ensures
that users are picking difficult passwords. With the network, clear text
passwords are a problem. Other possible choices: smart hubs (stops ethernet
network packet capture of the whole LAN) and one-time password technology.
- Check all the users' .rhosts and .forward files to make sure none of them
are weird or out of the ordinary. If .rhosts file contains '+ +', the account
can be accessed anywhere by anyone without a password. COPS has a scripts for
checking .rhosts.
find / -name .rhosts -ls -o -name .forward -ls
Look
also for all the files created/modified in the time you are suspecting the
break-in has taken place, eg:
find / -ctime -2 -ctime +1 -ls
To find all the files
modified not less than one day ago, but not more than 2.
All .login, .logout, .profile, .cshrc files are also worth looking at (at
least for the modification date/time). Make sure there are no '.rhosts' for
the locked or special accounts (like 'news', 'sundiag', 'sync', etc.) The
shell for such accounts should be something like '/bin/false' anyway (and not
'/bin/sh') to make them more secure. Also search for directories that have
like ". ", ".. " as names. They are usually found in /tmp , /var/tmp,
/usr/spool/* and any other publicly writeable directory.
- Check to make sure your NFS exports are not world writable to everyone.
NFSwatch available on harbor.ecn.purdue.edu:/pub/davy
, a program by David Curry, will log any NFS transactions that are taking
place. Try 'showmount -e' to see whether system agrees with your opinion of
what are you exporting and where. There are bugs in some nfsd implementations
which ignore the access lists when they exceed some limit (256 bytes). Check
also what are you IMPORTING!!! Use 'nosuid' flag whenever possible. You do not
want to be cracked by a sysadm from another host (or a cracker there) running
suid programs mounted via NFS, do you?
- Make sure you have implemented the newest sendmail daemon. Old sendmail
daemons allowed remote execution of commands on any Unix machine. See the
computer-security/security-patch FAQ.
- Try to install all the security patches available from the vendor on your
machine. See the computer-security/security-patch FAQ.
- Here is a check list of common ways that a machine is vulnerable:
- Do an rpcinfo -p on your machine to make sure it is not running any
processes that are not needed. (ie. rexd).
- Check for '+' in /etc/hosts.equiv.
- Check whether tftp is disabled on your system. If not - disable it, or
at least use '-s' flag to chroot it to some safe area, if you really can't
live without it (it is mostly used for booting up Xterminals, but sometimes
can be avoided by NFS-mounting appropriate disks). Under no circumstances
you should run it as root. Change the line describing it in /etc/inetd.conf
to something like:
tftp dgram udp wait nobody /usr/etc/in.tftpd in.tftpd -s
/tftpboot
or better yet, use tcpd wrapper program to protect it
from addresses which should not get access to tftp and log all other
connections:
tftp dgram udp wait nobody /usr/etc/tcpd in.tftpd -s /tftpboot
and edit appropriately /etc/hosts.allow to restrict access to
in.tftpd to only those addresses that really need it.
- Check crontabs and at-jobs. Make sure there are no delayed bombs which
will explode after you think you have got rid of all the nasty things left
by a intruder.
- Check /etc/rc.boot /etc/rc.local (SYSV: /etc/rc?.d/* ) and other files
cruicial for the system startup. (The best would be if you could compare
them with the copies kept off-line). Check all other files containing system
configuration (sendmail.cf, sendmail.fc, hosts.allow, at.allow, at.deny,
cron.allow, hosts, hosts.lpd, etc.) In 'aliases' look for aliases expanding
to some unusual programs (uudecode is one but example).
- Check your inetd.conf and /etc/services files to find if there are no
additional services set up by an intruder.
- Copy all the log files you still have (pacct, wtmp, lastlog, sulog,
syslog, authlog, any additional logs you have set up earlier) to some safe
place (offline) so you may examine them later. Otherwise, do not be
surprised if they disappear the next day when the cracker realises he forgot
to remove one of them. Use your own imagination to find what other traces he
could have left in your system (What about /tmp/* files? Check them BEFORE
you reboot).
- Make backup copy of /etc/passwd (best offline) then change all root
passwords (after verifying that 'su' and 'passwd' are not the trojan
versions left by an intruder). It may sound like a horrible thing to do
(especially if you have something like 2000 users) but *do* lock them all by
putting '*' in the password field. If the intruder has a copy of your
passwords file he may possibly sooner or later guess all the passwords
contained there (It is all the matter of proper dictionaries). In fact he
could have inserted few passwords that he only knows for some users who for
example have not logged in for a long time.
On the NIS servers check not only the real /etc/passwd /etc/groups etc
files but also those used for building NIS maps (if they are different).
- Check if your anonymous ftp (and other services) are configured properly
(if you have any of course) See the computer-security/anonymous-ftp FAQ.
- If you want to make your life easier next time (or if you still cannot
get rid of an intruder) consider installing 'ident' daemon. Together with
tcpd on a set of hosts it can be used to find what accounts the intruder is
using.
- Make sure the only 'secure' terminal is console (if at all). This way
you prevent root logins just from the net. Maybe it is not a big deal as if
somebody knows the root password he may already know other peoples'
passwords too, but maybe not?
- Check hosts.equiv, .rhosts, and hosts.lpd for having # as comments
within those files. If an intruder changes his hostname to #, it will be
considered a trusted host and allow him to access your machines.
- And remember... There are so many ways that somebody could have modified
your system, that you really have to have your eyes and ears wide open for a
loooooong long time. Above, are the pointers just to the most obvious things
to check.
- Mail all the sites that you were able to find out that the intruder was
going through and warn them. Also, CC: cert@cert.org. Check all the sites in
your near-by, ie. in your domain/institution/whatever. It's usually trivial
for a hacker to get to another system by a simple 'rlogin' if the two systems
have a common subset of users (and using .rhosts to make the access easier).
- A preventive from stopping many intruders from even trying your network is
to install a firewall.
Side-effects: Firewalls may be expensive; filtering may slow down the
network. Consider blocking nfs (port 2049/udp) and portmap(111/udp) on your
router. The authentication and access controls of these protocols is often
minimal. Suggestion: Block all udp ports except DNS and NTP ports. Kill all
source routing packets. Kill all ip-forwarding packets.