信息安全工程6-4 参考资料-Building a Logging Infrastructure

1、1Building a Logging Infrastructure2What are we here for?Using system & application logs to improve security and reliability on your networkBuilding a logging infrastructure that works, across UNIX & Windows environments3How Do We Get There?Generate Useful DataCollect and ArchiveExtract Wisdom4How do

2、 we get there?Picking the most efficient place to startGetting the data you need into your logsUnderstanding the UNIX syslog paradigm, and how it generalizes to other systemsIntegrating Windows Event Log data into your UNIX log management system5How do we get there? cont.Managing audit data in a het

3、erogeneous computing environmentReducing log content to human-readable quantitiesInterpreting the content of log files Keeping track of whats going on in your network!6AgendaThe Log ProblemGenerating Interesting DataCentralizing your log dataParsing system logsAttack signaturesCommon mistakes7The Lo

4、g Problem8The Log Problem“Go look at those logs!”Boatloads of data, most of it superfluous9The Log ProblemOn most OSes and apps, security events form less than 1% of total volume of log data“Intelligent” security devices IDS help, but dont eliminate the need for archiving host-based logsIgnoring the

5、 problem or the data doesnt make it go away10The Log Problem cont. Conservative minimum amount of operating system log data, for UNIX/NT servers, on a mid-sized corporate network:Not including Web server access logs, mail logs, IDS data, authentication records, etc.3.8 GB per day11The Log Problem co

6、nt.Successful attacks are often not loggedLog messages vary in quality, and not designed for machine parsingWhats “interesting” is very dependent on your environment12What does it take?Automated processingNominal status data usage patterns, capacity planning, etc off-line, batch processing okayCriti

7、cal event data security issues, hardware failures must be handled real-time or close to real-time13What does it take?The common item to look for when reviewing log files is anything that appears out of the ordinary.CERT Coordination Center Intrusion Detection Checklist14Generating Interesting Data15

8、Just starting out?What do you need to know? Start small. Pick one or two apps or types of devices.What kinds of events indicate security problems, performance issues or administrative changes?Are your favorite events recorded by the default logging configuration on your device?16Always watch forHard

9、ware failuresResource exhaustionReboots/restartsPatches or changes to system code or firmware or app software (upgrades or downgrades)Failed logins, esp to admin accounts17Panic AttackMar 15 23:22:45 enigma unix: paniccpu2/thread=2a1001bdd60: Mar 15 23:22:54 enigma unix: dumping to /dev/dsk/c0t2d0s2

10、, offset 1810694144Mar 15 23:26:08 enigma savecore: reboot after panic: zero18Patching Windows19UNIX Login AttemptsSep 12 10:17:11 kuspy PAM_pwdb17529: authentication failure; (uid=0) - tbird for ssh serviceSep 12 10:17:12 kuspy sshd17529: log: Password authentication for tbird accepted.20Failed Log

11、on to Win2k DomainEvntSLog:6388: AUF Wed Oct 10 10:57:15 2001: OSMOSIS/Security (675) - Pre-authentication failed: User Name: Administrator User ID: %S-1-5-21-776561741-2052111302-1417001333-500 Service Name: krbtgt/LAB Pre-Authentication Type: 0 x2 Failure Code: 0 x18 Client Address: EvntSLog:6389:

12、 AUF Wed Oct 10 10:57:15 2001: OSMOSIS/Security (529) - Logon Failure: Reason: Unknown user name or bad password User Name: Administrator Domain: LAB Logon Type: 2 Logon Process: User32 Authentication Package: Negotiate Workstation Name: OSMOSIS21UNIX System BootJul 8 01:46:52 evileye unix: SunOS Re

13、lease 5.7 Version Generic_106541-04UNIX(R) System V Release 4.022Windows System Reboot23Windows System Reboot cont.24Cisco IOS restart*Mar 1 00:00:24.716 UTC: %SYS-5-RESTART: System restarted Cisco Internetwork Operating System Software IOS (tm) C3500XL Software (C3500XL-C3H2S-M), Version 12.0(5.4)W

14、C(1), MAINTENANCE INTERIM SOFTWARE Copyright (c) 1986-2001 by cisco Systems, Inc. Compiled Tue 10-Jul-01 12:32 by devgoyal 25Always watch for:Creation of new accounts, esp those that “look like” system accounts, or have admin privilegesSignatures of known attacksthose that crash serversthose that do

15、nt crash serversobviously system specific26Where do I start?What systems to start with?Most vulnerable serversWeb servers (public, intranet, extranet)publicly-visible mail serversadministrators workstations (operating systems, applications, network equipment)27Where do I start? cont.Critical network

16、 infrastructure systemsDomain Name servers (or WINS, or whatever)Windows Domain Controllers, NetWare Directory ServicesRouters & switchesBackup servers, network attached storageInternal mail servers28Where do I start? cont.Perimeter devicesintrusion detection systems (theoretically highest bang-for-

17、buck security info)firewalls (often the first machines to detect probes and scans; access control points)remote access servers (account harvesting, brute force attacks)29Where do I start? cont.Any systems that store proprietary corporate datadatabase serversfile serverscode repositoriesdata warehous

18、es30Monitoring RoutersUser entering enable modeAccess control list changesEnable/disable/reconfigure interfacesFirmware downgraded/upgraded/patchedConditions that produce Traceback errorsrsh, rcp connection attempts31Monitoring FirewallsHost OS messages as applicableConfiguration changesAdds/deletes

19、/changes of admin accountsAdministrative traffic from “unexpected” locations (like the Internet)Connection logs (start/stop/amt of data)32Monitoring Database ServersInteractive DB access rather than scheduled jobs or automated processingAccess control changes (DBA granting themselves or other DBAs h

20、igher level of access to system)DB account access over networkAutomated reporting of network component versions33Monitoring Database Servers cont.Changes to scripts on DB serversPresence (?) and use of non-interactive DB accounts34File system fullset /kernel: pid 801 (mysqld), uid 88 on /var: file s

21、ystem full 35Monitoring Web ServersHost OS messagesMalicious signatures in access logs (artificial ignorance/content inspection)New virtual hosts addedNew listening ports or virtual IPs addedUnusual increase in inbound or outbound traffic (Nimda, anyone?)36Monitoring Web Servers cont.New scriptsNew

22、modulesNew contentParent or child processes dying with unexpected errorsWeb server action resulting from client request (i.e. how did that URL map to file system?)37Improving the Quality of Logs38Improving the quality of logsComplexity of configuration and invisibility of most attacks (especially th

23、e successful ones) make monitoring hardA good alarm improves the chances that youll see evil quickly, without overwhelming you with false positives39Why IDS isnt EnoughJan 2 16:19:23 yyy.yyy.yyy.yyy snort1260: RPC Info Query: :963 - xxx.xxx.xxx.xxx:111 Jan 2 16:19:31 yyy.yyy.yyy.yyy snort1260: spp_p

24、ortscan: portscan status from : 2 connections across 1 hosts: TCP(2), UDP(0) 40Buffer OverflowsJan 02 16:19:45 xxx.xxx.xxx.xxx rpc.statd351: gethostbyname error for XXYYZZbffff750 804971090909090687465676274736f6d616e797265206520726f7220726f66 bffff718 bffff719 bffff71a bffff71b!41Buffer Overflown?J

25、an 02 16:20:25 xxx.xxx.xxx.xxx adduser12152: new user: name=cgi, uid=0, gid=0, home=/home/cgi, shell=/bin/bash Jan 02 16:22:02 xxx.xxx.xxx.xxx PAM_pwdb12154: password for (cgi/0) changed by (null)/0)42Your Network is Talking43Improving quality of OS logsWhat conditions or “state changes” indicate ma

26、licious activity, component failure, or significant admin activity?Do default logging mechanisms detect and record them?If not, can we make them easier to detect?44Improving quality of OS logs cont.What kinds of events do we want to record?User logins and logouts, at least for administratorsChanges

27、to administrative accountspassword change on root/admin accountaddition of new user with root or admin privileges45Improving quality of OS logs cont.Application starts/restarts/shutdownsconfiguration changessecurity context (does it run as root or some other user?)network ports in usesystem files in

28、 use46Improving quality of OS logs cont.System boot/reboot/shutdownwho did it (if appropriate)hardware/software/admin changesResource issuesNetwork configuration changesIP address, MAC addressAccess Control Lists47Improving quality of OS logs cont.Invalid data input to applicationwhat sort of invali

29、d: data not present, too much data, improper formatresult: did app crash, spawn root shell, recover gracefullyInappropriate privilege transitions in kernel48RememberYou cant extract interesting information from your logs if its not there in the first place49Collecting and Archiving50Collecting.So no

30、w you have all of these systems spewing out tons of wonderfully useful information.Where do you put it?51Collecting and ArchivingMaking sure your logs are going somewhereMaking sure theyre being received somewhereMaking sure they dont disappear52Building a Central Logging InfrastructureCreate good d

31、ataBe sure you can detect the events you want to seeCollect good dataBuild a loghost, forward device/app logsExtract wisdom from the good dataReal-time monitoring for critical eventsBatch processing for trends, planning53Centralizing Your LogsWhy?easier to archiveeasier to correlatelog preservation

32、if host is attackedHomogeneous or mixed?Homogeneous: lucky youBuilt in mechanismsMixed54Centralizing Your Logs cont.Mixed environment:syslog may not be a good choicesecurityreliabilitysyslog may be the only choicemost supported logging mechanismSo its clearly the best choice!55syslog & Its Relatives

33、56syslogdConsolidated audit mechanism for UNIX kernel and application messagesGives application and OS developers a consistent interface for reporting significant eventsAllows local or remote storage of messages57syslogd cont./etc/syslog.conf controls how much data is recorded, and what becomes of i

34、tsyslog.conf format:selector actionselectors indicate whats sending the message, and what criticality the message has 58syslogd cont.facility the application or system component that generates a log messageuser default facility applied if nothing else is specified when message is writtenkern message

35、s generated by system processeslocal0local7 facilities available for customized processing59syslogd cont.level the severity of a message on the computer generating it, i.e. emerg system is or will be unusable if situation is not resolved (most severe)alert immediate action requirednotice a significa

36、nt but typically normal event that may merit investigationAssigned by the developer who implemented the logging60syslogd cont.action whats done with a message once its received from a facilityactions usually represent destinations message is written to a local file, a syslog daemon on another system

37、, the system console, or a user console61syslogd Historical OdditiesMany syslogds require as delimiter, not whitespace, & die gory, unpleasant, hard-to-detect deaths if s are not presentFixed in SDSC-syslog, syslog-ng, sysklogd, some OS implementations (FreeBSD)62Audit Caveatssyslog only records wha

38、t youve told it to recordVast majority of events on a system are not recorded events must generate logs to show up in log monitoringFailed attacks often leave tracks; successful attacks are often only recorded indirectly63Audit Caveats cont.Running automated attack tools (nessus, CyberCop Scanner) a

39、gainst base operating systems 15% of all probes logged by OS or application mechanisms, but at least record genuine system activityIDS, other network alarms really help to identify when further examination is warranted64syslogd IssuesNo default limitations on data sources (users or processes), so al

40、l log data is inherently unreliableNothing to prevent forged data from being inserted into data streamLimited number of actions possible on receipt of a particular message65syslogd ReplacementsImproved ability to filter and redirect inbound log messagesIntegrity checks on locally-stored logfilesStor

41、e more information about log data and eventsFix that whole problemRetain compatibility with classic syslog66syslogd Replacements cont.syslog-ng: most popular replacement; allows forwarding over TCP; remembers forwarding addresses; more granular message filteringmodular syslog: a syslog replacement t

42、hat includes data integrity checks, easy database integration, and output redirection using regular expressions67syslog the Protocolsyslog messages are sent to central loghost via syslog protocol (UDP/514)Relay architecture supported, but eliminates data from message originatorNo validation of messa

43、ge (headers or content)Data sent in cleartext: can be sniffed, can be modified in transit68Secure Protocol Initiativessyslog-reliable:TCP-based for reliable message deliveryAdd authentication and encryption to protect audit datasyslog-sign:authentication of message senderreplay protectionmessage int

44、egrity and delivery checks69Real-World Secure TransmissionSDSC-syslog implements syslog-sign and syslog-reliablensyslog TCP over SSLTunnelling over SSH or SSLClient: netcat -l -u -p syslog | netcat localhost 9999 loghost: netcat -l -p 9999 | netcat localhost -u syslogSerial cables70syslog OutputMess

45、age format is invented by developer whos creating logging capabilityNo standard message formats, but usually something like date time host/IP service message71Recording facility & levelMost UNIX syslogs dont include facility & level in messages, so hard to determine appropriate filters without patte

46、rn matchingIf you configure syslog.conf to send all emerg messages to logged in users, how do you know youll get what you expect?72Recording facility & level cont.Solaris 7 and later enables message tagging, controlled in /kernel/drv/log.conf73Recording facility & level cont.Enabling message tagging

47、 adds fields to syslog messagesID msgid facility.priority msgid = hash of message textAlso lists specific kernel module for facility, rather than kern74Recording facility & level cont.Without tagging:Oct 1 14:07:24 mars unix: alloc: /: file system full With tagging:Oct 1 14:07:24 mars ufs: ID 845546

48、 kern.notice alloc: /: file system full 75Recording facility & level cont.Using syslog-ng:destination my_file file(/var/log/messages template($DATE $FACILITY.$LEVEL $FULLHOST $MESSAGEn); ; 76Recording facility & level cont.Without tagging:Aug 30 01:20:56 bettiepage/bettiepage postfix/smtpd22956: dis

49、connect from 2100 06255078.8With tagging:Oct 27 11:41:22 bettiepage/bettiepage postfix/smtpd18020: connect from smtp8.Stanford.EDU577syslog Output cont.UNIX applications that use syslog:amddateftpdgatedinetdsendmailloginrloginnamedntpdpasswdsudotcpdvixie-cronlpdnnrpd78loggerUNIX command line utility

50、 writes arbitrary messages to syslog hathor:/var/log# logger this space intentionally left blankhathor:/var/log# Oct 27 13:05:41 localhathor tbird65: ID 702911 user.notice this space intentionally left blank79Windows Event LogWindows analog of syslogNo integrated capability for remote loggingBinary

51、file no grep!System default auditing is disabled80Windows Event Log cont.System Log: Startup and shutdown messages, system component data, critical servicesSecurity Log: Windows auditing system data only, including user & host auth, share access, printing, otherApplication Log: Nearly everything els

52、e81Windows Event Log cont.Any process can write to Application and System Event Logs “should” register message libraryOnly LSA and Event Log Service itself can write to Security Event LogSecurity log is more reliable forensic information than off the shelf syslog82Windows Application LogApplication

53、Log messages parsed via message dictionaryShould be provided by application developerFrequently isnt83Windows Application Log cont.84Windows Event Log cont.85Windows Event Log cont.86Windows Event Log cont.87Windows Event Log cont.logger equivalent for Windows: Win2000 Resource Kit tool logeventWrit

54、es an Event ID set by an administrator to the Application LogMessage severity is always InformationalAdiscons MonitorWare agent will forward data added to a Windows text based log to a syslog server88Windows Event Log cont.Another logger equivalent for Windows: Kiwis Syslog Message GeneratorSends ma

55、nually-generated syslog messages from a Windows command line or GUI to a syslog serverDoes not read data from Event Log, but useful for testing89WinNT Audit Configuration90NT vs. 2000 Audit Categories WinNT Win2kUser/Group Mgmt.Audit Account ManagementLogon and LogoffAudit logon eventsFile and Objec

56、t AccessAudit object accessSecurity Policy ChangesAudit policy changesUse of User RightsAudit privilege useAudit process trackingAudit process trackingRestart, Shutdown,Audit system events System+ Audit account logon events+ Audit directory service access91Win2k Event Log DetailsLocal policy setting

57、s applied first, then domain policy settings, then active directory settingsMay make local audit setting different from effective audit setting92Win2k Audit Configuration93syslog & Relatives SummaryUNIX: syslogd & syslog protocoluncontrolled, unverified datastreammost widely implemented logging stru

58、ctureWindows: Event Logreliable security informationproprietary formatBoth require OS configuration, possibly application configuration & tuning94Building a CentralLogging Infrastructure95Security SurveillancePerimeter devices detect port scans & vulnerability probes FW, router Improve network attac

59、k detection NIDSImprove host-based attack detection HIDS96Open Source Surveillance ToolsMonitoring system calls systrace, St. JudeFW ipfilter, TCP WrappersNIDS SnortHIDS logdaemonFile system integrity checkers tripwire, Samhain97portsentry LogNov 19 00:12:53 hosty portsentry17645: ID 702911 daemon.n

60、otice attackalert: Connect from host: .br/ 34 to TCP port: 8098Centralized LoggingBuilding a loghostPopular architecturesHave a good timeGetting data to the loghostConfiguring clients (OS & application)Transport mechanismsArchiving99Loghost DecisionsWhich operating system?Most experience = easiest t