1185 Peer-to-Peer Methods for Operating System Security Zoltán Czirkos Budapest University of Technology and Economics, Hungary Gábor Hosszú Budapest University of Technology and Economics, Hungary P IntroductIon The importance of the network security problems comes into prominence with the growth of the Internet. This article presents a special approach to the intrusion de- tection (ID) problem, which relies on the collaboration of the protection programs running on different hosts. Computers connected to networks are to be protected by various means (Kemmerer & Vigna, 2002). The collaboration of the elements of the proposed intrusion detection system uses the so-called peer-to-peer (P2P) communication model. The article first presents the us- age of the P2P paradigm for improving the protection of the operating systems (Bauer, 2005). In the following sections the most important in- trusion types and the developed intrusion detection methods are introduced. Intrusion attempts, based on their purpose, can be of different methods, but these techniques share things in common, for example, scan- ning networks ports or subnetworks for services, and making several attempts in a short time. This property can be used to detect these attempts and to prepare for protection. An attacker may be looking for sensitive data as well as resources. One common scenario for an attack is scanning many "neighboring" hosts, a network address range in a local area network for some security flaw or bad configuration. This is known as a portscan. One of the most important properties of these in- trusion attempts is that many hosts are usually under attack by a single attacker. In particular, this fact can also be used to create defense. If a host, whose protec- tion was strong enough to defend its own operating system, could send an alert to the other hosts, those could prepare themselves for the attack in advance, and could improve their protection, for example, by installing the necessary software. The novel application of P2P theory is easy to use; the nodes organize the P2P overlay automatically, and do not need any user interaction. The developed system is named Komondor, which is a very reliable Hungarian guard dog. The article demonstrates the effectiveness of the novel system and shows the roadmap for further de- velopment, too. tHe peer-to-peer coMMunIcatIon In order to demonstrate the P2P overlay network used in the proposed intrusion detection method the theory and the latest results of P2P networks are reviewed in this section. The P2P communication model is old in computing, its first example was the Usenet system. The Usenet has no central control. It is similar to a mailing list to some extent. It is mainly used by software developers and beginners to share their knowledge. This system relies on newsgroup servers, which are connected virtually to each other and exchange messages at spe- cific time intervals. The difference between the Usenet and the current P2P systems is that the Usenet applies dedicated servers in each organization, whereas the current P2P applications use the hosts as servers to share their resources. Theory of P2P and grid networks has gone through a great development since the most recent years. Both of them consist of peer nodes; however, usually registered and reliable nodes connect to a grid, while P2P networks can tolerate unreliability of nodes and quick change of their numbers (Uppuluri et al., 2005). The important parts of an application implementing a P2P overlay network can be seen on Figure 1 (Hosszú, 2005). In Figure 1, the layer "P2P Substrate" is responsible for the creation and the maintenance for the overlay network, while the task of the "P2P Application" layer is communication. The optional middleware layer has Copyright © 2008, IGI Global, distributing in print or electronic forms without written permission of IGI Global is prohibited.