A population-based incremental learning approach to network hardening

Abstract

Enterprise networks constantly face new security challenges. Obtaining complete network security is almost impossible, especially when usability requirements are taken into account. Previous research has provided ways to identify multi-stage attacks caused by network vulnerabilities and misconfigurations, but few have addressed ways to circumvent those multi-stage attacks, especially when usability requirements are taken into account. The latter problem is reckoned as Network Hardening problem [10] and is known to be an NP hard combinatorial problem. In this paper, we map the network hardening problem to a constrained optimization problem and resort to the theory of Population-Based Incremental Learning (PBIL) in order to solve it. We devise two approaches based on the PBIL, namely the Acceptance-Rejection approach, and the Penalty-based approach. Our aim is to tighten the security of the network by minimizing the number of privileges that an attacker can gain over network under some usability constraints measured in terms of the number of configurations in a network that can be activated or cannot be deactivated. The Acceptance-Rejection approach disqualifies configurations that violate the usability constraint while the Penalty-based approach relaxes the latter constraint by attempting to find a compromise between security and usability of the configuration. While the Acceptance-Rejection approach can be seen as a simple alternative to the state of the art MinCostSAT solution adopted in [10], the Penalty-based approach is, to the best of our knowledge, the first solution in the literature that tries to find such compromise. Experimental results show that the devised approaches are computationally efficient, scalable and reliable.