HOWTO check step-by-step a Layer 4 LVS-NAT setup for problems


Q.1 Can the real server ping client?

	rs# ping -n client

A.1 Yes => good
A.2 No => bad

	Some settings for the director:

	Linux 2.2/2.4:
	ipchains -A forward -s RIP -j MASQ

	Linux 2.4:
	iptables -t nat -A POSTROUTING -s RIP -j MASQUERADE

Q.2 Traceroute to client goes through LVS box and reaches the client?

	traceroute -n -s RIP CLIENT_IP

A.1 Yes => good
A.2 No => bad

	same ipchains command as in Q.1

	For client and server on same physical media use these
	in the director:

	echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
	echo 0 > /proc/sys/net/ipv4/conf/<DEV>/send_redirects


Q.3 Is the traffic forwarded from the LVS box, in both directions?

	For all interfaces on director:
	tcpdump -ln host CLIENT_IP

	The right sequence, i.e. the IP addresses and ports on each
	step (the reversed for the in->out direction are not shown):

	CLIENT
	   | CIP:CPORT -> VIP:VPORT
	   |		||
	   |		\/
 out	   | CIP:CPORT -> VIP:VPORT
 ||	LVS box
 \/	   | CIP:CPORT -> RIP:RPORT
 in	   |		||
	   |		\/
	   | CIP:CPORT -> RIP:RPORT
	   +
	REAL SERVER

A.1 Yes, in both directions => good (for Layer 4, probably not for L7)
A.2 The packets from the real server are dropped => bad:

	- rp_filter protection on the incoming interface, probably
	hit from local client
	- firewall rules drop the replies

A.3 The packets from the real servers leave the director unchanged

	- missing -j MASQ ipchains rule in the LVS box

	For client and server on same physical media:

	The packets simply does not reach the director. The real
	server is ICMP redirected to the client. In director:

	echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
	echo 0 > /proc/sys/net/ipv4/conf/<DEV>/send_redirects

A.4 All packets from the client are dropped

	- the requests are received on wrong interface with rp_filter
	protection
	- firewall rules drop the requests

A.5 The client connections are refused or are served from service
in the LVS box

	- client and LVS are on same host => not valid
	- the packets are not marked from the firewall and don't hit
	firewall mark based virtual service

Q.4 Is the traffic replied from the real server?

	For the outgoing interface on real server:

	tcpdump -ln host CLIENT_IP

A.1 Yes, SYN+ACK => good
A.2 TCP RST => bad, No listening real service
A.3 ICMP message => bad, Blocked from Firewall/No listening service
A.4 The same request packet leaves the real server => missing accept
	rules or RIP is not defined
A.5 No reply => real server problem:

	- the rp_filter protection drops the packets
	- the firewall drops the request packets
	- the firewall drops the replies

A.6 Replies goes through another device or don't go to the LVS
box =? bad

	- the route to the client is direct and so don't pass the LVS
	box, for example:

		- client on the LAN
		- client and real server on same host

	- wrong route to the LVS box is used => use another

	Check the route:

	rs# ip route get CLIENT_IP from RIP


The result: start the following tests

rs# tcpdump -ln host CIP
rs# traceroute -n -s RIP CIP
lvs# tcpdump -ln host CIP
client# tcpdump -ln host CIP


For more deep problems use tcpdump -len, i.e. sometimes the link layer
addresses help a bit.


For FTP:

	LVS-NAT in Linux 2.2 requires:

	- modprobe ip_masq_ftp (before 2.2.19)
	- modprobe ip_masq_ftp in_ports=21 (2.2.19+)

	LVS-NAT in Linux 2.4 requires:

	- ip_vs_ftp

	LVS-DR/TUN require persistent flag


	FTP reports with debug mode enabled are useful:

	# ftp
	ftp> debug
	ftp> open my.virtual.ftp.service
	ftp> ...
	ftp> dir
	ftp> passive
	ftp> dir

	There are reports that sometimes the status strings reported
	from the FTP real servers are not matched with the string
	constants encoded in the kernel FTP support. For example,
	Linux 2.2.19 matches
	"227 Entering Passive Mode (xxx,xxx,xxx,xxx,ppp,ppp)"


Julian Anastasov

	Document about supporting IPSec protocols with the
			Linux Virtual Server
			(Development HowTo)



	CONTENTS:

	1. Introduction
	2. Goals
	3. Current practice
	4. Implementation
	4.1 Direct Routing/Tunnel Methods
	4.2 NAT Method
	4.3 Denial of Service prevention



	1. Introduction

	Before going into the details, the reader needs to understand
all terms and internals for the following subjects: [1] Linux
Virtual Server (LVS), its forwarding methods, the Internet Protocol
Security (IPSec) terminology and usage, one of the Linux IPSec
implementations, [2]FreeSWAN. Look at the end of the document for
any URLs and references.



	2. Goals

	The main goals to add IPSec support to LVS are:

- build a cluster of VPN servers for IPv4

- use more CPU power for encryption and decryption, fill
your bandwidth with many real servers, high availability,
etc

- no more, isn't it enough? Read below

	But how? In the near feature there will be methods
where the both Secure Gateways will be able to automatically
switch to secure mode without any prearrangements or at least
with a low cost. We start to develop this support with the
goal to be ready for that time and with the ability to provide
all LVS features for all widely used protocols.

	We already know that the FreeSWAN team already proposed
a way to "automatically" establishing secure connections by
using authentication with RSA keys and with the help from the
Secure DNS: the "Opportunistic" encryption [3]. This scheme allows
the secure communication to take place if possible. If not,
the talks can continue in the normal unencrypted way. We
consider this as an Initiator's problem. If the requirements
are to build only encrypted channels then this can be arranged
by using the current methods but this is not a LVS issue, we
are going to cover the case where the the encryption takes
place. The unencrypted case is already implemented in LVS.

	Note that the Director does not need any IPSec support
in the IP Stack. It is needed only if LocalNode method is
used to deliver locally the IPSec traffic.

	3. Current practice

* Terminology in our context

	- Initiator: the VPN Client

	- Responder: the director (in the client's eyes) or
	the real server in director's eyes

* VPN Masquerade (in Linux 2.0 and 2.2)

	The VPN Masquerade [5] has the most difficult goal to
share the ISAKMP channel (UDP, port 500) for all its masqueraded
initiators. This is caused from the way the ISAKMP connection
is used: it is a host-to-host channel, from port 500 to port 500.
The both ends make negotiations for other talks. The ISAKMP
masquerading takes care of the fact that two initiators can
establish ISAKMP session to same remote secure gateway which
can break the protocol.

* IPSec and NAT

	There are some difficulties in forwarding the IPSec
protocols (ISAKMP, ESP and AH) through a NAT router. In some of
the modes the protocols can provide integrity checks which
involve the addresses included in the first IP header. We
can list the following rules which apply when using NAT
over the IPSec traffic:

- NAT works for ESP tunnel mode because the addresses in the
outer IP header are not protected from the protocol

- AH does not like address translations because the addresses are
protected

- NAT for ESP transport mode can work safely only for UDP
without checksums

- The TCP/UDP checksum checks at the end host can be disabled to
allow NAT for ESP transport mode. This is caused from the fact
that both TCP and UDP include pseudo header checksums which
break on address change.

- IKE negotiations with preshared key depend on source addresses, so
FQDN or ASN1 Identity types can be used for NAT

- We don't care for the self identification, we don't care
for the ID types but may be IPv4 is not suitable for NAT
purposes. They can be FQDN or DER_ASN1_DN.


* ISAKMP and NAT

- We can't alter the source port 500 in the ISAKMP packets on
NAT

- From the ISAKMP traffic we can't understand what negotiate
the both ends but by analyzing the traffic we can know whether
the initiator is valid: we can see (in NAT mode only) the
right responder's cookie in its next packets sent from the
initiator. This helps in preventing a spoofing for the ISAKMP
requests.

- The timeouts for the ISAKMP traffic should be set
according to the used rekey time

- Because the cookies can change on rekey, we don't need to
compare them each time, we need the match only for the first
packets.

- We can expect that keep-alive packets can be used to maintain the
IKE/IPSec alive. The bad thing is that if these keep-alive packets
are sent with low IP TTL with the intention not to make big traffic
and to reach only the local NAT router, the LVS-DR method can't
detect them (it works in one direction only, i.e. for out->in only).

- Note that we can't learn something useful from the ISAKMP
traffic because it is encrypted

* NAT Traversal

	With the above scheme of two states for ISAKMP it seems
we can support the NAT Traversal as defined in [4]. The ESP
traffic is provided in ISAKMP packet after the two cookies with
all zeroes. We can stop any traffic with initiator's cookie equal
to 0x00 if the entry is still in initial state.

	There is one question: can we encapsulate from ESP to
UDP on DNAT and from UDP to ESP on SNAT and by this way to
transparently provide NAT traversal for the real servers,
invisible to the external clients (suitable only for NAT method).

	4. Implementation

	We can say that according to the forwarding methods
and the kind of the IPSec traffic, not all cases can be
supported in LVS.

	In any case, for all forwarding methods, we can use the
following rules:

- We don't need to maintain connection entries for the ESP and
the AH protocols. It is enough to create one entry according to
the used virtual service type:

	UDP/CIP:CPORT->VIP:500

	with an optional connection template for persistent
	virtual services:

	IP/CIP->FWMARK

	If this disturbs the load balancing the problem should
	be handled differently but in any case maintaining
	one ESP and one AH entry per CIP is not a big problem,
	we even don't need to create additional entry for
	each client's SPI because all traffic from one client
	is sent to one real server. Of course, if we maintain
	one connection entry for each client's SPI we will know
	exactly the number of the SAs but with the help from
	dynamic weights for the real servers we can solve the
	problem with the imbalance in the cases when only a
	ISAKMP entry is maintained per each CIP. We even don't
	need to restart expiration timers for the ESP and AH
	traffic, we need only to follow the ISAKMP traffic and
	to treat the ESP/AH traffic as related and slave.

- The ISAKMP entry needs 2 states with different timeouts:
the first state needs short timeout to prevent the attackers
from creating many ISAKMP entries without replying with correct
responder's cookies. The first state covers the cases where the
initiator sends packets with rcookie=0. The second state should
be treated as established, i.e. in this state we can forward ESP
and AH traffic to the selected real server. The timeout for this
state should be tuned from the administrator according to the desired
lifespan and the rekey interval. If keep-alives are used, this
can help to maintain the ISAKMP entry in the LVS table.

	In short:

	When a ESP/AH packet comes we should lookup for ISAKMP
	entry. If such entry already exists and is in ESTABLISHED
	state (responder's cookie is already set), we can forward
	the ESP/AH packet to the ISAKMP's real server.

	When an ISAKMP entry is not found we should return
	ICMP_PROT_UNREACH error.

	When ISAKMP traffic is received we can restart the
	expiration timer for the current entry state.

- We can even define a per-service number of entries that should
be softly handled from the expire mechanism, i.e. we can allow
up to specified number of entries to live even after their normal
idle timeout period. In some cases this can cover the cases where
the rekey interval is not incorporated correctly in the ISAKMP
connection entries.

- Note that we don't want to mark the virtual service as persistent,
the persistence is implicitly required for the way the ISAKMP
protocol is used. In such cases the load balancing can be
maintained with dynamic weights for the real servers, only if
needed. Apply persistence if you extend the virtual service with
other kind of traffic and the redirection of the client to
same real servers both for the IPSec and the other traffic is
required.

	4.1 Direct Routing/Tunnel Methods

	Nothing to add here, these methods can forward any
kind of IPSec traffic because we don't change the addresses
in the outer IP header and the IPSec terminators will treat
the packet as correctly routed.

	As for other kind of traffic, in DR and TUN method it
is difficult to deduce only from the input traffic whether
the client is valid and we are not flooded from attacker
generating pseudo requests.

	We know that there are many different ways to replace
a NAT setup with Direct Routing setup without breaking any
security rules. The experience shows that in most of the
cases it is a matter of taste whether to select NAT or DR
setup. It is even believed that the NAT method is more
secure but the real benefits for the NAT method is that
it provides more valid way to analyze the in->out traffic.
The DR method sees the traffic only in one direction.

	In the LVS world we already know different ways to
route the real server's replies back through the director,
even for DR method without reaching the anti-spoofing checks
at routing layer. The different ways can include:

- using Layer 2 switching (Linux Bridging) to forward the
real server's replies through the director without checking
them at routing

- using other solutions such as modifing the reverse path
protection (forward_shared device flag) in the director to
allow spoofed traffic from the network interfaces used to
receive the traffic from the real servers

	There is a way even the real servers not to be reachable
from the public, we don't need any public IP addresses on the
real servers to use DR method, we can even provide the needed
firewall rules according to the desired firewall policy.

	As result, there are many ways to use Direct Routing
method for the IPSec traffic instead of NAT and by this way
to support any kind of its modes and protocols: ISAKMP, ESP
and even AH.

	4.2 NAT Method

	From the above list of exceptions we can say that it
is administrator's job to decide whether it is safe to use
NAT method for the specific ESP traffic (AH can not be used in
NAT mode). The ESP traffic in tunnel mode can be safely NAT-ed
but for transport mode the admins should take care of the
broken TCP/UDP checksums of the encrypted traffic caused from the
changes in the addresses of the IP headers on NAT. The TCP/UDP
receivers should ignore the broken pseudo checksums. Note that
this is not a recommendation but a solution.

	We should mention that the ISAKMP traffic can be safely
handled in this mode because we can match the responder's cookie
in the initiator's packet and we can know whether the initiator
is an attacker generating fake ISAKMP requests or a valid
client.


	4.3 Denial of Service prevention

	We can use many ways to prevent the attackers from
flooding the director with fake ISAKMP or ESP/AH traffic.
We know for some LVS and non-LVS ways:

- QoS ingress

	We can rate limit the IPSec request packets by adding
different rates for the ISAKMP, ESP and the AH traffic. We
hope that this barrier prevents wasting memory for LVS entries
and also protects the real servers.

- QoS egress

	This kind of handling prevents flooding the real servers
with unexpected number of requests but does not protect the
LVS connection table from growing in a bad way.

- LVS defense strategies

	We know that LVS already has different strategies to
defend against attacks. These strategies prevent overloading
the director's memory with fake entries but do not protect the
real servers from the actual traffic. For this, the LVS defense
can be combined with the above solutions or even with any other
kinds of denial of service preventions. We should mention that
the LVS dropentry strategy is universal way to keep the allocated
memory for LVS connection entries in reasonable size.


[1] http://www.linuxvirtualserver.org/ - The Linux Virtual Server
[2] http://www.freeswan.org/ - The Linux IPSec implementation
[3] draft-richardson-ipsec-opportunistic.txt - Opportunistic encryption
[4] draft-ietf-ipsec-udp-encaps-01.txt - IPSec NAT Traversal
[5] http://www.impsec.org/linux/masquerade/ip_masq_vpn.html - VPN
Masquerading for Linux 2.0 and 2.2

Authors:

Started from Julian Anastasov <ja@ssi.bg>, February 17, 2002

KERNEL COMPILE 하기

1.kernel source구하기
ftp://ftp.kernel.org 또는 http://ftp.kernel.org 에서 kernel source를 구한다.

2.kernel source 설치
download 한 kernel source를 아래와 같은 방법으로 압축을 푼다.
# cd /usr/src
# rm -f linux
# mkdir linux-2.4.19
# tar xvzf linux-2.4.19.tar.gz linux-2.4.19
# ln -s linux-2.4.19 linux

3.kernel patch하기
http://www.linuxvirtualserver.org/software/index.html에서 kernel version에 맞는 patch를 구한다.
ex)linux-<커널버전>-ipvs-<patch 버전>.patch.gz

# gzip -cd <커널patch file> | patch -p0
또는

# cd /usr/src/linux
# cat ../ <커널 patch file> | patch -p1

4.make mrproper
이전에 설정한 정보를 지우거나 새로운 커널 소스를 가져다 컴파일 할 때에는 이 명령을 먼저 사용해야 합니다. make mrproper 명령은 이전 커널을 컴파일할 때 만들어진 오브젝트 파일(*.o)과 의존성 설정, 컴파일 환경 설정값, 버전 정보 등 새로 시작하는 컴파일에 영향을 주는 이전 정보들을 삭제합니다.
이전에 설정된 컴파일환경을 재사용한다면 이 과정을 건너뜁니다.
# cd /usr/src/linux
# make mrproper

5.kernel option 추가(환경설정)
--xwindow 환경시...
# cd /usr/src/linux
# make xconfig
--text mode 환경시...
# make menuconfig
커널 컴파일 옵션 셋팅
----------------------
Code maturity LeveL optionS -->
----------------------
<Y> Prompt for deveLopment and/or incomplete code/drivers

----------------------
LoadabLe module SupporT -->
----------------------
<Y> EnaLbLe LoadabLe module Support
<Y> Set version information on aLL SymboLs for moduLes
<Y> KerneL module Loader

----------------------
Networking Options -->
----------------------
<Y> Packet socket
<Y> Packet socket:mmapped IO
<Y> Netlink device dmulation
<Y> Network packet filtering (replaces ipchains)
<N> Network packet filtering debugging
<Y> Socket Filtering
<Y> Unix domain sockets
<Y> TCP/IP networking
<Y> IP:multicastring
<Y> IP:advanced router
<Y> IP:policy routing
<Y> IP:use netfilter MARK value as routing key
<Y> IP:fast network address translation
<Y> IP:equal cost multipath
<Y> IP:use TOS value as routing key
<Y> IP:verbose route monitoring
<Y> IP:large routing tables
<Y> IP:kernel level autoconfiguration
<N> IP:DHCP support
<N> IP:BOOTP support
<N> IP:RARP support
<M> IP:tunneling
<N> IP:GRE tunnels over IP
<N> IP:multicast routing
<N> IP:ARP daemon support
<N> IP:TCP Explicit Congestion Notification support
<Y> IP:TCP syncookie support
------------------------------
IP:Netfilter Configuration -->
------------------------------
<M> Connection tracking
<M> FTP protocol support
<M> IRC protocol support
<M> Userspace queueing via NETLINK
<M> IP tables support
<M> limit match support
<M> MAC address match support
<M> netfilter MARK match support
<M> Multiple port match support
<M> TOS match support
<M> AH/ESp match support
<M> LENGTH match support
<M> TTL match support
<M> tcpmss match support
<M> Connection state match support
<M> Unclean match support
<M> Owner match support
<M> Packet filtering
<M> REJECT target support
<M> MIRROR target support
<M> Full NAT
<M> MASQUERADE target support
<M> REDIRECT target support
<N> NAT of local connections
<N> Basic SNMP-ALG support
<M> Packet mangling
<M> TOS target support
<M> MARK target support
<M> LOG target support
<M> ULOG target support
<M> TCPMSS target support
<M> ARP tables support
<M> ARP packet filtering
<N> ipchains support
<N> ipfwadm support
----------------------------------
IP:Virtual Server Configuration-->
----------------------------------
<M> virtual server support
<Y> IP virtual server debugging
[12] IPVS connection hash table size (the Nth power of 2)
------- IPVS scheduler
<M> round-robin schduling
<M> weighted round-robin scheduling
<M> least-connection scheduling scheduling
<M> weighted least-connection scheduling
<M> locality-based least-connection scheduling
<M> locality-based least-connection with replication scheduling
<M> destination hashing scheduling
<M> source hashing scheduling
------- IPVS application helper
<M> FTP protocol helper
<N> The IPv6 protocoL (EXPERIMENTAL)
IPv6: Netfilter Configuration-->
<N> Kernel httpd acceleration (EXPERIMENTAL)
[N] Asynchronous Transfer Mode (ATM) (EXPERIMENTAL)
<M> 802.1Q VLAN Support (EXPERIMENTAL)
---
<N> The IPX protocol
<N> Appletalk protocol support
Appletalk devices-->
<N> DECnet Support
<N> 802.1d Ethernet Bridging
<N> CCITT X.25 Packer Layer (EXPERIMENTAL)
<N> LAPB Data Link Driver (EXPERIMENTAL)
[Y] 802.2 LLC (EXPERIMENTAL)

----------------------
Network device SupporT -->
----------------------
<Y> Network device support
ARCnet devices --->
<M> Dummy net driver support
<N> Bonding driver support
<N> ELQ (serial line load balancing) support
<N> Universal TUN/TAP device driver support
<N> Ethertap network tap (OBSOLETE)
<N> General Instruments Surfboard 1000
------------------------------
Ethernet (10 or 100Mbit) -->
------------------------------
[Y] Ethernet (10 or 100Mbit)
<N> Sun Happy Meal 10/100baseT support
<N> Sun GEM support
[M] 3COM cards
<N> 3c501 "EtherLink" support
<N> 3c503 "EtherLink II" support
<N> 3c505 "EtherLink Plus" support
<N> 3c507 "EtherLink 16" support (EXPERIMENTAL)
<N> 3c509/3c529 (MCA)/3c579 "EtherLink III" support
<N> 3c515 ISA "Fast EtherLink"
<M> 3c590/3c900 series (592/595/597) "Vortex/Boomerang" support
<N> AMD LANCE and PCnet (AT1500 and NE2100) support
[N] Western Digital/SMC cards
[N] Racal-Interlan (Micom) NI cards
<N> AT1700/1720 support (EXPERIMENTAL)
<N> DEPCA, DE10x, DE200, DE201, DE202, DE422 support
<N> HP 10/100VG PCLAN (ISA, EISA, PCI) support
[N] Other ISA cards
[Y] EISA, VLB, PCI and on board controllers
.
.
.
<Y> EtherExpressPro/100 support
.
.
.
<M> RealTek RTL-8139 PCI Fast Ethernet Adapter support

----------------------
FiLe SyStemS -->
----------------------
[N] Quota Support
<N> Kernel automounter support
<Y> Kernel automounter version 4 support (also supports v3)
[Y] Reiser fs support
[Y] Have reiserfs do extra internal checking
[Y] Stats in /proc/fs/reiserfs
.
.
.
<Y> Ext3 journalling file system support
[Y] JBD (ext3) debbuging support
.
.
.
[Y] Virtual memory file system support (former shm fs)
<Y> ISO 9660 CDROM file system support
[Y] Microsoft joliet CDROM extensions
.
.
.
[Y] /proc file system support
[N] /dev file system support (EXPERIMENTAL)
[Y] /dev/pts file system for Unix98 PTYs
.
.
.
<Y> Second extended fs support
.
.
.
<N> UFS file system support (read only)
------------------------------
Network File Systems -->
------------------------------
<Y> Coda file system support (advanced network fs)
<N> InterMezzo file system support (experimental, replicating fs)
<Y> NFS file system support
.
.
.
<Y> NFS server support
(나머지 System Dependent한 사항은 http://kldp.org의 커널 컴파일
Howto를 참고하여 옵션을 셋팅할 것.)

6.청소 와 Compile
# cd /usr/src/linux
# make dep && make clean
&& 는 앞선 명령이 성공했을 때에만 뒤이은 명령을 실행하라는 뜻입니다
make dep : 새 커널 만들기를 시작하는 명령입니다. 컴파일을 위한 의존성 관계를 설정합니다.
make clean : 이전에 수행했던 컴파일과정에서 생성된 오브젝트 파일(*.[oas]), 커널, 임시파일(.tmp*), 설정값(configure) 등을 삭제합니다.

--Kernel compile
# cd /usr/src/linux
# make bzImage
--모듈 컴파일
모듈 컴파일을 시작하기 전에 이전 버전에서 설치한 모듈을 (커널 버전이 달라지지 않았다면) 이름을 바꾸어 두는 것이 좋습니다. 커널 버전이 다르다면 잠시 그냥 두었다가, 새 커널과 모듈이 제대로 동작하는 것을 확인하고 지웁니다.
# cd /usr/src/linux
# make modules && make modules_install && make install7.Boot Loader Setting(Grub.config)
정상적으로 compile과 option setting끝이 나면 boot loader를 수정한다.

#vi /boot/grub/grub.conf

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You do not have a /boot partition. This means that
# all kernel and initrd paths are relative to /, eg.
# root (hd0,0)
# kernel /boot/vmlinuz-version ro root=/dev/hda1
# initrd /boot/initrd-version.img
#boot=/dev/hda
default=0
timeout=10
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
title Red Hat Linux (2.4.19)
root (hd0,0)
kernel /boot/vmlinuz-2.4.19 ro root=/dev/hda1 /*이 부분을 부팅지정해준다*/
initrd /boot/initrd-2.4.19.img
title Red Hat Linux (2.4.18-14)
root (hd0,0)
kernel /boot/vmlinuz-2.4.18-14 ro root=/dev/hda1
initrd /boot/initrd-2.4.18-14.img
title Windows2000
rootnoverify (hd0,1)
chainloader +1

8. NAT방식의 클러스터링 로드 벨런서 시스템 구성

INTERNET

│
┌─┴─┐ eth0 192.168.3.108
│ │ GW/NETMASK/SUBNET는 회선
│ LB │ 공급자에게서 정보를 받아 세팅한다.
│ │
└─┬─┘eth1 10.0.0.1
│ GW 192.168.0.108
│
┌─────── HUB ────────┐
│ │
┌─┴─┐ eth0 10.0.0.2 ┌─┴─┐ eth0 10.0.0.3
│ │ GW 10.0.0.1 │ │ GW 10.0.0.1
│ RS1 │ │ RS2 │
│ │ │ │
└───┘ └───┘

-NAT방식의 클러스터링 구성을 위해서는 LB에 두개의 이더넷 카드를 사용해야하면 구성은 위의 그림과 같다.
-네트웍 환경이 구성되면 각각 핑테스트를 해본다.(테스트중 LB와 HUB 라인의 이상으로 상당 시간을 소비했다.)

NIC 관련 세팅 파일
/etc/sysconf/network-scripts/ifcfg-eth0
/etc/sysconf/network-scripts/ifcfg-eth1(LB경우만)

# vi ifcfg-eth0 (for RS1)

DEVICE=eth0
ONBOOT=yes
BOOTPROTO=static
IPADDR=10.0.0.2
NETMASK=255.0.0.0
GATEWAY=10.0.0.1

9 LVS 관리툴 ipvsadm

ipvsadm 설치

#rpm -ivh ipvsadm-1.21-3.src.rpm
#cd /usr/src/redhat/SOURCES
#tar xvzf ipvsadm-1.21.tar.gz
#cd ipvsadm-1.21
#vi Makefile
(INCLUDE 항목에서 include 경로를 수정)
INCLUDE = -l/usr/src/linux-2.4/include -l.. -l.
#cd /usr/src/redhat/SOURCES
#rm ipvsadm-1.21.tar.gz -f
#tar cvzf ipvsadm-1.21.tar.gz ipvsadm-1.21/
#rm -rf ipvsadm-1.21/
#cd /usr/src/redhat/SPECS
#rpmbuild -ba ipvsadm.spec
#cd /usr/src/redhat/RPMS/i386/
#rpm -ivh ipvsadm-1.21-3.i386.rpm

ipvsadm이 잘 설치되었는지 확인하기 위해
# ipvsadm

IP Virtual Server version 1.0.7(size=4096)
Prot Address:Port Scheduler Flags
->RemoteAddress:Port Forward Weight ActiveConn InActConn
LB에서..아래와 같이 실행시킨다.

#ipvsadm -A -t 192.168.3.108:80 -s wrr

#ipvsadm -a -t 192.168.3.108:80 -r 10.0.0.2:80 -m

#ipvsadm -a -t 192.168.3.108:80 -r 10.0.0.3:80 -m
-A : add service
-t : tcp_service_address
-s : scheduling을 사용하겠음
{
wrr : weight round robbing
wlc : weight least connection sc.
rr : round robbing
lc : least connection sc.
}
-a : add server
-r : real service
{
-m: NAT
-g : direct routing
-I : tunneling
}