有没有人配过 Apache2+Tomcat4 做 balance?

iceant
这是我的 workers2.properties 文件，但是很奇怪，会出现后面那些错误~~ 难道是我用的 jk2_module.dll 不支持这些属性吗？还是我配错了？请指正

# only at beginnin. In production uncomment it out
[logger.apache2]
level=DEBUG

[shm]
file=c:/jk2.shm
size=1048576

# Example socket channel, override port and host.
[channel.socket]
port=11009
host=127.0.0.1
lbfactor=150
tomcatId=tomcat1

[channel.socket]
port=12009
host=127.0.0.1
lbfactor=100
tomcatId=tomcat2

#define a load balancer work
[lb]
stickySession=0
attempts=5

# Uri mapping
[uri:/examples/*]

[uri:/*.jsp]


=================================================================================
错误
=================================================================================
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting channel.socket:2 lbfactor 150
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting channel.socket:2 lbfactor 100
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting channel.socket:5 lbfactor 150
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting channel.socket:5 lbfactor 100
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting lb:8 stickySession 0
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting lb:8 attempts 5
[Sat Nov 15 20:18:14 2003] [notice] Parent: Created child process 2864
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting channel.socket:2 lbfactor 150
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting channel.socket:2 lbfactor 100
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting channel.socket:5 lbfactor 150
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting channel.socket:5 lbfactor 100
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting lb:8 stickySession 0
[Sat Nov 15 20:18:14 2003] [notice] config.setAttribute() Error setting lb:8 attempts 5
[Sat Nov 15 20:18:14 2003] [notice] Child 2864: Child process is running
[Sat Nov 15 20:18:14 2003] [notice] jk2_init() Found child 2864 in scoreboard slot 0
[Sat Nov 15 20:18:14 2003] [notice] workerEnv.init() ok C:/Program Files/Apache Group/Apache2/conf/workers2.properties
[Sat Nov 15 20:18:14 2003] [error] mod_jk child init 1 0
[Sat Nov 15 20:18:15 2003] [notice] Child 2864: Acquired the start mutex.
[Sat Nov 15 20:18:15 2003] [notice] Child 2864: Starting 250 worker threads.

iceant 2003-11-15 21:16
以下是能正常工作的配置:

可以利用 http://host:port/jkstatus 看到相应的报告.
奇怪得很，我在 lb:lb0 里设置了 stickySession=0 但是从 log 里看(error.log)，这个 attribute 设置时出错了，是不是 apache 的文档并未反应开发的进度?

# only at beginnin. In production uncomment it out
[logger.apache2]
level=DEBUG

[shm]
file=c:/jk2.shm
size=1048576

# Example socket channel, override port and host.
[channel.socket:localhost:12009]
tomcatId=tomcat2
group=lb:lb0

[channel.socket:localhost:11009]
tomcatId=tomcat1
group=lb:lb0

#
# defined workers
#
[status:me]

[lb:lb0]
#stickySession=0

# Uri mapping
[uri:/examples/*]
group=lb:lb0

[uri:/*.jsp]
group=lb:lb0

[uri:/jkstatus]
group=status:me

iceant 2003-11-15 22:49
Apache2.* + Tomcat 4.* + Load Balance

Updated: 2003-11-15
Edition: 1.0

================================================================================
Introduction

The Tomcat servlet container will crashed in some strange situation recently in my office.
I must restart the server to supply the service as soon as possible. So, I got no time to
find the bug. I want to build a load balancing system to supply the service of 7*24.
That is mean, when one instance of tomcat server crashed another instance of tomcat server
should take over the responsibility to service the client's requests.

I don't want to reconstruct all the applications after setup the load balance sytem, so, I
choose the sticky session solution of load balance. That's mean when a request of one user
is responsed by tomcat1 server, then the following request should be responsed by the same
tomcat server, tomcat1. Anyway, if the tomcat1 server has crashed by some reason, the followed
request will be delegated to the second server, tomcat2.

I use linux box as server in office. At home I used windows as work-station. The following
steps are tested on windows box.

Here is the steps how do I setup the load balance system.


1. Download the required software

Apache2.x -- http://httpd.apache.org
Tomcat4.x -- http://jakarta.apache.org/tomcat/
The JK Module


2. Install and configure the Apache

2.1 Install Apache

Click downloaded install archive, for widows this should be *.msi. Follow the wizard to install.

2.2 Configure the JK2 Module in httpd.conf

Add following lines below the Dynamic Shared Object (DSO) support secion.

#
# Load JK Module
#
LoadModule jk2_module modules/mod_jk2-2.0.43.dll

2.3 Create the workers2.properties file under ${ServerRoot}/conf, which ${ServerRoot} should be
the directory that apache web server installed at.

# only at beginnin. In production uncomment it out
[logger.apache2]
level=DEBUG

[shm]
file=c:/jk2.shm
size=1048576

# Example socket channel, override port and host.
[channel.socket:localhost:12009]
tomcatId=tomcat2
group=lb:lb0

[channel.socket:localhost:11009]
tomcatId=tomcat1
group=lb:lb0

#
# defined workers
#
[status:me]

[lb:lb0]
#stickySession=0

# Uri mapping
[uri:/examples/*]
group=lb:lb0

[uri:/*.jsp]
group=lb:lb0

[uri:/jkstatus]
group=status:me

3. Install and configure the Tomcat

Unzip the downloaded tomcat archive.
unzip tomcat-4.*.zip tomcat1
copy tomcat1 tomcat2

In both tomcat1 and tomcat2, the same files will be modified. Here, I present the modifications
to the files in tomcat1. You should apply the same changes to the corresponding files located
under tomcat2.

3.1 Modify the catalina.bat to add following lines:
JAVA_HOME=c:/j2sdk1.4
CATALINA_HOME=c:/tomcat1

3.2 Modify conf/server.xml
3.2.1 Add unique jvmRoute to Tomcat Engine

Replace following line

with:

for tomcat2, set jvmRoute="tomcat2"

3.2.2 Modify the control port
replace:

with:

For the tomcat2 server, replace port 8005 with 12005. This will prevent the two servers from conflicting.

3.2.3 Change the AJP13 port
In the AJP 13 connector definition, replace:

port="8009"
with:

port="11009"

For the tomcat2 server, replace port 8009 with 12009.

3.2.4 Disable the standalone HTTP port
I don't want the server to respond the HTTP request directly. So, I comment out the HttpConnector section.

3.3 Modify conf/jk2.properties

3.3.1 Replace jk2.properties under c:\tomcat1\conf

# Set the desired handler list
handler.list=channelSocket,request

#
# Override the default port for the socketChannel
channelSocket.port=11009

# Default:
shm.file=c:/jk2.shm

For the tomcat2 server, make channelSocket.port to 12009

3.4 Create test jsp file (index.jsp)

3.4.1 Create a file named index.jsp and put it under c:\tomcat1\webapps\ROOT




<%= request.getSession().getId() %>

Tomcat 1



3.4.2 Create a file named index.jsp and put it under c:\tomcat2\webapps\ROOT




<%= request.getSession().getId() %>

Tomcat 2



4. Start Tomcat1, Tomcat2, Apache
c:\tomcat1\bin\startup.bat
c:\tomcat2\bin\startup.bat
c:\apache2\Apache.exe

5. Test your Installation

Now is the time to test your setup. First, verify that Apache serves static content.

Click on: http://localhost/. You should see the default Apache index.html page.

Now test that tomcat (either Tomcat 1 or Tomcat 2) is serving Java Server Pages.

Click on: http://localhost/index.jsp

Click on: http://localhsot/jkstatus to show the status of jk2 module in Apache.

Jevang 2003-11-16 01:05
Hi Iceant,
From your post, it's not clear if you have find the answer or not. But we have spent lots of time try to configu jk2 on NT and linux without luck, many infos get from web are incorrrect. So we turned back using mod_jk instead, that works beautifully on all platforms.

Cheers,
-Jevang

iceant 2003-11-16 22:57
Hello, Jevang

Yes, I have found the answer. jk is stable I think and there are lots of resource about how to configure jk. But apache said that "JK2 is a refactoring of JK and is much more powerfull" and "JK2 has been developed with Apache 2.0 in mind, and is better suited for multi-threaded servers". So, I think maybe jk2 is more suitable for Apache2.

I just make successfuly configuration of jk2 as the bridge between apache and tomcat on linux box and windows box through channel.socket mode.

If you need help, I can send you a copy of my configuration instruction.

iceant 2003-11-16 23:06
我的下一个问题是在一个 tomcat instance crash 掉以后，能不能发个 Email 通知我一下，要不然系统一直在工作，我都不知道系统的具体情况。不知道有没有人做过这样的工作?
Jevang 2003-11-17 01:54
Hi Iceant,
The work was actually done by one of my engineer, he was configuring Apache + Jboss clustering. If you made jk2 work, that's great, please mail the solution to me wanchun@jevang.com

About notification upon failure, I have not seen there is a centralized admin service offered by Apache to Tomcat, so I think you can't not expected the error msg be pushed out, maybe write a tomcat ping service using JMX?

Cheers
-Wanchun

banq 2003-11-17 11:45
是的，Jk+Apache 1.3比较稳定，这个论坛配置也是这样。
liloboy 2003-11-18 07:28
iceant:
试试检查一下Apache和Tomcat的版本,版本(比如说2.x.yy)差个(yy本本差一都可能有问题的).
iceant 2003-11-18 13:42
TO: jevang
Sorry, 昨晚很忙，没有给你 mail 过去，今晚再给你发 email.

TO:lilobay
单 Apache + 多个Tomcat 的方案我已经搞定~~，昨天和一个同事聊了一下，已经找到一个方案来侦测 Tomcat 的状态，下一步就是实现与测试。

不过我还有一个问题，就是 Tomcat 有时会因为某些资源(特别是网络资源)的问题而出现线程死掉的情况，那么当你运行 shell 中的 shutdown.sh后，再用 ps 会看到 jvm 还在哪，并没有 shutdown 掉，可能它正在等待某些资源的释放~~, 那有什么办法能让这个jvm 安全有效地poweroff?

iceant 2003-11-18 16:27
TO: Jevang

I found a document described more detail than the one I wrote.
http://www.cardon.biz/docs/tomcat/index.html

Maybe it's useful for you.

iceant 2003-11-18 21:28
fail over 应该还包括 session replication
前面我所做的仅仅是 take over.

我记得以前是看过一篇文章介绍如何做 Tomcat 的 session replication. 今天终于找到了，试了一下，先启动两个 tomcat instance ，用户访问其中一个 instance, 这时 SessionManager 会将 Session replicate 到另一个 instance上，我们停掉用户访问的 instance,然后再让用户访问系统，这时可以看到用户访问的 sessionId 还是 stop 掉的 tomcat instance 的sessionId

以下是如何配置 tomcat session replication 的详细介绍:

http://www.theserverside.com/resources/article.jsp?l=Tomcat

iceant 2003-11-18 21:43
其实我更倾向于 N+1 的模式来做 HA
比如 2 个 tomcat Instance + 1 个 Session replicater
也就是只用一台 server 来做 session 的热备，这样可以减少服务器内存的占用。只是实现的算法要稍微复杂一点而已。有空再详细了解一下~~
liloboy 2003-11-19 00:16
能不能说具体一点,什么资源?如果不太重要的话.

要是我的话就在shell里面再 & run一个
sleep N
kill -9
的script,大不了check一下pid的uptime>N就是了.
应该不难.


===============================================================
不过我还有一个问题，就是 Tomcat 有时会因为某些资源(特别是网络资源)的问题而出现线程死掉的情况，那么当你运行 shell 中的 shutdown.sh后，再用 ps 会看到 jvm 还在哪，并没有 shutdown 掉，可能它正在等待某些资源的释放~~, 那有什么办法能让这个jvm 安全有效地poweroff?
===============================================================

iceant 2003-11-19 10:06
谢谢 Liloboy, 但是，如果要跨平台呢?