Laravel在CentOS服务器环境下使用beanstalkd队列服务

发表于 | 分类于

Beanstalk 是一个高性能、轻量级的分布式内存队列系统，最初设计的目的是想通过后台异步执行耗时的任务来降低高容量Web应用系统的页面访问延迟。

安装与配置beanstalkd

安装

从官网下载最新版的源码，当前最新版为 1.10。参考下面命令，编译安装。

1
2
3
4
5
wget https://github.com/kr/beanstalkd/archive/v1.10.tar.gz
tar zxvf v1.10.tar.gz
cd beanstalkd-1.10
make
mv beanstalkd /usr/local/bin/

上面将 beanstalkd 可执行程序放置在 /usr/local/bin/ 目录下。

启动脚本

下面使用 systemctl 来配置启动脚本。

创建 /lib/systemd/system/beanstalkd.service 文件，写入以下内容（注意相关路径和配置参数）：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[Unit]
Description=Beanstalkd

[Service]
User=root
ExecStart=/usr/local/bin/beanstalkd -l 127.0.0.1  

[Install]
WantedBy=multi-user.target
``` 

然后执行下面命令启用 `beanstalkd` 服务和查看它的进程。

```bash
systemctl enable beanstalkd
systemctl start beanstalkd
ps ax | grep beanstalkd

安装beanstalkd console

beanstalkd console 是使用 php 语言编写的 beanstalkd 队列控制台程序。

从官网下载 beanstalk_console 安装包 ，下载完解压之后，部署个域名，注意 nginx 绑定的 root 的为 beanstalk_console 的 public 目录，然后修改 config.php 配置，为了安全起见建议启用 Basic Authentication 认证。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
<?php

$GLOBALS['config'] = array(
    /**
     * List of servers available for all users
     */
    'servers' => array('Local Beanstalkd' => 'beanstalk://localhost:11300'),  // 启用本地server
    /**
     * Saved samples jobs are kept in this file, must be writable
     */
    'storage' => dirname(__FILE__) . DIRECTORY_SEPARATOR . 'storage.json',
    /**
     * Optional Basic Authentication
     */
    'auth' => array(
        'enabled' => true,  // 修改为true开启
        'username' => 'admin',  // 用户名修改成自己的id名或者服务名
        'password' => 'password',  // 密码修改成强类型的密码，并牢记
    ),
    /**
     * Version number
     */
    'version' => '1.7.10',
);

安装与配置supervisord

安装与基本配置

为了更好地管理 Laravel 相关队列服务进程，推荐使用 supervisord 工具，安装与配置方法如下：

1
2
3
4
5
yum -y install python-setuptools
easy_install supervisor
mkdir /etc/supervisor/
mkdir /etc/supervisor/conf.d
echo_supervisord_conf > /etc/supervisor/supervisord.conf

supervisor 的配置文件放置在 /etc/supervisor/supervisord.conf 中。编辑它，在最后面修改添加上：

1
2
[include]
files = /etc/supervisor/conf.d/*.conf

以后，supervisor 管理的服务进程脚本可以放在 /etc/supervisor/conf.d/ 目录下，以 .conf 后缀名的文件中。

启动与服务脚本

创建 /lib/systemd/system/supervisor.service 文件，写入以下内容：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[Unit]
Description=Supervisor daemon
After=network.target

[Service]
Type=forking
ExecStart=/usr/bin/supervisord -c /etc/supervisor/supervisord.conf
ExecStop=/usr/bin/supervisorctl $OPTIONS shutdown
ExecReload=/usr/bin/supervisorctl $OPTIONS reload
KillMode=process
Restart=on-failure
RestartSec=42s

[Install]
WantedBy=multi-user.target

然后执行下面命令设置自动开机启动。

1
2
3
systemctl enable supervisor.service
systemctl daemon-reload
chmod 766 supervisor.service

创建 /etc/init.d/supervisor 脚本，写入以下内容。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
#!/bin/bash
#
# supervisord   This scripts turns supervisord on
#
# Author:       Mike McGrath <mmcgrath@redhat.com> (based off yumupdatesd)
#               Jason Koppe <jkoppe@indeed.com> adjusted to read sysconfig,
#                   use supervisord tools to start/stop, conditionally wait
#                   for child processes to shutdown, and startup later
#               Mikhail Mingalev <mingalevme@gmail.com> Merged
#                   redhat-init-jkoppe and redhat-sysconfig-jkoppe, and
#                   made the script "simple customizable".
#               Brendan Maguire <maguire.brendan@gmail.com> Added OPTIONS to
#                   SUPERVISORCTL status call
#
# chkconfig:    345 83 04
#
# description:  supervisor is a process control utility.  It has a web based
#               xmlrpc interface as well as a few other nifty features.
#               Script was originally written by Jason Koppe <jkoppe@indeed.com>.
#

# source function library
. /etc/rc.d/init.d/functions

set -a

PREFIX=/usr

SUPERVISORD=$PREFIX/bin/supervisord
SUPERVISORCTL=$PREFIX/bin/supervisorctl

PIDFILE=/tmp/supervisord.pid
LOCKFILE=/var/lock/subsys/supervisord

OPTIONS="-c /etc/supervisor/supervisord.conf"

# unset this variable if you don't care to wait for child processes to shutdown before removing the $LOCKFILE-lock
WAIT_FOR_SUBPROCESSES=yes

# remove this if you manage number of open files in some other fashion
ulimit -n 96000

RETVAL=0


running_pid()
{
    # Check if a given process pid's cmdline matches a given name
    pid=$1
    name=$2
    [ -z "$pid" ] && return 1
    [ ! -d /proc/$pid ] && return 1
    (cat /proc/$pid/cmdline | tr "\000" "\n"|grep -q $name) || return 1
    return 0
}

running()
{
# Check if the process is running looking at /proc
# (works for all users)

    # No pidfile, probably no daemon present
    [ ! -f "$PIDFILE" ] && return 1
    # Obtain the pid and check it against the binary name
    pid=`cat $PIDFILE`
    running_pid $pid $SUPERVISORD || return 1
    return 0
}

start() {
        echo "Starting supervisord: "
        
        if [ -e $PIDFILE ]; then 
                echo "ALREADY STARTED"
                return 1
        fi

        # start supervisord with options from sysconfig (stuff like -c)
        $SUPERVISORD $OPTIONS
        
        # show initial startup status
        $SUPERVISORCTL $OPTIONS status
        
        # only create the subsyslock if we created the PIDFILE
        [ -e $PIDFILE ] && touch $LOCKFILE
}

stop() {
        echo -n "Stopping supervisord: "
        $SUPERVISORCTL $OPTIONS shutdown
        if [ -n "$WAIT_FOR_SUBPROCESSES" ]; then 
            echo "Waiting roughly 60 seconds for $PIDFILE to be removed after child processes exit"
            for sleep in  2 2 2 2 4 4 4 4 8 8 8 8 last; do
                if [ ! -e $PIDFILE ] ; then
                    echo "Supervisord exited as expected in under $total_sleep seconds"
                    break
                else
                    if [[ $sleep -eq "last" ]] ; then
                        echo "Supervisord still working on shutting down. We've waited roughly 60 seconds, we'll let it do its thing from here"
                        return 1
                    else
                        sleep $sleep
                        total_sleep=$(( $total_sleep + $sleep ))
                    fi

                fi
            done
        fi

        # always remove the subsys. We might have waited a while, but just remove it at this point.
        rm -f $LOCKFILE
}

restart() {
        stop
        start
}

case "$1" in
    start)
        start
        RETVAL=$?
        ;;
    stop)
        stop
        RETVAL=$?
        ;;
    restart|force-reload)
        restart
        RETVAL=$?
        ;;
    reload)
        $SUPERVISORCTL $OPTIONS reload
        RETVAL=$?
        ;;
    condrestart)
        [ -f $LOCKFILE ] && restart
        RETVAL=$?
        ;;
    status)
        $SUPERVISORCTL $OPTIONS status
        if running ; then
            RETVAL=0
        else
            RETVAL=1
        fi
        ;;
    *)
        echo $"Usage: $0 {start|stop|status|restart|reload|force-reload|condrestart}"
        exit 1
esac

exit $RETVAL

注意修改上面脚本中 supervisor 配置文件路径（包括 PIDFILE 文件）为你的 supervisor 的配置文件里的相关路径。

然后，修改文件权限为 755，并设置开机启动。

1
2
chmod 755 /etc/init.d/supervisor
chkconfig supervisor on

后续，你就可以使用下面命令来管理 supervisor 了。

1
2
service supervisor help
service supervisor {start|stop|status|restart|reload|force-reload|condrestart}

更多其它 Linux 发行版开机启动脚本：https://github.com/Supervisor/initscripts 。

安装与配置Laravel

Laravel 安装与使用，这里不做过多赘述，参考社区中文文档。

Laravel 启用 beanstalkd 队列支持，需要依赖 pda/pheanstalk ~3.0 扩展包，请在 composer.json 追加上，然后 composer update 一下。修改 .env 文件，将 QUEUE_DRIVER=sync 由默认的同步 sync 改为 beanstalkd 。

1
QUEUE_DRIVER=beanstalkd

现在，你可以编写 supervisor 管理队列服务的脚本了。这里演示了一个名叫 laravel-worker 的 supervisor 进程管理的脚本示例，文件路径为 /etc/supervisor/conf.d/laravel-worker.conf 。

1
2
3
4
5
6
7
8
9
[program:laravel-worker]                                                                                                                                      
process_name=%(program_name)s_%(process_num)02d                                                                                                               
command=/usr/local/php/bin/php /data/wwwroot/laravel/artisan queue:work beanstalkd --sleep=3 --tries=3 --daemon                                          
autostart=true                                                                                                                                                
autorestart=true                                                                                                                                              
user=root                                                                                                                                                     
numprocs=8                                                                                                                                                    
redirect_stderr=true                                                                                                                                          
stdout_logfile=/data/wwwroot/laravel/storage/logs/laravel-worker.log

其中 /usr/local/php/bin/php 为服务器中 php 二进制文件的绝对路径，/data/wwwroot/laravel/ 为 Laravel 网站项目的路径，artisan queue:work beanstalkd --sleep=3 --tries=3 --queue=default --daemon 为队列处理器命令参数，它支持传入更多参数，请执行 php artisan queue:work --help 获取。以上参数请根据实际情况修改。

配置完上面，就可以使用 supervisorctl 来启用与重启 laravel-worker 了：

1
2
3
4
supervisorctl reread
supervisorctl update
supervisorctl start laravel-worker:*
supervisorctl restart laravel-worker:*

为了实现队列平滑重启，建议手动运行 php artisan queue:restart 命令之后，再重启 laravel-worker 。

1
2
php artisan queue:restart 
supervisorctl restart laravel-worker:*
坚持原创分享，您的支持将鼓励我继续创作！