開始セロリの芽20人の作業員。どうやって？

Question

私はデーモン化しているセロリを展開しています。以下は私のcelerydファイルです。

CELERYD_NODES="worker1 worker2 worker3 worker4" 

CELERY_BIN="/usr/local/bin/celery" 

CELERY_APP="proj:app" 

CELERYD_CHDIR="/path/to/proj/" 

CELERYD_OPTS="-n %N@%h --config=proj.celeryconfig -l DEBUG --without-heartbeat" 

CELERYD_LOG_FILE="/var/log/celery/%N.log" 
CELERYD_PID_FILE="/var/run/celery/%N.pid" 

CELERYD_USER="celery" 
CELERYD_GROUP="celery" 

CELERY_CREATE_DIRS=1 

私はcelerydを起動すると、20個のプロセスが生成されます。なぜ、どのように？私のサーバーには4つのCPUがあります。続いて、以下のコマンド

ps aux|grep 'celery worker' 

を発射する際

が出力されます。

gfile=/var/log/celery/worker1.log --pidfile=/var/run/celery/worker1.pid --hostname=worker1@worker1@worker1@worker1@%h 
celery 26652 5.7 0.8 270340 66892 ?  S 18:41 0:02 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker2.log --pidfile=/var/run/celery/worker2.pid --hostname=worker2@worker2@worker2@worker2@%h 
celery 26662 1.0 0.7 860128 59364 ?  Sl 18:41 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker1.log --pidfile=/var/run/celery/worker1.pid --hostname=worker1@worker1@worker1@worker1@%h 
celery 26663 1.1 0.7 565456 59584 ?  S 18:41 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker1.log --pidfile=/var/run/celery/worker1.pid --hostname=worker1@worker1@worker1@worker1@%h 
celery 26664 1.6 0.7 860384 59624 ?  Sl 18:41 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker1.log --pidfile=/var/run/celery/worker1.pid --hostname=worker1@worker1@worker1@worker1@%h 
celery 26665 0.2 0.7 270272 59672 ?  S 18:41 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker1.log --pidfile=/var/run/celery/worker1.pid --hostname=worker1@worker1@worker1@worker1@%h 
celery 26668 5.3 0.8 270340 66656 ?  S 18:41 0:02 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker3.log --pidfile=/var/run/celery/worker3.pid --hostname=worker3@worker3@worker3@worker3@%h 
celery 26682 0.3 0.7 270272 58048 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker2.log --pidfile=/var/run/celery/worker2.pid --hostname=worker2@worker2@worker2@worker2@%h 
celery 26683 0.1 0.7 270272 58044 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker2.log --pidfile=/var/run/celery/worker2.pid --hostname=worker2@worker2@worker2@worker2@%h 
celery 26684 0.3 0.7 270272 58044 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker2.log --pidfile=/var/run/celery/worker2.pid --hostname=worker2@worker2@worker2@worker2@%h 
celery 26685 0.2 0.7 270272 58108 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker2.log --pidfile=/var/run/celery/worker2.pid --hostname=worker2@worker2@worker2@worker2@%h 
celery 26687 5.5 0.8 270340 66824 ?  S 18:42 0:02 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker4.log --pidfile=/var/run/celery/worker4.pid --hostname=worker4@worker4@worker4@worker4@%h 
celery 26696 0.1 0.7 270272 58036 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker3.log --pidfile=/var/run/celery/worker3.pid --hostname=worker3@worker3@worker3@worker3@%h 
celery 26697 1.3 0.7 861124 60648 ?  Sl 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker3.log --pidfile=/var/run/celery/worker3.pid --hostname=worker3@worker3@worker3@worker3@%h 
celery 26698 0.1 0.7 270272 58032 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker3.log --pidfile=/var/run/celery/worker3.pid --hostname=worker3@worker3@worker3@worker3@%h 
celery 26699 0.1 0.7 270272 58100 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker3.log --pidfile=/var/run/celery/worker3.pid --hostname=worker3@worker3@worker3@worker3@%h 
celery 26701 0.0 0.7 270272 57720 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker4.log --pidfile=/var/run/celery/worker4.pid --hostname=worker4@worker4@worker4@worker4@%h 
celery 26702 0.0 0.6 270080 53260 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker4.log --pidfile=/var/run/celery/worker4.pid --hostname=worker4@worker4@worker4@worker4@%h 
celery 26703 0.0 0.7 270272 57268 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker4.log --pidfile=/var/run/celery/worker4.pid --hostname=worker4@worker4@worker4@worker4@%h 
celery 26704 0.0 0.7 270272 57336 ?  S 18:42 0:00 /usr/bin/python -m celery worker --without-heartbeat -l DEBUG --config=c26_message.celeryconfig --loglevel=INFO --logfile=/var/log/celery/worker4.log --pidfile=/var/run/celery/worker4.pid --hostname=worker4@worker4@worker4@worker4@%h 
root  26740 0.0 0.0 14196 984 pts/5 S+ 18:42 0:00 grep --color=auto celery worker 

セロリの花も20人のうち16人がオフラインで、4人がオンラインです。どのように可能ですか？

Answer 1

根本的な原因は4つのノードが作成され、CPUの数に基づくデフォルトの並行性の設定です。したがって、各ノードが5つのプロセスを作成したことがわかります。

CELERYD_OPTS="-c 4 -c:worker4 1 -c:worker3 1 -c:worker2 1 -c:worker1 1"

がセロリのドキュメントの構成例を参照することができる：http://docs.celeryproject.org/en/latest/userguide/daemonizing.html#example-configuration

セットとして、以下の例を参照することにより、各並行処理ノードは、CPUごとにノードを設定します