由于django-celery这个模块我在django1.10的版本内运行不起来,只能使用野生的Celery,Celery4支持django1.8以上的版本,1.8以下的版本请使用Celery3,整个配置过程并不复杂:
整个目录结构:
一、安装模块:
pip install celery pip install django-celery-results
django-celery-results作用是将Celery的运行结果存入数据库
二、建立Celery入口文件(celery.py):
from __future__ import absolute_import, unicode_literals import os from celery import Celery # set the default Django settings module for the ‘celery‘ program. os.environ.setdefault(‘DJANGO_SETTINGS_MODULE‘, ‘Mir2Admin.settings‘) app = Celery(‘Mir2Admin‘) # Using a string here means the worker don‘t have to serialize # the configuration object to child processes. # - namespace=‘CELERY‘ means all celery-related configuration keys # should have a `CELERY_` prefix. app.config_from_object(‘django.conf:settings‘, namespace=‘CELERY‘) # Load task modules from all registered Django app configs. app.autodiscover_tasks() @app.task(bind=True) def debug_task(self): print(‘Request: {0!r}‘.format(self.request))
三、把celery.py中的app加入__init__.py文件中,确保django运行的时候加载到它:
from __future__ import absolute_import, unicode_literals # This will make sure the app is always imported when # Django starts so that shared_task will use this app. from .celery import app as celery_app __all__ = [‘celery_app‘]
四、配置settting.py:
# celery配置 CELERY_BROKER_URL = ‘redis://:lihuipeng@192.168.x.x:6379/3‘ #: Only add pickle to this list if your broker is secured #: from unwanted access (see userguide/security.html) CELERY_ACCEPT_CONTENT = [‘json‘] CELERY_RESULT_BACKEND = ‘django-db‘ CELERY_TASK_SERIALIZER = ‘json‘ INSTALLED_APPS = [ ...... ‘django_celery_results‘, ...... ]
我的broker用redis,没有redis的google撸一个
CELERY_RESULT_BACKEND这一项只有装了django-celery-results这个模块这里才能配置‘django-db‘,把结果存入数据库
五、生成数据表:
migrate django_celery_results
六、在自己的app中建立tasks.py文件,添加需要异步执行的函数:
# Create your tasks here from __future__ import absolute_import, unicode_literals from celery import shared_task @shared_task def add(x, y): return x + y @shared_task def mul(x, y): return x * y @shared_task def xsum(numbers): return sum(numbers)
五、启动测试(在django工程的manage.py同层目录执行):
/usr/local/python27/bin/celery -A Mir2Admin worker -l info
六、简单测试:
python manage.py shell >>> from myapp.tasks import add >>> add.delay(2, 2)
从第五步的前台日志可以看到Celery执行结果,django-celery-results会把结果存进数据库
七、生产环境配置(附件):
添加启动脚本:
chmod +x /etc/init.d/celeryd
添加配置文(/etc/default/celeryd):
# Names of nodes to start # most people will only start one node: CELERYD_NODES="Mir2AdminCelery" # but you can also start multiple and configure settings # for each in CELERYD_OPTS #CELERYD_NODES="worker1 worker2 worker3" # alternatively, you can specify the number of nodes to start: #CELERYD_NODES=10 # Absolute or relative path to the ‘celery‘ command: CELERY_BIN="/usr/local/python27/bin/celery" # App instance to use # comment out this line if you don‘t use an app CELERY_APP="Mir2Admin" # or fully qualified: #CELERY_APP="proj.tasks:app" # Where to chdir at start. CELERYD_CHDIR="/data/www/Mir2Admin" # Extra command-line arguments to the worker CELERYD_OPTS="--time-limit=300 --concurrency=8" # Configure node-specific settings by appending node name to arguments: #CELERYD_OPTS="--time-limit=300 -c 8 -c:worker2 4 -c:worker3 2 -Ofair:worker1" # Set logging level to DEBUG #CELERYD_LOG_LEVEL="DEBUG" # %n will be replaced with the first part of the nodename. CELERYD_LOG_FILE="/data/www/Mir2Admin/logs/%n%I.log" CELERYD_PID_FILE="/data/www/Mir2Admin/%n.pid" # Workers should run as an unprivileged user. # You need to create this user manually (or you can choose # a user/group combination that already exists (e.g., nobody). CELERYD_USER="root" CELERYD_GROUP="root" # If enabled pid and log directories will be created if missing, # and owned by the userid/group configured. CELERY_CREATE_DIRS=1
CELERYD_NODES:名称,影响日志文件名及pid文件名
CELERY_BIN:celery路径
CELERY_APP:django工程名称
CELERYD_CHDIR:django工程路径
CELERYD_OPTS:参数
CELERYD_LOG_FILE:日志路径
CELERYD_PID_FILE:PID文件路径
最后启动一发:
/etc/init.d/celeryd start
参考文章:http://docs.celeryproject.org/en/latest/django/first-steps-with-django.html
本文出自 “运维笔记” 博客,请务必保留此出处http://lihuipeng.blog.51cto.com/3064864/1893463
原文地址:http://lihuipeng.blog.51cto.com/3064864/1893463