besoin d'aide concernant django/celery

vohu, samedi 27 janvier 2018 à 21h43

Bonjour,

Je cherchais à faire quelques test avec celery avec les exemples qu’on trouve un peu partou, et je reste coincé depuis un certain temps. Lorsque le lance les appels dans un ./manage.py shell. la console reste bloquée reste bloqué. Par exemple : hello.delay("ok") s’execute de façon infinie…

Pourtant, les crontab semblent fonctionner car je vois les appels dans celery -A projet beat -l info :

celery -A integration beat -l info
celery beat v4.1.0 (latentcall) is starting.
__    -    ... __   -        _
LocalTime -> 2018-01-27 20:38:51
Configuration ->
    . broker -> redis://:**@127.0.0.1:6379/0
    . loader -> celery.loaders.app.AppLoader
    . scheduler -> celery.beat.PersistentScheduler
    . db -> celerybeat-schedule
    . logfile -> [stderr]@%INFO
    . maxinterval -> 5.00 minutes (300s)
[2018-01-27 20:38:51,951: INFO/MainProcess] beat: Starting...
[2018-01-27 20:38:51,971: INFO/MainProcess] Scheduler: Sending due task add-every-5-seconds (multiply_two_numbers)
[2018-01-27 20:38:56,959: INFO/MainProcess] Scheduler: Sending due task add-every-5-seconds (multiply_two_numbers)
[2018-01-27 20:38:56,961: INFO/MainProcess] Scheduler: Sending due task add-every-minute-contrab (multiply_two_numbers)
[2018-01-27 20:39:00,000: INFO/MainProcess] Scheduler: Sending due task add-every-minute-contrab (multiply_two_numbers)
[2018-01-27 20:39:01,959: INFO/MainProcess] Scheduler: Sending due task add-every-5-seconds (multiply_two_numbers)

`

Les logs de redis-server semblent muets :

tail -qf /var/log/redis/redis-server.log 
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

18521:M 27 Jan 21:41:35.088 # Server started, Redis version 3.2.6
18521:M 27 Jan 21:41:35.088 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
18521:M 27 Jan 21:41:35.088 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
18521:M 27 Jan 21:41:35.090 * DB loaded from disk: 0.002 seconds
18521:M 27 Jan 21:41:35.090 * The server is now ready to accept connections on port 6379

`

Est ce que ce problème vous parle ? Je ne sais plus trop quoi faire.

J’utilise les versions suivantes des librairies python : django==1.11.9 celery==4.1.0 django-celery==3.2.2 django-celery-beat==1.1.0 django-celery-monitor==1.1.2 django-celery-results==1.0.1

Sur ma machine j’ai installé python 3.5 et redis-server 3:3.2.6-1

Voici les extraits de codes concernés :

Fichier projet/settings.py

...

# librairies
INSTALLED_APPS += (
    ...
    #'celery',
    'django_celery_monitor',
    'django_celery_results',
    'tasksasync',
    ...
)

...
BROKER_TRANSPORT = "redis"
BROKER_HOST = "redis://:mdp@127.0.0.1:6379/0"  # Maps to redis host.
BROKER_PORT = 6379 
BROKER_VHOST = "0"

CELERY_BROKER_URL = BROKER_HOST
CELERY_RESULT_BACKEND = BROKER_HOST
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = TIME_ZONE
CELERY_IGNORE_RESULT = True

projet/celery.py

#from __future__ import absolute_import, unicode_literals
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'projet.settings')

app = Celery('projet')
app.config_from_object('django.conf:settings', namespace='CELERY')

app.autodiscover_tasks()

from celery.schedules import crontab
app.conf.beat_schedule = {
    'add-every-minute-contrab': {
        'task': 'multiply_two_numbers',
        'schedule': crontab(),
        'args': (16, 16),
    },
    'add-every-5-seconds': {
        'task': 'multiply_two_numbers',
        'schedule': 5.0,
        'args': (16, 16)
    },
    'add-every-30-seconds': {
        'task': 'tasks.add',
        'schedule': 30.0,
        'args': (16, 16)
    },
}

projet/init.py

#from __future__ import absolute_import, unicode_literals

# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app

__all__ = ['celery_app']

tasksasync/tasks.py

# Create your tasks here
#from __future__ import absolute_import, unicode_literals
import random

from celery.decorators import task
from celery import shared_task

@shared_task(name="hello")
def hello(name):
    return "Hello %s" % name

@task(name="sum_two_numbers")
def add(x, y):
    return x + y

@task(name="multiply_two_numbers")
def mul(x, y):
    total = x * (y * random.randint(3, 100))
    return total

@task(name="sum_list_numbers")
def xsum(numbers):
    return sum(numbers)

`

Merci d’avance pour votre aide

27/01/18 à 21h43

+0 -0

django 1.11, celery, redis

Pas encore membre ?