celery - Is there a way to trigger a task that will run on the same server? -

i have many celery nodes running on different linux boxes. simplified cluster looks like:

i have long-running taska downloads (huge) file remote file-server, , want trigger taskb once taska finishes job on same machine because taska downloaded file input taskb. know celery supposed distributed, in cases, 1 having true distributed solution expensive.

notice in diagram taska on nodea triggers taskb on same node, while taska runs on nodeb triggers taskb on nodeb (same node runs "parent" task).

apply() not work because want queue taskb() run when there spare worker process (on same node!) it.

i started exploring possibility of having unique queue on each node (so nodea has queuea, nodeb has queueb, etc), not know how available queue names taska can queue task on queue on same node. (this solve problem.)

i trying find solution in celery documentation, not find helps me. idea appreciated.

1. you trigger taskb in withing taska directly using apply method.

   this method execute task locally without sending broker.

   note equivalent have 1 task more advanced retry policy.

2. first task published generic queue while schedule second task within taska running on serverabc , passing specific queue name "file_processing_server_abc" worker running on serverabc configured consumer. in way taska run on whatever server picks while b forced tu run on specific queue belonging worker taska running. more on manual routing here.

   edit

   as suggested aaron answer use environment variable store queue machine x , retrieve within taska when scheduling task b.