Grafy News
Try Grafy free

Edge‑Enabled Multi‑LLM Agentic AI Introduces Supervisor‑Worker Scheduling

Grafy Research Lab unveils an edge‑centric architecture where a supervisor LLM breaks requests into sub‑tasks and dispatches them to worker LLMs.

Illustration for the news story: Edge‑Enabled Multi‑LLM Agentic AI Introduces Supervisor‑Worker SchedulingSource: Grafy Research Lab

A new research paper from Grafy Research Lab proposes an edge‑focused multi‑large language model (LLM) architecture that assigns a supervisory AI to split user requests and coordinate a fleet of worker AIs. The design aims to keep computation close to the user while managing limited edge resources such as battery life and bandwidth.

What is the proposed edge‑centric multi‑LLM architecture?

According to the paper, the system consists of a single supervisor AI agent and multiple worker AI agents that run on edge hardware. The supervisor receives the original user prompt, analyses it, and decomposes it into a sequence of smaller sub‑tasks. Each sub‑task is then handed off to a worker that already hosts an LLM capable of handling the specific type of request (e.g., summarisation, code generation, image description). By keeping the heavy lifting at the edge, the architecture reduces latency compared with cloud‑only solutions and avoids sending raw data to remote servers.

How does the supervisor dispatch tasks to workers?

The paper describes two distinct agentic workflows:

  • Parallel workflow – The supervisor launches several independent sub‑tasks at once, allowing multiple workers to run simultaneously. This maximises throughput when the edge device has enough compute cores or when tasks do not depend on each other.
  • Serial workflow – For dependent operations, the supervisor queues sub‑tasks so that each worker starts only after the previous one finishes. This ensures correct ordering for pipelines such as "extract entities → generate a summary → produce a visualisation".

Both workflows are driven by a dynamic dispatch algorithm that evaluates the current load, estimated execution time, and energy budget of each worker before assignment. The algorithm can switch between parallel and serial modes on‑the‑fly if conditions change, for example when a battery level drops or network bandwidth fluctuates.

What reliability mechanisms are built into the system?

To guard against failures common in edge environments (e.g., intermittent connectivity, hardware throttling), the authors introduce a two‑level retry mechanism. The first level retries a sub‑task on the same worker if it fails due to transient errors. If the second attempt also fails, the second level re‑assigns the sub‑task to a different worker that has a comparable capability profile. This layered approach improves overall success rates without requiring the user to repeat the request manually.

How does the paper address energy consumption?

The researchers formulate an energy minimisation problem that jointly optimises two variables: the scheduling of sub‑tasks across workers and the power‑state configuration of each edge device. By treating task assignment and device throttling as a coupled optimisation, the system can decide, for instance, to run a low‑intensity language model on a battery‑saving core while delegating a high‑priority sub‑task to a more power‑hungry accelerator only when necessary. The resulting schedule aims to minimise total energy expenditure while respecting user‑defined latency constraints.

What practical implications does this have for creators, developers, and researchers?

  • Creators can build AI‑powered applications that run offline or with minimal cloud dependence, preserving user privacy and reducing data‑transfer costs. For example, an on‑device video‑captioning tool could use the supervisor to break a long video into segments, dispatch each segment to a specialised captioning worker, and then re‑assemble the final transcript.
  • Developers gain a template for constructing multi‑agent pipelines that balance performance and power. The paper’s scheduling algorithm can be integrated into edge‑oriented frameworks, allowing developers to focus on task design rather than low‑level resource management.
  • Researchers receive a concrete test‑bed for studying how large language models cooperate in constrained environments. The two‑level retry strategy and the joint energy‑scheduling formulation open new avenues for evaluating reliability and sustainability of AI at the edge.

How to explore the concepts with Grafy tools?

Grafy’s Graph app lets users visualise branching prompt structures, making it a natural fit for experimenting with the supervisor‑worker decomposition described in the paper. By creating a graph where a root node represents the user request and child nodes represent sub‑tasks, creators can manually emulate the parallel and serial workflows before automating them. For a hands‑on test, open the Graph workspace at /graph and build a simple two‑step pipeline (e.g., "summarise article → generate illustration"). Pair this with Chat at /chat to route each sub‑task to a different LLM model, observing latency and token usage differences. If you prefer to dive into the paper itself, the Reader app at /reader can ingest the PDF, enable keyword search, and let you ask follow‑up questions to the AI about the scheduling equations.

The architecture also aligns with Grafy’s broader vision of multimodal AI workspaces: the supervisor can invoke the Creator app for image generation, the Music Studio for audio synthesis, or the Film Studio for stitching together video clips, all while staying on the edge device. By leveraging Grafy’s modular apps, developers can prototype end‑to‑end edge pipelines without building each component from scratch.


Source: https://arxiv.org/abs/2611.20915

Frequently asked questions

Does the supervisor AI always run on the same edge device as the workers?

The paper assumes the supervisor resides on a device with enough compute to handle request parsing and scheduling, but it can be colocated with a worker or run on a slightly more powerful hub that communicates with nearby workers over a local network.

Can the two‑level retry mechanism increase overall latency?

Yes, retries add extra execution time when failures occur. However, the design balances this by only invoking the second‑level retry when the first attempt fails, and by selecting alternate workers that are likely to succeed quickly, keeping latency spikes modest.

Is the energy‑minimisation formulation applicable to non‑LLM edge workloads?

The optimisation problem is model‑agnostic; it treats each task as a workload with known energy characteristics. Therefore, the same scheduling logic can be applied to vision models, speech recognisers, or any compute‑intensive edge service.


Source: Grafy Research Lab — Multi-LLM Agentic AI at the Edge: Joint Agent Scheduling and Resource Allocation