Internet Research Task Force Y-G. Hong
Internet-Draft Daejeon Univ.
Intended status: Informational J-S. Youn
Expires: 6 May 2027 DONG-EUI Univ.
S-W. Hong
ETRI
P. Martinez-Julia
NICT
Q. Wu
Huawei
2 November 2026
Considerations of network/system for AI services
draft-irtf-nmrg-ai-deploy-04
Abstract
As AI technology has matured and has been applied in various fields,
the execution environment of AI services has evolved from dedicated
high-performance servers to commodity servers and affordable, small-
scale hardware, including microcontrollers, low-performance CPUs, and
AI accelerators. At the same time, generative AI based on large and
small language models, and Agentic AI systems that plan and act
autonomously, have become major AI workloads. This document outlines
how to configure the network and system for AI inference services
provided in a distributed manner across local machines, edge devices,
and cloud servers. It describes the factors to consider when a
client requests an AI service from a cloud server or an edge device,
including hardware, AI model, communication method, generative AI
workload characteristics, model distribution, and agent-driven
selection of the execution point. It also describes use cases for
deploying network-based AI services, such as self-driving vehicles,
network digital twins, and Physical AI.
This document is a product of the Network Management Research Group
(NMRG) of the Internet Research Task Force (IRTF). It is not a
candidate for any level of Internet Standard and is published for
informational purposes.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Hong, et al. Expires 6 May 2027 [Page 1]
Internet-Draft Deploying AI services November 2026
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 6 May 2027.
Copyright Notice
Copyright (c) 2026 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document.
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Procedure to provide AI services . . . . . . . . . . . . . . 6
4. Network configuration structure to provide AI services . . . 8
4.1. AI inference service on local machine . . . . . . . . . . 8
4.2. AI inference service on cloud server . . . . . . . . . . 9
4.3. AI inference service on edge device . . . . . . . . . . . 10
4.4. AI inference service on cloud server and edge device . . 10
4.5. AI inference service on horizontal multiple servers . . . 12
4.6. Network-side utilization for AI learning . . . . . . . . 14
5. Considerations of network/system for AI services . . . . . . 15
5.1. Considerations of the functional characteristics of the
hardware . . . . . . . . . . . . . . . . . . . . . . . . 15
5.2. Considerations for the characteristics of the AI model . 16
5.3. Considerations for the characteristics of the communication
method . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.4. Considerations for generative AI and LLM inference
workloads . . . . . . . . . . . . . . . . . . . . . . . . 18
5.5. Considerations for AI model distribution and lifecycle
management . . . . . . . . . . . . . . . . . . . . . . . 19
5.6. Considerations for Agentic AI-driven autonomous
offloading . . . . . . . . . . . . . . . . . . . . . . . 20
6. Addressing challenges for coupling AI and NM . . . . . . . . 22
Hong, et al. Expires 6 May 2027 [Page 2]
Internet-Draft Deploying AI services November 2026
6.1. Problem characteristics (C1-C6) . . . . . . . . . . . . . 22
6.2. High-level challenges in adopting AI in NM . . . . . . . 24
7. Use cases of deploying network-based AI services . . . . . . 25
7.1. Deploying AI services for self-driving vehicles . . . . . 25
7.2. Deploying AI services for network digital twins . . . . . 27
7.3. Deploying AI services for Physical AI . . . . . . . . . . 30
8. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 32
9. Security Considerations . . . . . . . . . . . . . . . . . . . 32
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 34
11. Informative References . . . . . . . . . . . . . . . . . . . 34
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 37
1. Introduction
In the Internet of Things (IoT), the volume of generated data has
exploded alongside the rapid proliferation of devices due to
industrial digitization and the development and dissemination of new
devices. Various methods have been tried to effectively handle the
growing number of IoT devices and the data they produce. One of them
is to provide IoT services close to IoT devices and users, shifting
away from centralized cloud computing frameworks that transmit all
data generated by IoT devices to a cloud server [RFC9556].
IoT services have also been moving away from the traditional method
of analyzing collected IoT data in the cloud and delivering the
analyzed results back to IoT objects or devices. AIoT (Artificial
Intelligence of Things), a combination of IoT technology and
artificial intelligence (AI) technology, has been discussed at
international standardization organizations such as ITU-T. AIoT, as
discussed by the ITU-T CG-AIoT group, is defined as a technology that
combines AI technology and IoT infrastructure to achieve more
efficient IoT operations, improve human-machine interaction, and
improve data management and analysis [CG-AIoT].
The first work started by the IETF to apply IoT technology to the
Internet was to research a lightweight protocol stack instead of the
existing TCP/IP protocol stack so that various types of IoT devices,
not traditional Internet terminals, could access the Internet
[RFC6574] [RFC7452]. These technologies have been developed by the
6LoWPAN, 6lo, 6TiSCH, and CoRE working groups and the T2TRG research
group, among others. Just as IoT technology was integrated into
resource-constrained devices and connected to the Internet, AI
technology has also changed from running only on very high-
performance servers to running on small hardware, including
microcontrollers, low-performance CPUs, and AI accelerators. This
direction of technology development is called On-device AI, TinyML,
or, more broadly, edge AI [EDGE-AI]. More recently, generative AI
models, including Small Language Models (SLMs), have also begun to
Hong, et al. Expires 6 May 2027 [Page 3]
Internet-Draft Deploying AI services November 2026
run on end devices and edge devices equipped with neural processing
units, while Large Language Models (LLMs) continue to be served
mainly from cloud data centers.
In this document, we consider how to configure the network and system
for AI inference services. The technology of collecting sensing data
from various sensors and delivering it to the cloud has already been
studied by many standardization organizations including the IETF, and
many standards have been developed. After an AI model has been
created based on the collected data, how to deploy this AI model as a
system has become a main research question. Until now, it has been
common to collect data and perform inference on the servers used for
training, but as AI services spread, using expensive servers for
every AI service is not appropriate. In addition, since the servers
that collect data and train models mainly exist in the cloud,
connecting a large number of terminals to these cloud servers to
request AI services raises many problems. Therefore, requesting an
AI service from an edge device located nearby, rather than from an AI
server located in a distant cloud, may support real-time services,
reduce network traffic, and keep important data local [RFC9556].
Even if an edge device is used to serve AI services, it is still
important to connect to an AI server in the cloud for tasks that take
a lot of time or require a lot of data. Therefore, offloading
techniques that properly distribute the workload between the cloud
server and the edge device are also being actively studied.
Furthermore, beyond the deployment of basic AI inference on
distributed devices, the concept of Agentic AI is rapidly emerging.
Agentic AI refers to AI systems capable of autonomous goal setting,
complex task decomposition, planning, tool invocation, execution, and
continuous learning and reflection, rather than simple one-off
inference responses. Deploying such Agentic AI systems in a
distributed environment (cloud, edge, and local) is expected to be an
important building block for autonomous and intelligent network
management and service provision. The considerations for distributed
AI deployment in this document also apply to the network
infrastructure required by Agentic AI. This document does not
specify agent architectures or agent communication protocols.
In this document, for the network structures described in Section 4,
the points to be considered when a client connects to a server or an
edge device and requests an AI service are derived and described.
The following considerations and options are derived:
* AI inference service execution entity
Hong, et al. Expires 6 May 2027 [Page 4]
Internet-Draft Deploying AI services November 2026
* Hardware specifications of the machine that performs AI inference
services
* Selection of AI models to perform AI inference services
* Method of providing AI services from cloud servers or edge devices
* Communication method to transmit data to request AI inference
services
* Characteristics of generative AI and LLM inference workloads
* Distribution and lifecycle management of AI models across the
cloud continuum
* Autonomous, agent-driven selection of the execution point
(offloading)
The proposed considerations are applied to describe the use cases of
self-driving vehicles, network digital twins, and Physical AI, all of
which can benefit from providing AI services in a distributed manner.
2. Terminology
This document uses the following terms:
AI inference service: A service that executes a trained AI model on
input data provided by a client and returns the inference result.
AI client module: A software module that requests an AI inference
service and receives its result.
AI server module: A software module that executes an AI model to
perform an AI inference service.
Local machine: The machine on which the AI client module runs (e.g.,
an end device, a vehicle, or a user terminal).
Edge device: A computing node located in the edge network, close to
the local machine, that can host an AI server module.
Cloud server: A computing node located in a cloud data center that
can host an AI server module.
Cloud continuum: The set of local machines, edge devices, and cloud
servers across which the components of an AI service can be
placed. Also called the cloud-edge-local continuum.
Hong, et al. Expires 6 May 2027 [Page 5]
Internet-Draft Deploying AI services November 2026
Offloading: Transferring all or part of an AI task from one
execution point to another (e.g., from a local machine to an edge
device or a cloud server).
LLM / SLM: Large Language Model / Small Language Model. Generative
models whose inference produces output tokens sequentially. SLMs
are small enough to run on end devices or edge devices.
AI agent: A software entity that uses an AI model (typically an LLM)
to perceive its environment, plan, invoke tools, and act toward a
goal with some degree of autonomy.
Agentic AI: AI systems composed of one or more AI agents.
AI agent planning module: The function of an AI agent that
decomposes tasks and decides where (local, edge, or cloud) each
task is executed. It corresponds to the Intelligence Core
described in Section 5.6.
Tool: An external capability (e.g., an API or a network management
interface) that an AI agent can invoke to obtain information or
perform an action.
Multi-Agent System (MAS): A system in which multiple AI agents
interact, negotiate, and collaborate toward common or individual
objectives.
Physical AI: AI systems that perceive the physical environment
through sensors, perform AI inference, and act on the physical
world through actuators (e.g., robots, drones, autonomous
vehicles, and industrial automation systems).
In-time delay-sensitive AI service: An AI service whose result must
be delivered before a deadline.
On-time delay-sensitive AI service: An AI service whose result must
be delivered within a specified time range (neither too early nor
too late).
3. Procedure to provide AI services
Given the extensive history of research on AI services, various
architectures exist for providing them. However, due to the nature
of AI technology, a system for providing AI services generally
consists of the following steps [AI_inference_architecture].
Hong, et al. Expires 6 May 2027 [Page 6]
Internet-Draft Deploying AI services November 2026
+----------+ +----------+ +----------+ +----------+ +----------+
|Collect & | |Analysis &| | Train | | Deploy & | |Monitor & |
| Store |->|Preprocess|->| AI model |->|Inference |->| Maintain |
| data | | data | | | | AI model | | Accuracy |
+----------+ +----------+ +----------+ +----------+ +----------+
|<-------->| |<---------------------->| |<-------->| |<-------->|
Sensor, DB AI Server Target AI Srvr &
machine Tgt. machine
|<-------->| |<---------------------->| |<-------->| |<-------->|
Internet Local Internet Local &
Internet
Figure 1: AI service workflow
* Data collection & Storage
* Data Analysis & Preprocessing
* AI Model Training
* AI Model Deploy & Inference
* Monitor & Maintain Accuracy
In the data collection step, data required for training is prepared
by collecting data from sensors and IoT devices or by using data
stored in a database. Equipment involved in this step includes
sensors, IoT devices, servers that store their data, and database
servers. Since the operations performed at this step are conducted
through the Internet, many IoT technologies developed by the IETF are
suitable for this step.
In the data analysis and preprocessing step, the features of the
prepared data are analyzed and preprocessing for training is
performed. Equipment involved in this step includes a high-
performance server equipped with a GPU and a database server, and
this step is mainly performed in a local network.
In the AI model training step, a model is created by applying an
algorithm suitable for the characteristics of the data and the
problem to be solved. Equipment involved in this step includes high-
performance servers equipped with GPUs or other AI accelerators, and
this step is mainly performed in a local or data center network.
In the AI model deployment and inference service provision step, the
problem to be solved (e.g., classification, regression, or content
generation) is solved using the trained AI model. Equipment involved
in this step may include a target machine, a client, edge devices,
Hong, et al. Expires 6 May 2027 [Page 7]
Internet-Draft Deploying AI services November 2026
and cloud servers that provide AI services. Since various equipment
is involved in this step, it is conducted through the Internet. This
document summarizes the factors to be considered at this step.
In the accuracy monitoring step, if performance deteriorates due to
new data, a new model is created through re-training, and the AI
service quality is maintained by using the newly created model. This
step repeats the model training and model deployment steps described
above.
Two aspects have become prominent in addition to these classical
steps. First, many AI services are no longer built by training a
model from scratch; instead, a pre-trained foundation model is
adapted to the target task through fine-tuning (including parameter-
efficient methods) or is complemented at inference time by retrieving
external knowledge (Retrieval-Augmented Generation, RAG). Second,
the distribution of models, adapters, and their updates to many
inference points has become a networking task in its own right, as
discussed in Section 5.5.
4. Network configuration structure to provide AI services
In general, after an AI model has been trained, it can be deployed on
a local machine to provide AI inference services. Alternatively, AI
models can be placed on cloud servers or edge devices, and AI service
requests can be made remotely. In addition, for overall service
performance, some AI service requests can be sent to the cloud server
and others to edge devices through appropriate load balancing.
4.1. AI inference service on local machine
The following figure shows a case where an AI client module on a
local machine requests an AI service from an AI server module on the
same machine.
+---------------------------------------------------------------------+
| |
| +-----------------+ Request AI +-----------------+ |
| | Client module | Inference service | Server module | |
| | for AI service |----------------------->| for AI service | |
| | |<-----------------------| | |
| +-----------------+ Reply AI +-----------------+ |
| Inference result |
+---------------------------------------------------------------------+
Local machine
Figure 2: AI inference service on local machine
Hong, et al. Expires 6 May 2027 [Page 8]
Internet-Draft Deploying AI services November 2026
This method is often used when configuring a system focused on
training AI models to improve their inference accuracy and
performance, without particular consideration of AI service
deployment and inference. In this case, since the AI client module
that requests the AI inference service and the AI server module that
performs it are on the same machine, the communication/network
environment and the service provision method need little
consideration. This method can also be used to implement a self-
contained AI inference service that is not expected to change, such
as on an embedded or customized machine.
In this case, the hardware does not need to be capable of training
the AI model, but it must be capable of running the AI inference
service.
4.2. AI inference service on cloud server
The following figure shows the case where the AI client module that
requests the AI service and the AI server module that performs the AI
service run on different machines.
+--------------------------------------+
+------------------------+ | +---------------------------+ |
| +-----------------+ | | | +-----------------+ | |
| | Client module |<-+------+-----+---->| Server module | | |
| | for AI service | | | | | for AI service | | |
| +-----------------+ | | | +-----------------+ | |
+------------------------+ | +---------------------------+ |
Local machine | Server machine |
+--------------------------------------+
Cloud (Internet)
Figure 3: AI inference service on cloud server
In this case, the AI client module runs on the local machine, and the
AI server module runs on a separate server machine located in the
cloud network. The performance of the local machine does not need to
be high because the local machine only needs to request the AI
inference service and, if necessary, deliver the data required for
the request. The AI server module can be hosted on a self-operated
AI server or on a commercial cloud service.
Hong, et al. Expires 6 May 2027 [Page 9]
Internet-Draft Deploying AI services November 2026
4.3. AI inference service on edge device
The following figure shows the case where the AI client module and
the AI server module are separated, and the AI server module is
located on an edge device. As more and more data is generated at the
edge, it is necessary to perform AI processing at the edge due to
performance, privacy, and the cost of moving data to a central
location. For example, some enterprises owning both local machines
and edge networks may hesitate to upload confidential datasets to the
central cloud due to privacy and security concerns. Instead, they
retrieve AI models from the cloud and use them for AI inference
locally at the edge device.
+--------------------------------------+
+------------------------+ | +---------------------------+ |
| +-----------------+ | | | +-----------------+ | |
| | Client module |<-+------+-----+---->| Server module | | |
| | for AI service | | | | | for AI service | | |
| +-----------------+ | | | +-----------------+ | |
+------------------------+ | +---------------------------+ |
Local machine | Edge device |
+--------------------------------------+
Edge network
Figure 4: AI inference service on edge device
In this case, the AI client module runs on the local machine, and the
AI server module runs on an edge device in the edge network. The AI
server module on the edge device can be deployed on a self-configured
edge device or on a commercial edge computing platform.
Compared with the cloud case, the edge device is usually closer to
the client but has lower computing performance than a cloud server.
Therefore, the edge case has an advantage in data transfer time,
whereas its inference throughput (the number of inference requests
processed per unit time) and the size of the models it can host are
usually smaller.
4.4. AI inference service on cloud server and edge device
The following figure shows the case where AI server modules are
distributed over the cloud and edge devices.
Hong, et al. Expires 6 May 2027 [Page 10]
Internet-Draft Deploying AI services November 2026
+--------------------------------------+
+------------------------+ | +---------------------------+ |
| +-----------------+ | | | +-----------------+ | |
| | Client module |<-+-+----+-----+---->| Server module | | |
| | for AI service |<-+-+ | | | for AI service | | |
| +-----------------+ | | | | +-----------------+ | |
+------------------------+ | | +---------------------------+ |
Local machine | | Edge device |
| +--------------------------------------+
| Edge network
|
| +--------------------------------------+
| | +---------------------------+ |
| | | +-----------------+ | |
+----+-----+---->| Server module | | |
| | | for AI service | | |
| | +-----------------+ | |
| +---------------------------+ |
| Server machine |
+--------------------------------------+
Cloud (Internet)
Figure 5: AI inference service on cloud server and edge device
AI server modules in the cloud and on edge devices differ in AI
inference service performance. Therefore, the client may distribute
its AI inference service requests between the cloud and the edge
device appropriately to obtain the desired AI service. For example,
a request for which a short inference time matters more than the
highest inference accuracy can be sent to the edge device.
The distribution of AI inference service requests between the cloud
and edge devices is a critical decision. In the context of Agentic
AI, the AI client module can be extended with an AI agent planning
module. This module determines the execution point (local, edge, or
cloud) based on a dynamic assessment of inference accuracy
requirements, latency constraints (e.g., in-time/on-time delay
sensitivity), and resource costs. Since the decision making of an
LLM-based agent itself consumes time and resources, the timescale at
which such decisions are made needs to be considered, as discussed in
Section 5.6.
Hong, et al. Expires 6 May 2027 [Page 11]
Internet-Draft Deploying AI services November 2026
4.5. AI inference service on horizontal multiple servers
The configurations in Sections 4.1 to 4.4 place AI server modules at
different levels of a vertical hierarchy of local machines, edge
devices, and cloud servers. Because the capabilities of each machine
are different, the overall performance of a vertical hierarchy
depends on each machine. Generally, a cloud server has the most
powerful computing capabilities, followed by an edge device.
In contrast, in a horizontal configuration, multiple server machines
at the same level (e.g., several cloud server machines or several
edge devices) can serve the same AI inference service. In this
configuration, the AI service may show different performance
according to the load level of each server, the computing capability
of each server machine, and the link state between the local machine
and each server machine. Thus, to find the server machine that can
provide the best AI service, a network element is required to monitor
the network link state and the current computing capability of the
server machines, along with a load balancer that can enforce a
scheduling policy. When a client requests an AI inference service,
the request is routed to a single server machine based on the load
balancer's decision. Alternatively, the request can be processed by
several server machines that each contribute to the final inference
result.
The Computing-Aware Traffic Steering (CATS) framework [RFC10053]
provides an architecture for this kind of joint compute- and network-
aware selection of a service instance. The monitoring element and
the load balancer described above map to the CATS functional
components that collect computing metrics and network metrics and
select a service contact instance. AI inference services can benefit
from CATS, and AI-specific metrics (e.g., the model hosted by an
instance, its expected accuracy, and its current inference latency or
queue length) are candidate inputs to such selection.
These functions can be placed at different locations depending on the
deployment. First, they can be collocated with the edge device or
with the first network element that the request from the local
machine reaches (e.g., an access router or an edge gateway), which
allows the selection to be made close to the client and is suitable
when the candidate server machines are edge devices. Second, they
can be placed at the network element that interconnects the edge
network and the cloud (e.g., a gateway or an ingress router of a
provider network), which is suitable when the candidates include both
edge devices and cloud servers. Third, a logically centralized
controller can collect the computing and network metrics and compute
the selection policy, which is then enforced by the network elements
on the path. Finally, the local machine itself can select the server
Hong, et al. Expires 6 May 2027 [Page 12]
Internet-Draft Deploying AI services November 2026
machine based on information obtained from the network. In terms of
the CATS framework [RFC10053], the collection of computing metrics is
typically collocated with the server machines, the collection of
network metrics is performed by network elements or a controller, and
the selection of the service instance is performed either at the
ingress network element closest to the local machine or by a
centralized controller.
When utilizing horizontal multiple servers, Multi-Agent Systems (MAS)
based on Agentic AI can also be applied. Instead of relying solely
on a centralized load balancer, multiple AI agents, each deployed on
a separate server machine or edge device, can negotiate tasks and
collaborate toward a common objective. This structure allows a
complex AI task to be decomposed into sub-tasks, with each agent
specializing in a part, leading to a collectively processed final
result, improved resilience, and dynamic resource optimization. Such
collaboration requires agents to discover each other, exchange tasks
and results, and resolve conflicting decisions. The interoperability
of agent communication is being addressed by agent communication
protocols such as MCP [MCP] and A2A [A2A] and by the IETF AGENTPROTO
working group [AGENTPROTO].
The following figure shows the case where the local machine requests
an AI service from horizontal multiple cloud servers.
Hong, et al. Expires 6 May 2027 [Page 13]
Internet-Draft Deploying AI services November 2026
+--------------------------------------+
| +---------------------------+ |
| | +-----------------+ | |
+----+-----+---->| Server module | | |
| | | | for AI service | | |
| | | +-----------------+ | |
| | +---------------------------+ |
| | Server machine 1 |
| +--------------------------------------+
| Cloud (Internet)
|
| +--------------------------------------+
+------------------------+ | | +---------------------------+ |
| +-----------------+ | | | | +-----------------+ | |
| | Client module |<-+-+----+-----+---->| Server module | | |
| | for AI service |<-+-+ | | | for AI service | | |
| +-----------------+ | | | | +-----------------+ | |
+------------------------+ | | +---------------------------+ |
Local machine | | Server machine 2 |
| +--------------------------------------+
| Cloud (Internet)
|
| +--------------------------------------+
| | +---------------------------+ |
| | | +-----------------+ | |
+----+-----+---->| Server module | | |
| | | for AI service | | |
| | +-----------------+ | |
| +---------------------------+ |
| Server machine 3 |
+--------------------------------------+
Cloud (Internet)
Figure 6: AI inference service on horizontal multiple servers
4.6. Network-side utilization for AI learning
Collecting and preprocessing data and training an AI model require
high-performance resources such as CPU, GPU, power, and storage. To
mitigate this requirement, a network-side configuration can be
utilized. Federated learning is a machine learning technique that
trains an AI model across multiple decentralized servers, in contrast
to traditional centralized machine learning techniques where all
local datasets are uploaded to one server. Federated learning
enables multiple network nodes to build a common machine learning
model. These network nodes or servers can be located in the same
data center network or in multiple data center networks. The
security implications of federated learning, such as model poisoning,
Hong, et al. Expires 6 May 2027 [Page 14]
Internet-Draft Deploying AI services November 2026
are discussed in Section 9.
Transfer learning is a machine learning technique that focuses on
storing knowledge gained while solving one problem and applying it to
a different but related problem. In transfer learning, a network
configuration can be utilized to transfer common information and
knowledge between different network nodes within one data center
network or across multiple data center networks.
5. Considerations of network/system for AI services
As described in the previous section, the AI server module that
performs AI inference services can run on a local machine, a cloud
server, or an edge device.
In theory, if the AI inference service is performed on a local
machine, the AI service can be provided without communication delay
or packet loss, but a certain level of hardware performance is
required to perform AI inference. As AI services become widespread,
the cost of the machines that perform AI services becomes important.
Therefore, whether the AI inference service is performed on the cloud
server or on an edge device with lower infrastructure cost can be a
determining factor in the system configuration.
5.1. Considerations of the functional characteristics of the hardware
When an AI inference service request is made to a distant cloud
server, transmission may take a long time, but the cloud server can
process many AI inference requests in a short time and can host
larger models, which typically provide higher inference accuracy.
Conversely, when an AI service request is made to a nearby edge
device, the transmission time is short, but the edge device cannot
process as many requests at once and usually hosts smaller models
with lower inference accuracy. Note that the inference accuracy is
determined by the deployed model, not by the location itself.
Therefore, by analyzing the characteristics and requirements of the
AI service to be performed, it is necessary to determine whether to
perform the AI inference service on a local machine, a cloud server,
or an edge device.
The hardware characteristics of the machines performing AI services
vary. In general, cloud servers are regarded as having higher
performance than edge devices. However, the performance of an AI
inference service depends on how the hardware, such as CPU, RAM, GPU
or other AI accelerators, and network interfaces, is configured for
each cloud server and edge device. Without cost constraints, a
Hong, et al. Expires 6 May 2027 [Page 15]
Internet-Draft Deploying AI services November 2026
system for AI services would use the machines with the best hardware
performance, but in reality, cost must always be considered.
Therefore, the performance of the local machine, cloud server, and
edge device must be determined according to the characteristics and
requirements of the AI service to be performed.
Performance evaluation is possible through the performance metrics
presented by ETSI [MEC.IEG006]. These metrics are divided into two
groups. Functional metrics assess user performance and include
classical indexes such as latency in task execution, device energy
efficiency, bit rate, loss rate, jitter, and Quality of Service
(QoS). Non-functional metrics focus on the deployment and management
of the MEC (Multi-access Edge Computing, formerly Mobile Edge
Computing) network. Non-functional metrics include service life-
cycle (instantiation, service deployment, service provisioning,
service update (e.g., service scalability and elasticity), and
service disposal), service availability and fault tolerance
(reliability), service processing/computational load, global mobile
equipment host load, number of API requests (more generally, number
of events) processed per second on the mobile equipment host, delay
to process API requests (north and south), and number of failed API
requests. The sum of service instantiation, service deployment, and
service provisioning provides the service boot time.
For generative AI services, additional metrics are commonly used,
such as time to first token (TTFT), time per output token, token
throughput (tokens per second), energy consumed per inference or per
token, and cost per inference or per token. Energy consumption in
particular is becoming an important criterion when choosing an
execution point, together with accuracy, latency, and cost.
5.2. Considerations for the characteristics of the AI model
Although not directly related to communication or networking, the AI
model used is the biggest factor influencing an AI inference service.
For example, for image classification there are various AI models,
such as ResNet, EfficientNet, VGG, and Inception, and for language
tasks there are language models ranging from SLMs with a few billion
parameters or fewer to LLMs with hundreds of billions of parameters.
These models differ not only in inference accuracy but also in model
size, memory footprint, and inference time. Models with the highest
inference accuracy typically have very large sizes and long inference
times. Therefore, when constructing an AI service system, choosing
the model with the highest inference accuracy is not always
appropriate, and the model must be selected according to the
characteristics and requirements of the AI service.
Hong, et al. Expires 6 May 2027 [Page 16]
Internet-Draft Deploying AI services November 2026
Model compression techniques, such as quantization, pruning, and
knowledge distillation, can reduce model size and inference time at
some cost in accuracy, and allow a family of model variants to be
derived from one model. A common practice is to deploy a model with
high inference accuracy on the cloud server and a compressed or
smaller model that provides fast inference, with somewhat lower
accuracy, on the edge device.
How AI services are delivered on cloud servers or edge devices also
needs to be considered, although this is partly an implementation
issue. AI server modules can be built with general-purpose web
frameworks or with serving runtimes specialized for AI inference.
General-purpose web frameworks run on many types of machines, but
since they are designed for general web services, they do not provide
inference-specific optimizations. Specialized inference serving
runtimes provide optimizations such as request batching, hardware
acceleration, and, for LLMs, efficient management of the key-value
(KV) cache, but they may require specific hardware features (e.g.,
particular CPU instruction sets or GPUs). Therefore, the AI server
module method should be decided in consideration of the hardware
characteristics of the AI system that can be built.
5.3. Considerations for the characteristics of the communication method
The communication method used to transfer data when requesting an AI
inference service is also an important decision in constructing an AI
system. The traditional REST method can be used with various
machines and services, but its performance is generally inferior to
gRPC. gRPC uses HTTP/2 [RFC9113] and protocol buffers, providing low
latency, efficient data serialization, and efficient transfer of
large data compared to REST. QUIC [RFC9000] or other transport
protocols can also be used to request an AI inference service. For
generative AI services, the results are typically streamed token by
token over long-lived connections, and agent-based services may
maintain long-lived sessions between agents and tools. The selection
or specification of a particular transport protocol is beyond the
scope of this document.
Cloud-edge-endpoint collaboration-based AI service development is
actively underway. In particular, for AI services that are sensitive
to network delay, such as object recognition and autonomous vehicle
services, (micro)services for inference are placed on edge devices
close to the client to obtain fast inference results. As such, in
the development of intelligent IoT services, various devices that
provide computing services within the network, such as edge devices,
are being added as network elements, and the number of IoT devices
using them is rapidly increasing. Therefore, new functions for
computing resource management and operation are required to provide
Hong, et al. Expires 6 May 2027 [Page 17]
Internet-Draft Deploying AI services November 2026
computing services within the network. In addition, to operate
distributed AI services on the network, network policies for
collaboration are required between edge devices, or between edge
devices and endpoint devices, that provide computing resources for AI
services.
From a network policy perspective, to efficiently support distributed
AI services, the network needs to support the collaboration of AI
services between edge devices, such as multi-edge network
configurations for AI-service-aware traffic steering. This allows
the client to receive distributed AI services efficiently in various
network environments and allows network resources and edge computing
resources to be adjusted dynamically. The CATS framework [RFC10053]
is applicable to such AI-service-aware traffic steering. For
example, AI task messages must be exchangeable between edge devices,
or between edge devices and endpoint devices, to provide collective
AI inference. Another example is caching at edge devices between the
client and the central cloud: a copy of an accessed file or model is
kept at the edge device and served for subsequent requests from the
same or other clients. For generative AI services, caching can also
be applied to results of semantically similar requests and to
reusable inference state such as the KV cache of a shared prompt
prefix.
There are also various delay-sensitive AI services based on edge
devices in the network. They are divided into in-time delay-
sensitive AI services with a deadline and on-time delay-sensitive AI
services with a time range. In particular, on-time delay-sensitive
AI services need to return prediction results within the time range.
Therefore, distributed AI services must provide appropriate delay
performance through the network, and the client should be able to use
both in-time and on-time delay-sensitive services by interacting with
the edge devices where distributed AI services are built. To support
on-time delay-sensitive AI services, the underlying network may need
to interact with deterministic networking capabilities, such as
DetNet [RFC8655] or IEEE 802.1 Time-Sensitive Networking (TSN)
[IEEE-TSN], to guarantee bounded latency.
5.4. Considerations for generative AI and LLM inference workloads
Inference of generative AI models, in particular LLMs and SLMs, has
characteristics that differ from classical inference (e.g., security
risk identification, anomalous flow behavior detection, and
application identification, which are widely used in networks as
embedded or lightweight AI capabilities) and that affect where it
should be executed:
Hong, et al. Expires 6 May 2027 [Page 18]
Internet-Draft Deploying AI services November 2026
* Two-phase execution: LLM inference consists of a prefill phase,
which processes the whole input context and is compute-bound, and
a decode phase, which generates output tokens one by one and is
memory-bandwidth-bound. These phases may be placed on different
machines, in which case the intermediate state (KV cache) must be
transferred between them over the network.
* Asymmetric and variable traffic: the input context (e.g.,
documents, images, or retrieved knowledge) can be large in the
uplink, while the output is streamed in small increments over a
period that depends on the output length. The response time is
therefore not fixed in advance.
* Collaborative edge-cloud inference: an SLM at the edge or on the
local machine can generate candidate tokens that are verified by
an LLM in the cloud (speculative decoding), or simple requests can
be answered at the edge while complex ones are escalated to the
cloud. In such schemes, each round trip between the edge and the
cloud adds network latency, so the benefit depends on the RTT and
on how often escalation or verification is needed.
* Split inference: a model can be partitioned so that the first
layers run on the local machine or edge device and the remaining
layers run in the cloud. The size of the intermediate data at the
split point and the available bandwidth determine whether such a
split is beneficial.
* Context and state: multi-turn and agent-based services maintain a
growing context. Moving a session from one execution point to
another (e.g., due to mobility) requires either transferring this
context or recomputing it.
These characteristics mean that the network metrics relevant for
placing generative AI workloads include not only latency and loss but
also sustained bandwidth for context and state transfer and the
stability of long-lived streams.
5.5. Considerations for AI model distribution and lifecycle management
When AI inference is distributed over many edge devices and local
machines, the models themselves must be distributed and kept up to
date. Model files, including those of compressed models, range from
megabytes to many gigabytes, and adapters produced by fine-tuning add
further artifacts. Distributing them to a large number of inference
points consumes significant network resources and needs to be
planned, for example by scheduling updates, using caching or
multicast-like distribution at the edge, and transferring only the
differences between model versions where possible.
Hong, et al. Expires 6 May 2027 [Page 19]
Internet-Draft Deploying AI services November 2026
Consistency is also a concern. When a service uses models in the
cloud and at the edge together, the versions of these models need to
be compatible, and a mechanism to roll back to a previous version is
needed when a new version degrades the service. To support such
rollback, each deployed version of a model, including its adapters
and configuration, needs to be stored as a checkpoint or snapshot,
for example in a model registry, so that an inference point can
revert to any previously saved version. Where these checkpoints are
stored also matters: keeping recent versions cached at the edge
allows a fast rollback without transferring the model again from the
cloud, at the cost of additional storage at the edge. The integrity
and authenticity of distributed models need to be verifiable by the
inference points, as discussed in Section 9. Interoperability
challenges in deploying the same AI solution across different
environments are discussed in Section 7.2 of
[I-D.irtf-nmrg-ai-challenges].
5.6. Considerations for Agentic AI-driven autonomous offloading
Traditional offloading mechanisms predominantly rely on static, rule-
based thresholds (e.g., hardcoded values for network latency or
available bandwidth) to determine whether an AI task should be
executed locally, at the edge, or in the cloud. However, such
approaches may fail to adapt to highly dynamic network conditions,
volatile computing resource availability, and the nuanced accuracy
requirements of complex AI applications. To address these
limitations, distributed AI deployments can consider an Agentic AI-
driven autonomous offloading framework.
An Agentic AI-driven framework complements hardcoded logic with a
closed-loop reasoning system consisting of an LLM-based Intelligence
Core (the AI agent planning module), a Contextual Memory component,
and a Tool Use mechanism for environmental sensing:
* Intelligence Core and Planning: The LLM-based core interprets
high-level application goals or network intents, rather than
simple binary commands. When a complex AI task is requested, the
core performs task decomposition, breaking the objective into
sequential or parallel sub-tasks. It then formulates an execution
plan by evaluating trade-offs among model inference accuracy,
latency bounds (in-time/on-time sensitivities), and energy and
financial costs.
* Contextual Memory: The memory component maintains both short-term
environmental context and long-term historical knowledge. Short-
term memory tracks immediate telemetry fluctuations, while long-
term memory retains historical performance data of specific edge
and cloud nodes, previous offloading success rates, and semantic
Hong, et al. Expires 6 May 2027 [Page 20]
Internet-Draft Deploying AI services November 2026
context (e.g., geographical routing patterns in vehicular
networks). This allows the agent to anticipate network
degradation and make proactive, rather than reactive, offloading
decisions.
* Tool Use for Resource Sensing: To ground its reasoning in the
actual state of the system, the AI agent uses a predefined set of
tools, such as APIs and network management interfaces. Through
tool invocation, the agent queries the network link state (such as
RTT, packet loss rate, and jitter) and the processing capacity of
available nodes (such as CPU/GPU utilization, memory usage, and
queueing delay). Existing network telemetry [RFC9232] and YANG-
based management interfaces can be exposed to the agent as tools,
for example through an agent-to-tool protocol such as MCP [MCP].
By orchestrating planning, memory, and tool-based sensing, the AI
agent continuously perceives the cloud continuum. For example, it
can decide to withdraw a cloud offloading plan and fall back to a
lightweight on-device model when a tool detects a sudden drop in
wireless link quality, improving the resilience of the AI service.
The following points need to be considered when applying this
framework:
* Timescale separation: the reasoning of an LLM-based agent
typically takes from hundreds of milliseconds to several seconds,
which can exceed the deadline of the tasks being offloaded (e.g.,
about 33 ms per frame at 30 FPS in Section 7.1). Therefore, the
agent is better suited to a slow control loop that sets policies,
plans, and placements, while per-request or per-frame offloading
decisions are made by a fast control loop based on lightweight
rules or learned models that follow the policies set by the agent.
* Predictability and validation: LLM outputs are not deterministic
and may be incorrect. Decisions produced by the agent need to be
validated against constraints (e.g., resource limits and policies)
before they are enforced, and a safe default behavior (e.g., local
execution with a fallback model) needs to be defined for when the
agent fails or does not respond in time. Approaches such as the
incremental application of decisions with rollback and the
verification of decisions against safety bounds are discussed in
Section 5.5 of [I-D.irtf-nmrg-ai-challenges].
* Human oversight: actions with a significant impact on the network
or on other services need to be subject to approval, auditing, and
the possibility of intervention by operators, as discussed in
[I-D.cui-nmrg-llm-nm].
Hong, et al. Expires 6 May 2027 [Page 21]
Internet-Draft Deploying AI services November 2026
* Overhead: the agent itself consumes computing resources and
generates telemetry and tool traffic. This overhead needs to be
accounted for when evaluating the benefit of agent-driven
offloading.
6. Addressing challenges for coupling AI and NM
The document [I-D.irtf-nmrg-ai-challenges] describes the challenges
of coupling AI and network management (NM). It first characterizes
difficult NM problems with six problem characteristics (C1 to C6),
and then describes the high-level challenges in adopting AI in NM,
which are detailed in Sections 5, 6, and 7 of
[I-D.irtf-nmrg-ai-challenges]. This section describes how the
concepts discussed in the present document are linked to those
characteristics and challenges.
In particular, Section 5.4 of [I-D.irtf-nmrg-ai-challenges]
identifies distributed AI as a research challenge in two directions.
The first is network management for efficient distributed AI, in
which the communication requirements of a distributed AI application
are mapped to network requirements such as latency, bandwidth, and
confidentiality, leading to research on AI-aware network management
and network-aware AI applications. The second is distributed AI for
network management. The present document elaborates the first
direction for AI inference services and AI agents deployed across the
cloud continuum (Section 4 and Section 5), while the network digital
twin use case in Section 7.2 illustrates the second. In addition,
lightweight AI, as described in Section 5.3 of
[I-D.irtf-nmrg-ai-challenges], in which AI tasks are offloaded to
network devices while costly learning tasks remain on high-end
servers, corresponds to the placement of AI server modules on local
machines and edge devices described in Section 4.
The structures presented in this document can also be combined with
management systems that introduce AI functions into network
management cycles and express management goals as network intents
[RFC9315]. An example of such a framework is AINEMA
[I-D.pedro-nmrg-ai-framework], and an example of such a management
system is shown in [TNSM-2018].
6.1. Problem characteristics (C1-C6)
C1 (Fundamental Algorithmic Complexity) refers to a solution space
that explodes combinatorially with the scale and topological
footprint of the network. The present document proposes to tackle
part of such complexity by separating the AI work across multiple
elements of the network and by decomposing complex AI tasks into sub-
tasks handled by different servers or agents (Section 4.5). The
Hong, et al. Expires 6 May 2027 [Page 22]
Internet-Draft Deploying AI services November 2026
separation is asymmetrical, so that some elements (cloud side)
contribute more computation power to the distributed service.
C2 (Uncertainty and Unpredictability) refers to highly dynamic and
non-deterministic variables, such as shifting user traffic patterns,
limited visibility into end-to-end network state, and multi-domain
interdependencies. The present document addresses part of this
characteristic by confining AI models and data within pre-defined
client-edge-cloud structures, so that AI functions executed on local
machines or edge devices can rely on locally observed state rather
than on an incomplete global view. In addition, the uncertainty of
the outputs of AI agents themselves is mitigated by validating their
decisions against constraints and by defining safe fallback behaviors
(Section 5.6).
C3 (Real-Time Constraints and Determinism) refers to computing
solutions and executing closed-loop control actions within strictly
bounded or deterministic timeframes. The key aspect discussed in
this document to address this characteristic is the integration of
end-node, in-network, edge, and cloud computing services, which
allows AI computations to be performed in the best place possible.
The time needed to obtain responses for easy operations is minimized
by executing them in end nodes (e.g., vehicles) and edge devices,
while the time needed to compute more complex operations is minimized
by offloading them to cloud computing services. For on-time delay-
sensitive AI services, the network may interact with deterministic
networking capabilities (Section 5.3). Furthermore, the separation
between a slow, agent-driven control loop and a fast control loop
described in Section 5.6 allows Agentic AI to be used without placing
LLM reasoning on the critical path of time-sensitive decisions. In
Physical AI services (Section 7.3), this characteristic applies to
the entire closed loop from sensing to actuation.
C4 (Data Dependency and Heterogeneity) refers to processing and
analyzing massive volumes of network telemetry that are
heterogeneous, possibly incomplete, and generated at high velocity.
This characteristic is specific to each information domain and
problem. However, the difficulties in C4 can be relieved by data
processing functions, such as data homogenization, filtering, and
aggregation, instantiated in the cloud continuum and used
transparently by AI services. Moreover, processing data on edge
devices or network devices close to where it is generated reduces the
volume of data that needs to be collected and stored centrally
(Section 7.2).
C5 (Integration with Existing Practice) refers to interoperating with
pre-existing automated procedures, legacy deterministic control
loops, heterogeneous operator workflows, and established human-in-
Hong, et al. Expires 6 May 2027 [Page 23]
Internet-Draft Deploying AI services November 2026
the-loop structures. The flexibility of the structures presented in
this document allows them to be connected to existing systems. In
the agent-driven framework described in Section 5.6, the fast control
loop can be realized by existing deterministic mechanisms while the
AI agent adjusts their policies, and actions with a significant
impact are subject to operator approval. The result is exemplified
in the support for network digital twin and vehicle environments.
C6 (Economics and Computational Footprint) refers to ensuring cost-
effectiveness across limited physical and operational resources,
including link bandwidth, node CPU, and human overhead, particularly
when model training or inference is distributed at the network edge.
Enabling AI services to be deployed in the cloud continuum supports
the maximization of effectiveness by dynamically relocating services
to the cheapest point that is able to meet the computation
requirements. This relocation can be guided by intents [RFC9315] and
implemented by a management cycle that is independent of the AI
services, as shown in [TNSM-2018]. As a result, the cost of a
distributed AI system is continuously checked and adjusted by
requesting the underlying infrastructure to modify the point of
instantiation of each part of the AI system. The energy and cost
metrics in Section 5.1 can be used as inputs to such decisions, and
the overhead of model distribution (Section 5.5) and of the AI agents
themselves (Section 5.6) needs to be accounted for.
6.2. High-level challenges in adopting AI in NM
Section 4 of [I-D.irtf-nmrg-ai-challenges] describes three high-level
challenges in adopting AI in NM, which are detailed in Sections 5, 6,
and 7 of [I-D.irtf-nmrg-ai-challenges], respectively.
The first high-level challenge is that AI advances have historically
been driven by other communities, such as image/video and natural
language processing, so network-tailored AI has been overlooked and
AI techniques must be extensively adapted for network problems.
Although this is a complex challenge, this document supports its
resolution by enabling AI models and algorithms to evolve separately.
Multiple providers can offer multiple solutions, and they can be
stitched together by following the structures introduced above.
The second high-level challenge concerns network data as input to AI:
AI techniques require sufficiently representative data, and the
analysis of network data can expose sensitive information and raise
privacy concerns. Although the present document does not directly
address data representation, the structures presented in this
document help alleviate the burden by relocating some AI processes as
close as possible to the end points (e.g., vehicles and NDT). The
same structures support the implementation of measures to protect
Hong, et al. Expires 6 May 2027 [Page 24]
Internet-Draft Deploying AI services November 2026
privacy. The AI services that are closer to the edges deal with
sensitive data and ensure privacy protection by, for example,
anonymizing it. Those services are instantiated within the data
domain, so private data does not cross the boundaries of its data
domain.
The third high-level challenge concerns the seamless integration of
AI into existing NM processes and its acceptability to administrators
and operators. Giving administrators and operators control over AI
operations increases such acceptance. The structures presented in
this document support the involvement of administrators and operators
in AI system processes through the provision of policies and network
intents. In particular, aligning the distributed AI system described
in this document with intent-based networking [RFC9315] enables
administrators and operators to use high-level specifications (namely
network intents) to communicate the policies, requirements, and goals
that must be enforced on the AI system constructed over the cloud
continuum. For Agentic AI, the validation, human oversight, and
fallback mechanisms described in Section 5.6 further support
acceptance by keeping operators in control of the actions taken by AI
agents. These mechanisms correspond to the safeguards discussed in
Section 5.5 of [I-D.irtf-nmrg-ai-challenges], such as the incremental
application of AI decisions with rollback, verification against
safety bounds, and the use of digital twins as a sandbox, and to the
human-in-the-loop approaches discussed in Section 7.3 of
[I-D.irtf-nmrg-ai-challenges].
7. Use cases of deploying network-based AI services
7.1. Deploying AI services for self-driving vehicles
Various sensors are used in self-driving vehicles, and the final
judgment is made by combining their data. Among them, camera-based
object detection complements equipment such as LiDAR and RADAR.
Camera-based object detection performs various tasks: in addition to
lane recognition for keeping and changing lanes, it supports safe
driving and parking assistance by recognizing pedestrians, signs, and
parked vehicles along the road.
To perform such driving assistance and autonomous driving, object
detection needs to be performed in real time. A commonly used
minimum frame rate for real-time object detection in autonomous
driving is 30 FPS (Frames Per Second) [Object_detection]. No matter
how high the accuracy is, a detector cannot be used for autonomous
driving if it does not meet this reference value.
Hong, et al. Expires 6 May 2027 [Page 25]
Internet-Draft Deploying AI services November 2026
Task offloading refers to transferring computing tasks to other
processing devices or systems. Task offloading can quickly process
tasks that exceed the performance limits of a resource-constrained
device by delivering them from devices with limited computing power,
storage, and energy to devices that are rich in computing resources.
For a device with low hardware performance, such as an embedded
board, performing all operations locally without task offloading may
not achieve the frame rate required for object-detection-based
autonomous driving. On the other hand, if task offloading is applied
so that object detection is performed on a device with high hardware
performance and the rest of the work is performed on the client, the
required frame rate can be achieved, provided that the communication
delay between the client and the offloading target is sufficiently
small.
For AI services such as object detection that are difficult to
perform on resource-constrained devices, the task offloading
structure can thus be effective. However, offloading between network
nodes affects the end-to-end time, because larger data leads to
greater communication latency. In addition, a vehicle moves between
the coverage areas of different access points and edge devices, so
the offloading target may change during a service session, and the
context and state of the service may need to be transferred or
recomputed. Therefore, in such a distributed network environment, AI
services should be designed in consideration of data size, link
quality, mobility, and the capabilities of the available nodes.
Figure 7 shows an example of distributed AI deployment for a self-
driving vehicle that does not have enough capability to perform
object detection in real time and offloads some tasks to edge devices
and cloud servers.
Hong, et al. Expires 6 May 2027 [Page 26]
Internet-Draft Deploying AI services November 2026
+--------------------------------------+
+------------------------+ | +---------------------------+ |
| +-----------------+ | | | +-----------------+ | |
| |Object detection |<-+-+----+-----+---->|Object detection | | |
| | service |<-+-+ | | | service | | |
| +-----------------+ | | | | +-----------------+ | |
+------------------------+ | | +---------------------------+ |
Vehicle | | Edge device |
| +--------------------------------------+
| Edge network
|
| +--------------------------------------+
| | +---------------------------+ |
| | | +-----------------+ | |
+----+-----+---->|Object detection | | |
| | | service | | |
| | +-----------------+ | |
| +---------------------------+ |
| Server machine |
+--------------------------------------+
Cloud (Internet)
Figure 7: Distributed object detection service in self-driving
vehicle
7.2. Deploying AI services for network digital twins
Network digital twins also need distributed AI services. The purpose
of a network digital twin is described in
[I-D.irtf-nmrg-network-digital-twin-arch]. In particular, the
network digital twin provides network operators with technology that
enables data-driven network management and allows real-time
interaction between the physical network and the twin network. To
achieve this, the network digital twin uses AI capabilities for
various purposes such as scenario planning, impact analysis, and
change management.
Various AI functions will be applied for optimal network operation
and management. However, an actual physical network consists of
various network devices and has a complex structure with various
types of data, such as topology data, configuration data, and state
data. In addition, in a large-scale network environment, collecting
and storing information from many network devices in a centralized
manner, and creating and changing network management policies based
on it, causes very large and expensive network overhead.
Hong, et al. Expires 6 May 2027 [Page 27]
Internet-Draft Deploying AI services November 2026
Therefore, there is a need for a method to apply AI functions in a
distributed form for network operation and management. In
particular, since an actual physical network is built as a logical
hierarchical structure, it is necessary to apply a distributed AI
method that considers this logical hierarchical network structure.
To optimally perform network operation and management through
distributed AI methods, it is necessary to generate AI-based network
operation and management topology models and policy models, and an
operational method to distribute the generated AI-based network
policies across different networks or administrative domains. In
particular, to operate a network digital twin in a large-scale
network environment, AI-based network policy models need to be
generated in a distributed manner. A federated learning algorithm or
a transfer learning algorithm that can learn large-scale networks in
a distributed manner can be applied.
+-----------------------------------------------------+
| |
| Distributed network learning model |
| in large-scale network environment |
| |
| +-------------+------------+ |
| | Master | |
| | (AI based Policy model) | |
| +-------------+------------+ |
| | |
| +-----------+-----+-----+-----------+ |
| | | | | |
| +----+----+ +----+----+ +----+----+ +----+----+ |
| | Worker | | Worker | | Worker | | Worker | |
| | (Agent) | | (Agent) | | (Agent) | | (Agent) | |
| +----+----+ +----+----+ +----+----+ +----+----+ |
| | | | | |
| +----+----+ +----+----+ +----+----+ +----+----+ |
| | Local | | Local | | Local | | Local | |
| | Data | | Data | | Data | | Data | |
| | Repo- | | Repo- | | Repo- | | Repo- | |
| | sitory | | sitory | | sitory | | sitory | |
| +---------+ +---------+ +---------+ +---------+ |
| |
+-----------------------------------------------------+
Figure 8: Distributed learning model of network learning for
network digital twins
Hong, et al. Expires 6 May 2027 [Page 28]
Internet-Draft Deploying AI services November 2026
As shown in Figure 8, to learn a large-scale network through a
distributed learning method, a local data repository to store network
data must be established in each region, for example, based on
location or AS (Autonomous System). The distributed learning method
learns through each worker (agent) based on the local network data
stored in the local data repository, and generates a large-scale
network policy model through the master. This method can reduce the
network overhead of centralized data collection and storage, and
reduce the time required to create AI models for network operation
and management policies for large-scale networks. In addition, the
network policy model generated by a worker can be used as a locally
optimized network policy model that provides AI-based network
operation and management policy services optimized for local network
operations.
Trained AI network policy models can be deployed on network devices
that manage and operate the local network, to minimize network data
movement. For example, in a large-scale network consisting of
multiple ASes, AI network policy models can be deployed per AS to
optimize network operation and management. Figure 9 shows an example
of operating and managing a network by distributing AI network policy
models by AS.
+---------------------------------------------------------+
| +-------------+------------+ |
| | Master | |
| | (AI-based Network Policy | |
| | model management) | |
| +-------------+------------+ |
| | |
| +-------------+-------------+ |
| Worker | Worker | |
| +----------+----------+ +----------+----------+ |
| | Network device | | Network device | |
| | (AI-based network | | (AI-based network | |
| | Policy model) | | Policy model) | |
| +----------+----------+ +----------+----------+ |
| | | |
| +----------+----------+ +----------+----------+ |
| | AS_1 | | AS_2 | |
| +---------------------+ +---------------------+ |
+---------------------------------------------------------+
Figure 9: Distributed deployment of trained AI network policy models
The workers in Figure 8 and Figure 9 can also be realized as AI
agents that use the network digital twin as a tool, for example to
validate a candidate policy in the twin network before it is applied
Hong, et al. Expires 6 May 2027 [Page 29]
Internet-Draft Deploying AI services November 2026
to the physical network. An architecture that integrates Agentic AI
with the network digital twin is described in
[I-D.wmz-nmrg-agent-ndt-arch]. The considerations in Section 5 apply
to the placement of such agents and of the AI models they use across
the network.
7.3. Deploying AI services for Physical AI
Physical AI refers to AI systems that perceive the physical
environment through sensors, perform AI inference, and act on the
physical world through actuators. Examples include service and
industrial robots, drones, industrial automation systems, and
autonomous vehicles; the self-driving vehicle use case in Section 7.1
is one instance of Physical AI. Unlike conventional AI applications,
Physical AI continuously collects multimodal sensor data from
physical devices and generates control decisions that directly affect
physical systems. Therefore, the performance and reliability of
Physical AI services depend not only on the capabilities of AI models
but also on the availability, allocation, and coordination of the
underlying communication and computing resources.
A Physical AI service may consist of multiple distributed components,
including sensors, end devices, edge computing platforms, AI
accelerators, cloud infrastructures, and actuators. These components
cooperate to execute a closed loop of sensor data acquisition, data
processing, AI inference, decision generation, and control command
delivery, as shown in Figure 10. To meet application-specific timing
requirements, the supporting infrastructure needs to provide
predictable communication and computing performance across the entire
execution path of this loop, not only on its individual segments. In
particular, end-to-end latency, latency variation (jitter), packet
loss, and computing execution time need to be considered together
when determining whether the required service performance can be
achieved. The metrics described in Section 5.1 and the in-time and
on-time delay sensitivity described in Section 5.3 therefore apply to
the loop as a whole.
Hong, et al. Expires 6 May 2027 [Page 30]
Internet-Draft Deploying AI services November 2026
+----------------------+ +------------------------+
| Physical device | Sensor data | Edge device |
| (robot, drone, etc.) | (video, LiDAR, | |
| | measurements) | +----------------+ |
| +--------------+ |--------------->| | AI inference | |
| | Sensors | | | | and decision | |
| +--------------+ | | +----------------+ |
| +--------------+ | Control | |
| | Actuators | | commands | |
| +--------------+ |<---------------| |
+----------------------+ +-----------+------------+
|
Model updates, | Telemetry,
policies | non-time-critical
| tasks
|
+-----------+------------+
| Cloud server |
| (training, planning, |
| global coordination) |
+------------------------+
Figure 10: Closed-loop Physical AI service across a physical
device, an edge device, and a cloud server
The communication network supporting a Physical AI service carries
heterogeneous traffic within the same service, including high-
resolution video streams, LiDAR data, sensor measurements, control
messages, telemetry, and AI inference requests. These traffic types
have different requirements in terms of bandwidth, latency,
reliability, and priority. Traffic classification and differentiated
forwarding, as in the Differentiated Services architecture [RFC2475],
QoS-aware resource allocation, and scheduling mechanisms can be used
to satisfy these requirements. For latency-sensitive and safety-
critical operations, such as the delivery of control commands,
resource reservation, admission control, and traffic prioritization
can be used to maintain predictable communication performance under
varying network conditions, for example by using the deterministic
networking capabilities described in Section 5.3.
In addition to communication resources, Physical AI services require
distributed computing resources, such as CPUs, GPUs, NPUs, memory,
storage, and other AI accelerators deployed across end devices, edge
platforms, and cloud infrastructures, that can execute data
processing and AI inference within application-specific time
constraints. Depending on the computational complexity of AI
workloads and their latency requirements, AI functions can be
executed locally or distributed across multiple computing
Hong, et al. Expires 6 May 2027 [Page 31]
Internet-Draft Deploying AI services November 2026
environments, as described in Section 4. Since cloud-based execution
may introduce additional communication delay and variability,
latency-sensitive functions, such as perception and control in the
closed loop, are preferably executed close to the physical devices,
while functions with looser timing constraints, such as model
training, task planning, and global coordination, can be executed in
the cloud. The computing infrastructure therefore needs to support
workload placement, execution scheduling, and dynamic resource
allocation based on computing requirements, resource availability,
and network conditions, for which the CATS framework [RFC10053] is
applicable (Section 4.5). When AI agents are used for task planning
in Physical AI, the timescale separation described in Section 5.6
applies, so that the reasoning of the agent is kept outside the
closed control loop.
8. IANA Considerations
This document has no IANA actions.
9. Security Considerations
Distributing AI services across local machines, edge devices, and
cloud servers increases the attack surface compared to a centralized
deployment. Even an AI service performed entirely on a local machine
is not free from security issues: the model and its inputs can be
manipulated, and the machine itself can be compromised. The
following threats need to be considered:
* Exposure of AI service endpoints: when AI services are provided
through cloud servers or edge devices, their endpoints are
reachable from the network and can be targets of unauthorized
access and denial-of-service attacks. Inference requests can be
computationally expensive, so resource exhaustion attacks can be
effective even at low request rates. Authentication,
authorization, and rate limiting of AI service requests are
needed.
* Confidentiality and integrity of offloaded data: data offloaded
from local machines to edge devices or cloud servers (e.g., camera
images from vehicles, or network telemetry) can be sensitive. It
needs to be protected in transit (e.g., by TLS [RFC8446] or QUIC
[RFC9000]) and at the execution point.
* Attacks on AI models: adversarial inputs can cause incorrect
inference results, and repeated queries can be used to extract a
model or to infer information about its training data. Models
deployed on edge devices, which may be physically accessible, are
particularly exposed to theft and tampering.
Hong, et al. Expires 6 May 2027 [Page 32]
Internet-Draft Deploying AI services November 2026
* Integrity of distributed models: models and their updates
distributed to many inference points (Section 5.5) can be tampered
with in transit or at rest. Inference points need to verify the
integrity and origin of models (e.g., by digital signatures)
before using them.
* Attacks on distributed learning: in federated learning
(Section 4.6, Section 7.2), a compromised participant can send
poisoned updates that degrade or bias the global model. Robust
aggregation and the authentication of participants are needed.
* Risks specific to Agentic AI: AI agents that invoke tools and act
on the network introduce additional risks. Content obtained
through tools (e.g., telemetry, logs, or documents) can contain
instructions crafted to manipulate the agent (prompt injection),
so such content needs to be treated as data, not as instructions.
Agents need to be identified and authenticated, and granted only
the minimum privileges necessary for their tasks. Actions with a
significant impact on the network need to be validated, logged for
auditing, and, where appropriate, approved by operators
(Section 5.6). In multi-agent systems, messages from other agents
need to be authenticated, and the impact of a compromised agent
needs to be contained.
* Safety of Physical AI: in Physical AI services (Section 7.3),
incorrect inference results, manipulated sensor inputs, or delayed
or lost control commands can directly cause physical harm. In
addition to protecting sensor data and control messages, physical
devices need to transition to a safe state when control commands
are not received in time or when communication with the edge
device or cloud server is lost.
AI inference requests often carry personal or confidential data, such
as images of people and vehicles, location information, user prompts,
or operator network data, and where AI inference is executed
determines which parties can access this data. Executing inference
on the local machine or on an edge device within the same
administrative or data domain can prevent such data from leaving that
domain, as discussed for the second high-level challenge in
Section 6.2. When data needs to be offloaded to another domain, data
minimization, such as sending extracted features instead of raw data
or anonymizing data, needs to be considered. The context and memory
maintained by AI agents, the caches maintained at edge devices, and
the telemetry collected to support offloading decisions may also
contain personal or confidential data, such as location and mobility
patterns, and their retention and access need to be controlled.
Hong, et al. Expires 6 May 2027 [Page 33]
Internet-Draft Deploying AI services November 2026
When providing AI services by utilizing machines in the network such
as cloud servers and edge devices, it is necessary to analyze the
characteristics of the modules to be used, identify their security
vulnerabilities, and take countermeasures.
10. Acknowledgements
Experts from the IETF/IRTF NMRG provided valuable input during the
development of this document.
11. Informative References
[A2A] Linux Foundation, "Agent2Agent (A2A) Protocol",
.
[AGENTPROTO]
IETF, "Agent Communication Protocols (agentproto) Working
Group", October 2026,
.
[AI_inference_architecture]
IBM, "IBM Systems, AI Infrastructure Reference
Architecture",
.
[CG-AIoT] ITU-T, "ITU-T CG-AIoT", .
[EDGE-AI] EDGE AI FOUNDATION, "EDGE AI FOUNDATION (formerly tinyML
Foundation)", .
[I-D.cui-nmrg-llm-nm]
Cui, Y., Xing, M., and L. Zhang, "A Framework for LLM
Agent-Assisted Network Management with Human-in-the-Loop",
Work in Progress, Internet-Draft, draft-cui-nmrg-llm-nm-
02, 1 July 2026, .
[I-D.irtf-nmrg-ai-challenges]
François, J., Clemm, A., Papadimitriou, D., Fernandes, S.,
and S. Schneider, "Research Challenges in Coupling
Artificial Intelligence and Network Management", Work in
Progress, Internet-Draft, draft-irtf-nmrg-ai-challenges-
06, 6 July 2026, .
Hong, et al. Expires 6 May 2027 [Page 34]
Internet-Draft Deploying AI services November 2026
[I-D.irtf-nmrg-network-digital-twin-arch]
Zhou, C., Yang, H., Duan, X., Lopez, D., Pastor, A., Wu,
Q., Boucadair, M., and C. Jacquenet, "Network Digital Twin
(NDT): Concepts and Reference Architecture", Work in
Progress, Internet-Draft, draft-irtf-nmrg-network-digital-
twin-arch-13, 1 July 2026,
.
[I-D.pedro-nmrg-ai-framework]
Martinez-Julia, P., Homma, S., and D. Lopez, "Artificial
Intelligence Framework for Network Management", Work in
Progress, Internet-Draft, draft-pedro-nmrg-ai-framework-
05, 20 October 2024,
.
[I-D.wmz-nmrg-agent-ndt-arch]
Wu, Q., Zhou, C., Contreras, L. M., Han, S., and Y. Hong,
"Network Digital Twin and Agentic AI based Architecture
for AI driven Network Operations", Work in Progress,
Internet-Draft, draft-wmz-nmrg-agent-ndt-arch-04, 21 May
2026, .
[IEEE-TSN] IEEE 802.1, "Time-Sensitive Networking (TSN) Task Group",
.
[MCP] Agentic AI Foundation, "Model Context Protocol",
.
[MEC.IEG006]
ETSI, "Mobile Edge Computing; Market Acceleration; MEC
Metrics Best Practice and Guidelines", ETSI GS MEC-IEG 006
V1.1.1, January 2017.
[Object_detection]
Lewis, G., "Object Detection for Autonomous Vehicles",
2016.
[RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.,
and W. Weiss, "An Architecture for Differentiated
Services", RFC 2475, DOI 10.17487/RFC2475, December 1998,
.
[RFC6574] Tschofenig, H. and J. Arkko, "Report from the Smart Object
Workshop", RFC 6574, DOI 10.17487/RFC6574, April 2012,
.
Hong, et al. Expires 6 May 2027 [Page 35]
Internet-Draft Deploying AI services November 2026
[RFC7452] Tschofenig, H., Arkko, J., Thaler, D., and D. McPherson,
"Architectural Considerations in Smart Object Networking",
RFC 7452, DOI 10.17487/RFC7452, March 2015,
.
[RFC8446] Rescorla, E., "The Transport Layer Security (TLS) Protocol
Version 1.3", RFC 8446, DOI 10.17487/RFC8446, August 2018,
.
[RFC8655] Finn, N., Thubert, P., Varga, B., and J. Farkas,
"Deterministic Networking Architecture", RFC 8655,
DOI 10.17487/RFC8655, October 2019,
.
[RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based
Multiplexed and Secure Transport", RFC 9000,
DOI 10.17487/RFC9000, May 2021,
.
[RFC9113] Thomson, M., Ed. and C. Benfield, Ed., "HTTP/2", RFC 9113,
DOI 10.17487/RFC9113, June 2022,
.
[RFC9232] Song, H., Qin, F., Martinez-Julia, P., Ciavaglia, L., and
A. Wang, "Network Telemetry Framework", RFC 9232,
DOI 10.17487/RFC9232, May 2022,
.
[RFC9315] Clemm, A., Ciavaglia, L., Granville, L. Z., and J.
Tantsura, "Intent-Based Networking - Concepts and
Definitions", RFC 9315, DOI 10.17487/RFC9315, October
2022, .
[RFC9556] Hong, J., Hong, Y., de Foy, X., Kovatsch, M., Schooler,
E., and D. Kutscher, "Internet of Things (IoT) Edge
Challenges and Functions", RFC 9556, DOI 10.17487/RFC9556,
April 2024, .
[RFC10053] Li, C., Ed., Du, Z., Boucadair, M., Ed., Contreras, L. M.,
and J. Drake, "A Framework for Computing-Aware Traffic
Steering (CATS)", RFC 10053, DOI 10.17487/RFC10053,
October 2026, .
Hong, et al. Expires 6 May 2027 [Page 36]
Internet-Draft Deploying AI services November 2026
[TNSM-2018]
Martinez-Julia, P., Kafle, V. P., and H. Harai,
"Exploiting External Events for Resource Adaptation in
Virtual Computer and Network Systems", IEEE Transactions
on Network and Service Management Vol. 15, No. 2, pp.
555-566, 2018.
Authors' Addresses
Yong-Geun Hong
Daejeon University
62 Daehak-ro, Dong-gu
Daejeon
34520
South Korea
Phone: +82 42 280 4841
Email: yonggeun.hong@gmail.com
Joo-Sang Youn
DONG-EUI University
176 Eomgwangno Busan_jin_gu
Busan
614-714
South Korea
Phone: +82 51 890 1993
Email: joosang.youn@gmail.com
Seung-Woo Hong
ETRI
218 Gajeong-ro Yuseong-gu
Daejeon
34129
South Korea
Phone: +82 42 860 1041
Email: swhong@etri.re.kr
Pedro Martinez-Julia
National Institute of Information and Communications Technology
4-2-1, Nukui-Kitamachi, Koganei, Tokyo
184-8795
Japan
Phone: +81 42 327 7293
Email: pedro@nict.go.jp
Hong, et al. Expires 6 May 2027 [Page 37]
Internet-Draft Deploying AI services November 2026
Qin Wu
Huawei
101 Software Avenue, Yuhua District
Nanjing
210012
China
Email: bill.wu@huawei.com
Hong, et al. Expires 6 May 2027 [Page 38]