Distributed AntNet: Stigmetric Control for Communications Networks Gianni Di Caro & Marco Dorigo Journal of Artificial Intelligence Research 1998
Presentation by Tavaris Thomas
Presentation Contents
Introduction/Background Model Description AntNet: An Adaptive Agent-based Routing Algorithm Other Routing Algorithms Experimental Networks Used Results Conclusions and Future Work
Introduction/Background
Increase in the supply and demand of network communication services Network Control online and off-line monitoring and management of the network resources Routing process or method of determining and prescribing incoming packets to an outgoing path (forwarding messages)
Swarm Intelligence (SI)
New research field Collective behavior of social insects and other organisms
ants, honey bees states/actions
Stimergy Complex and intelligent behavior performed through the interaction of thousands of autonomous swarm members
Ant Colony Optimization(ACO)
Foraging behavior of ants and is used successfully to solve combinatorial optimization problems.
traveling salesman genome matching routing in telecommunications networks load balancing
Model Description
WAN Irregular topology connection-less network Network communication is mapped on a directed weighted graph with N processing/forwarding nodes Links characterized by bandwidth (bit/sec) and transmission delay (sec) 2 types of packets (routing and data) routing have greater priority C++ based discrete event driven simulator
AntNet
Adaptive, distributed, and mobile agentbased routing algorithm Reinforcement learning problems with hidden state (Bertsekas & Tsitsiklis, 1996; Kaelbling, Littman, & Moore, 1996; McCallum, 1995).
AntNet Algorithm Overview
Mobile agents are asynchronously launched towards randomly selected destination nodes. Each agent searches for a minimum cost path joining its source and destination nodes. Each agent moves step-by-step towards its destination node. At each intermediate node a greedy stochastic policy is applied to choose the next node to move to. The policy makes use of (i) local agentgenerated and maintained information, (ii) local problem-dependent heuristic information, and (iii) agent-private information. While moving, the agents collect information about the time length, the congestion status and the node identifiers of the followed path.
AntNet Algorithm Overview
Once they have arrived at the destination, the agents go back to their source nodes by moving along the same path as before but in the opposite direction. During this backward travel, local models of the network status and the local routing table of each visited node are modified by the agents as a function of the path they followed and of its goodness. Once they have returned to their source node, the agents die.
Routing Table Contents
nNk
P
Pnd Tk
nd
= 1, d [1, N ]
N k = { neighbors (k )}
Goodness (desirability) Routing table
2 M k d , d ,Wd
(
)
Mean, variance, and best
Array of ds defining parametric statistical model for the traffic distribution over the network as seen by local node k
AntNet Algorithm
The heuristic correction ln is a [0,1] normalized value proportional to the length qn (in bits waiting to be sent) of the queue of the link connecting node the k with its neighbor n: The value of alpha weights the importance of the heuristic correction with respect to the probability values stored in the routing table. Agent's decisions are taken on the basis of a combination of a long-term learning process and an instantaneous heuristic prediction. Ideal alpha between 0.2 and 0.5
AntNet Algorithm
The backward ant updates the routing table and arrays stored at each node as it propagates through network.
Pfd ' Pfd ' + r (1 - Pfd ' )
Positive reinforcement Negative reinforcement
Pnd ' Pnd ' - rPnd ' , n N k , n f
r r (T , M k )
r ( 0,1]
Reinforcement to be a function of the goodness where
Other Routing Algorithms Compared
OSPF (static, link state)Open Shortest Path First SPF (adaptive, link-state) Shortest Path First BF (adaptive, distance-vector) Bellman Ford Q-R (adaptive, distance-vector): Q-Routing PQ-R (adaptive, distance-vector): is the Predictive Q-Routing algorithm Daemon (adaptive, optimal routing): is an approximation of an ideal algorithm
Networks Used
SimpleNet (1.9, 0.7, 8)
10Mbit/s and propagation delay of 1msec mean shortest path distance, in terms of hops, between all pairs of nodes, the variance Of this average, and the total number of nodes
Networks Used
NFSNET(2.2,0.8,14)
1.5Mbps propagation delays 4-20 msec
Networks Used
NTTnet(6.5,3.8,57) 6Mbps propagation delay 1 to 5 msec
Metrics for Performance Evaluation
Throughput Delay Distribution- the authors used whole
empirical distribution or to use the 90th percentile statistic, which allows one to compare the algorithms on the basis of the upper value of delay they were able to keep the 90% of the correctly delivered packets
Network Capacity Usage (as expressed by the as the sum of the link capacities divided total available link capacity)
SimpleNet Throughput Results
SimpleNet: Comparison of algorithms for FCBR traffic directed from node 1 to node 6) The delay distribution showed similar...