Vanilla Netrek Server Development Archive
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[VANILLA-L:1014] Client to Metaserver Recoding

I'm about 80% done on this.
Want technical comments.
It's a big read, but please do.

Introduction
------------

This adjustment to the metaserver and client code will allow for;

	(a) faster fetching of the server list,
	(b) simultaneous communication with multiple metaservers,
	(c) factor of four reduction in metaserver/client packet load.

In short, the client will use UDP to the metaservers to request a server
listing, and the metaservers will send the server list back by UDP to
the client.


Code Changes
------------

Code is added to the metaserver that will handle a different packet
type on the 3521 UDP port that the servers use.  The packet is one byte
long, and consists of a question mark "?".  The metaserver dumps the
server list into a UDP packet to send back to the client.  Very simple
stuff.  [and I've finished coding it]

Code is also added to the client that will resolve the multiple
addresses of metaserver.netrek.org.  For each address, a UDP packet "?"
will be sent to the metaserver.  The client will expect to receive a
reply from each metaserver, and present the resulting merged list to the
user.

The client will wait for up to ten seconds in the case of no response
from _any_ metaserver.  So if the network is broken between the client
and the rest of the 'net, ten seconds is the delay.

However, if at least one metaserver responds, the delay will be trimmed
to two seconds.  So when one or more metaservers are down, the login
time will be extended by two seconds.

Further, if all metaservers contacted respond, the client will not delay
and will immediately continue presenting the merged list.

[the client code is done up-to but not including the server list
merging]


Client/Metaserver UDP Protocol
------------------------------

The protocol is;

client to metaserver, UDP packet, source port any, destination port 3521

        "?"

metaserver to client, UDP packet, from port 3521 to client's port

	"r,count-of-servers\n"
        "name-of-server,port-number,player-count,queue-size,type\n"
        "name-of-server,port-number,player-count,queue-size,type\n"

It's a comma delimited set of lines.  One header line and one line per
server.  Sent as one UDP packet.  The header includes the count of
servers in order to avoid complex malloc/realloc stuff in the client.


Packet Analysis
---------------

Packet analysis of existing TCP/IP client to metaserver communication
... edited output from tcpdump, removed DNS and arp packets.  This
intranet metaserver lists three active servers.

0.941697 doosh.1026 > ogg.3521: S 3980:2974 523980(0) win 512
0.941697 ogg.3521 > doosh.1026: S 4000:2090 944000(0) ack 3981 win 33580
0.941697 doosh.1026 > ogg.3521: . ack 1 win 32120 (DF)
1.029537 ogg.3521 > doosh.1026: P 1:917(916) ack 1 win 33580
1.031489 ogg.3521 > doosh.1026: F 917:917(0) ack 1 win 33580
1.031489 doosh.1026 > ogg.3521: . ack 918 win 32120 (DF)
1.045153 doosh.1026 > ogg.3521: F 1:1(0) ack 918 win 32120
1.046129 ogg.3521 > doosh.1026: . ack 2 win 33580

Packet sequence for new protocol ...

0.941697 doosh.5454 > ogg.3521: udp 1
1.029537 ogg.3521 > doosh.5454: udp 90

The client-friendly format is fairly compressed when compared to the
existing TCP protocol.  The savings come from both the swap to UDP, and
the conversion to an easily parsable format.

metaserver.us.netrek.org just then delivered me 2778 bytes, and another
metaserver delivered me 2790 bytes.  That's 2.8k delivered to each
client when they start, most likely as more than one packet, unlike the
trace above.  Converted to the new protocol, this 2.8k  bytes reduces to
784 bytes; easily enough to fit in one Ethernet packet.  A 70%
compression.

Concerns;
	- effect of packet loss,
	- effect of packet fragmentation,
	- scheduling of changes,
	- default behaviour of clients.


Effect of Packet Loss
---------------------

If the packet loss is confined to the client side of the network, then
chances are they won't want to play anyway.

We have three metaservers.  Loss to any one of the three will simply
cause a two second delay for the client.  Loss to all three would
indicate a pathological situation that would typically affect the
client's connection to servers as well.


Effect of Packet Fragmentation
------------------------------

If the server list grows considerably from it's current size, then the
UDP reply packet may reach the magic 1460 value that corresponds to the
10mb/s Ethernet fragmentation point.

When this happens, the chances of packet loss are increased, because
there are two packets in flight instead of one.


Scheduling of Changes
---------------------

The metaserver guys can dictate the timing of the changes, and anyway,
I'm not proposing that we remove the old TCP based stuff.  All we have
to say to rec.games.netrek, when we are done, is that the new method
will work faster and better.

We have to wait for Frank to come back from vacation anyway.  ;-)


Default Behaviour of Clients
----------------------------

Once we have this working fine on all the metaservers, I'll be proposing
the patch to the COW list, along with a default to use the new protocol,
but with a fallback to the old protocol.  This fallback can also be
forced with a new -M flag.


-- 
James Cameron                              (james.cameron@digital.com)
Digital Equipment Corporation (Australia) Pty. Ltd. A.C.N. 000 446 800
+
++ Vanilla-l Mailing List ++
To unsubscribe: send "unsubscribe vanilla-l" to majordomo@real-time.com
For more information: http://archives.real-time.com
Follow-Ups: