Network Architectural Design for Networked Gaming - Part 1

Part 1:  Network Hardware and Cabling Requirements 

Hardware Selection 

So we can just plug in some network equipment, connect the slot machines and servers and deploy "server-based gaming", right? 

 

In a word, no.  In networking, as in most things, you get what you pay for.  Slot machines or EGMs, associated player tracking and slot accounting systems, as well as TITO systems, and a variety of other networked systems and applications are all critical to a casino's operation today.  The network that supports these systems must have very high availability, particularly as more and more of the critical systems and applications for casino operation are connected to the "networked casino floor". 

Ask yourself: 

  • "How important is it that the network be operational at 7am?  5pm? 10pm?" 
  • "What are the consequences and effects of a wide-spread network failure at say, 10pm on Friday night?" 
  • "What happens to progressive slot systems which use the network for communication if there is a network failure?" 
  • "How much revenue is lost if a bank of popular and well-placed slots were to fail and stay down for an extended period?  4 hrs?  12 hrs?"     

 These questions highlight the impact a network failure could have on a server-based gaming casino floor.  Several years ago, while working for Aristocrat Technologies, I asked myself some of these same questions while developing the Aristocrat recommendations for building ethernet casino floor networks to support the Aristocrat Sentinel 3 player tracking system EGM monitor device and to support server-based gaming or "downloadables", as they are sometimes called.  Aristocrat refined these over the years and provides their recommendations to customers if you ask for them. I expect that IGT, Bally Technologies and some of the others do as well.

 

My recommendation is to build casino floor networks using Cisco equipment, and use Layer 3 access switches (3560 or better) as bank switches, building a Layer 3 routed network design.  I'll get into the particulars in a later section, but suffice it to say:  You want to build a converged network on the casino floor, right?  One that can support server-based gaming, player tracking and slot accounting systems, TITO, IP telephony, IP video surveillance, and potentially wireless gaming and wireless guest access, not to mention future cashless kiosk machines, all on a single network?

Impossible you say?   No, very possible, and able to comply with regulatory compliance segmentation requirements while sharing a common physical network.  This is the power of the so-called Network Virtuailization  effort on the part of Cisco and some otehr vendors (notably Juniper's Infranet initiative). 

 

As a casino operator, what sort of network hardware vendor would you want to bet your business on?  The network vendor that spends $3B (that's billion with a B) on R&D, or the one that spends $2M?  The one whose main focus is building server hardware?  The one whose main  customer market is home-users?  The one with hundreds of thousands of network engineers skilled in use of their products?  Or the vendor that has maybe a few hundred?   Does this lack of traned personnel mean the products are bad?  No, but it does mean that it could cost your business more to support a network  if you have trouble finding personnel trained on the equipment.   What's the ROI on network hardware replaced early for lack of features to build the converged network described above?

 

GSA Network Wiring Suggestions 

The GSA recently released their  "GSA Network Wiring Suggestions for Implementing G2S v1.0 ", dated 11/07/2007.  While I think it is a good start, I believe that the recommendations fall short of what is needed in the long term.

 

What the GSA got right 
The GSA put several recommendations in the document which I agree with:
  • Layer 3 switches as bank switches capable of multicast and multicast routing protocols - Surprise!  Multicast is functionality that G2S uses for particular communication functions (notably, progressive communication). What is multicast?   Essentially, a specialized form of broadcast where a subset of endpoints forms a multicast group.   Servers must be behind stateful firewalls per regulatory compliance dictates in NV GCB Reg 14 and GLI-21/GLI-11/GLI-13.  Multicast routing capability will be required to use multicast between the server and EGMs, as they are at least a Layer 3 hop apart.  While it is possible to create multicast capable L2 networks, doing so on low-end equipment without adequate features to control flooding of multicast group and IGMP messages will result in a very poor-performing network.
  • 802.1x for access and distribution/core switches - IEEE 802.1x is a means of performing port-based Network Access Control, based on who or what the device attached to the network is.   see http://en.wikipedia.org/wiki/802.1x  for some background.   What this comes down to is being able to allow or block access to the casino floor network based on the EGM or other attached device being able to authenticate properly... This is a very critical capability to allow access to EGMs, but deny access to an intruder attempting to compromise the server-based gaming systems

 

 What's Missing from the GSA's Recommendations or what should be changed?

  • No mention of critical network and security management capabilities... NTP, AAA via RADIUS/TACACS+,  SSHv2 in-band CLI access, SNMP status monitoring, central syslog, central configuration change management and audit capabilities, configuration archive
  • Document title is GSA Network Wiring Suggestions, however, the document delves into firewalls, IPS, multicast routing 802.1x in addition to discussing cabling, yet it stops short of providing the whole network and security management picture.
  • TACACS+ should be recommended as the sole or preferred administrative Authntication, Authorization and Accounting (AAA) protocol for network equpment... TACACS+ provides an audit trail or who did what when, in addition to the authentication of username/password that RADIUS and LDAP-based AAA may do.  Most network equipment supports TACACS+ and RADIUS, yet relatively little supports LDAP.  The bottom line is that TACACS+ provides a central AAA system which provides an audit trail and may replicate between geophratically dispersed AAA servers... try that with MS AD-based IAS RADIUS servers
  • Fiber distances are misleading... the distance overwhich communication via fiber works is highly dependent upon the  core size and modal bandwidth of the fiber and the operating wavelength of the fiber optic module (GBIC or SFP).  Compare the distance for 62.5 micron 160 Mhz/km 850nm   to the distance using LX/LH SFP modules... LX/LH modules are of course more than double the price of SX modules.

Here is some information from:   http://www.cisco.com/en/US/prod/collateral/modules/ps5455/ps6577/product_data_sheet0900aecd8033f885.html

Product

Wavelength (nm)

Fiber Type

Core Size (μm)

Modal Bandwidth (MHz* Km)

Operating Distance (m)

1000BASE-SX

850

MMF

62.5

160

220 (722 ft)

62.5

200

275 (902 ft)

50

400

500 (1,640 ft)

50

500

550 (1,804 ft)

1000BASE-LX/LH

1300

MMF*

62.5

500

550 (1,804 ft)

50

400

550 (1,804 ft)

50

500

550 (1,804 ft)

SMF

-**

-

10,000 (32,821 ft)

1000BASE-ZX

1550

SMF

-

-

Approximately 70 km depending on link loss

1000BASE-BX-D

1310

SMF

-**

-

10,000 (32,821 ft)

1000BASE-BX-U

1490

SMF

-**

-

10,000 (32,821 ft)

 

  •  No discussion of 10Gbps.  10Gbps Fiber?   yes... any network which converges multiple applications such as VoIP, server-based gaming, video signage, streaming video-on-demand, and IP Video surveillance will be congesting uplinks, particularly when doing very high-quality video surveillance over the WAN.    Couldn't we just bundle multiple 1Gbps interface into a port-channel bundle?   yes, HOWEVER,   four 1Gbps in a port-channel bundle is not the same as 4 Gbps... the port-channel load-balancing algorithm does a sort of round-robin of flows on the links in the bundle, so the maximum a flow may use is 1Gbps.  In larger layer 2 access-based networks, spanning tree logical ports due to number of trunks and numbers of vlans  could become a critical issue.
  • 10Gbps  should probably be discussed and recommended...   Cabling is capital infrastructure improvement which must be depreciated over a relatively long cycle... 5-7 years or more... With this sort of life, longevity and ROI is likely as important as initial capital cost.
  • 10Gbps distances are shorter than 1Gbps on the same fiber...  see the chart below from:

 http://www.cisco.com/en/US/prod/collateral/modules/ps5455/ps6574/product_data_sheet0900aecd801f92aa.html

Table 1. X2 Port Cabling Specifications

Cisco X2

Wavelength (nm)

Cable Type

Core Size (microns)

Modal Bandwidth (MHz*km)

Cable Distance*

Cisco X2-10GB-CX4

-

CX4 (copper)

-

-

15m

Cisco X2-10GB-SR

850

MMF

62.5

62.5

50.0

50.0

50.0

160

200

400

500

2000

26m

33m

66m

82m

300m

Cisco X2-10GB-LRM

1310

MMF

62.5

50.0

50.0

500

400

500

220m

100m

220m

Cisco X2-10GB-LX4

1310

MMF

62.5

50.0

50.0

500

400

500

300m

240m

300m

Cisco X2-10GB-LR

1310

SMF

G.652

-

10 km

Cisco X2-10GB-ER**

1550

SMF

G.652

-

40 km***

  •  Distances at some casino properties may require use of single-mode fiber