Oxford University IPv6 deployment

Our full deployment schedule is more complex and will change, but the list below gives an approximate guide to the work which is currently underway.

The focus right now is on preparing the core services for IPv6 support. Part of this will require some funding from the university, which IT Services has applied for.

1. Older switch hardware performing IPv6 functions in software will be replaced. Since the last backbone upgrade all major devices (i.e. the routers and FroDos) support IPv6 routing and switching in hardware. This item applies to older devices in remote data centres.
2. Routing of IPv6 will be enabled on the university backbone.
3. Underlying core service software (e.g. DNS) will be upgraded and tested.
4. Supporting services (e.g. NTP, Web Proxy) will be upgraded to support IPv6 in readiness, despite not being reachable via IPv6 at the time of configuration.
5. A test wireless network will be the first network to have IPv6 enabled, as it does not depend on the IP Address Management systems and is separate to other production university network services.
6. IPv6 will be ready for university IT support staff to deploy locally.

The university has an IPv6 allocation of 2001:630:440::/44.

The university was originally allocated 2001:0630:0051::/48 by JANET. While planning the IPv6 deployment it became apparent that the unique structure of IT administration in Oxford University meant more subnets would be required, assuming units were to be making use of EUI-64 addressing and multiple internal subnets. A /48 would have only allowed for 256 departments or colleges and there are already 210 in the collegiate University.

An application was made through JANET to RIPE and as a result the university now has the larger 2001:630:440::/44 allocation.

With IPv6 it helps to think in terms of the number of subnets, not hosts as we currently do with IPv4. This is because the standard IPv6 subnet size of /64 effectively supports an unlimited number of connected hosts. The /64 subnet size comes from a technique known as EUI-64 whereby the MAC address of an interface is transposed into the IPv6 address's lower half, allowing an IPv6 address to be automatically assigned without collision (i.e. no DHCP for address allocation, as in IPv4). EUI-64 uses 64 bits for this transposition, hence the standard subnet/host boundary is at /64.

IPv6 addresses are written in hexadecimal notation, and this is also used for naming the reverse DNS zones. These zone files tend to be split on what is called the nibble boundary (a nibble is four bits, i.e. one hexadecimal character). The fallout from DNS zone names being limited to nibble boundaries is that when the hierarchy of IPv6 within Oxford university is created, there are potential constraints on where the network separation takes place. The allocation cannot be split at any single bit as can be done with IPv4 (/22, /23, /24, etc). Rather, splitting is constrained to the nibble (four bit) boundaries (/44, /48, /52, /56, /60 and /64). This influences what size IPv6 network is assigned to a department or college, and then of course how many departments or colleges can be supported overall.

Another point to note is that IPv6 has no ARP, but instead uses something called Neighbor Discovery. Related to this is that there's no concept of broadcast traffic. The equivalent is instead multicast to a well-defined group, an example of which happens to be the all hosts on this subnetgroup. The core details are covered in RFC3513. To deploy IPv6 you should have multicast capable switches.

Our planned constraints are:

• The standard and smallest subnet size is /64
• Hierarchy is established on the nibble boundary
• A unit may very well require more than 16 active routed networks
• Support for EUI-64 addressing

Based on the above each department and college will be allocated a /56 network, providing it with 256 subnets for internal use. This shall be the single and maximum allocation to any department or college/

Other than the departments and colleges, there are a number of uses for subnet allocations within the university:

• Router loopback addresses - /128s
• Central servers (data centre networks) - /64s
• Location independent device networks (e.g. wireless) - /64s
• Transfer Networks (aka linknets or transfernets) - /64s

There are varying opinions on the correct size of a Transfer Network (what is typically a /30 router-to-router link under IPv4). For technical reasons the "obvious" choice of /127 is actually not usable, nor in fact is anything smaller than a /120 (see RFC 3627).

The three common choices are /126, /112 and /64. JANET-UK are using /126 transfernets to connect to sites, as are other ISPs such as clara.net. RFC 3627 gives reasons why /126 and /112 transfernets may be at best fiddly to manage, or worse, break certain applications.

Although the /64 is seen as wasteful, it's extremely unlikely that we will encounter a router in use on a department/college network which will not support that subnet size, as it's the RFC defined default. Masks other than /64 are not common, so may not be supported by naive hardware vendors. However, /112 is now becoming more popular, and so it's possible to assign over one /64 to contain 2^32 transfernets of size /112. Only bits 81 through 112 can be used, 65 through 80 will need to remain at zero (for reasons covered in RFC 3627).

One recent finding is that access control lists on some switches, implemented in hardware, often are limited to matching addresses of either /64 or /128 in size, or some fixed point in between such as /80. The technical reason is that these devices allocate TCAM memory in chunks suitable for IPv4 addresses and so supporting IPv6 addresses cannot be done with full bitmask flexibility. Implementing infrastructure access lists might be awkward with /112 masks on such platforms.

To maintain the most flexibility and avoid potential problems we will settle on using /64 for linknets.

The university's allocation will be split into 16 x /48 networks:

This single /48 provides subnets required on the university backbone network independently of a department or college. The most common example of this is a server network for a central service provider such as IT Services, BSP or NSMS. The network is split into 32 /53 sections, each providing up to 2048 /64 service networks.

In the first /53 section, one /64 subnet will be reserved for providing the loopback addresses used by routing devices on these networks. A further 1023 /64 subnets are available for server networks, and a further 1024 subnets will be available for linknets as explained earlier in this document.

It is likely that the first block of server networks (with more memorable and recognisable numbers) will be reserved until we have a clear picture of the eventual allocation strategy for central services.

Each unit (that is, a department, college, PPH) shall be allocated one /56 network and no more. It is expected, and hoped, that this will be the only network ever allocated to the unit. A multi-site unit will be expected to use this allocation for all sites. Routing can be hosted within the university backbone by IT Services, or within the unit, in the same way as the current IPv4 system allows.

According to the standard model of using /64 subnets, each unit will have 256 subnets available for use within their site(s). An example of the use of such subnets might be the common present day division of staff, students, libraries and so on. This might extend to, say, research groups in the future.

It is the opinion of some that layer 3 functionality will get pushed down to the edge of the network, over the next 10-15 years, meaning it becomes more common to have many routing devices within a site. This would allow for the example of a research group being given their own /64 subnet.

This initial model supports up to 768 units within the collegiate University. Networks will be allocated to units in strict sequential order as they are requested. The last two nibbles of the /56 network will not encode any special meaning (though the 0441 - 0443 denotes this is a unit prefix, at least).

Although a complete /56 will be allocated and routed via a linknet to a unit on request, if annexe sites are in use it might be appropriate to allocate a /60 section to each site and route that locally, supporting up to 16 sites or fewer if additional networks are required at one site.

For some large scale projects such as OWL, eduroam and potentially a commercial Internet connection and Residential Network (ResNet) we need to deploy a set of subnets across the whole backbone network. The most straightforward way to achieve this is to allocate large sections of IP space and automate the provision on each backbone network router.

With IPv4 this could easily be done with RFC1918 prefixes but with IPv6 we instead reserve some large blocks for Site-wide Service Networks. There is one /48 for networks we might trust as being internal to the university, such as a student residential service, and another /48 for external untrusted networks (eduroam, etc).

Within the /48 we have up to 16 services of that type (internal or external), and the division below that is by router device allowing for an aggregation point at the /57 boundary.

It can be useful to classify hosts as University or non-University, for the purposes of firewalling etc. A convenient aspect of the current University IPv4 allocation is that it's possible to make an easy division between the class B and C networks. With the single IPv6 allocation we must arbitrarily draw a line somewhere in the /44 pool, but without knowing what potential IP uses might dramatically change the addressing needs of the university in the future.

In the meantime a simple distinction can still be achieved to start with. The External Site-wide Service networks are of course non-University addresses, conveniently placed in the highest /46 of our allocation. Therefore the prefix 2001:0630:044c::/46 is non-university and the remaining is (somewhat trusted) IP space.

Router Advertisements will (from 2011-2012) be off by default on IPv6 networks added by the network team unless the IT officers for the network in question request otherwise.

In the immediate short term the networks team suggest servers should ignore router advertisements [e.g. to avoid disruption from accidentally configured production equipment, or rouge devices advertising], unless the IT officer has requested Router Advertisements on the network. Staff should be aware this advice/policy may change in 2012 onwards.

Client machine networks (e.g. desktops, laptops) are expected (with the approval of local IT support staff) to be on a network running Router Advertisements for client auto configuration. Hence it's recommended that client machines such as desktops and laptops should listen to Router Advertisements on the network[1].

[1] This is merely a clarification of the advice and policy in regards to Router Advertisements and does not rule out the use of DHCPv6 if the local Network Administrator prefers this.

In regards to DHCP v6, the university DHCP service and IP address management system that provides a management interface to IT officers must be upgraded first, there isn't a date for this yet but the project is ongoing.